When I was a junior developer, the sort of books and blogs I read off-the-clock tended to be about design patterns, abstractions, test-driven development (TDD) and so on. These topics haven’t gone away, but these days we tend to focus more on rapid application delivery, continuous integration, feedback loops and continuous improvement in the development process of that software, not necessarily the software itself.
Much of the classic developer reading material has strong craftsman-like overtones, the excellent Pragmatic Programmer being the most obvious example. On the other hand, some of the more recent material, like Continuous Delivery or The DevOps Handbook, describe the process of developing software as a factory production line to be optimized, inspired by the lean manufacturing revolution at the likes of Toyota in the 1980s.
Is the change in imagery from the master artisan to the fine-tuned production line a sign of progress or something to be worried about? It looks like the software industry’s Industrial Revolution might have started.
Wait, wasn’t that about steam engines?
The Industrial Revolution is a period of history around the late 18th to the early 19th centuries where hand-production methods were replaced with machines in the factory setting. Prior to the industrial age, the major economies of the world were craft economies. Skilled craft workers (I’ll refer to them as Artisans) were highly specialized manual workers who had sufficient experience and aptitude to be considered an artist in their chosen discipline. Blacksmiths, carpenters, bakers and watchmakers are some examples. All of these crafts still exist today, but none are the primary means of production for their chosen products anymore. They pretty much all have niche status.
The majority of products in the craft economy were unique, but despite their artistic quality this turned out to be detrimental to customers. Replacement parts had to be manufactured from scratch by the person who crafted the original work. There was no standardization, no economies of scale and lots of subjectivity around quality. Craftsmen were also quite siloed into their particular craft and unable to resolve issues outside of their narrow (but deep) focus.
The fall of the artisanal methods can be linked to the waste that this situation caused. The Industrial Revolution brought simple mass production techniques, a greater focus on repeatability, workers that were less specialized but more flexible, standardization, reduced time to market, reduction of waste and reduced cost of producing goods. Arguably, the products of industrial methods did not have the artistic quality of an artisanal product, they might even be less robust, but they won in the marketplace regardless and have become the primary driver of the global economy ever since.
Signs of waste
You might be in a situation where you have a really smart team that produces high-quality work, but still have a lot of waste in your development process and risk being overtaken by a more streamlined competitor.
Signs that indicate a wasteful development process include:
Normally when we talk about premature optimization in software development we’re talking about algorithm efficiency and why we shouldn’t try to solve a scaling problem in code before we even have a scaling problem. This undoubtedly is a cause of waste, but an even greater source of waste than this is the premature optimization of architecture.
Gifted engineers are hard-wired to look for opportunities to apply a sophisticated technique that they know to solve a problem, it gives them a kick. However, when taken too far the link between the original problem and the benefits of the design technique employed to solve it can be quite strained, it becomes a case of Maslow’s Hammer. The work becomes more about demonstrating cleverness in the solution than something appropriate to the size of the original problem, leading to overengineered solutions.
Very talented engineers can descend into self-indulgence quite easily unless they also have a good eye for this sort of situation. Cleverness and pragmatism are not mutually exclusive, but engineering self-indulgence is one of the biggest causes of waste that I have seen regularly. It matches quite well with the image of the Artisan who was only concerned with the artistic quality of their work.
The best way to avoid this is to form a culture of taking decisions based on data, not on engineering sentiment. Are your users asking for you to support a different type of database? Can you prove it will improve the bottom line for the business? If so, then maybe it’s time to look at implementing the abstraction that will enable that cleanly.
“Not Built Here” Mentality
I have spent plenty of time in my career (mostly the unglamorous moments) working on some old legacy code that is designed to solve the same problem as some much more widely used components that might be available in the open source world. Why have your own in-house solution for something when you can follow the well-trodden path of others? Unless you can link it to some competitive advantage for your product, it is of no use. For me, the cost argument for not replacing it doesn’t hold water. It’s the long-term cost of not paying your technical debt that hurts you, rather than the short-term cost of just getting it done.
Using standard components was one of the key competitive advantages that mass production had over the craft economy when the Industrial Revolution took hold. Reducing your waste in these areas will allow your engineers to focus on those areas which do differentiate your product from the competition.
Lack of Automation
“So that ten men, by the aid of machinery, can accomplish with uniformity, celerity and ease, what formerly required labour of one hundred and ten.”
Richard Beamish, Assistant to Isambard Kingdom Brunel.
The most obvious difference between a team of pre-industrial craftsmen and a factory is the use of machines and automation. By automating as much of the manufacturing process as possible a small team of factory workers could achieve more consistent results, more quickly than a much larger team of craftsmen with no automation.
I’ve heard critics of test automation (it’s hard to believe there are any, but there are a few) complaining about the cost of maintaining automation that has replaced other forms of testing. It’s true that some workers need to reskill to look after the machines, just as they did in the industrial age, but when those machines are built well this should be just a fraction of the cost of what the automation replaces. History is certainly not on the side of those who consider it a fruitless endeavor.
Automation is not just limited to testing or quality control, it applies to build, application delivery and live monitoring as well. If your team is lacking in automated processes assisting in daily work, it’s a sign that you could be overtaken by a more productive competitor in future.
A lone craftsman toiling away in their workshop for weeks is an image that easily links the pre-industrial age worker with a modern-day developer who works alone in a feature branch for weeks before committing everything back to the mainline branch in one go.
Such a working pattern is extremely common in the industry, but it is incompatible with modern production methods. A production line works well because it is made up of continuous, small units of change that can be verified and quantified at every stage. Small units of change aids in detecting defects accurately, as soon as they are introduced, which reduces the cost of resolving them. The software practice that most accurately reflects this ideal is Continuous Integration, it promotes a methodology that can be measured scientifically.
Infrequent, large units of change are an indicator that you are neither making the most of your feedback loops or measuring the effects of those changes effectively.
The software industry is full of specialists. To name but a few:
- Developers who specialize in a particular programming language, or maybe even a particular framework.
- Testers. Automated, manual, performance specialists or maybe even test analysts.
- Product Owners and Scrum Masters.
- Sysadmins, network engineers and database administrators.
Workers who know their onions in the areas they contribute real value are extremely valuable to their employers. Expertise is necessary for solving the most difficult problems, however, expertise in a narrow field is rarely required 100% of the time for any project. If your experts are not comfortable outside of their silo, how do you deal with the waste that causes?
The industry recognized some time ago that inter-team dependencies in a project were terrible for the predictable, reliable delivery of that project. Many more teams these days are formed around the end-to-end value stream of their project rather than the functional discipline of its members, in order to avoid having to coordinate different teams with different immediate priorities towards a common goal.
Flexibility was a key advantage of factory workers versus craft workers in the industrial era. The team of workers could adapt more easily and efficiently to the immediate needs of production than the siloed experts could.
Signs of progress or the death of the expert?
I think a lot of us have been conditioned, by culture or otherwise, to have quite a romantic idea of the master craftsman in their workshop and a more negative image of the factory worker. The reality of history is that the craftsmen were inefficient and the factory workers had a greater impact on our quality of life. For example, Henry Ford and his workers made it possible for the average American to buy a car for the first time in the early 20th century.
As engineers and specialists, we like to think of ourselves with the same romantic notion we have for master craftsmen, but we shouldn’t be afraid of the change required to develop our products in more efficient ways, even if it might be to the cost of something that we hold quite dear to ourselves. It might be that by focusing only on our favorite tool or area of software development, or just being stuck in how we have always worked, we hold our teams back from making greater gains for our business and society in general.