In my last post I talked about Velocity Anchors, and how applying metaphors such as technical debt takes away from the real problem, that developers want to get stuff done, and there’s things slowing them down. I also mentioned a couple of causes, WTF and Code Jenga. WTF representing code that developers waste time having to figure out before they can get stuff done, and Code Jenga being the fear that even the smallest change will cause cascading failures and a lot of wasted time figuring out the problem. Time that could better be spent getting stuff done. To continue the series, I’d like to talk about one of the biggest causes of WTFs: Speculative Generality*. What speculative generality means is the inclination to speculate that some piece of functionality will be generally useful. I found a great illustration of this concept recently, even if it wasn’t named that, from the chaosinmotion blog: How (not) to write factorial in java. Its a great post that uses some solid coding examples to illustrate the point, and I think his conclusion really nails it:

The biggest complaint I have with many Java developers is that they develop a whole bunch of really bad habits. Specifications are unclear, or they think someday the code may need to be extended into a different direction. So they write a whole bunch of overblown architectural nonsense, sight unseen, thinking that the additional crap someday will help out and make things easier. And Java as a language lends itself to doing this very quickly and easily, so that (as the theory goes) it’s easy for us to build architectures that will someday make it easier on us in the future.

But the future never gets easier, does it?

Later, in responding to comments, he presents his theory on why this tends to happen so often in programming:

Java is simple to program. With a good IDE like Eclipse, Java is a pleasure to write programs in. It’s also quite easy to write some very complicated algorithms, and building a client/server stack is absolutely trivial.

But programmers like complexity. We like things that are complicated: the flashing complicated lights on the control panels on Star Trek, or the joys of figuring out a complicated calculator with nearly a hundred buttons or playing with cell phones that have dozens of modes and menus and buttons. Programmers are puzzle solvers, and we love puzzles. We love taking things apart, putting things together again: many of us are the ones who were punished at 8 for disassembling the family computer or television set or radio, despite the fact that we nearly put it together in working order.

We love complexity.

His essential theory is that Java is easy to crank out code and that deep down developers love complexity. There’s probably something to that. However, with the exception of assembler, there hasn’t been a real restriction in programming that prevents developers from writing too much code, at least programmatically. Perhaps years ago when you had to code in punch cards and mail them off to be compiled, or wait for cpu cycles on a mainframe, then your development environment was limiting you. Nowadays, I think any language could be guilty of making it easy to slap around a lot of architecture or abstractions you may not need. I also agree fundamentally with his argument about certain programmers loving complexity. There is certainly some nerd pride there, and in some cases legitimate attempts by developers to make themselves too necessary to fire. However, for the sake of argument, let’s assume all developers are acting with integrity. That they’re not doing something malicious to guarantee themselves a job, but are honestly doing what they think is right.

In my experience, what has caused the majority of the type of code he describes in his blog is the statement I’ve heard every-time I’ve questioned overly complex code: ‘I know that we’re going to need to do X in the future’ There’s always something that the developer just knows for 100% sure will need to be done. Next month we’ll need to be able to swap in a new implementation of this algorithm. There might be a performance issue, so we’ll definitely need to be able to swap out this strategy. I know there could be a lot of references in response to this that: You ain’t gonna need it. However, I don’t think that gets to the heart of the issue. I think the more important question is what’s causing them to think that they are going to need it? Why aren’t they thinking about the complexity cost they’re putting into the code? The technical debt they’re adding. Or WTF count, or whatever. The fact that the next developer in 6 months has to figure out why you made something simple into a pluggable architecture, figure out how it works, and then try and bolt code onto it. Why?

Architecture Astronauts

One motivation certainly comes from the architecture astronaut camp. You can spot these types in a few ways. Sometimes they’ll come out and say: “I prefer providing frameworks, not mucking around with business rules”. They’ll usually also exist in some kind of special architecture group that delivers ‘frameworks’ to ‘business groups’. They’re also at varying levels of the organization. Some ‘enterprise architects’ make decisions to use things like SoA or to split your web application into 10 war files or a host of other decisions that lead to complexity. Their heart is usually in the right place, and they honestly believe they’re making things better, but they’re usually so separated from delivering actual functionality, to actual end users, that they don’t realize how much pain and complexity they’re causing. They don’t have to deal with the outcomes. There are also architects actually writing framework code. But when you’re an architect, architecture is the solution to every problem. You look for additional places to add architecture. You also almost never have to actually use it, so you don’t feel the pain. And finally, there’s the cause I’ve heard most often: ‘The average developers here aren’t capable enough to work with [insert technology here] so we need to provide an architecture to make it easier for them”. Even though it sounds like it, in most cases they aren’t trying to belittle their coworkers. In many organizations architects are the senior developers and ‘application developers’ are the junior developers, at least from an experience perspective. There could even be a case where architects are on-shore and application developers are off-shore, which is a whole other level of organizational pain. However, my main point is that they live outside of delivering business value. Their job is to deliver complex frameworks. I’ll leave out the organizational challenges that often lead to these positions being created, but in my experience its one of the biggest causes of overly complex code. I could talk for a long time about this particular subject, but I’ll try and stay on topic and move on to the next cause of speculative generality.

Inability to effectively refactor the codebase

Even when working with an application developer who is focused on actually delivering working code to end users and truly wants to deliver it quickly, you will find they have implemented a crazy abstraction for no good reason. As I mentioned earlier in this post, the common reasoning is: ‘I know I’m going to need feature x, so I created the strategyfatoryplugabblewidget’. Ok then you say, why can’t you add that new feature next month when you need it? In my experience, the true reason is generally because they know they’ll be afraid to next month. The codebase is finicky, there aren’t a lot of tests, the feature they’re working on right now will have been signed off on, whatever. Bottom line is, they won’t want to change it in the future for fear of breaking the existing functionality. The overall system makes it difficult to refactor any code at all. So, anytime they put in any new feature, they try and future proof all of it, so they just need to add one new class, or extend one abstract class or implement some interface and wire it in with spring. Whatever it is, they never change or improve the existing codebase. Their coding is just one long exercise in bolting in new classes and fixing bugs in existing code. Sometimes the motivation can be worthwhile. They don’t want to waste time ‘refactoring’, they want to provide new functionality. Of course, this generally leads to complex and buggy code, which eventually catches up with them, causing work to grind to a halt, but they don’t want to think about that now. And in their defense, without reasonable automated testing or an army of manual testers, refactoring code is really really hard. And the longer you let things go, the harder it is to refactor. It’s a vicious cycle.

So what’s the conclusion here? Of course, keeping hammering on the yagni principles. Continue to push to make sure you’re doing the simplest thing that could possibly work. However, make sure you code is reasonably easy to refactor. Encourage refactoring in whatever way you can. Although, it’s probably a separate post in and of itself about how to work refactoring into a general feature development cycle, but it can be done. And ditch the architects. Everyone is delivering business value, everyone is feeling the same pain. And don’t accept complexity just because the code works. It has it’s own cost.

* Credit where credit is due, I got this term from Ram Singaram, a former colleague at ThoughtWorks.