One of the great things about Grails is how easy it is to do integration testing. When running the integration tests, the grails container will be started up, including an in-memory version of hsqldb. (and it looks to go to h2 in 1.4) When you combine that with controller mocking, you can easily test from you controller down to the database. However, because you’re getting your entire system, minus only the web server, it can take a while to startup. It’s not too bad for a CI build, but can get annoying when you’re writing and running tests in your development environment. One of the big culprits in terms of time is spring security. I’m not sure why that is, but it is possible to disable it in your integration test environment. The first step is to set the active configuration property to false in Config.groovy:

test {
  grails.plugins.springsecurity.active = false        
}

Now, when you run ‘grails test-app integration:’ you should see something like this:

grails test-app integration:
Welcome to Grails 1.3.6 - http://grails.org/
Licensed under Apache Standard License 2.0
Resolving dependencies...
Dependencies resolved in 5123ms.
Environment set to test
Starting integration test phase ...
Spring Security is disabled, not loading

The important thing to notice is the last line: Spring Security is disabled, not loading

Of course, there’s now a new problem, any code you have that is referencing springSecurityService will now fail. In general, the most likely method to cause an issue is getCurrentUser(). However, it’s simple to create a quick stub for it:

class StubSpringSecurityService {

    def currentUser

    Object getCurrentUser() {
        return currentUser
    }

    String encodePassword(String password, salt = null) {
        return password
    }

}

In my case, I only needed to stub getCurrentUser() and encodePassword, however, you may need to add more, and the general approach can be applied to any additional methods you may need. It is also worth noting that I debated about using something like stubFor, but decided it was more straightforward to make a simple stub.

Now that you have a stub, you need to make sure that it’s being set in the test environment. You can do this by adding the following to resources.groovy:

if (GrailsUtil.environment == "test"){
  springSecurityService(StubSpringSecurityService)
}

Now for the last step, setting up the current user in your bootstrap:

if (GrailsUtil.environment == "test") {
      def testUser = new User()
      testUser.save(failOnError: true)
      springSecurityService.currentUser = testUser;
}

Any code running during an integration test, will now get back the saved user if it calls getCurrentUser()

I was reading an article I saw linked to from a blog I follow:

http://devblog.blackberry.com/2011/02/thanks-for-the-open-letter-to-rim-developer-relations/

Ironically, it’s a response to an open letter to RIM about how difficult it is to use their PlayBook developer tools. I was doing other things when I noticed that page will still loading, so I took a quick peak into the html, and saw this:

<script src="http://code.jquery.com/jquery-latest.js"></script>
<script src="http://dev.jquery.com/view/trunk/plugins/validate/jquery.validate.js" type="text/javascript"></script>

This should go without saying, but don’t do that! That’s not the only offense, but certainly the most costly in terms of loading a page. There is actually another section that reference jquery locally, so I’m not sure how that got in there, but it has a pretty huge negative impact to have to download the unminified version of jquery from jquery.com.

I’ve been having some fun lately with multi-platform development. Personally, I develop on a Mac. However, I also have an Ubuntu workstation that runs a MySQL instance among other things that I can connect to at home. It’s a nice setup, since production is ultimately running on ubuntu server and I have always advocated developing on an environment as close to production as possible. (Our build server is Ubuntu as well) However, one of the other devs uses Windows and has MySQL installed on his Windows machine as a development environment. This can create interesting case-sensitivity problems. Anyone using the default Mac setup (HSF+) or Windows have case-insensitive file systems. (Although, I run a case-sensitive variant of HSF+ on my mac) This can be important if you are also running MySQL on that system, as described below:

In MySQL, databases correspond to directories within the data directory. Each table within a database corresponds to at least one file within the database directory (and possibly more, depending on the storage engine). Consequently, the case sensitivity of the underlying operating system plays a part in the case sensitivity of database and table names. This means database and table names are not case sensitive in Windows, and case sensitive in most varieties of Unix.

This can be especially interesting in Spring Batch. By default, SB uses all uppercase table names. An example being BATCH_JOB_EXECUTION. If for example, your windows colleague creates the tables in lower case on his Windows system, they will work fine for him/her, since the underlying file system isn’t case sensitive. However, if you take a data dump from his system and put it into your linux mysql instance, running batch jobs will result in the following error:

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'dbmaster.BATCH_JOB_INSTANCE' doesn't exist

Or something similar. What could get even wackier is if you change the prefix of your Spring Batch tables. Setting the prefix to something like ‘awesome’, would result in SB querying for a table such as ‘awesome_JOB_EXECUTION’, which again, would work fine on Windows but blow up anywhere else. (Unless you happened to create the table in alternating caps)

There are some options, however. There is a property in MySQL called ‘lower_case_table_names’ that allows you to configure how MySQL deals with table name resolution:

• 0 - tables are stored on disk using the letter case defined in your create table statements and all checks are case sensitive. (This setting could be problematic on non-case sensitive systems)
• 1 - all table names are stored on disk as lowercase regardless of how you define them in your create statement, and any table names in queries will be converted to lower case
• 2 - table names are stored as defined in the create statement but mysql converts them to lowercase on lookup. Which obviously won’t make a difference on a case-insensitive system.

Note: lower_case_table_names was introduced in MySQL 4.0.17

On my ubuntu install of MySQL lower_case_table_names is set to 0, which I assume to be the default. However, you can check your system easily by entering: ‘show variables;’ into mysql.

On my local ubuntu instance I create a file called: “/etc/mysql/conf.d/lower_case_table_names.cnf”. The .cnf ending is important, as that is what my.cnf is setup to look for, at least in my setup. In the file is the following:

[mysqld]
lower_case_table_names=1

The [mysqld] is important as there are numerous engines which are configured in the same file. For example, you can all create settings for mysqldump specifically as well. Next time you bounce mysql, if you make another call to ‘show variables’ you should see the updated setting. After this change, even though Spring Batch is making SQL statements in all caps and the tables are actually stored in lower case, the queries will still work.

Amazon RDS

It’s worth noting that if you use Amazon’s RDS service that you must use their web console (or command line tools) to modify this setting. It’s under parameter groups. The following guide from amazon should help you: http://aws.amazon.com/articles/2935

First: Watch this hilarious VW spot on YouTube: http://www.youtube.com/watch?v=R55e-uHQna0

Second: Press the share button below the clip.



On the top we’ve got how many people shared this link on twitter, facebook or buzz. Do we really care about that?

Then the most important bit, the actual link, which we can shorten. Although I don’t know why, doing so cuts the size down by 25%. I’ve always thought YouTube links were some of the simplest out there: youtube.com?watch?v=SomeRandomNumber. “Shortening” it turns it into: youtu.be/SomeRandomNumber. Go Belgium I guess.

And then there’s no less than 9 places I can share it. Half of them I haven’t heard of. And really, Myspace?

Can I just have a link?

In my last post I talked about Velocity Anchors, and how applying metaphors such as technical debt takes away from the real problem, that developers want to get stuff done, and there’s things slowing them down. I also mentioned a couple of causes, WTF and Code Jenga. WTF representing code that developers waste time having to figure out before they can get stuff done, and Code Jenga being the fear that even the smallest change will cause cascading failures and a lot of wasted time figuring out the problem. Time that could better be spent getting stuff done. To continue the series, I’d like to talk about one of the biggest causes of WTFs: Speculative Generality*. What speculative generality means is the inclination to speculate that some piece of functionality will be generally useful. I found a great illustration of this concept recently, even if it wasn’t named that, from the chaosinmotion blog: How (not) to write factorial in java. Its a great post that uses some solid coding examples to illustrate the point, and I think his conclusion really nails it:

The biggest complaint I have with many Java developers is that they develop a whole bunch of really bad habits. Specifications are unclear, or they think someday the code may need to be extended into a different direction. So they write a whole bunch of overblown architectural nonsense, sight unseen, thinking that the additional crap someday will help out and make things easier. And Java as a language lends itself to doing this very quickly and easily, so that (as the theory goes) it’s easy for us to build architectures that will someday make it easier on us in the future.

But the future never gets easier, does it?

Later, in responding to comments, he presents his theory on why this tends to happen so often in programming:

Java is simple to program. With a good IDE like Eclipse, Java is a pleasure to write programs in. It’s also quite easy to write some very complicated algorithms, and building a client/server stack is absolutely trivial.

But programmers like complexity. We like things that are complicated: the flashing complicated lights on the control panels on Star Trek, or the joys of figuring out a complicated calculator with nearly a hundred buttons or playing with cell phones that have dozens of modes and menus and buttons. Programmers are puzzle solvers, and we love puzzles. We love taking things apart, putting things together again: many of us are the ones who were punished at 8 for disassembling the family computer or television set or radio, despite the fact that we nearly put it together in working order.

We love complexity.

His essential theory is that Java is easy to crank out code and that deep down developers love complexity. There’s probably something to that. However, with the exception of assembler, there hasn’t been a real restriction in programming that prevents developers from writing too much code, at least programmatically. Perhaps years ago when you had to code in punch cards and mail them off to be compiled, or wait for cpu cycles on a mainframe, then your development environment was limiting you. Nowadays, I think any language could be guilty of making it easy to slap around a lot of architecture or abstractions you may not need. I also agree fundamentally with his argument about certain programmers loving complexity. There is certainly some nerd pride there, and in some cases legitimate attempts by developers to make themselves too necessary to fire. However, for the sake of argument, let’s assume all developers are acting with integrity. That they’re not doing something malicious to guarantee themselves a job, but are honestly doing what they think is right.

In my experience, what has caused the majority of the type of code he describes in his blog is the statement I’ve heard every-time I’ve questioned overly complex code: ‘I know that we’re going to need to do X in the future’ There’s always something that the developer just knows for 100% sure will need to be done. Next month we’ll need to be able to swap in a new implementation of this algorithm. There might be a performance issue, so we’ll definitely need to be able to swap out this strategy. I know there could be a lot of references in response to this that: You ain’t gonna need it. However, I don’t think that gets to the heart of the issue. I think the more important question is what’s causing them to think that they are going to need it? Why aren’t they thinking about the complexity cost they’re putting into the code? The technical debt they’re adding. Or WTF count, or whatever. The fact that the next developer in 6 months has to figure out why you made something simple into a pluggable architecture, figure out how it works, and then try and bolt code onto it. Why?

Architecture Astronauts

One motivation certainly comes from the architecture astronaut camp. You can spot these types in a few ways. Sometimes they’ll come out and say: “I prefer providing frameworks, not mucking around with business rules”. They’ll usually also exist in some kind of special architecture group that delivers ‘frameworks’ to ‘business groups’. They’re also at varying levels of the organization. Some ‘enterprise architects’ make decisions to use things like SoA or to split your web application into 10 war files or a host of other decisions that lead to complexity. Their heart is usually in the right place, and they honestly believe they’re making things better, but they’re usually so separated from delivering actual functionality, to actual end users, that they don’t realize how much pain and complexity they’re causing. They don’t have to deal with the outcomes. There are also architects actually writing framework code. But when you’re an architect, architecture is the solution to every problem. You look for additional places to add architecture. You also almost never have to actually use it, so you don’t feel the pain. And finally, there’s the cause I’ve heard most often: ‘The average developers here aren’t capable enough to work with [insert technology here] so we need to provide an architecture to make it easier for them”. Even though it sounds like it, in most cases they aren’t trying to belittle their coworkers. In many organizations architects are the senior developers and ‘application developers’ are the junior developers, at least from an experience perspective. There could even be a case where architects are on-shore and application developers are off-shore, which is a whole other level of organizational pain. However, my main point is that they live outside of delivering business value. Their job is to deliver complex frameworks. I’ll leave out the organizational challenges that often lead to these positions being created, but in my experience its one of the biggest causes of overly complex code. I could talk for a long time about this particular subject, but I’ll try and stay on topic and move on to the next cause of speculative generality.

Inability to effectively refactor the codebase

Even when working with an application developer who is focused on actually delivering working code to end users and truly wants to deliver it quickly, you will find they have implemented a crazy abstraction for no good reason. As I mentioned earlier in this post, the common reasoning is: ‘I know I’m going to need feature x, so I created the strategyfatoryplugabblewidget’. Ok then you say, why can’t you add that new feature next month when you need it? In my experience, the true reason is generally because they know they’ll be afraid to next month. The codebase is finicky, there aren’t a lot of tests, the feature they’re working on right now will have been signed off on, whatever. Bottom line is, they won’t want to change it in the future for fear of breaking the existing functionality. The overall system makes it difficult to refactor any code at all. So, anytime they put in any new feature, they try and future proof all of it, so they just need to add one new class, or extend one abstract class or implement some interface and wire it in with spring. Whatever it is, they never change or improve the existing codebase. Their coding is just one long exercise in bolting in new classes and fixing bugs in existing code. Sometimes the motivation can be worthwhile. They don’t want to waste time ‘refactoring’, they want to provide new functionality. Of course, this generally leads to complex and buggy code, which eventually catches up with them, causing work to grind to a halt, but they don’t want to think about that now. And in their defense, without reasonable automated testing or an army of manual testers, refactoring code is really really hard. And the longer you let things go, the harder it is to refactor. It’s a vicious cycle.

So what’s the conclusion here? Of course, keeping hammering on the yagni principles. Continue to push to make sure you’re doing the simplest thing that could possibly work. However, make sure you code is reasonably easy to refactor. Encourage refactoring in whatever way you can. Although, it’s probably a separate post in and of itself about how to work refactoring into a general feature development cycle, but it can be done. And ditch the architects. Everyone is delivering business value, everyone is feeling the same pain. And don’t accept complexity just because the code works. It has it’s own cost.

* Credit where credit is due, I got this term from Ram Singaram, a former colleague at ThoughtWorks.

A former colleague of mine from my ThoughtWorks days, Aaron Erickson recently wrote a blog post about Engineering practices and how they impact a project’s velocity. The post is good, and talks about some common engineering practices to help improve velocity. However, the problem I’ve found on numerous projects, regardless of their level of engineering maturity is that they often don’t see that they even have any velocity issues. The closest most projects get to is: ‘We want to release by date x, with feature-set y and at our current velocity that isn’t possible’. Aaron’s post is a an example of positive ways to improve velocity, rather than the unsustainable practices of throwing more people at it or making everyone work 100+ hours a week. But what really is the proverbial anchor slowing the velocity ship down?

Technical Debt?

Martin Fowler wrote a recent blog post related to software craftsmanship where he talked about how much he hates using metaphors to describe software development. Technical debt is an often used metaphor for describing what is essentially causing velocity anchors in the codebase. This is of course completely separate from process issues. (7 design meetings and signoff sessions before even starting to code will kill the velocity of even the cleanest codebase) It’s a tempting metaphor to use. Everyone understands the concept of debt from their own lives. Your income is severely handicapped if every month a large percentage has to go to paying down debt. The same can be applied mentally too, paying down debt every-time you code. However, there’s a number of problems with using this metaphor.

People don’t understand debt

Earlier I said “Everyone understands the concept of debt from their own lives” No, no they don’t. Especially not in America. In many ways our attitude towards debt in our personal lives mirrors the problems in codebases. Can’t afford the new TV? It’s cool, Best Buy has financing. In fact, in many ways the concept of “I’m taking on acceptable debt now to get a feature out, and I’ll totally going pay it off next quarter” is what gets most companies into trouble in the first place.

The code fairies don’t send you a bill

A credit card bill is tangible. They mail you a piece of paper that says: “You have to pay this much right now or bad stuff is going to happen.” And you know exactly what that bad stuff is. Everyone knows what the end of this path is, and speaking as someone who has recently watched family go through it, bankruptcy sucks. There is no such obviously painful end point to accumulating too much debt in your code. The only thing I can tell you is: you’ll know it when you see it. When you’ve spent years working on a codebase, and have a small percentage of the features you want done, and your retention rate is falling rapidly, you’ll know you have too much debt. However, its really hard to make someone understand that when they’re staring at all the screens in best buy.

Software development is subjective

Is that piece of code technical debt or a really clever architecture? I bet you could take any random snippet from your codebase and ask two developers whether its technical debt or not and get two completely different answers. Usually because one of them thinks you’ll “definitely need to let users add their own custom widgets in the future”. (I’ll save the speculative generality rant for another day) My point is that you could spend a lot of time trying to define what is debt vs. ‘good architecture’ and end up being one of those people who hasn’t actually written code in years and has a calendar where every hour is triple booked.

Its not really that hard of a concept really. I’m not sure it even needs a metaphor. Your developers are trying to get stuff done. There’s a bunch of crap in the way making that take longer than it rightfully should. If you care about developer productivity, you try and hunt it down and fix it. If you don’t, then stop reading this and get back to your gant chart. I bet if you put one more developer on that task the earn/burn ratio will look way better. (Another rant for another day though)

So, what are some of the things getting in the way of getting stuff done?

WTF?

That’s the sound your developer makes when they are looking at a piece of code and can’t figure out what it’s doing. it might be a bug fix or a new feature or whatever. The bottom line is that they’re trying to get something done, but instead they spend hours digging through file after file trying to figure out why there are 7 variables named ‘chart’ that all mean completely different things depending upon which part of the application they’re used in. It’s a simple equation. The number of WTFs is inversely proportional to developer productivity. We could probably even make a Sonar widget for it.

Code Jenga

Have you ever touched a piece of code to make what seemed like a simple bug fix only to realize that you just committed an atrocity? One variable changes, and now the login page has no text? It makes developers afraid to do anymore than the bare minimum required. And if anything more is required, it takes forever, because they now have to climb the mountain of WTFs to accomplish the feature/bug fix.

I’m sure I could come up with some more if I kept thinking about it. But then I’ve seen plenty of teams brought to their knees because they spend the entire time arguing about which design pattern to use, or what’s the most productive IDE, or which BBQ restaurant is best.

The only thing I’ve ever seen work is empowering developers* to make the right decision when the time comes. They know the codebase, they know what’s slowing them down, let them make the decision. Yes, stuff still has to get done. Yes, releases still need to get out. But by and large, if you let developers code and focus on solving business problems, most of the time they’ll make the right call.

*Architects aren’t developers. If they aren’t solving business problems, they’re not a developer. I know, a bit hypocritical coming from someone who wrote a framework.

Unless you have a team full of developers who don’t realize all the little things out of their day that’s killing their velocity. 15 minutes to do a build? 7 properties files to update for each change? 10 Different components that all inter-depend upon each other and require a certain build order for no good reason? People get used to the craziest things, and unless someone points it out to them, won’t even notice. And even if you do, they might just shrug and say: ‘I don’t mind’. Then again, telling them they have too much technical debt doesn’t help much either.

And maybe that’s my point really. Regardless of all the engineering practices or programming language, or IDEs, there’s no guarantees. Switching to Ruby won’t make your project ship on time. Using TDD doesn’t mean you won’t have quality issues. Meticulously planning every second of every day of the project from now until completion doesn’t mean you’ll make that date either. The only thing you can do is focus on getting stuff done, hire the best, most passionate people you can, and hope for the best.

This post should have been made a month ago, so I apologize for the delay. However, last month I left ThoughtWorks to pursue an opportunity a start-up called Fundspire. I have accepted the position of CTO there and will be leading the technical direction of the software. It’s a small company, so that’s really a fancy way of saying that I’ll be the one coding. I plan to continue blogging and contributing to open source, with the same blessing from Fundspire as I received from ThoughtWorks. The Fundspire solution has been implemented in Grails on Amazon EC2, so there should be more posts in the future around those technologies.

Consider the following code snippet:

def someList = ["75", 32, 3.2]

println someList.max {item ->
 return null
}

While working on a Grails project I started investigating groovy meta-programming. I wanted to be able to add one method per class for every Enum in the containing class. (I’ll have an example below)

The official groovy documentation on the topic is here: http://groovy.codehaus.org/ExpandoMetaClass

Most examples show something like the following:

//Add new method call printHello
SomeClass.metaClass.printHello = { print "Hello" }
def instance = new SomeClass()

Which of course when run will print “Hello”.

However, I didn’t know the method was called at runtime, I just wanted to add one for every entry in an enum. It’s not hard, but there isn’t an example out there, after a little playing in the groovy console, I figured it out:

enum Status {A,B,C}

class User{
   def status
}

Status.each{ statusEnum ->
   User.metaClass."is${statusEnum.name()}" = {
       return status == statusEnum
   }
}

def user = new User()
println user.isA()
user.status = Status.A
println user.isA()
println user.isB()
user.status = Status.B
println user.isB()

While will output:

false
true
false
true

Of course, this isn’t exactly complex, and I may find that I don’t like how this turns out, but I couldn’t find an example of this type of Groovy meta-programming example, so I thought I would share.

There has been an interesting discussion going on recently in the Maven user group list about Continuous Delivery:

http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-td3245370.html

My colleague here at ThoughtWorks, Jez Humble, contends that the SNAPSHOT model that Maven uses is prohibitive to many of the principles espoused in his book. And I have to say, I largely agree with him.

Before diving into why I agree with him, it’s worthwhile to add a bit of context. I’m one of a minority of people in ThoughtWorks that prefer Maven to Ant. That’s not to say that we don’t deliver projects using Maven, but if given a choice, most ThoughtWorkers prefer Ant. Personally, I have had a long history of using Maven, starting with my work with Spring Batch. That’s not to say that Maven doesn’t have it’s problems, but if given a choice, I’ll start a project with Maven. (Why that’s the case would certainly be worthy of another blog post) I mention all of this because I am not bringing up issues with the release model of Maven to bash it because I hate the tool. I bring it up because I think there can be improvements in how Maven thinks about and deals with releases.

I remember the first time I had to cut a release with Maven oh so many years ago. I had my project working, everything was in exactly the state I wanted it to be into for release. All I needed to do was create the final cut. Like a good Maven user, I went to check out the documentation for how Maven approached releases. I quickly found the section on the release plugin and thought ‘Aha, I just need to use this plugin, and my release will be painless, thanks Maven!’ Of course, reality is never that simple. The maven release project has two ‘phases’ prepare, and then the actual release. Below is a list of what Maven does during the prepare phase, shamelessly pulled from the official plugin site:

• Check that there are no uncommitted changes in the sources
• Check that there are no SNAPSHOT dependencies
• Change the version in the POMs from x-SNAPSHOT to a new version (you will be prompted for the versions to use)
• Transform the SCM information in the POM to include the final destination of the tag
• Run the project tests against the modified POMs to confirm everything is in working order
• Commit the modified POMs
• Tag the code in the SCM with a version name (this will be prompted for)
• Bump the version in the POMs to a new value y-SNAPSHOT (these values will also be prompted for)
• Commit the modified POMs

The SCM related bits make sense. Of course you need to create a tag that coincides with the release. However, the majority of the remaining bullets are around modifications to the POM files themselves, and checking that other POM files aren’t in a ‘SNAPSHOT’ state.

I didn’t understand that process years ago, and I still don’t understand it today. I have my codebase, I’ve tested it thoroughly, both with unit, integration, automated, and manually testing. I just want to take the ‘binary’ that I’ve been using, and promote it to production while also taging the SCM revision that created it. Why does a snapshot state even matter? I suppose with external dependencies there are some concerns about APIs changing underneath you. However, what does it matter, you have working code, right now, surely the changing API is a problem for tomorrow, not the release of today?

In my time at ThoughtWorks, and other consulting experience, I have always prognosticated that the same build that a developer makes on his machine, and ultimately what will come out of CI (because he or she commits that revision) should look as identical as possible to what will be in production. Why add an extra step to change anything at all?

In short, I agree with Jez, why can’t every artifact created from mvn deploy be potentially a release into production? The only value I’ve seen the snapshot concept add is away to describe a dependency as ‘whatever the latest CI has produced is’. It seems to me its just as easy to say: ‘any release greater than x.y’ and call it a day, which is something Maven supports now as well, at least from a dependency declaration perspective.