Recently, my 8 year old linux server I was running mysql on died. My new computer is more than capable of running mysql in the background, but I didn’t want to mess with moving everything over. Now I’m forced to. It’s extremely easy on Ubuntu to install mysql. However, I run my operating system and all development related applications off of a 64GB SSD. There isn’t room, or a need to run it on the SSD. I have a second platter based hard drive as well. It should be fairly straightforward to store MySql data there. Most instructions for doing so will tell you to do the following:

1. Copy the current data directory to your new partition. (Current directory is usually /var/lib/mysql on Ubuntu)
2. Make sure it is still owned by the mysql user. ‘chown -R mysql:mysql newdatadir’
3. Change the mysql configuration to point to the new directory. Inside of my.cnf is a value datadir, which is what needs to be changed to the new directory.

Sounds easy, right? It is, for the most part. If you try these steps on ubuntu, it won’t work. MySql won’t start up. If you’re like me, the first thing you’ll look is syslog, where you’ll find something like:

[1064644.473299] type=1400 audit(1305502269.192:364):
apparmor="DENIED" operation="open" parent=1 profile="/usr/sbin/mysqld" 
name="/media/data1/mysql-data/" pid=24578 comm="mysqld" requested_mask="r" 
denied_mask="r"fsuid=1001 ouid=1001

Your first instinct is to check the permissions, but they’re all fine. So, what’s going on? AppArmor is preventing access. If you type ‘ls -la /etc/apparmor.d’ you’ll see the following:

drwxr-xr-x   8 root root  4096 2011-05-15 19:18 ./
drwxr-xr-x 141 root root 12288 2011-05-15 18:20 ../
drwxr-xr-x   3 root root  4096 2011-04-11 18:38 abstractions/
drwxr-xr-x   2 root root  4096 2011-05-15 17:38 cache/
drwxr-xr-x   2 root root  4096 2010-10-07 11:06 disable/
drwxr-xr-x   2 root root  4096 2010-08-06 23:19 force-complain/
-rw-r--r--   1 root root   986 2010-09-13 03:07 gdm-guest-session
drwxr-xr-x   2 root root  4096 2011-05-15 17:38 local/
-rw-r--r--   1 root root  2052 2010-08-06 23:18 sbin.dhclient3
drwxr-xr-x   3 root root  4096 2011-04-11 18:38 tunables/
-rw-r--r--   1 root root  2052 2010-09-27 17:58 usr.bin.evince
-rw-r--r--   1 root root  4006 2011-04-29 04:00 usr.bin.firefox
-rw-r--r--   1 root root  4158 2010-10-01 04:58 usr.sbin.cupsd
-rw-r--r--   1 root root   989 2010-11-10 00:51 usr.sbin.mysqld
-rw-r--r--   1 root root  1172 2010-08-06 12:45 usr.sbin.tcpdump

If you open up usr.sbin.mysqld, and add the new directory to the list, your problems will be solved. Something like:

 /mysql-data/ r,
/mysql-data/** rwk,

I recently created a private repository on GitHub, and wanted to create a build for it on my Jenkins server. I ran into some interesting problems that I couldn’t find reasonable solutions to via google, so I thought I would write about my experience.

The first and most important thing to keep in mind is that there are no settings in Jenkins to control or manage SSH keys for Git. You might get confused when you see the SSH section in the general settings, thinking you can upload your git RSA key there and it will work. It won’t. The only thing that will work is setting it in the .ssh directory of the user that is running Jenkins. If you’re running it standalone, it will be the Jenkins user. In the case of a standard Ubuntu install, this was in /var/lib/jenkins/.ssh. If you are running Jenkins inside of a tomcat container, then it will be the tomcat user. (tomcat6 generally in Linux)

The second thing I learned is: If you’re running Jenkins on Windows, it’s going to be painful. Mind you, I’m not running it that way, but nearly every single Google result on this issue was coming back with issues for Windows. It’s not really even Jenkins fault either. I’ll stop short of bashing Windows, but if you’ve ever had to deal with running any type of service on Windows, you’re already familiar with the pain anyway.

And the final lesson I learned? You can’t use a passphrase in your RSA key file. I lost hours to this one. I thought I had some other issue with the key. Until I stumbled on this blog post. It makes sense when you think about it. When you’re running git on your machine you might get prompted for access to a keychain, but after that its seamless so it can be easy to forget that its there. If you follow the default instructions for creating the key on github, it will guide you to create the passphrase. Ignore that and follow the advice on the previously mentioned post. Git should work without issue after that. I would still recommend using a passphrase for your personal account though.

***Update - 4/11/2013***

I finally got around to updating and publishing this plugin: http://www.lucasward.net/2013/04/grails-envers-plugin-update.html

I recently needed to add auditing to my grails application and decided to use envers, mostly for its ability to keep updates to multiple hibernate objects tied together in a way that is query-able afterwards. Unfortunately, because Grails exerts somewhat tight control of Hibernate, and because of how Envers was implemented, getting it to work with grails can be a bit tricky. I did a cursory glance around and couldn’t find anyone that had created a plugin to help with Envers and Grails. So while implementing it in my application, I wrote the envers specific code as a plugin, which I have published on github:

https://github.com/lucaslward/grails-envers-plugin

Eventually, I need to write actual documentation for it, and try and get it in the official grails plugin repository. However, for now, I’ll write up how to use it based on the integration tests.

If you’re pulling it down from the git repo above, you need only do a simple ‘grails package-plugin’, which will create a zip file, which can be then be installed into your grails application via: ‘grails install-plugin /path/to/zip/grails-envers-plugin-0.1.x.zip’. The plugin sets up envers via Hibernate event listeners, and provides gorm style dynamic methods that can be used to query for revisions.

In order to configure your domain objects to use Envers, you need to annotate them with @Audited. (I haven’t written any code to try and make that work with static members a la gorm) I’ll use the sample domain from the plugin:

@Audited
class Customer {

  String email
  String name
  Address address
  SortedSet orders = new TreeSet()

  static constraints = {
    address (blank: true, nullable: true)
    address component: true
  }

  static mapping = {
    orders cascade: "all,delete-orphan"
  }
}

@Audited
class Address {
  String city
  String zip
}

@Audited
class OrderEntry implements Comparable{

  Date date
  double amount
  int numberOfItems
  Customer customer

  static belongTo = [customer:Customer]

  @Override
  int compareTo(OrderEntry o) {
    date.compareTo(o.date)
  }
}

In this example, I have three domain classes: Customer, Address, and OrderEntry. Customer in this class is the main class, who has a one to one relation ship with an Address, and a collection of Orders. Now, let’s assume that we create a customer with an address, then modify the customer and the address twice in separate transactions:

Customer customer
Customer.withTransaction {
  def address = new Address(city: "Chicago", zip: "60640")
  address.save()
  customer = new Customer(name: "PureGorm", email: "tester@gorm.org", address: address)
  customer.save(flush: true)
}

Customer.withTransaction {
  customer = Customer.findByName("PureGorm")
  customer.email = "tester2@gorm.org"
  customer.address.city = "New York"
  customer.save(flush: true)
}

Customer.withTransaction {
  customer = Customer.findByName("PureGorm")
  customer.email = "tester3@gorm.org"
  customer.address.zip = "10003"
  customer.save(flush: true)
}

Note: If you’re trying to test envers in your application you have to turn off transactions for your test and wrap individual calls in transactions as above. (with an appropriate tear-down to clean up afterwards of course)

In this scenario, we should have 3 entries in the audit tables for Customer and Address (customer_aud and address_aud respectively). We can query for this methods with the findAllRevisions dynamic method:

def results = Customer.findAllRevisions()

assert results.size() == 3
def r = results[0]
assert r.name == "PureGorm"
assert r.email == "tester@gorm.org"
assert r.address.city == "Chicago"
assert r.address.zip == "60640"
assert r.revisionType == RevisionType.ADD
r = results[1]
assert r.email == "tester2@gorm.org"
assert r.address.city == "New York"
assert r.revisionType == RevisionType.MOD
r = results[2]
assert r.email == "tester3@gorm.org"
assert r.address.zip == "10003"
assert r.revisionType == RevisionType.MOD

As you can see, findAllRevisions returns an array of three results. The plugin will automatically take the envers RevisionEntity and RevisionType, and set them on the returned revisions, which is how revisionType is being checked. As you can see, all three revisions are present. By default its sorted by the earliest revision first. First is an add, followed by two ‘mods’ (update). The revision entity is also accessible via the revisionEntity property: r.revisionEntity.getRevisionDate(). It’s also worth noting that even a custom envers RevisionEntity will be accessible this way. (See the envers documentation for more details on how to do this)

findAllRevisions also supports sorting in the same way as gorm queries. Assuming the same audit trail as above, you could query it this way:

def results = Customer.findAllRevisions(sort:"email",order:"asc")

assert results.size() == 3
assert results[0].email == "tester2@gorm.org"
UserRevisionEntity entity = results[0].revisionEntity
assert entity.getUserId() == currentUser.id
assert results[0].revisionEntity.getUserId() == currentUser.id
assert results[1].email == "tester3@gorm.org"
assert results[2].email == "tester@gorm.org"

You still get back 3 results. But this time the first result is the first update, followed by the second, following by the initial create. This is because its being sorted by email ascending.

As with sorting, you can also use max and offset:

    
def results = Customer.findAllRevisions(max:1)

assert results.size() == 1
assert results[0].email == "tester@gorm.org"

results = Customer.findAllRevisions(max:1,offset:1)

assert results[0].email == "tester2@gorm.org"

results = Customer.findAllRevisions(max:1,offset:2)

assert results[0].email == "tester3@gorm.org"

Besides sorting and paginating, you can also search for revision by property name:

Customer.findAllRevisionsById(customer.id)

Customer.findAllRevisionsByAddress(customer.address)

Customer.findAllRevisionsByName("PureGorm",[sort:"email",order:"asc", max:2])

The first case above is searching by id, and requires a long. The second is searching by the address, which requires an attached hibernate entity. The final is searching by a simple property name, in this case: “PureGorm”, with some additional sorting, etc tacked on. As with gorm, you can use any property name on the object to query by. At this point, the plugin only supports querying by one property. If you need to query by more, you will need to use the Envers classes directly. (i.e. AuditReader)

The final method available statically is getCurrentRevision():

Customer.getCurrentRevision()

Which gets the current revision number. Meaning, what is the last revision of any customer?

There are also two methods that are applicable on domain class instances as well:

     
List revisions = customer.getRevisions()
assert revisions != null
assert revisions.size() == 3

Customer oldCustomer = customer.findAtRevision(revisions[1])
assert oldCustomer.email == "tester2@gorm.org"
assert oldCustomer.address.city == "New York"

The first method will return all revisions of a particular class. (The same as Customer.findAllRevisionById(customer.id)) And the second will find a particular customer at a particular revision.

There is certainly some functionality I’m missing here, as I’m using the integration tests as an example. I will add more blog entries on the subject as I discover the corner cases I have missed. Hopefully, I can also get time to write actual documentation, and get the plugin in the official repository as well.

We use Ext Js as our main Javascript library at Fundspire. There’s a couple of really cool features we use a lot. One is the GridPanel. It gives you a nice looking ‘table’ with sorting, column resizing, etc. There are also some interesting ways to make multiple ‘panes’ in your application. One example being the border layout. (The layout browser shows all the layouts pretty well) A good example is the official ‘forum example’ from Ext Js: http://dev.sencha.com/deploy/dev/examples/forum/forum.html I’m still relatively new to the framework, but have been able to do some interesting things with it. After a couple more months I hope to write up a more detailed review.

I’ve been working with a lot of these components this week and was having a lot of trouble manipulating the column and border size defaults. My changes just didn’t seem to have an impact. We use jawr, which renames the javascript when there’s changes to force a reload, so that wasn’t the issue. It turns out that the border and column size information is stored in a cookie on the user’s browser! It’s something that’s easily setup via an Ext Js CookieProvider Once you have a cookie provider setup, your state across a lot of components is saved automatically. Which, if you think about it is pretty slick. If someone is using a smaller monitor they can change the sizes to be what they want and not have to worry about it again. However, I went down a lot of rabbit holes when my changes didn’t appear to be having an affect, and I haven’t seen this behavior documented anywhere in the Api docs for the components I’m using. I knew we were saving state in cookies, but didn’t realize that it included these components automatically. So, if you’re using Ext Js and your layout changes don’t appear to be having an effect, make sure that it’s not storing a cookie to save that information.

I ran into a bit of a weird scenario recently with grails. I have a couple of classes similar to those below:

class Foo {
Bar bar
}

class Bar {
static hasMany = [foos: Foo]
}

Essentially, I have a many to one relationship. Now, consider the following code:

Foo foo = Foo.get(1)
Bar bar = foo.bar

Assuming that there is actually a bar set on this particular foo, what will you get?

You’ll get the actual Bar, completely instantiated. Now, imagine the exact same scenario in normal java with the exact same equivalent hibernate settings via annotations. (I’m too lazy to model what that would look like) What would you get back for a similar call? You would get back a proxy! Then, if you called something on the proxy, hibernate would go and fetch the actual object for you.

Gorm unwraps the proxy when the object itself is called via a property.

If you take a look at the HibernatePluginSupport class, which configures all of the dynamic Gorm methods (and the first place I go when I want to see how a particular method is actually working) you will see this in action:

static final LAZY_PROPERTY_HANDLER = { String propertyName ->
  def propertyValue = PropertyUtils.getProperty(delegate, propertyName)
  if (propertyValue instanceof HibernateProxy) {
      return GrailsHibernateUtil.unwrapProxy(propertyValue)
  }
  return propertyValue
}

/**
 * This method overrides a getter on a property that is a Hibernate proxy 
 * in order to make sure the initialized object is returned hence avoiding 
 * Hibernate proxy hell
 */
static void handleLazyProxy(GrailsDomainClass domainClass, 
                            GrailsDomainClassProperty property) {
    String propertyName = property.name
    def getterName = GrailsClassUtils.getGetterName(propertyName)
    def setterName = GrailsClassUtils.getSetterName(propertyName)
    domainClass.metaClass."${getterName}" = 
        LAZY_PROPERTY_HANDLER.curry(propertyName)
    domainClass.metaClass."${setterName}" = { 
        PropertyUtils.setProperty(delegate, propertyName, it)  
    }

    for (GrailsDomainClass sub in domainClass.subClasses) {
        handleLazyProxy(sub, sub.getPropertyByName(property.name))
    }
}

So, anytime you access your properties, the above code is going to unwrap it.

What whoever wrote this is trying to avoid is the ‘proxy hell’ mentioned in the comment. The hibernate docs have a good explanation of it:

http://docs.jboss.org/hibernate/core/3.3/reference/en/html/performance.html#performance-fetching-lazy

As does the Grails docs:

http://grails.org/doc/latest/guide/5.%20Object%20Relational%20Mapping%20%28GORM%29.html#5.5.2.8%20Eager%20and%20Lazy%20Fetching

They’re more eagerly fetching the lazy relationships than Hibernate does to avoid some of the instanceof check issues outlined in the above links.

Now, back to my issue. What happens if you want to obtain the identifier of your relationship without actually resulting in a database call to get it? The proxy has the identifier already, that’s how it loads it up when you call it. In pure hibernate there’s a few ways to do it:

1. Session.getIdentifier
2. entity.getId() (assuming you are using property based configuration)
3. Cast it to a HibernateProxy, get the LazyLoadInitializer and get the Identifier from it

The problem with all of these solutions is that you have to get around the Gorm code above that unwraps any properties when you call a getter. You can’t just add setBar and getBar. However, you can add another method that can get ‘raw’ access to the field. And before anyone posts a comment about it you can’t use load either because it requires the id, and that’s the whole point of this exercise (getting the id that is).

I didn’t find this out until I had wasted quite a few hours, but apparently this was a known issue: http://jira.grails.org/browse/GRAILS-2570 I never saw this when googling for it originally, but you can essentially say:

foo.barId

And you will get the id of the bar without the proxy being unwrapped. I’m not sure this is referenced anywhere in the docs though.

In my new gig at Fundspire, I generally work from home. For awhile this has been on my 4 year-old Macbook Pro. However, during my time at ThoughtWorks, I got extremely accustomed to using extremely powerful desktop machines running Ubuntu. In fact, I think the ones I used on my last project were running two xeon processors. Regardless, rather than get a new laptop, since I’m working from home I decided to build a new desktop. I’ve built a number of machines over the years, and it’s always been a favorite hobby of mine. I enjoy spending time getting back up to speed with what all the major chipsets are, and the actual build can be a lot of fun too. Below, I’m going to list out everything I bought and why, with a total at the end. All of the prices and links are to NewEgg. I’ve been using them for years and have always been happy with their prices, selection and service.

Case

There’s two areas I never scrimp on when building a new computer: The motherboard and the case. The case can make the build and future upgrades to the machine simple, or a nightmare. When looking for a case, three things are important to me:

1. Cable management. Besides the whole airflow thing, it makes it a lot easier to work inside of the case, or do simple upgrades when the cables aren’t in a huge jumble in front of the motherboard. I can’t count the number of times I accidentally knocked a cable loose and didn’t realize it.
2. Sound proofing. Maybe I’ve just been using laptops too long, but I’m really over having a tower that sounds like a jet taking off when I turn it on.
3. Lots of space for additional drives, etc.

I have no intentions of overclocking, so I’m not particularly worried if it has things like water cooler hookups or whatever. And I really don’t want a light show in my office. So, I went with a full tower by Fractal Design.

CPU

Since the CPU really determines the motherboard nowadays, I had to decide that next. However, since the new Intel Sandy Bridge processors came out a couple of months ago, it was a no-brainer. I went with the Core i7 Sandy Bridge 2600K. The K in this instance means the chip has an unlocked modifier, which is why all the overclockers you see tend to get either it or the i5. And while I’m not going to overclock it, there was a combo deal on my motherboard for the K version, which would normally be $20 more than the regular 2600 variant, making it the same price, so why not?

Motherboard

This was the most difficult spot for me. As I said above, it’s probably the most important piece in the system, and completely affects how you can upgrade in the future, so it’s important. The first thing I noticed when getting back into the latest hardware was how much it seems like every generation of new processors from either Intel or even AMD seems to have a completely different socket. Maybe they’re more interchangeable than I think, but it at least seems pretty specific. For instance, take a look at this wikipedia page: http://en.wikipedia.org/wiki/List_of_Intel_Core_i7_microprocessors. Here’s the sockets of all the different Intel i7 chips:

Lynnfield: LGA1156
Bloomfield/Gulftown: LGA 1366
Sandybridge: LGA1155

Obviously, I wanted to go with an LGA1155 socket motherboard, since that’s the chip I got, but it’s really easy to get confused. There’s also a noticeable difference in the motherboards for Bloomfield processors. Most of the them are triple channel. While all the newer Sandy Bridge are all dual channel. It’s a bit confusing at first, but SB represented a pretty big change in the memory controller. Namely, it’s on the processor itself now and not the northbridge. That being the case, the only option for motherboards is whatever the chip has, which is dual channel. Also, from the numbers I’ve read, it seems that SB chipsets running 1333 DDR3 have higher memory thoroughputs than the previous generation triple channel setups, so it seems to work out.

I went with Gigabyte for my motherboard: http://www.newegg.com/Product/Product.aspx?Item=N82E16813128478

I really just wanted something that would support a lot of memory for the future, (This one goes up to 32 GB) SATA III (6.0 GBs) and USB 3.0 ports. I also tend to favor the ‘enthusiast’ motherboards a bit more, as their tendency to be used in overclocking setups means they have gratuitous amounts of heatsinks on all the chips. Having overheated more than one motherboard in the past, it’s nice to have that even if you’re not going to overclock.

Video Card

I’m going to be running Ubuntu, so the video card selection is extremely important. Video cards and linux have always had a checkered past, and even in 2011 you can easily pick up a card that doesn’t work right with a particular kernel or something. (It happened to me last year with a Fedora setup I had) Personally, I think the only option is an NVidia setup. I used to use Radeons exclusively, but starting with something like the Geforce 8xxx chipsets, Nvidia added this thing called PureVideo. This has support for VDPAU (Video Decode and Presentation API for Unix) Which allows *nix machines to use hardware decoding of video on the video cards themselves. With that in mind, I bought a Geforce GT 430. It was introduced last fall, is extremely cheap and focuses on being able to push video, which is all I want. I watch mlb.tv on another screen while coding sometimes, without hardware video decoding, it would take up a lot of resources.

Everything else is pretty straightforward.

Powersupply: OCZ ModXStream Pro 700W You can now get ‘modular’ PSUs, which means that you only have the cables you plug into the unit, and don’t have a tangle of wires for all the connections you’re not currently using.

SSD: Crucial RealSSD 64GB Who in their right mind would build a system today and not use an SSD? Especially if you’re a programmer. I went with 64 Gb to save money. It’s enough to run ubuntu on the SSD, and any code projects. For everything else, I picked up a normal platter drive for $40.

Ram: 4 x 4GB DDR3 PC1333 (I did a quantity of 2 on the setup I linked to for a total of 16 GB) Most of the nicer SandyBridge setups support faster ram, but since the sandy bridge architecture is at 1333, I’m assuming they’re getting those speeds by overclocking, and it wasn’t something I wanted to deal with.

Everything else is pretty normal, I picked up a new DVD drive for $20, mostly because I needed one with a SATA hookup. (All my existing ones are PATA) I also picked up a heatsink, thermal paste, etc as well. They’re generally cheap and worth it to make sure you don’t fry your cpu. With all of that, the total including shipping was: $1,337.62

It’s not too bad for what is essentially a top of the line system. The only way to go higher is to get into server grade territory, with two CPUs and 12 ram slots. You could reduce the cost on the setup I have by going i5 and halving the ram, but that only saves you $200, and as a programmer, most of my work is CPU and Ram bound so I think it’s money well spent. You can also go cheaper on the motherboard and the case, but you’ll always regret it. Of course, if you also want to play games on the computer, you’ll probably want to dump way more money than I did in a video card.

Yesterday, I posted about how to upgrade hibernate versions in grails. This solution worked great for me locally. (i.e. using run-app) But we were still getting weird issues in our staging environment. It’s not uncommon for this kind of thing to happen, and I used my local tomcat setup to try and nail down the problem. When I cracked open the WAR, I found two version of hibernate, 3.3.1 and 3.3.2. Upon closer inspection of the problem, it turned out to be the same original issue I had upgraded Hibernate to fix, somewhat hidden by some additional ‘noise’ in the log file. I tried about 10 different things to get the war to generate correctly, until I stumbled onto this blog post:

http://www.anyware.co.uk/2005/2009/01/21/excluding-files-from-a-war-with-grails-the-right-way/

Grails gives you the option to declare a closure to modify the war resources. I added the following to the top of my buildConfig.groovy, and it fixed the issue:

grails.war.resources = { stagingDir ->
  delete(file:"${stagingDir}/WEB-INF/lib/hibernate-core-3.3.1.GA.jar")
}

While using Hibernate Envers, I ran into an issue that was fixed in Hibernate 3.3.2. Unfortunately, Grails is using Hibernate 3.3.1. Luckily, it’s fairly straightforward to exclude and declare a new dependency in your BuildConfig.groovy:

inherits("global") {
 excludes 'hibernate'
}
dependencies {
 compile ('org.hibernate:hibernate-core:3.3.2.GA'){
     excludes 'ehcache', 'xml-apis', 'commons-logging'
 }
}

Update: turns out it’s not as straightforward as I thought. The above will work fine with ‘grails run-app’ but the 3.3.1 version will still be in the War file: http://www.lucasward.net/2011/03/excluding-jars-from-grails-generated.html You can fix that by adding the following to your BuildConfig.groovy:

grails.war.resources = { stagingDir ->
 delete(file:"${stagingDir}/WEB-INF/lib/hibernate-core-3.3.1.GA.jar")
}

Unfortunately, in the 3.3.2 release, ““org.hibernate.dialect.DialectFactory” was moved to “org.hibernate.dialect.resolve.DialectFactory”. Which was part of the following issue: http://opensource.atlassian.com/projects/hibernate/browse/HHH-2933

This is apparently an issue many have been seeing: http://grails.1312388.n4.nabble.com/update-hibernate-version-td3002449.html

While it may seem crazy that the Hibernate team moved something to a new package in a point release. I’m pretty sure they consider this class to be internal and not part of their API. The issue the change was made in was to expose to users a way to resolve their own dialects programmatically. Which, if you look at where the Grails Hibernate plugin is creating the offending spring bean, is the same thing Grails is doing:

if (ds && ds.dialect) {
  if (ds.dialect instanceof Class) {
      hibProps."hibernate.dialect" = ds.dialect.name
  }
  else {
      hibProps."hibernate.dialect" = ds.dialect.toString()
  }
}
else {
  dialectDetector(HibernateDialectDetectorFactoryBean) {
      dataSource = ref("dataSource")
      vendorNameDialectMappings = vendorToDialect
   }
   hibProps."hibernate.dialect" = dialectDetector
}

The above is taken from HibernatePluginSupport.groovy, line 128. (At least in Grails 1.3.6) Essentially they’ve written a Spring bean to programmatically determine the dialect by calling the DialectFactory themselves. You can bypass this issue completely by specifying your dialect, rather than having it detected. If you’re using mysql, it looks something like:

dataSource {
driverClassName = "com.mysql.jdbc.Driver"
dialect = "org.hibernate.dialect.MySQLDialect"
}

At the beginning of this week I started work on a new feature. For the purposes of this post it doesn’t matter what the feature was, but I started in on it on Monday and things went extremely well. I was in the zone. It’s hard to describe, but I think if you’re a programmer you’ll understand the feeling. I started out with just the right amount of testing in the right places, built something simple and added on top of it, while refactoring periodically as I discovered new angles on the problem. Everytime I moved forward, I was able to run my tests and tell immediately that my last fix had broken something. I was moving forward in small steps and it was all falling together nicely.

And then I got greedy. I suppose it makes me a dork to admit it, but I was having fun. I had been somewhat behind on the feature because I was debating on a couple different paths, but now that I had chosen, I was moving forward with wreckless abandon. It’s somewhat intoxicating to be in the zone. So, I pushed myself hard. I worked from around 9 AM until 1 or 2 in the morning every night for 3 days straight. (I work from home, so it’s a bit easier to work later like that)

This morning I woke up begrudgingly. The comforter was very, very heavy. It was the weird programmer kind of fatigue, where you’ve done nothing but sit on your but all day, but your mind has been going too fast for too long, so you feel mentally exhausted but physically fine. When you hit that point, no amount of coffee is going to make it better. Now, I’ve been on some pretty hellish death marches before. Some of them spanned months, so it’s not like working hard for 3 days is all that terrible. But the programming gods are vengeful. I might have been tired, but I was excited to get the feature working. I was in the home stretch. My sister is coming to town this weekend and it’s her first time in Chicago since turning 21, so I wanted to get this thing done and party like it was…2003?

So I got sloppy. I ran into a weird bug. It was a NullPointerException with no stack trace. Which sometimes interchanged itself with an InovcationTargetException. When you see something like that happen, it’s time to stop the production line. I should have stopped, immediately, and attempted to recreate the conditions in my testing setup. I had a perfect, isolated, test suite setup where I could have recreated this thing in a controlled environment, but I didn’t do that. Instead, I fiddled with idea after idea. They were all well informed, reasonable things to be trying. But when you’re doing this kind of thing inside your application, there can be too many factors. Production code is complicated, and there’s just too many variables.

At around 6PM I stopped myself, and did what I should have done in the first place. In an integration test, I slowly recreated the problem, piece by piece. Within 15 minutes I had found the problem. I forgot to put an annotation on a particular class.

I. forgot. an. annotation…..$#$%#$%#$!!!!!!!!!!!

I suppose I could get mad at the guy who wrote the open source framework I was using. But I would be one heck of a hypocrite if I did. I can’t imagine how many people have been in the same situation with Spring Batch, and probably felt the same way about me. I hit a scenario he didn’t think of, so there wasn’t specific error handling for it, and the results were nondeterministic.

No, I was tired. I pushed myself too hard and I made a stupid mistake, and the funny thing is, I lost all of the time I had saved by working so much! Awesome.

And when has that not been the case? I can’t think of time where I worked too many hours and this exact same thing hasn’t happened. I was tired and made a stupid mistake. A mistake that I would have easily caught if fully rested. And that mistake had cost all the time I had spent working late. And it’s compounded by the number of people overworking themselves. The more people doing it, the worse the effect is. Because your stupid mistake made while trying to fix someone else’s stupid mistake quickly snowballs.

But sometimes, it’s just hard to grok. I don’t think the human mind is fully capable of thinking that way. If you’re running somewhere, running faster gets you there quicker, baring the limits of endurance. But overall, if you can run faster, running faster will get you there sooner. Or at least is seems that way when you’re coding at 1 in the morning on an interesting problem. And I know all about sustainable pace, and I agree with it fully. And I wasn’t under the spell of an evil project manager either. It’s a small company, and I’m the CTO. The only drive I have is to get features done quicker to support our sales efforts. And I know, from personal experience, what happens when programmers work unsustainable paces.

You know what it’s like? It’s like white castle. At least once or twice a year I’ll decide eating at White Castle is a good idea. I may or may not be sober when I make this decision, but I make it anyway. And I pay for it. The next day you wake up and you *know* you made a terrible mistake. And for days, maybe even weeks, you’ll swear to never eat at white castle again. But then you’ll be at your cousin’s wedding and there just so happens to be a white castle within walking distance, and the terrible cycle repeats itself.

I ran into this issue while unit testing my controllers (many of which only return JSON) in Grails. I’ve reported the issue to grails: http://jira.codehaus.org/browse/GRAILS-7339 but it’s also worth noting here, so anyone else left scratching their head as to why the rendering doesn’t appear to be working in their unit tests can find the workaround as well:

If you are writing a unit or integration test in grails and mocking the controller, which can be done either by extending ControllerUnitTestCase (or using mockController) you might run into this issue. As you probably know, there are multiple ways to render JSON in grails. One is to take a normal map and render as JSON:

def jsonMap = [success:true]
render jsonMap as JSON

This will work fine everywhere, because you’re effectively passing it in as a converter. However, if you use the more verbose (and in many ways more powerful) rendering, you will run into issues:

render(contentType: "text/json") {
  success(true)
  results {
    result.each {
    def period = it.getPeriod() 
    b(id: it.id,name: it.name)
    }
  }
}         

In my case, my test looks like this:

assert JSON.parse(this.controller.response.contentAsString)?.success == true

This won’t work, because the content on the mock response will be null. With a little digging into how mockController() works, you will find the problem lies in MockUtils. Specifically, in the method that handles a map with render options and a closure:

clazz.metaClass.render = {Map map, Closure c ->
  renderArgs.putAll(map)

  switch(map["contentType"]) {
    case null:
    break

    case "application/xml":
    case "text/xml":
      def b = new StreamingMarkupBuilder()
      if (map["encoding"]) b.encoding = map["encoding"]

      def writable = b.bind(c)
      delegate.response.outputStream << writable
      break

    default:
      println "Nothing"
      break
  }
}

As you can see, the case of ‘text/json’ isn’t handled. I was able to workaround this issue by creating a custom unit test case class to extend from the Grails ControllerUnitTestCase:

class CustomControllerTestCase extends ControllerUnitTestCase{

    protected void setUp() {
        super.setUp()
        controller.class.metaClass.render = {Map map, Closure c ->
            renderArgs.putAll(map)

            switch(map["contentType"]) {
            case null:
                break

            case "application/xml":
            case "text/xml":
                def b = new StreamingMarkupBuilder()
                if (map["encoding"]) b.encoding = map["encoding"]

                def writable = b.bind(c)
                delegate.response.outputStream << writable
                break

            case "text/json":
                new JSonBuilder(delegate.response).json(c)
                break
            default:
                println "Nothing"
                break
            }
        }
    }

}

Since it’s groovy, you could pretty much replace this anywhere by overriding your mock controller’s meta class.