My Octopress Blog

A blogging framework for hackers.

Grails Envers Plugin

***Update - 4/11/2013***

I finally got around to updating and publishing this plugin: http://www.lucasward.net/2013/04/grails-envers-plugin-update.html



I recently needed to add auditing to my grails application and decided to use envers, mostly for its ability to keep updates to multiple hibernate objects tied together in a way that is query-able afterwards. Unfortunately, because Grails exerts somewhat tight control of Hibernate, and because of how Envers was implemented, getting it to work with grails can be a bit tricky. I did a cursory glance around and couldn’t find anyone that had created a plugin to help with Envers and Grails. So while implementing it in my application, I wrote the envers specific code as a plugin, which I have published on github:

https://github.com/lucaslward/grails-envers-plugin

Eventually, I need to write actual documentation for it, and try and get it in the official grails plugin repository. However, for now, I’ll write up how to use it based on the integration tests.

If you’re pulling it down from the git repo above, you need only do a simple ‘grails package-plugin’, which will create a zip file, which can be then be installed into your grails application via: ‘grails install-plugin /path/to/zip/grails-envers-plugin-0.1.x.zip’. The plugin sets up envers via Hibernate event listeners, and provides gorm style dynamic methods that can be used to query for revisions.

In order to configure your domain objects to use Envers, you need to annotate them with @Audited. (I haven’t written any code to try and make that work with static members a la gorm) I’ll use the sample domain from the plugin:
@Audited
class Customer {

String email
String name
Address address
SortedSet orders = new TreeSet()

static constraints = {
address (blank: true, nullable: true)
address component: true
}

static mapping = {
orders cascade: "all,delete-orphan"
}
}

@Audited
class Address {
String city
String zip
}

@Audited
class OrderEntry implements Comparable{

Date date
double amount
int numberOfItems
Customer customer

static belongTo = [customer:Customer]

@Override
int compareTo(OrderEntry o) {
date.compareTo(o.date)
}
}
In this example, I have three domain classes: Customer, Address, and OrderEntry. Customer in this class is the main class, who has a one to one relation ship with an Address, and a collection of Orders. Now, let’s assume that we create a customer with an address, then modify the customer and the address twice in separate transactions:

Customer customer
Customer.withTransaction {
def address = new Address(city: "Chicago", zip: "60640")
address.save()
customer = new Customer(name: "PureGorm", email: "tester@gorm.org", address: address)
customer.save(flush: true)
}

Customer.withTransaction {
customer = Customer.findByName("PureGorm")
customer.email = "tester2@gorm.org"
customer.address.city = "New York"
customer.save(flush: true)
}

Customer.withTransaction {
customer = Customer.findByName("PureGorm")
customer.email = "tester3@gorm.org"
customer.address.zip = "10003"
customer.save(flush: true)
}

Note: If you’re trying to test envers in your application you have to turn off transactions for your test and wrap individual calls in transactions as above. (with an appropriate tear-down to clean up afterwards of course)

In this scenario, we should have 3 entries in the audit tables for Customer and Address (customer_aud and address_aud respectively). We can query for this methods with the findAllRevisions dynamic method:

def results = Customer.findAllRevisions()

assert results.size() == 3
def r = results[0]
assert r.name == "PureGorm"
assert r.email == "tester@gorm.org"
assert r.address.city == "Chicago"
assert r.address.zip == "60640"
assert r.revisionType == RevisionType.ADD
r = results[1]
assert r.email == "tester2@gorm.org"
assert r.address.city == "New York"
assert r.revisionType == RevisionType.MOD
r = results[2]
assert r.email == "tester3@gorm.org"
assert r.address.zip == "10003"
assert r.revisionType == RevisionType.MOD


As you can see, findAllRevisions returns an array of three results. The plugin will automatically take the envers RevisionEntity and RevisionType, and set them on the returned revisions, which is how revisionType is being checked. As you can see, all three revisions are present. By default its sorted by the earliest revision first. First is an add, followed by two ‘mods’ (update). The revision entity is also accessible via the revisionEntity property: r.revisionEntity.getRevisionDate(). It’s also worth noting that even a custom envers RevisionEntity will be accessible this way. (See the envers documentation for more details on how to do this)

findAllRevisions also supports sorting in the same way as gorm queries. Assuming the same audit trail as above, you could query it this way:

def results = Customer.findAllRevisions(sort:"email",order:"asc")

assert results.size() == 3
assert results[0].email == "tester2@gorm.org"
UserRevisionEntity entity = results[0].revisionEntity
assert entity.getUserId() == currentUser.id
assert results[0].revisionEntity.getUserId() == currentUser.id
assert results[1].email == "tester3@gorm.org"
assert results[2].email == "tester@gorm.org"


You still get back 3 results. But this time the first result is the first update, followed by the second, following by the initial create. This is because its being sorted by email ascending.

As with sorting, you can also use max and offset:

    
def results = Customer.findAllRevisions(max:1)

assert results.size() == 1
assert results[0].email == "tester@gorm.org"

results = Customer.findAllRevisions(max:1,offset:1)

assert results[0].email == "tester2@gorm.org"

results = Customer.findAllRevisions(max:1,offset:2)

assert results[0].email == "tester3@gorm.org"


Besides sorting and paginating, you can also search for revision by property name:

Customer.findAllRevisionsById(customer.id)

Customer.findAllRevisionsByAddress(customer.address)

Customer.findAllRevisionsByName("PureGorm",[sort:"email",order:"asc", max:2])


The first case above is searching by id, and requires a long. The second is searching by the address, which requires an attached hibernate entity. The final is searching by a simple property name, in this case: “PureGorm”, with some additional sorting, etc tacked on. As with gorm, you can use any property name on the object to query by. At this point, the plugin only supports querying by one property. If you need to query by more, you will need to use the Envers classes directly. (i.e. AuditReader)

The final method available statically is getCurrentRevision():

Customer.getCurrentRevision()


Which gets the current revision number. Meaning, what is the last revision of any customer?

There are also two methods that are applicable on domain class instances as well:

     
List revisions = customer.getRevisions()
assert revisions != null
assert revisions.size() == 3

Customer oldCustomer = customer.findAtRevision(revisions[1])
assert oldCustomer.email == "tester2@gorm.org"
assert oldCustomer.address.city == "New York"


The first method will return all revisions of a particular class. (The same as Customer.findAllRevisionById(customer.id)) And the second will find a particular customer at a particular revision.

There is certainly some functionality I’m missing here, as I’m using the integration tests as an example. I will add more blog entries on the subject as I discover the corner cases I have missed. Hopefully, I can also get time to write actual documentation, and get the plugin in the official repository as well.

Comments

bdbull
Has anyone come up with an efficient way to add a "revision" property to the domain class? It seems a little involved if I want to actually attach the revision to the domain class itself.
Prashant Potluri
I am using @Audited on getter methods instead of at class level for auditing specific properties only. However i am unable to use any of the dynamic finder methods findAllRevisionsById() etc. It says cannot resolve property revision. If i annotate the class @Audited, it works.
Is there a work around for this without annotating the class?
Prashant Potluri
There is @Transactional on controller. Also tried the programmatic way using Entity.withTransaction.
I still don't see any data written to audit tables. DataSource.groovy contains multiple data sources:

development {
dataSource {
dbCreate = "update"
url = "jdbc:mysql://localhost/dev"
username = "xxxxx"
password = "xxxx"
}
dataSource_audit {
dbCreate = "update"
url = "jdbc:mysql://localhost/audit"
username = "xxxx"
password = "xxxx"
}
}
Lucas Ward
Usually when it's not pushing to the AUD tables it's a transactional issue, because it only pushes those entries on transactional boundaries.
Prashant Potluri
If there are multiple data sources defined in DataSource.groovy, the plugin is not saving any data in the audit tables. Any suggestions?
Tomasz Kalkosiński
Hi Lucas

I wrote an interesting blog post about working with Grails 2.1 and Hibernate Envers together. It'd be great if you can read, comment and spread a word about my work. My post can be found here:

http://refaktor.blogspot.com/2012/08/hibernate-envers-with-grails-210.html

Greetings,
Tomasz Kalkosiński
Manuel B.
Jay Hogan has forked the repository and upgrade envers plugin for grails 2.0

URL: https://github.com/jayhogan/grails-envers-plugin

Regards
Lucas Ward
korsbecker,

Getting the plugin to work with Grails 2.0 is a whole other problem. Grails 2 jumped the hibernate version up to 3.6 which is something like 4 years worth of Hibernate. I haven't looked into that at all yet (but probably will in 3 or 4 months) but there's no telling what issues might pop up. The error you have looks more related to issues cause by this upgrade than from anything else.
korsbecker
Smita,

Did you solve your problem with

Caused by ClassCastException: org.codehaus.groovy.grails.orm.hibernate.cfg.GrailsAnnotationConfiguration cannot be cast to org.hibernate.cfg.AnnotationConfiguration

I have exact the same problem with Grails 2.0. I have upgraded the plugin to to grails 2.0 and change to Hibernate 3.6.7.Final and explicitly added a runtime for hibernate-envers:3.6.7.Final

But still with the same problem??

cheers
henrik
Lucas Ward
Sorry for the delay in responding to some of these. I took a long vacation and have just now gotten back.

@Smita - The plugin definitely won't work below grails 1.3, and even that can get tricky at times. The problem is that Envers was integrated with hibernate a couple of years ago. Grails is really behind on hibernate releases (for obvious reasons). It will all be much more straightforward with Grails 2.0 I think.

@Gmacmullin, I'm glad it worked out for you. I really need to get this thing into the official plugin repo…
gmacmullin
Looks like I had a typo where I declared the Envers plugin as a compile plugin dependency. It works great now.

Thanks for creating the plugin Lucas!

Glen
gmacmullin
I've been using the Envers plugin with Graisl 1.3.7, but I have to download the source code and upgrade it to Grails 1.3.7

The weird thing is the plugin works perfectly when I run "grails run-app", but when I deploy it as a war file in Tomcat, the _aud tables are not created. I turned on logging for Envers and the plugin, but I don't see any errors/problems.

Lucas, have you had any problems running the plugin in a WAR file?

Thanks,
Glen
Smita Saraswat
Anyone as run this plug in with the latest version of grails ?
Thanks.
Smita Saraswat
Lucas,

I am running grails 2.0M2 version and when I installed your plugin, I get the following error :

Caused by ClassCastException: org.codehaus.groovy.grails.orm.hibernate.cfg.GrailsAnnotationConfiguration cannot be cast to org.hibernate.cfg.AnnotationConfiguration

Thanks.
Lucas Ward
@euvitudo

Transactions in Grails are a bit weird anyway, but yes, nothing will be written to the audit tables until a commit. That's because a common revision ties together all the work done in a single transaction. Of course, if you do this work in a service, that method is transactional, you can use the @transactional annotation as you already mentioned, and there is a transactional controller plugin that let's you set which methods on a controller should be transactional. (which is what I use in my app)
euvitudo
Regarding the transactional context:

I've been experimenting with your plugin (thanks for the plugin, btw), and have only simplistic examples using grails scaffolding. I found that after adding @Transactional (org.springframework.transaction.annotation.Transactional) to the controllers, updates to the audit tables are committed.

The drawback to this method (at least, while using scaffolding) is that all methods will be transactional.

Annotating a service or service method in this way will do the same.
morungos
Thanks Lucas. I'd be OK doing that if I was a little more Groovy-savvy. Couldn't find much in the way of annotation at anything other than the class level. I was about 2/3 way through improvising auditing with Groovy MOP hacks, an event listener, and a static field, so I might for now go that way, but I'll certainly track this plugin, which appears to be much better for the long term.
Lucas Ward
@morungus

The plugin doesn't actually address configuration, and relies on envers itself. As I mentioned in a previous comment, I would love to do more gorm style mappings, but will likely wait until grails 1.4.

This problem is kind of tricky for groovy in general (and why I suspect grails uses static fields). There's a few posts you can find on the topic, but because groovy generates your getters and setters, etc, annotations on a field don't do what you expect (or in this case what envers expects). Your best bet is to create the getters and setters yourself, and annotate them. It's still tricky, but that's your best bet for individual field annotations on gorm classes. Of course, it will work straight away on java classes configured with hibernate xml or entity annotations. Although, I'm sure that's not a great option for many. You can test this yourself by annotating your groovy classes and using reflection in java to see if you can get at them.
morungos
This is almost ideal, but I need to be selective about which fields to track for audit. I tried the plugin but it always seemed to cough on a relation to a non-audited object. is this known to work?
Lucas Ward
@Fabio

Sorry for the delayed response.

Regarding the UserRevisionEntity. It's really tricky. The weirdest thing is it will only work if you make it a java class and register it directly with hibernate. If you look at the plugin code, you'll see a hibernate config file,where the revision entity is the only configured class. I have no idea why @Audit works fine on gorm classes, but @RevisionEntity doesn't. The listener via the annotation is also really silly of them. It prevents the listener from wiring it up. Once the plugin is released completely, and seems to be stable, my next big release will likely be to write my own configuration for it, probably using a combination of normal plugin config in config.groovy and static methods a la gorm on domain classes. But I probably won't even try that until grails 1.4, since envers changes to a first class citizen in hibernate 3.5. (1.4 will go to 3.6 I believe)

Oh, and it isn't necessary to use Spring Security. That's just what I use for my app at work, but really whatever you're using for authentication will work.
Fábio Miranda
Hi Lucas!

Nice post and nice plugin!

The only thing that is not clear is how to configure UserRevisionEntity to track the user that performed changes to the database.

First, only 'rev' and 'revtstmp' columns were created at the database.

Second, even after I found SpringSecurityRevisionListener, SpringSecurityServiceHolder and StubSpringSecurityService in the sources, it's not clear how to use them.

Third, is it necessary to integrate with a particular technology (Spring Security)?

Thanks!
Fábio Miranda.
daniel
Lucas,
that was the problem. When using a grails service, everything works as expected.

Thanks a lot for your help and the plugin ;-)

Daniel
Lucas Ward
@daniel

I'm assuming by empty _aud tables you mean that when doing an update to a database table, you aren't seeing a corresponding entry in the corresponding audit table?

My first thought is that it has to be something to do with transactions. For example, I have seen this happen with MySql myisam tables. You can write to them without transactions, and since Envers is tied completely to transactional hooks, it will never be called. I'm will to be if you breakpointed the Envers AuditEventListener class, which is what is actually listening to Hibernate events. If you replicate the tests I have in the blog post, using .withTransaction around your calls, you should see the entries. If that works in an integration test, then you need to probably take a look at your Grails controllers. Remember, by default they are not transactional.
Lucas Ward
@DM Sorry about that, I guess I left out the imports when I cut and pasted.
daniel
Hi Lucas,
I was recently integrating envers myself into Grails, but had a problem with empty _aud tables. This way I stumbled upon your plugin. Trying it with your plugin unfortunately yields the same result (empty _aud tables). I upgraded your plugin to Grails version 1.3.7 though, but I don't think this is causing the problem.

Any hint for me what is going wrong?

Cheers,
Daniel
DM
Okay - sorted. Just need to add:
"import org.hibernate.envers.Audited".
DM
G'day Lucas
Using eclipse (STS) and running through the blog, after installing the plugin, and attempting to implement the Customer class, the @Audited annotation is marked as an error … is there some extra configuration I need to do?