Groovy's groupBy and inject methods

groupBy

Aman Aggarwal blogged about a powerful method from the java.util.Collection class. The groupBy method converts a collection into a Map with the keys being returned by the closure passed to the method.

Example:

def map = invoices.groupBy { it.invoiceDate.format("MM/yyyy") }
map.each { k, v ->
  println "There are ${v.size()} invoices in ${k}"
}

Incidentally, the example above uses another cool function from java.util.Date: format. The method takes a date pattern (from the specification of java.text.SimpleDateFormat) and returns a String.

inject

While writing about groupBy, I thought I might as well talk about the inject() method too. It is a method that is best explained with an example. If I want to add all the numbers from 1 until 10, i.e. 1 + 2 + 3 + 4 + ... + 10, I can do it in one line:

def sum = (1..10).inject(0) { a, b -> a += b }

0 is passed as the initial value. In each iteration, the value from the last iteration is passed as a while the current value is passed as b. If we add a println statement, e.g.

def sum = (1..10).inject(0) { a, b ->
  println "${a} + ${b} = ${a + b}"
  a += b
}

The output will be:

0 + 1 = 1
1 + 2 = 3
3 + 3 = 6
6 + 4 = 10
10 + 5 = 15
15 + 6 = 21
21 + 7 = 28
28 + 8 = 36
36 + 9 = 45
45 + 10 = 55

Another typical example is to calculate the product of a range of numbers. For example, to calculate the product of 1 to 10 (the factorial of 10):

def product = (1..10).inject(1) { a, b -> a *= b }

The result is the expected 3628800 (= 10!).

New find and findAll methods for String in Groovy 1.6.1

In Ted Naleid's blog, he describes the patches to the String class that he contributed to the Groovy community recently. The patches have been released as Groovy 1.6.1 on April 7, 2009.

He listed the following new methods:

• eachMatch(Pattern pattern) { fullMatch, group1, ... -> ... }


• find(String regex)


• find(String regex) { fullMatch, group1, ... -> ... }


• find(Pattern pattern)


• find(Pattern pattern) { fullMatch, group1, ... -> ... }


• findAll(String regex)


• findAll(String regex) { fullMatch, group1, ... -> ... }


• findAll(Pattern pattern)


• findAll(Pattern pattern) { fullMatch, group1, ... -> ... }

In other words, the find/findAll methods now accepts a regular expression and the matching groups are passed to the closure.

Thanks for the patches, Ted.

Worst Usability Example

To create a Windows service from the command line, one would normally use the sc command. I wasted almost half an hour today while settings Subversion's svnserve service. So I thought I had better document my ordeal.

To start with, sc has the worst command line syntax that I have seen in my entire life as a software developer. The syntax to create a service for Subversion, type:

sc create svn binPath= "c:\subversion\bin\svnserve.exe -r c:\repo" start= auto DisplayName= "Subversion Server" depend= Tcpip

Notice there is whitespace after the "=" character? No, it's not "start=auto". If you type that, sc will print out the command line syntax and the service is not created. Add in the space, you will have svnserve running as a Windows service.

Can someone explain why the syntax is so weird?

A simple example of the power of polymorphism

I often see codes written by the less experienced programmers that contain a lot of if/else blocks where they do something differently with different types of objects. For example,

public class Coordinator {

  public void doSomethingWithList(List list) {
    for (Object o : list) {
      if (o instanceof A) {
        A a = (A) o;
        doSomethingWithA(a);
      }
      else if (o instanceof B) {
        B b = (B) o;
        doSomethingWithB(b);
      }
      // and so forth for class C, D, E, F, ...
    }
  }
}

In the above codes, the current method is making all the coordination work as it determines what to do with different types of objects. By introducing a superclass or an interface, we can refactor the code to something more extensible, and you don't have to change the loop each time a new class is introduced.

First, declare an interface (or a superclass that all the different types of objects will inherit from):

public interface Foo {
  public void workWith(Coordinator coordinator);
}

Change types A, B, C, D, ... to implement this interface:

public class A implements Foo {
  public void workWith(Coordinator coordinator) {
    // do something specific to the A class
  }
}

Change Coordinator.doSomethingWithList() to:

public void doSomethingWithList(List<Foo> list) {
  for (Foo o : list) {
    o.workWith(this);
  }
}

You'll never have to change the loop again when you introduce new types. The Coordinator will invoke the workWith() method polymorphically since all implementations of Foo must define that method.

Testing Controllers That Are Protected By JSecurity Plugin

It can be a challenge to write a test case to test an action that depends on the current user. For example, the list action only returns the records that is accessible by the current user.

If you use the JSecurity Plugin, chances are you will use SecurityUtils.getSubject() to refer to the current user. If the user logs in through the login form provided by the plugin, the user name is stored as SecurityUtils.getSubject().principal. What we want to do is to find a way to inject the "current" user name during the test.

This is how I did it. In FooController, I define getSubject and list like this:

class FooController {

  def getSubject = {
    return SecurityUtils.getSubject()
  }

  def list = {
    def subject = getSubject()

    // Restrict results in listing
    ...
    [fooInstanceList: results, fooInstanceListTotal: total]
  }
}

and the test:

void testList() {
  FooController.metaClass.getSubject = { return [principal: 'user1'] }
  def ctrl = new FooController()
  ctrl.list()
  ...
}

In line 1, just before the FooController instance is created, the getSubject closure is modified to return a map that contains principal as the key. When the list action is invoked, the getSubject() method will return the map. That way, the listing will be restricted to the records which are accessible by the user user1.

How to rollback a transaction in a service

The Grails Framework Reference Documentation only mentions that a method in a service class can be set as transactional. However, it fails to mention how one should rollback a transaction.

It is actually easier than I thought. If the method throws an exception, the transaction will rollback. Although you can perform multi-table updates in a controller, it is good practice to do it in a service method.

Difference between findAll and findAllBy

You may have already known the obvious difference between the domain class findAll() and findAllByXXX() method. findAll() expects a query as the first parameter, followed by the parameter list and the pagination parameters whereas findAllByXXX() expects variable list of objects, ending with an optional map for pagination purpose.

There is a small difference, and I got caught once by it.

As findAll allows you to write the query anyway you want, which may include the "order by ... desc" fragment. Therefore, findAll() ignores the "sort" and "order" entry in the pagination map. This won't return a list of countries in descending order:

def list = Country.findAll("from Country c where c.continent = ?",
                           [ continent ],
                           [sort: 'name', order: 'desc']);

To make it work, I did the following:

def sort = params.sort ?: 'name'
def order = params.order ?: 'asc'
def q = "from Country c where c.continent = ? order by ${sort} ${order}"
def list = Country.findAll(q, [ continent ], params)

The params is still passed in to the function because the max and offset parameters are used.

If the query is simple enough like the query above, it is better to write:

def list = Country.findAllByContinent(continent, params)

Integration test of Grails controller

The general pattern to test a controller is:

void testBar() {
    def ctrl = new FooController()
    def testParams = ...
    ctrl.params.putAll(testParams)
    ctrl.bar()
    ...
}

Different outcomes are expected from the call to the action:

Redirection

If redirection is expected, the redirected URL can be obtained from ctrl.response.redirectedUrl. Keep in mind that unless GRAILS-4270 is resolved, you will have to make sure that the controller name is specified in the parameter list of redirect(). For example,

def index = {
    redirect(action: 'list', params: params)
}

won't work. You need to change it to:

def index = {
    redirect(controller: 'foo', action: 'list', params: params)
}

A View Is Rendered

If a view is expected, the parameters passed to render() is actually stored in Spring's ModelAndView object. E.g. if you have the following statement in your controller:

render(view: 'show', model: [ fooInstance: foo ])

You can access foo via:

def foo = ctrl.modelAndView.model.fooInstance

Text Is Rendered Directly

If the action renders some texts directly, the texts can be accessed by

def text = ctrl.response.contentAsString

A Map Is Returned

If a map is returned, as is the case for the scaffolded list action, you can access the map like this:

def map = ctrl.list()

Groovy wrapper for Snipplr API

I created a Groovy wrapper for Snipplr API this afternoon. It is called, rather boringly, gsnipplr. I wanted to try out Groovy's XMLRPC module and use Maven to build the Groovy project. I threw in quite a comprehensive set of tests (IMO) and had a lot of fun playing with the testing code.

A lot of ideas were taken from arcturus' SnipplrPy. Thanks for the great example!

The source code is hosted at http://bitbucket.org/sjtai/gsnipplr.

To check out:

hg clone http://bitbucket.org/sjtai/gsnipplr

Testing Ajax pages with WebTest

I just learned that you need to add some delay to the test script after making an Ajax call. (Isn't it logical? Why didn't I think of that?).

For example, assuming there is an onChange handler in the combo box, you should add a sleep step after setSelectField. Groovy code below:

setSelectField(htmlId: 'foo', text: 'bar')
sleep(seconds: 5)
...

Shell Script in Groovy

It is possible to write a shell script using groovy. To do that, just write the script in a text editor, but remember to add the shebang line. For example, the following script will print "Hello World", groovy style:

#!/usr/bin/env groovy
println "Hello World"

The script can be named anything. It doesn't have to end with .groovy. Caveat: the script will not work if groovy is not in the PATH.

Writing Testable Code

If you search the web for advice on how to write testable code, you will end up with a lot of articles describing the strategies and the dos and donts. But, if there's only one thing you need to remember about making your code testable, then take this simple advice: write the tests first, then write your code.

As you start to write your tests, you'll start to think about how to use your code and the APIs that your code provides. You may find that your code has too many dependencies, instead of writing tests, your test case may end up with mostly the codes that build the object graph.

Think of how you will test your code as you design your application, then you'll realize that not all designs can be tested easily. You'll need to decide whether to use the design or to think of another design that is easier to test. All things being equal, choose the design that is easier to test.

I'm not suggesting that you should set strict rules about the writing tests. If you can't test all 10 areas of your code, test 9. If 9 is not possible, test 8. If you at least think about the tests before writing your code, you will have a higher chance of producing better codes.

This post and this elaborate on the issue in more details.

How to access config or metdata outside Grails controller?

grailsApplication is injected in controllers and service classes only. To access config and metadata from other classes, use ConfigurationHolder and ApplicationHolder respectively.

Reference: http://jira.codehaus.org/browse/GRAILS-2545

Convert MDB file in Linux

I have some old, standalone Java projects which used Access MDB file as database. As I have always wanted to use an installation-free database for my application, I was looking for ways to convert the data in the MDB file to HSQLDB. I choose HSQLDB because it's simple to use and cross platform.

I started off researching with unixODBC but ended up installing odbcinst. Eventually I stumbled upon mdbtools. That solves my problem. After installating mdbtools on my Ubuntu laptop, I exported the schema

mdb-schema data.mdb > schema.sql

I edited the schema to HSQLDB syntax and created the new database in HSQLDB.

Then, I only need to do

mdb-export -I data.mdb table | sed -e "s/)$/);/" > table.sql

for each of the tables in the MDB file. The sed command is to append a ';' as mdb-export doesn't end the insert statements with ';'.

I then edit the INSERT statements and run them in HSQLDB.

This is simpler than most of the other approaches I have read on the web.

XmlParser trim whitespace by default

I just found out that groovy.util.XmlParser trims whitespace by default. When parsing XML files with text nodes that contain trailing whitespace, for example, whitespace is removed in the Node returned by the parser.

def parser = new XmlParser()
def doc = parser.parseText("ABC ")
assert doc.data.text() == "ABC" // Not "ABC "!

To preserve whitespace, set the trimWhitespace property to false:

def parser = new XmlParser(trimWhitespace: false)
def doc = parser.parseText("ABC ")
assert doc.data.text() == "ABC "

View the source of compiled GSP

While grails is running in development mode, you can view the source of the compiled GSP by appending showSource to the URL. E.g.

http://localhost:8080/myapp/mycontroller/action?showSource

This trick is mentioned in http://www.piragua.com/2008/03/15/viewing-the-source-of-a-compiled-gsp-in-grails/ and was requested as an improvement in JIRA http://jira.codehaus.org/browse/GRAILS-2729.

I wonder how many more of these tricks are in grails.

Getting uptime in Windows

In Linux environment, we can get the uptime of the workstation with the command uptime. But how do you find out how long your Windows has been running?

I found out recently that you can do this:

net stats server | find "Statistics since"

It doesn't tell you exactly how long your PC has been up and running, but it does tell you the time your Windows started.

Adding jar to classpath at run time

Whenever I want to test some groovy ideas, I use groovyConsole. However, I don't want to copy the dependent jar files into my $HOME/.groovy/lib, and as of groovy 1.5.6, it is still not possible to add -classpath to the command line of groovyConsole.bat.

I found a solution on the web. Just add this line into the start of the code:

this.getClass().classLoader.rootLoader.addURL(new File("file.jar").toURL())

Better still, use a list:

[ "file1.jar", "file2.jar" ].each {
  this.getClass().classLoader.rootLoader.addURL(new File(it).toURL())
}

SQL Keywords As Column Names

Once in a while, I got caught by my own mistake by using SQL keywords as column names. For example, this looks pretty harmless (but it fails in GORM):

class Person {
  String userName
  String password
  String group
}

There are 2 problems:

1. password is a keyword in MySQL.


2. group is an SQL keyword.

Although it usually doesn't take me too long to realize my mistake, it is a nuisance that I'd rather live without. There is a more systematic way to detect such problem.

Create recursive markup with MarkupBuilder

I just found a way use Groovy's MarkupBuilder to build an XML file with nested elements. Assuming I have a container with a collection of X objects and a Y object. The XML output of the container can be generated as follows:

import groovy.xml.*

class X {
  private int id
  private String name

  def toXML = { builder ->
    builder.x(id: id, name)
  }
}

class Y {
  private Date date = new Date()
  def toXML = { builder ->
    final sdf = new java.text.SimpleDateFormat("yyyyMMdd'T'HH:mm:ssZ")
    builder.y(date: sdf.format(date))
  }
}

def w = new StringWriter()
def xml = new MarkupBuilder(w)
xml.doubleQuotes = true
def xa = [ new X(id: 1, name: "x-name"), new X(id: 2, name: 'x-name2') ]
def yObj = new Y()
xml.root() {
  agent(id: 1, name: 'agent1', "X")
  xa.each { xo ->
    xo.toXML(xml)
  }
  yObj.toXML(xml)
}
w.close()

The variable w is equivalent to:

<root>
 <agent id="1" name="agent1">X</agent>
 <x id="1">x-name</x>
 <x id="2">x-name2</x>
 <y date="20080318T23:56:54+0800" />
</root>

Notice that to get the builder to output double quotes for attributes, I set the doubleQuotes property to true.