The Linux cut command

cut is one of the lesser-known commands in the same category of tools such as grep, awk and sed. Nevertheless, I find it more convenient than the other tools in some circumstances. Let's say I have the following lines with ":" as the field delimiter:

abc:12:DEF
ghi:3a4:JKLM
opqr:56b:STUV

To print out the values in the second column, I can use the command

cut -d: -f2

where -d: specifies that the field delimiter is ":", -f2 specifes that the field to print is 2.

If the delimiter is the space character, the command will become

cut -d\  -f2

Notice there are 2 space characters after the backslash. The backslash will escape the space character that follows it.

0.0 Is A BigDecimal In Groovy!

When you are so used to the convenience offered by Groovy and Grails, sometimes you are caught by simplicity itself.

I created a domain class recently:

class Foo {
    ...
    double price

    static constraints = {
        price(min: 0.0)
    }
}

Boom! The application failed to run with the error message:

Parameter for constraint [min] of property [value] of class [class Foo] 
must be the same type as property: [double]

The cause of the problem is that I declared price to be a double, but 0.0 is a BigDecimal in Groovy. To fix the problem, change the value to 0.0d. I found the answer from this thread.

When Class.forName() Can't Find The Domain Class

I was trying to use reflection to create a domain class:

try {
    def clazz = Class.forName(domainClassName)
    def obj = clazz.newInstance()
}
catch (ClassNotFoundException ex) {
    ...
}

To my surprise, I caught ClassNotFoundException at line 2. After searching for a while, I found an explanation in this thread. Burt Beckwith explained that classes loaded without a ClassLoader will not resolve dynamically added classes such as the domain classes. He suggested to use the other variant:

try {
    def clazz = Class.forName(domainClassName, true, Thread.currentThread().contextClassLoader)
    obj = clazz.newInstance()
}
catch (ClassNotFoundException ex) {
    ...
}

Thanks for the tips, Burt.

Self-printing Groovy Program

It always amazes me when I read about how people write a C program that print itself. The self-printing program, or more formally, quine, prints out its source code.

Having used Groovy for a while, I wonder if there is any self-printing Groovy program on the Internet. After some googling around, I found some Java programs on this page. After spending about 15 minutes, I came up with what I think is the first self-printing Groovy program to date. I used Bertram Felgenhauer's 2nd example as my starting point. Here is my version in Groovy (the code is written in one line):

class S{public static void main(a){def s="class S{public static void main(a){def s=;char c=34;println(s.substring(0,41)+c+s+c+s.substring(41));}}";char c=34;println(s.substring(0,41)+c+s+c+s.substring(41));}}

See if you can make it shorter.

Updates

Chrigel suggested the removal of "public". I have to adjust the parameter to substring() too. Here is a shorter version (195 bytes):

class S{static void main(a){def s="class S{static void main(a){def s=;char c=34;println(s.substring(0,34)+c+s+c+s.substring(34));}}";char c=34;println(s.substring(0,34)+c+s+c+s.substring(34));}}

paulk's version is 97 bytes:

def s="def s=;char c=34;println s[0..5]+c+s+c+s[6..-1]";char c=34;println s[0..5]+c+s+c+s[6..-1]

paulk's 2nd version is 46 bytes:

s="s=%c%s%c;printf s,34,s,34";printf s,34,s,34

paulk's 3rd version is 42 bytes. Keep going, paulk!

s='s=%c%s%1$c;printf s,39,s';printf s,39,s

Gavin Grover found a 38-char solution at http://golf.shinh.org/p.rb?Quine:

printf _='printf _=%c%s%1$c,39,_',39,_

Are we approaching the limit?

Grails Quick Reference

I have just finished version 1.1.0 of Grails Quick Reference. For your download convenience, here's the zip file containing the latest source of the quick reference: http://bitbucket.org/sjtai/grailsqr/get/tip.zip. The PDF can be downloaded from here.


Grails Quick Reference by Tai Siew Joon is licensed under a Creative Commons Attribution 2.5 Malaysia License.

The Y292M problem

In about 292 million years from now, we will face the Year 292 Million problem in all our Java/Groovy programs. Luckily it falls on a Sunday, so I think the impact on business will be minimum.

I use new Date(Long.MAX_VALUE) to calculate that date.

Groovy's groupBy and inject methods

groupBy

Aman Aggarwal blogged about a powerful method from the java.util.Collection class. The groupBy method converts a collection into a Map with the keys being returned by the closure passed to the method.

Example:

def map = invoices.groupBy { it.invoiceDate.format("MM/yyyy") }
map.each { k, v ->
  println "There are ${v.size()} invoices in ${k}"
}

Incidentally, the example above uses another cool function from java.util.Date: format. The method takes a date pattern (from the specification of java.text.SimpleDateFormat) and returns a String.

inject

While writing about groupBy, I thought I might as well talk about the inject() method too. It is a method that is best explained with an example. If I want to add all the numbers from 1 until 10, i.e. 1 + 2 + 3 + 4 + ... + 10, I can do it in one line:

def sum = (1..10).inject(0) { a, b -> a += b }

0 is passed as the initial value. In each iteration, the value from the last iteration is passed as a while the current value is passed as b. If we add a println statement, e.g.

def sum = (1..10).inject(0) { a, b ->
  println "${a} + ${b} = ${a + b}"
  a += b
}

The output will be:

0 + 1 = 1
1 + 2 = 3
3 + 3 = 6
6 + 4 = 10
10 + 5 = 15
15 + 6 = 21
21 + 7 = 28
28 + 8 = 36
36 + 9 = 45
45 + 10 = 55

Another typical example is to calculate the product of a range of numbers. For example, to calculate the product of 1 to 10 (the factorial of 10):

def product = (1..10).inject(1) { a, b -> a *= b }

The result is the expected 3628800 (= 10!).

New find and findAll methods for String in Groovy 1.6.1

In Ted Naleid's blog, he describes the patches to the String class that he contributed to the Groovy community recently. The patches have been released as Groovy 1.6.1 on April 7, 2009.

He listed the following new methods:

• eachMatch(Pattern pattern) { fullMatch, group1, ... -> ... }


• find(String regex)


• find(String regex) { fullMatch, group1, ... -> ... }


• find(Pattern pattern)


• find(Pattern pattern) { fullMatch, group1, ... -> ... }


• findAll(String regex)


• findAll(String regex) { fullMatch, group1, ... -> ... }


• findAll(Pattern pattern)


• findAll(Pattern pattern) { fullMatch, group1, ... -> ... }

In other words, the find/findAll methods now accepts a regular expression and the matching groups are passed to the closure.

Thanks for the patches, Ted.

Worst Usability Example

To create a Windows service from the command line, one would normally use the sc command. I wasted almost half an hour today while settings Subversion's svnserve service. So I thought I had better document my ordeal.

To start with, sc has the worst command line syntax that I have seen in my entire life as a software developer. The syntax to create a service for Subversion, type:

sc create svn binPath= "c:\subversion\bin\svnserve.exe -r c:\repo" start= auto DisplayName= "Subversion Server" depend= Tcpip

Notice there is whitespace after the "=" character? No, it's not "start=auto". If you type that, sc will print out the command line syntax and the service is not created. Add in the space, you will have svnserve running as a Windows service.

Can someone explain why the syntax is so weird?

A simple example of the power of polymorphism

I often see codes written by the less experienced programmers that contain a lot of if/else blocks where they do something differently with different types of objects. For example,

public class Coordinator {

  public void doSomethingWithList(List list) {
    for (Object o : list) {
      if (o instanceof A) {
        A a = (A) o;
        doSomethingWithA(a);
      }
      else if (o instanceof B) {
        B b = (B) o;
        doSomethingWithB(b);
      }
      // and so forth for class C, D, E, F, ...
    }
  }
}

In the above codes, the current method is making all the coordination work as it determines what to do with different types of objects. By introducing a superclass or an interface, we can refactor the code to something more extensible, and you don't have to change the loop each time a new class is introduced.

First, declare an interface (or a superclass that all the different types of objects will inherit from):

public interface Foo {
  public void workWith(Coordinator coordinator);
}

Change types A, B, C, D, ... to implement this interface:

public class A implements Foo {
  public void workWith(Coordinator coordinator) {
    // do something specific to the A class
  }
}

Change Coordinator.doSomethingWithList() to:

public void doSomethingWithList(List<Foo> list) {
  for (Foo o : list) {
    o.workWith(this);
  }
}

You'll never have to change the loop again when you introduce new types. The Coordinator will invoke the workWith() method polymorphically since all implementations of Foo must define that method.

Testing Controllers That Are Protected By JSecurity Plugin

It can be a challenge to write a test case to test an action that depends on the current user. For example, the list action only returns the records that is accessible by the current user.

If you use the JSecurity Plugin, chances are you will use SecurityUtils.getSubject() to refer to the current user. If the user logs in through the login form provided by the plugin, the user name is stored as SecurityUtils.getSubject().principal. What we want to do is to find a way to inject the "current" user name during the test.

This is how I did it. In FooController, I define getSubject and list like this:

class FooController {

  def getSubject = {
    return SecurityUtils.getSubject()
  }

  def list = {
    def subject = getSubject()

    // Restrict results in listing
    ...
    [fooInstanceList: results, fooInstanceListTotal: total]
  }
}

and the test:

void testList() {
  FooController.metaClass.getSubject = { return [principal: 'user1'] }
  def ctrl = new FooController()
  ctrl.list()
  ...
}

In line 1, just before the FooController instance is created, the getSubject closure is modified to return a map that contains principal as the key. When the list action is invoked, the getSubject() method will return the map. That way, the listing will be restricted to the records which are accessible by the user user1.

How to rollback a transaction in a service

The Grails Framework Reference Documentation only mentions that a method in a service class can be set as transactional. However, it fails to mention how one should rollback a transaction.

It is actually easier than I thought. If the method throws an exception, the transaction will rollback. Although you can perform multi-table updates in a controller, it is good practice to do it in a service method.

Difference between findAll and findAllBy

You may have already known the obvious difference between the domain class findAll() and findAllByXXX() method. findAll() expects a query as the first parameter, followed by the parameter list and the pagination parameters whereas findAllByXXX() expects variable list of objects, ending with an optional map for pagination purpose.

There is a small difference, and I got caught once by it.

As findAll allows you to write the query anyway you want, which may include the "order by ... desc" fragment. Therefore, findAll() ignores the "sort" and "order" entry in the pagination map. This won't return a list of countries in descending order:

def list = Country.findAll("from Country c where c.continent = ?",
                           [ continent ],
                           [sort: 'name', order: 'desc']);

To make it work, I did the following:

def sort = params.sort ?: 'name'
def order = params.order ?: 'asc'
def q = "from Country c where c.continent = ? order by ${sort} ${order}"
def list = Country.findAll(q, [ continent ], params)

The params is still passed in to the function because the max and offset parameters are used.

If the query is simple enough like the query above, it is better to write:

def list = Country.findAllByContinent(continent, params)

Integration test of Grails controller

The general pattern to test a controller is:

void testBar() {
    def ctrl = new FooController()
    def testParams = ...
    ctrl.params.putAll(testParams)
    ctrl.bar()
    ...
}

Different outcomes are expected from the call to the action:

Redirection

If redirection is expected, the redirected URL can be obtained from ctrl.response.redirectedUrl. Keep in mind that unless GRAILS-4270 is resolved, you will have to make sure that the controller name is specified in the parameter list of redirect(). For example,

def index = {
    redirect(action: 'list', params: params)
}

won't work. You need to change it to:

def index = {
    redirect(controller: 'foo', action: 'list', params: params)
}

A View Is Rendered

If a view is expected, the parameters passed to render() is actually stored in Spring's ModelAndView object. E.g. if you have the following statement in your controller:

render(view: 'show', model: [ fooInstance: foo ])

You can access foo via:

def foo = ctrl.modelAndView.model.fooInstance

Text Is Rendered Directly

If the action renders some texts directly, the texts can be accessed by

def text = ctrl.response.contentAsString

A Map Is Returned

If a map is returned, as is the case for the scaffolded list action, you can access the map like this:

def map = ctrl.list()

Groovy wrapper for Snipplr API

I created a Groovy wrapper for Snipplr API this afternoon. It is called, rather boringly, gsnipplr. I wanted to try out Groovy's XMLRPC module and use Maven to build the Groovy project. I threw in quite a comprehensive set of tests (IMO) and had a lot of fun playing with the testing code.

A lot of ideas were taken from arcturus' SnipplrPy. Thanks for the great example!

The source code is hosted at http://bitbucket.org/sjtai/gsnipplr.

To check out:

hg clone http://bitbucket.org/sjtai/gsnipplr

Testing Ajax pages with WebTest

I just learned that you need to add some delay to the test script after making an Ajax call. (Isn't it logical? Why didn't I think of that?).

For example, assuming there is an onChange handler in the combo box, you should add a sleep step after setSelectField. Groovy code below:

setSelectField(htmlId: 'foo', text: 'bar')
sleep(seconds: 5)
...

Shell Script in Groovy

It is possible to write a shell script using groovy. To do that, just write the script in a text editor, but remember to add the shebang line. For example, the following script will print "Hello World", groovy style:

#!/usr/bin/env groovy
println "Hello World"

The script can be named anything. It doesn't have to end with .groovy. Caveat: the script will not work if groovy is not in the PATH.

Writing Testable Code

If you search the web for advice on how to write testable code, you will end up with a lot of articles describing the strategies and the dos and donts. But, if there's only one thing you need to remember about making your code testable, then take this simple advice: write the tests first, then write your code.

As you start to write your tests, you'll start to think about how to use your code and the APIs that your code provides. You may find that your code has too many dependencies, instead of writing tests, your test case may end up with mostly the codes that build the object graph.

Think of how you will test your code as you design your application, then you'll realize that not all designs can be tested easily. You'll need to decide whether to use the design or to think of another design that is easier to test. All things being equal, choose the design that is easier to test.

I'm not suggesting that you should set strict rules about the writing tests. If you can't test all 10 areas of your code, test 9. If 9 is not possible, test 8. If you at least think about the tests before writing your code, you will have a higher chance of producing better codes.

This post and this elaborate on the issue in more details.

How to access config or metdata outside Grails controller?

grailsApplication is injected in controllers and service classes only. To access config and metadata from other classes, use ConfigurationHolder and ApplicationHolder respectively.

Reference: http://jira.codehaus.org/browse/GRAILS-2545

Convert MDB file in Linux

I have some old, standalone Java projects which used Access MDB file as database. As I have always wanted to use an installation-free database for my application, I was looking for ways to convert the data in the MDB file to HSQLDB. I choose HSQLDB because it's simple to use and cross platform.

I started off researching with unixODBC but ended up installing odbcinst. Eventually I stumbled upon mdbtools. That solves my problem. After installating mdbtools on my Ubuntu laptop, I exported the schema

mdb-schema data.mdb > schema.sql

I edited the schema to HSQLDB syntax and created the new database in HSQLDB.

Then, I only need to do

mdb-export -I data.mdb table | sed -e "s/)$/);/" > table.sql

for each of the tables in the MDB file. The sed command is to append a ';' as mdb-export doesn't end the insert statements with ';'.

I then edit the INSERT statements and run them in HSQLDB.

This is simpler than most of the other approaches I have read on the web.