Sunday, December 16, 2012

Converting CDs to digital from the command line

One of the easiest to use and well featured command line CD ripping utilities around is called jack.

This is not to be confused with the relatively sophisticated general purpose open source audio software called Jack or the windows software called "Jack the CD ripper".

You should be able to find jack in the repositories of most linux distributions. In Ubuntu it is called, unsurprisingly, jack.

The jack program is written in python and makes use of a number of other software packages to do the heavy work. It performs the following:

  • Scan the CD
  • Download album and song information from the internet into a file alongside the tracks
  • Extract the tracks into a directory named using a specified format (eg. artist/album)
  • Convert the tracks them to the desired format (mp3, ogg, flac, etc)
  • Add the downloaded tag information to the files
  • Rename the files using a specified format (eg. 01 - song title.xyz)

Although most the preferences can be specified from the command line, it is more practical to place them in either the user specific config file ~/.jack3rc or the system wide config file /etc/jackrc. I have defined the following settings in my ~/.jack3rc file:


cd_device:/dev/sr0
base_dir:~/jack
ripper:cdparanoia
encoder:flac
rename_fmt:%n - %t
dir_template:%a/%l
unusable_chars:['/', '"', ':']
replacement_chars:['_', '\'', '-']
query_on_start
cont_failed_query


The man page for jack has explanations for all of these (and more). The ones I considered important to change were:

  1. encoder - my preference is to use flac files as my master digital copy
  2. dir_template + rename_fmt - defines my preferred directory and file names formats
  3. query_on_start - perform lookup of album/song information before ripping tracks
  4. cont_failed_query - continue with ripping if lookup of information failed
  5. unusable_chars + replacement_chars - prevent troublesome characters in file names

The last is important and a common feature in CD ripping utilities, even some of the more sophisticated GUI ones. As I use my digital files on a range of operating systems, it is important that they do not contain certain characters that are allowed in directory or file names on one system but not on another. You may even want to add more characters to these options.

If jack is interrupted or stopped by an error (eg. by the user, a power failure, a disc than needs cleaning, etc), simply run it again and it will pick up from where was up to.

There are quite a number of options jack has that I am not making use of (yet). For example, the ripping can be done without any lookup of information (or even encoding) and then the rest of the job done later.

A couple of areas jack could be improved are:

  • Fetching album cover art along with album and song information
  • Allowing files to be re-tagged and renamed after the downloaded information is edited

To make up for this I simply download my on album cover art found on google images and use the metaflac command line utility to make any necessary changes (usually modifying album titles to be consistent).

Friday, June 8, 2012

Fix for hibernate AssertionFailure - collection was not processed by flush()

I recently had to trouble shoot the cause of a hibernate exception that took a while to figure out. It turns out that this was caused by a problem in hibernate that has existed for many years and has only just recently been addressed.

The exception was

org.hibernate.AssertionFailure: collection [package.entity.collectionProperty] was not processed by flush()
    at org.hibernate.engine.CollectionEntry.postFlush(CollectionEntry.java:228)
    at org.hibernate.event.def.AbstractFlushingEventListener.postFlush(AbstractFlushingEventListener.java:352)
    at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:52)
    at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216)
    at org.hibernate.ejb.AbstractEntityManagerImpl.flush(AbstractEntityManagerImpl.java:960)

The error log also showed the following:

org.hibernate.AssertionFailure - an assertion failure occured (this may indicate a bug in Hibernate, but is more likely due to unsafe use of the session)

This confused me because I thought I had finished with the data access layer and hadn't changed anything in that area - certainly nothing that could be categorised as "unsafe use of the session".

We have had this exception before when using hibernate event listeners if lazy loaded collections where touched in the event listener itself. That wasn't the case here.

After removing changes made by others in the data access layer, the problem remained. After testing against daily builds, I confirmed that the problem was in my new code, specifically one line of code in one JPA entity:

@Size(max = 60)

This is clearly not "unsafe use of the session". It isn't even on the collection property reported in the session !!!

This annotation is a little unusual because the it is on a getter method that isn't a persistent property, but it should work without this exception. The problem in this case must be triggered because the getter method includes fetching the properties of an associated entity which is lazy loaded. Somehow that must result in the reported collection property also being loaded.

It appears that in hibernate reading lazy loaded collection properties during a flush results in the above assertion exception. Given that validation needs to happen after any pre-update events and before the actual persistence, the validation in this case is calling the getter method during the flush, triggering loading of the lazy collection property and therefore triggering the hibernate exception.

Solutions include separating and/or moving the validation logic to properties that are not lazy loaded or making the relevant properties eagerly loaded, but these solutions are not always possible or desirable.

In my case I was able to remove the annotation from the getter method.

This was occurring in hibernate 3.6.9. Hibernate 4 has fixed things so this doesn't happen any more (reading a lazy loaded collection during flush doesn't trigger the exception). After upgrading I can put @Size back on!

See https://hibernate.onjira.com/browse/HHH-2763 for the long history of this problem.