This post discusses why I switched my Rimu Markup project’s build tool to Jake, having previously ported the same project from Make to Grunt.

NOTE: This post has been revised since original publication to more clearly explain Jake synchronicity and to show how I now process task shell commands in parallel.

Why Jake instead of Grunt

Jake (like Grunt) is a JavaScript application for building JavaScript projects. Jake is modeled after Rake (a Make-like program implemented in Ruby). If you know Rake or Make you’ll be right at home with Jake.

Grunt is currently the most popular JavaScript build tool and this is one of the reasons I first ported my project to Grunt, but at the end of the day Grunt’s limitations and its philosophical bent pushed me to Jake.

Here’s why I switched to Jake:

• Grunt is plugin-centric — its default usage paradigm is plugins and configuration over scripting — this is one of those things that sounds good in theory but is not so good in practice (see my previous post).
• Jake files are JavaScript code which is more readable and (crucially) more flexible than Grunt’s JSON configuration file format.
• Grunt does not have built-in file dependency management and only rudimentary task dependency management — dependency management is what build tools should be all about and these are real deficiencies (especially if you come from Make, Rake or Jake).

So why isn’t Jake more popular? The herd mentality aside, I think part of the answer is that Jake is not as immediately accessible as Grunt — the Grunt website does an excellent job of introducing Grunt and getting users started. Another reason may be that the command-line is not the primary UI for many Windows developers.

Here’s the full Jakefile.js and here’s the legacy Gruntfile.js.

Task synchronicity

Tasks should execute serially, but shell commands within a task should be run in parallel.

Shell commands executed with jake.exec (or Node’s child_process.exec) are synchronous i.e. they return immediately after starting the command — command completion is handled with callbacks. This can be good because it allows you to execute multiple shell commands in parallel, but it can also be bad because the next task will start before the current task has completed. Task dependencies become impossible to manage unless tasks execute sequentially.

Jake has a clever technique for ensuring that tasks with asynchronous code execute sequentially (i.e. the next task is not started until the current task has finished)): Set the task’s async option to true and then call complete() once all task processing has finished (this signals Jake that the task is done and the next task can be started). Making all tasks execute sequentially also ensures task dependencies are run in declaration order.

I wrote the following exec wrapper task to execute multiple shell commands in parallel:

/*
  Execute shell commands in parallel then run the callback when they have all finished.
  `callback` defaults to the Jake async `complete` function.
  Abort if an error occurs.
  Write command output to the inherited stdout (unless the Jake --quiet option is set).
  Print a status message when each command starts and finishes (unless the Jake --quiet option is set).

  NOTE: This function is similar to the built-in jake.exec function
  but is twice as fast.
*/
function exec(commands, callback) {
  if (typeof commands === 'string') {
    commands = [commands];
  }
  callback = callback || complete;
  var remaining = commands.length;
  commands.forEach(function(command) {
    jake.logger.log('Starting: ' + command);
    child_process.exec(command, function (error, stdout, stderr) {
        jake.logger.log('Finished: ' + command);
        if (!jake.program.opts.quiet) {
          process.stdout.write(stdout);
        }
        if (error !== null) {
          fail(error, error.code);
        }
        remaining--;
        if (remaining === 0) {
          callback();
        }
      });
    });
}

Here’s an example of its use (SOURCE is an array of TypeScript source file names):

desc('Lint TypeScript source files.');
task('tslint', {async: true}, function() {
  var commands = SOURCE.map(function(file) { return 'tslint -f ' + file; });
  exec(commands);
});

By default exec executes the Jake complete function once all shell commands have finished. The above example runs over seven times faster than it would if the tslint commands were run sequentially.

Use jake.exec to run interactive tasks

For example if you want your Git commit task to solicit your commit message using the editor:

jake.exec('git commit -a', {interactive: true}, complete);

Use file tasks to suppress unnecessary commands

Use Jake file tasks to avoid rerunning commands whose prerequisite files have not changed — typically doing things like compiling and documention generation. For example the Rimu project build command executes up to twice as fast using file tasks (the Jake -B option forces unconditional task execution):

$ time jake build -B -q

real    0m17.280s
user    0m8.909s
sys     0m7.580s

$ time jake build -q

real    0m7.939s
user    0m4.112s
sys     0m5.204s

Use shell commands not plugins

Use the ShellJS library to leverage your existing knowledge and to make your Jakefiles easier to read. Plugins are a systemic problem and I avoid them like the plague — see Use ShellJS instead of Grunt plugins.

What about Gulp?

Gulp is the new kid on the block and is kind of interesting:

• Gulp configuration files are JavaScript code (a Node.js application).
• Gulp tasks are composed using Node.js streams which is quite different to Grunt’s sequential task-by-task execution.

I ported my Rimu Jake file to Gulp but abandoned the effort because of plugin problems.

References

• Intro to Jake.
• Jake website.
• Introducing Jake.

This post documents my experience porting a Makefile to a Gruntfile in my Rimu Markup project.

NOTE: Since first writing this post I have switched the same project from Grunt to Jake.

Why Grunt

My reasons for porting were to:

1. Learn about Grunt (it seems to be the de facto nodejs/JavaScript build tool).
2. Make the build process cross-platform.
3. End up with a build script that is more widely understood.
4. End up with a build script that is easier to write, maintain and test.

What differentiates Grunt from Make

• Grunt is similar to Make in that it provides a mechanism for running tasks (JavaScript functions) along with helpers that assist the definition and execution of tasks.
• Unlike Make, Grunt does not have built-in file dependency management and only rudimentary task dependency management — in short Grunt, on it’s own, doesn’t do a lot.
• Grunt files are cross-platform whereas Make files are only cross-platform across UNIX type systems (e.g. Linux, BSD, OSX). GNU Make can be run under Cgywin on Windows but, in my opinion, it’s not as good the real thing.
• Gruntfiles can be written in JavaScript or CoffeeScript — for me this is the biggest advantage over Makefiles (which are an arcane mix of UNIX shell script and Makefile syntax). There’s just no comparison between the two when it comes to writing non-trivial scripts especially when coupled with the huge ecosystem of nodejs libraries.

Use ShellJS instead of Grunt plugins

My first porting attempt was based on the de facto Grunt scripting model i.e. use Grunt Plugins to run development tools. I soon became frustrated, for every build tool I wanted to run I had to:

1. Find a corresponding Grunt plugin (often having to choose from competing plugins).
2. Install the plugin and add the devDependencies to the package.json file.
3. Learn the plugin’s JSON configuration syntax and mentally map it onto the tool’s command-line arguments.
4. Load the plugin in the Gruntfile.
5. Cross my fingers and hope that the plugin developers keep the plugin bug free and in sync with both new releases of the build tool and Grunt.

For example, my Makefile’s lint target (task) runs a jshint command:

lint:
        jshint test/spans.js test/blocks.js bin/rimuc.js

Aside from installing jshint that’s it, it’s that simple — plus I know exactly what it does because I already know how to use the jshint command.

Now consider doing it with Grunt using the grunt-contrib-jshint plugin:

grunt.initConfig({
  jshint: {
    files: ['test/spans.js', 'test/blocks.js', 'bin/rimuc.js'],
    options: {
      jshintrc: true
    }
  }
});

grunt.loadNpmTasks('grunt-contrib-jshint');

All this crazy extra work and stuff to learn for nothing! I was about to pack it in when I discovered ShellJS, a great library that gives you cross-platform UNIX-like shell capabilities. The above plugin code can now be replaced by:

grunt.registerTask('lint', 'Lint Javascript files', function() {
  shelljs.exec('jshint test/spans.js test/blocks.js bin/rimuc.js');
});

No plugin, nothing new to learn, easy to read, easy to verify from command-line — what’s not to like!

See the full Gruntfile.js for more examples of the use of ShellJS functions.

Lessons Learnt

1. Do not use Grunt plugins that are just wrappers for existing command-line tools (I’m not the only one who thinks this way, see here and here.
2. Be very selective of the plugins you do use — weigh carefully plugins verses your own JavaScript code.
3. Use the ShellJS library to leverage your existing knowledge and to make your Gruntfiles easily readable by users that are not Grunt experts.