Typically you’d just have a makefile deal with this, maybe like so:

compile:
./rebar compile; escript header_gen


But that kind of breaks other people’s builds if they want to use rebar themselves. The problem is that rebar doesn’t yet provide a mechanism for overriding the build/compile command for dependencies (at all, let alone individually) and thus if your build requires a makefile to be run then your consumers build has to incur the same kind of complexity. Fortunately there is a work-around. Rebar provides a compile_post_script hook which you can specify in your rebar.config file. You specify a shell command and rebar will run this hook immediately after compilation. Unfortunately it also runs it when the compile command is applied to all your dependencies (again this is something which isn’t controllable yet) which typically will break because your shell command is not relevant to these apps. I spent a little time trying to do something clever with erl --eval '...' but in the end that didn’t play nicely at all. I then tried to encode the whole thing in a compound shell command like if [ -f hrlgen ]; then escript hrl; fi but then discovered that rebar_utils:sh/1 (which runs the shell command) wraps the supplied command text in an exec system call (which obviously isn’t going to work for a complex if..then expression). In the end, it was a comedic reptile that came to the rescue:

{compile_post_script,
"env ERL_LIBS=deps python -c \"import os; os.path.isfile('hrlgen') and os.system('escript hrlgen') or None\""}.


This decision paid off recently when I tried integrating badrabbit unit tests into a django application. Django has built in support for running unit tests as part of the development lifecycle. The manage.py script that gets generated when you run django’s equivalent of Rails’ scaffolding, takes one of a several commands and one of these is ./manage.py test <appname>.

This has the effect of going off a running all your tests, where the tests for an application are either in your model definition (models.py) a file named tests.py, or contained in (multiple) modules in a package of the same name. By default as described here, django will go off and look for tests defined with the unittest and doctest frameworks, both of which are part of python‘s stdlib. You can override this behavior by providing your own test runner and doing so is, in fact, ridiculously easy. Nevertheless, this isn’t neccessary because badrabbit is 100% compatible with unittest. And so all you need do to use it in your tests is put and import statement in your script, subclass badrabbit.testmagic.autotest instead of unittest.TestCase and you’re off. Here’s an abridged example:

#!/usr/bin/env python

import badrabbit
from badrabbit import *
from users.models import ServiceUser

class ServiceUserTests(autotest):
    @test
    def it_should_set_absolute_uri_correctly(self):
        user = ServiceUser(username='Johannes')
        assert_that(user.get_absolute_url(),
            equal_to("/users/Johannes/"))

Of course, django has its own subclass of unittest.TestCase already, making subclassing badrabbit.testmagic.autotest less attractive (not least since they share the same superclass!) so there is another way. If you were to go take a peek at the implementation of badrabbit‘s autotest class you would soon notice that it does nothing apart from setting the metaclass to AutoTest:

class AutoTestCase(unittest.TestCase):
    __metaclass__ = AutoTest
autotest = AutoTestCase

So instead of subclassing autotest you can just as well set the metaclass (in exactly this way) on your own test class which can then happily subclass django’s TestCase instead.

I can’t imagine the integration being much simpler really. Because badrabbit depends on hamcrest-python, which doesn’t appear to come with an installer, we stuck it a couple of directories above our code and run the manage.py utility using a wrapper shell script which configures the PYTHON_PATH environment variable to include this first.

We generate our Python SDK using our (sort of) open source SDK generation system, which is hosted by our friends at osmosoft. You can find the SDK generation code here.

Anyway, there has been much debate about langauge proliferation within our group. As one of the most enthusiastic exponents of a polyglot programming, I perform quite a bit of evangelism within our group. Most recently, I’ve started the work of setting up a Python focus group, and in the course of doing so, have begun setting some “homework” for some of my keen fellow engineers.

Always ahead of the game, Kerry has already started blogging about some of these exploits. The second exercise is an interesting one. I’ll discuss the thinking behind the exercise, followed by my solution.

Here’s the exercise itself:

Write a recursive function that calculates the value of Fibonacci numbers. These are your acceptance criteria:

• Calculating fib(250) must return 7896325826131730509282738943634332893686268675876375
• The function must use recursion. No intermediary data structures, etc.
• The implementation must be written in pure python – no C extension modules, that’s cheating.
• The function must calculate the 250th Fibonacci number in under one second.

  You will get extra points if:

• You can also demonstrate a proof for the Reciprocal Fibonacci constant, meeting the following conditions
  • Your proof must also run in under one second
  • Your proof must not duplicate any of the concerns addressed by your original Fibonacci function implementation.
  • Your proof is allowed to call into the Fibonacci function though!

The reciprocal Fibonacci constant is defined here.

Here’s a couple of hints about things to look for:

• Memoization
• Function decorators

Well now. What’s the point of this exercise?

The point here is to get people thinking about function composition. Re-use is so often quoted as being important in our work, and yet in our industry at large, we often see developers failing to apply even basic functional (de!)composition, let alone OOP and other high level programming paradigms.

Properly composed functions are a basic unit of abstraction in most languages, and Python is no exception.

Implementation

The performance requirements of the spec are really a red herring. The key to this exercise is to avoid duplicating the performance enhancing code, which is required to avoid the slowdown (and possible program failure) resulting from excessive recursion. I decided to abstract a simple caching algorithm as a function decorator, which is then applied to both recursive functions (fibonacci and the fib’ constant implementation). You could use the ‘memoized’ decorator anywhere you have a function that is referentially transparent. Here’s the code:

#!/usr/bin/env python

def memoized(fun):
  target = fun
  cache = {}
  def memoized(*args, **kwargs):
      if cache.has_key(args[0]):
          return cache[args[0]]

      result = target(*args, **kwargs)
      cache[args[0]] = result
      return result
  return memoized

@memoized
def fib(n):
  if n < 2: return n
  return fib(n - 1) + fib(n - 2)

@memoized
def fibConstant(n):
  if(n == 1):
      return (1.0 / fib(n))
  else:
      return (1.0 / fib(n)) + fibConstant(n - 1.0)

if __name__=='__main__':
  import profile
  profile.run('print fib(250)')
  profile.run('print fibConstant(93.0)')