Not in code

A 2 week break over christmas/new years and pairing with the incandescent Mark Ryall seemed to make our unit test (spec?) suite time of 200 seconds feel excruciatingly slow. I’m only kidding Mark, I really do enjoy pairing with you. :)

(How do you punctuate around a smiley?)

Feeling even more (irrationally) intolerant than usual, I convinced Mark to let us switch away from the reams of text produced by rSpec’s ‘specdoc’ format to its more succinct ‘progress’ format and we immediately noticed something odd. Each example was being run 7 times, in fact the entire suite was being run several times with runs interleaved!

Some perplexed investigation and educated guessing finally revealed the cause.

While rSpec gives you a SpecRunner for free via an at_exit hook, we use a script to explicitly invoke a CommandLine::SpecRunner. This is in part because our project is built by Ant (it’s a polygot-project; a mishmash of Java, Ruby, Python and, somewhat unfortunately IMO, Bash) and in part just because …

rSpec’s at_exit hook allows you to run your specs directly via the ruby command (e.g. $ ruby three_is_greater_than_four_spec.rb) and is responsible enough to ensure you haven’t already run these specs with a custom runner. Except if you call Process.fork during an example.

When you fork in an example the at_exit hook gets replicated in the child process and when this child process tries to exit, you guessed it, it runs your specs. Turns out we had quite a few unstubbed forks in our unit level specs. The immediate remedy was to stub these; they are unit level specs which should after all be orthogonal.

To make sure we aren’t inadvertently bitten again we’ve intercepted rSpec’s at_exit hook from any child processes by creating our own at_exit hook which exits without executing any other at_exit blocks, i.e. :at_exit { exit! }

Our 830ish specs now run in 15 seconds. The numbers don’t quite add up (200 seconds / 7 processes > 15 seconds), but since I’m certain none are being skipped, I’m content with this outcome.