Writing an orthogonal test for a target Class is easy enough, you simply mock/stub any methods the Module provides.

Testing the Module elegantly however is less obvious, atleast it wasn’t obvious to me.

One approach I’ve used is to create a fake class for the test, include the Module and finally instantiate this for the test. But there’s the issue of testing the Module’s interaction with the including Class. For this you could partially mock/stub the test class instance, but there’s a simpler approach that avoids the test class altogether: create an ordinary rSpec mock instance and extend it with the Module.

– snip –

First about unit and integration tests.

• I write unit tests for focused feedback; i.e. tell me exactly what broke. To keep them focused I try to keep them orthogonal which usually means using fakes of any collaborators.
• I write integration tests where I need more safety than a unit test will offer. They seem even more important when I’m stubbing and mocking a lot in a dynamically typed language like Ruby.
• I write both types of tests to help convey intent and understand the problem better. I TDD with either a unit test or integration test, whichever feels natural.

Then about interaction and state based testing.

• I pick the approach that feels natural at the time, favouring neither by default. I struggle with rules about when to use which.
• I dislike interaction tests that look suspiciously similar/symmetrical/coupled to the code they refer to. I expect a test to earn its right to exist, and therefore add to the size of the codebase and build’s time, by either conveying intent that is difficult to express in the code itself (which is why I love the term ‘example‘) or addressing some other consciously identified risk.

Your thoughts?

(How do you punctuate around a smiley?)

Feeling even more (irrationally) intolerant than usual, I convinced Mark to let us switch away from the reams of text produced by rSpec’s ‘specdoc’ format to its more succinct ‘progress’ format and we immediately noticed something odd. Each example was being run 7 times, in fact the entire suite was being run several times with runs interleaved!

Some perplexed investigation and educated guessing finally revealed the cause.

While rSpec gives you a SpecRunner for free via an at_exit hook, we use a script to explicitly invoke a CommandLine::SpecRunner. This is in part because our project is built by Ant (it’s a polygot-project; a mishmash of Java, Ruby, Python and, somewhat unfortunately IMO, Bash) and in part just because …

rSpec’s at_exit hook allows you to run your specs directly via the ruby command (e.g. $ ruby three_is_greater_than_four_spec.rb) and is responsible enough to ensure you haven’t already run these specs with a custom runner. Except if you call Process.fork during an example.

When you fork in an example the at_exit hook gets replicated in the child process and when this child process tries to exit, you guessed it, it runs your specs. Turns out we had quite a few unstubbed forks in our unit level specs. The immediate remedy was to stub these; they are unit level specs which should after all be orthogonal.

To make sure we aren’t inadvertently bitten again we’ve intercepted rSpec’s at_exit hook from any child processes by creating our own at_exit hook which exits without executing any other at_exit blocks, i.e. :at_exit { exit! }

Our 830ish specs now run in 15 seconds. The numbers don’t quite add up (200 seconds / 7 processes > 15 seconds), but since I’m certain none are being skipped, I’m content with this outcome.