NET to Ruby: Learning How to Write Tests, Part II

Part 1 of this post covered my experiences in .NET when writing tests, and how that helped me getting productive in Ruby within a short period of time. Part 2 covers my experiences in Ruby, going from how I got started writing tests as soon as I started doing anything with Ruby, until my current state, writing better tests and improving my approach to how I develop software.

I know that I still have tons to learn, and that there are Ruby wizards out there who can certainly do things a lot better than what I’ll show you here. These things will always be the case. My intention is solely to come out and say what my experiences have been, and hopefully we can also get some feedback in the comments section and get some experience exchange going.

Tests Helped Me Learn Ruby

As I mentioned in on my post here, with Ruby being a dynamic language, I couldn’t rely on the compiler giving me at least some direction as to where I’m messing things up. I could use the Interactive Ruby Shell (IRB) to try things out (and I do that once in a while), but I’d rather write tests. Why? Because they become “exploratory” tests. They let me try things out until I get the tests to pass. Once they do, it becomes documentation of the last thing I tried that actually worked.

As time goes by and I learn better ways to get something done in Ruby, I can go back to my exploratory tests, try out the better way, and see if my tests still pass. Not only do I go back and improve my “production” code, but I also go back and improve my test code.

Picking a Test Framework

Just like in the .NET community, the Ruby community also has a bunch of test frameworks available. I decided to pick the one that most of my buddies use: RSpec. I learned just the minimal, just so I could get by, not too worried whether it was “the Ruby way” or not. As long as I could instantiate classes, call methods, and assert some expectations, life was good. So I started writing tests like this:

describe Calculator do

  it "should sum two numbers" do
    calculator = Calculator.new
    result = calculator.sum(1, 1)
    result.should == 2
  end

  it "should subtract two numbers" do
    calculator = Calculator.new
    result = calculator.subtract(5, 3)
    result.should == 2
  end
end

The code above should be readable by any C# developer. Writing tests like that I managed to learn enough Ruby to get a bunch of things done.

When running the tests, I can pass a switch to RSpec and tell it to format the output as a document, and this is what it looks like:

Calculator
  should sum two numbers
  should subtract two numbers

And if any of my examples (that’s what each test is called in RSpec, by the way) fail, I get this type of output:

Calculator
  should sum two numbers (FAILED - 1)
  should subtract two numbers

Once I gained some momentum, I started to look for ways to improve my test code a little. I was hoping to resemble the better tests I was writing in C#, with SubSpec, the “should” extension methods and such. Slightly improved tests started to look like this:

describe Calculator do
  context "Given a calculator" do
    before(:each) do
      @calculator = Calculator.new
    end

    describe "when told to add two numbers" do
      it "should return the sum of the two numbers" do
        result = @calculator.sum(1, 1)
        result.should == 2
      end
    end

    describe "when told to subtract two numbers" do
      it "should return the difference of the two numbers" do
        result = @calculator.subtract(5, 3)
        result.should == 2
      end
    end

  end
end

A couple of things to notice:

The use of context to provide the “Given” part of my spec
The use of before(:each) to set up my context before each example (each test)

With such structure, now my test output looks like:

Calculator
  Given a calculator
    when told to add two numbers
      should return the sum of the two numbers
    when told to subtract two numbers
      should return the difference of the two numbers

Let’s say I needed to change the design of that calculator so that it would do two extra things: store the inputs give into an array, and store the last result calculated. I could rewrite the specs like this:

describe Calculator do

  context "Given a calculator" do

    before(:each) do
      @calculator = Calculator.new
    end

    describe "when told to add 1 + 1" do

      before(:each) do
        @calculator.sum(1, 1)
      end

      it "should have two input values" do
        @calculator.input_values.length.should == 2
      end

      it "should have last result of 2" do
        @calculator.last_result.should == 2
      end
    end

  end
end

The output for those specs still looks nice:

Calculator
  Given a calculator
    when told to add 1 + 1
      should have two input values
      should have last result of 2

At that point I felt like I had my tests at the same level I did in .NET. After writing tests like that for some time, though, I came to learn about some specific RSpec tricks (tricks that are courtesy of Ruby’s awesome power and flexibility!). Such tricks allow my test code to be more compact, but still express intent, and still get good output when running the specs. And best of all, I’m writing less code!

describe Calculator do

  context "Given a calculator" do

    subject { Calculator.new }

    describe "when told to add 1 + 1" do

      before { subject.sum(1, 1) }

      it { should have(2).input_values }
      its(:last_result) { should == 2 }
    end

  end
end

Things to notice:

Instead of instantiating Customer and storing it to an instance variable in a before(:each), I use the subject block, which stores whatever is returned from it to a subject property on the spec;
Instead of calling the method being tested (e.g., subject.sum(1, 1)) in a before(:each), I just use a before {}, which does the very same thing, but is a little shorter;
Instead of providing strings to the it blocks, I let RSpec create the strings based on what Ruby code is inside the block (you’ll see the results shortly);
it intrinsically refers to subject;
Instead of checking the length of the input_values array, I use the *have* matcher. At the end of the day, RSpec turns *should have(2).input_values* into subject.input_values.length.should == 2, but I like the way the former reads much better;
In order to check a property that hangs off the subject, I use the its method (e.g., its(:last_result) { should == 2 }

Make sure to compare the prior two versions of the example side by side. Notice that the second one has less strings explaining what I’m testing, it has more Ruby code, but the Ruby code there reads in such a way that we don’t need to parse the code mentally in order to understand what it does. Last but not least, the output of the revised spec still looks good:

Calculator
  Given a calculator
    when told to add 1 + 1
      should have 2 input_values
      last_result
        should == 2

That stuff was all great, and it was inline with the things I was doing in .NET (but with much better syntax!). However, I was still focusing solely on “object” behavior. Users don’t really care about how an “object” behaves; they care about how the “application” behaves.

So That is What BDD Was All About…

I’ve talked about the “Given some context, When certain thing happens, Then some expectation should be met” idea. An important part missing in that is the answer to the following question: “What is the business value in that expectation being met?”. I was aware of writing user stories like so:

In order to something that’ll produce business value
As a type of person
I want to have the application do something for me

I couldn’t really tell how that could be incorporated into my workflow as a developer. I was missing the whole point of “stop thinking about objects, and start thinking about the real business value behind certain feature”.

I wasn’t aware of how to encode something like that as tests for my application in the .NET arena. I actually did know of some initiatives about it, but they didn’t seem to be getting much traction.

Next stop: Cucumber.

Executable acceptance tests

Sometime after spending a couple of weeks writing tests in RSpec, I learned how to write “feature” tests (or “feature stories”) in Cucumber. Cucumber is built on top of RSpec, so it wasn’t to hard to learn it, as I already knew some things. A feature story looks something like this (example straight off Cucumber’s site):

Feature: Addition
  In order to avoid silly mistakes
  As a math idiot
  I want to be told the sum of two numbers

  Scenario: Add two numbers
    Given I have entered 50 into the calculator
    And I have entered 70 into the calculator
    When I press add
    Then the result should be 120 on the screen

Pretty much plain English. It has the “In order to… As a… I want to…” part, which describes the business value for the feature, and it has “scenarios” described with “given-when-then”. This part of the equation covers what the user, the people you’re building the application for, needs.

Where does Ruby or RSpec come in? In the implementation of those scenario “steps”:

Given /I have entered (.*) into the calculator/ do |n|
  calculator = Calculator.new
  calculator.push(n.to_i)
end

Writing the steps is the other part the equation, this time covering what code I, the developer, would like to have. If I find myself having too much code within a step, that’s a good indication I’m not designing my objects properly. It’s a chance for me to think harder about the step and come up with something simpler.

At this point, when I run the test I get errors, since the objects and methods haven’t been created yet. The code for this sample could look like this:

class Calculator
  def push(n)
    @args ||= []
    @args << n
  end
end

After creating the class and implementing the methods (only those required by the current step I’m working on), then the step should pass, and I can move on to the next one. I repeat the process until all the steps in the feature are passing, and therefore, the feature is complete.

Outside-In

There will be times when my objects will require more thorough and granular tests. At that point, I drop into RSpec and write my tests there. In other words, with “feature stories” created using Cucumber I am testing the behavior of the “application”, while keeping an eye on having the most minimal object API’s to enable the feature. On the other hand, with specs created using RSpec, I am testing the behavior of my “objects”, while keeping an eye on the sorts of collaboration and dependencies my objects need in order to perform their job.

This approach is also known as “outside-in” development: I start from the outside of the application, where I’m very close to the people who’ll be using my application. Then, I move into the innards of my objects that enable the features we’re requesting from the outside. For me, this is a departure from the “think about the objects first” approach I was following before. Users of my applications do not care about objects!

Building a Feature Without Running the Application

In the past I’ve built several applications like this:

Create the view
Run the application
Click through numerous menus and screens until I get to the view created in step #1
Go in the code and drop a breakpoint
Click some button on the application
Wait until that breakpoint has been hit
Go to the debugger and look things around
Repeat steps several times and eventually my feature is done

There was a lot of time wasted there. Following this outside-in approach with Cucumber and RSpec, several times I don’t even get to run my application until my feature is completed (which means I run my tests and they all pass). Once my tests tell me the feature is complete, I run the application, and voila, things work. Yes, sometimes it may not work, but that’s the exception, not the rule (usually the problem was due to me messing something up).

Building and Testing REST API’s

Just recently I’ve done work creating a REST API in one of my Rails applications. This API served up JSON consumed by mobile applications. I’ve described the API with Cucumber feature stories, which worked as documentation for my API, and had the nice “side effect” of being automated tests to ensure the API works.

I worked pretty much exclusively with my tests for a couple of days. Once all of my tests were passing, I’ve deployed my API and our mobile application worked just fine consuming data and posting data to the API.

Here’s a small example of a feature supported by this API:

Feature: User authentication

  In order to only allow authenticated users to use our applications
    As an API client
    I want to be able to authenticate a user based on his credentials

  Scenario: Authentication Failed

  Given invalid credentials
    When I send a request for user authentication
    Then the response should be "200"
     And I should receive message indicating authorization failure

  Scenario: Authorization Succeeded

  Given valid credentials
    When I send a request for user authentication
    Then the response should be "200"
     And I should receive message indicating authorization success

And here’s an example of how the steps are implemented:

Given /^invalid credentials$/ do
  @login = "INVALID_LOGIN"
  @password = "INVALID_PASSWORD"
end

Given /^valid credentials$/ do
  @user = Factory(:athlete_user)
  @login = @user.email
  @password = @user.password
end

When /^I send a request for user authentication$/ do
  post "api/v3/user_authentication.json?login=#{@login}&password=#{@password}"
end

Then /^I should receive message indicating authorization success$/ do

  success = {:user_authentication =>
               {:status => "success",
                :errorcode => nil,
                :errormessage => nil,
                :login => @login,
                :profile => 'athlete'}}

  last_response.body.should == success.to_json
end

Then /^I should receive message indicating authorization failure$/ do

  error = {:user_authentication =>
               {:status => "failed",
                :errorcode => "UNKNOWN_LOGIN",
                :errormessage => "Invalid credentials.",
                :login => nil,
                :profile => nil}}

  last_response.body.should == error.to_json
end

The time I spent writing this is so well worth it. Again, it works as documentation for my API while testing the API to make sure it works.

Is That What All Ruby Developers Do?

No!! That is not what all Ruby developers do. Some developers don’t like RSpec and/or Cucumber at all, and use other test frameworks instead. People use whatever makes sense to them. What I’ve describe here is what I currently believe to be the best processes and tools for the work I do, in the projects on which I’m currently working. It’s definitely not a one-size-fits-all solution, and it doesn’t mean I’m not constantly looking for different things people are doing or using.

Do I Write Tests for Everything?

I do not write tests for everything. If I have requirements that are clear and well understood between me and my client, then by all means I’ll do my best to write tests as appropriate. However, there are cases where my client isn’t sure of what they need. There are sparse ideas, but nothing written in store. In such case, I may prototype some things in a new branch, and show it to the client. For that, I do not write tests. It’d be wasted time, since those tests would be very likely to get thrown away very soon.

Once I show the “live” prototype to the client, get some feedback, and discuss the options, we try to formalize the mutual understanding for what they really need. Then, I write the feature stories for it.

Resources

The best resources I’ve used to learn more about the topic I discussed here were The RSpec Boook and The Cucumber Book. I keep hearing good things about Specification by Example, so that’s up on my “to read” list.

What do you do?

I’d like to know of your experiences with testing, SubSpec, RSpec, Cucumber, TDD, BDD… Anything we touched on in this blog post, really. What worked for you? What didn’t? Why? What kind of experiences have you tried? Often I hear people say “I tried this testing thing but it didn’t work for me.”. Well, that’s not enough. It’d be great to know how exactly the person tried it, what was the type of application, language, client, context, etc. I know several things I tried didn’t work, but I’d talk to peers about my experiences, and more often then not we’d identify the likely reasons as to why it didn’t work, so I could make corrections and try again. :)