The open source software movement has come a long way and I’m particularly excited by the improvements made in automated code QA and how it empowers a small team (sometimes, a team of one) to develop software that, just a few years ago, required a large group of programmers and testers and long development times mostly comprising of manual code reviews.
DISCLAIMER: These two projects are still in alpha/beta stage. Still, that shouldn’t stop us from appreciating the power of automated software QA.
I am a believer of the TDD and BDD movement so when I develop, I make sure to write the test first before I write code. As a high-level overview, here’s how my tests and code are organized for RedStack.
RedStack’s main code resides under
/lib whereas the tests for it reside under
/test/lib. The file
view on the right side is showing a section of
project_test.rb which contains all the tests for
project.rb. I use MiniTest for my unit
and integration tests, specifically minitest/spec, because of its simplicity and ease of use.
So a quick walkthrough of how I implement the features for RedStack would be:
it 'creates a project' do...endblock that you see above, pretending that the method being tested already exists. In the above example, the first half of the
creates a projecttest is where I call the method (
project = Project.create(...) and the second half is where I check its output.
project.rbto satisfy the test.
Oftentimes, more than one test is needed to ensure that the method I’m writing is bug free. In that case, I still follow the same test-first-before-code process.
The beauty about this methodology is that writing a test is like adding a teammate to your project whose job is to repeatedly go through your code as many times as you need him to and tell you immediately if it’s broken or not.
So far, though, your ‘teammate’ is not very proactive: it will only check your code if you remember to run it and, sometimes, when you are so engrossed with coding, it’s very easy to forget this. When that happens, minutes have passed and when you run your tests again, you might find a lot of broken code that’s hard to debug. This leads you to ignore your tests ‘for now’ which will eventually lead you to ignore it altogether. Before you know it, your tests are useless.
This is where I put continuous testing to good use. For this, I use Guard to monitor file modifications in my project and to automatically execute one or more tests depending on what changed. Here’s what I get when I write code and all tests are satisfied:
There are a few things happening in the above screenshot. Let me highlight the important ones:
That’s all cool, but so far we haven’t seen how Guard automatically runs my tests. For that, I changed my code to deliberately fail. Guard detected the file change and automatically ran the related tests. Here’s what happened afterwards:
And that’s how to ensure that your tests stay fresh and relevant.
So far, we have tests and we have a way to ensure that they stay useful. However, just because all tests pass doesn’t mean that your code is bug free. How is that possible, you ask? Well, it may be the case where your test suite isn’t testing most of your code. This is where code coverage comes into play. With code coverage, you can see which parts of your code was exercised by your test.
For this, I use SimpleCov which you saw in action in the above screenshots but probably didn’t notice it. Did you wonder about the line that says “Coverage report generated for…?” Let’s look at a sample coverage report generated by SimpleCov:
The above summary tells me that about 94% of my code is being exercised by my tests. Cool. Clicking on one of the items in that list will show me the exact lines in my code that are being exercised or not:
In the above screenshot, lines that are green were exercised by my tests whereas lines that are red were not. Red lines can tell me one of two things: 1) I need to test that code, or 2) That code is probably not necessary and I should delete it. Whichever it is depends on the situation.
Just because your code has a high code coverage doesn’t guarantee that your code is bug free. Here’s a real-world example that I didn’t notice was a bug until much later. It was not detected by my tests and SimplCov marked it (correctly) as covered.
What’s wrong with that line of code? That error message is in the User class, not the Project class, so it should say “…not authorized to delete user.” It’s a bug, but because I was not specifically testing the message and because SimpleCov was marking it green, I didn’t notice it until much later.
Automated code QA has made some great strides these past few years thanks to the many contributions of some of the great minds in the open source community. The above walkthough doesn’t do justice to the tools mentioned so here are some bonus items:
I’ve been saying this for some time now: It’s a good time to be a programmer, thanks to the tools that are available to anyone for free these days.