At software startups in particular, companies tend to shy from bureaucracy and hard and fast policies. Employees generally formed together in the first place to avoid that sort of work environment. In turn, in order to motivate people on a project given a proverbial carrot and stick, the carrot is almost always the tool of choice.
Work is generally fun and rewarding at companies like this, but eventually you get to a point where work is legitimately work, and it’s not all that fun. Superb test coverage is probably one of those things that most people consider “work.” Tests are great and everyone loves them – as long as someone else writes them. They’re some of the most tedious and sometimes boring tasks associated with software development.
So how do you get people to write tests? Do you force them to? Do you require layers of approval to get code merged? NAY! You want to establish a culture of superb test coverage, and my way to encourage such a culture is to gamify the test coverage stats on a repository.
“Gamification” is simply the process of adding gamelike elements to something to encourage participation. If a task is no longer a “task” but instead a “game,” it changes the dynamic of the said task.
The “gamelike elements” you can associate with tasks can be anything, but typically all you might need to do is establish incentives, competition, and notoriety.
My idea is simple: For all of the authors in a repo, publicly display the top contributors with respect to number of lines written and test coverage percent. In this way, you’re not embarassing anyone by displaying the top 3rd, and you’re incentivizing people with as much interest as they have in the game itself. Writing more tests would then have the added benefit of advancing in “the game.”
How to Tabulate Results Conceptually
The general concept behind tabulating results is fairly straightforward: take existing modules that allow you to find out test coverage percent by file and combine that with the results of git blame. Then associate test coverage by author instead of the corresponding lines.
Get the Files to Score in the Repo
Before getting started, it’s necessary to be able to list the files that you want in your dataset. In my case, I only wanted Python files and I didn’t want this to include test files or library files. So for starters, we’ll be using a few different shell commands to get some results, and that will require piping from one command to another. So I wrote this simple function that will take in a shell command inclusives of pipes and output the textual result:
From there you can use that helper function to collect your list of files:
We’ll be using the file list both to get the test coverage and to get the authors on from Git Blame.
Install and use the Coverage Module
Install the coverage module to use in conjunction with your tests:
From then you can run coverage. Instead of:
You can do:
Once run, you can collect stats on a file. For example:
And you’ll get the output:
Parse the Coverage File
For our purposes, we care about a file and the line numbers that are missing coverage. For that I just put together a class to parse results and output the missing lines:
So now if I just run this:
I’ll get some simple output of a list of numbers:
Now we need to take those lines and associate them with Git authors.
Getting Authors from Git Blame
There’s probably multiple ways to do this, but this was my method.
If I run this command:
I’ll get results that look something like this:
So now I basically want to associate each line number with its author so that I can take the line numbers excluded and associate it with a person. I will also use this information to tabulate the total line counts for each author so I can express test coverage as a percent. The code to write that is relatively straightforward and should be relatively easy for you to express if you’re reading this blog. So now, you can simply tabulate the results and then output them as you please. In this example I have the columns for author, test coverage percent, and total (non-test) lines written.
I smudged all the names for anonymity in this example. But you should get the idea.
I’d like to see about packaging this whole process and uploading the results to a web application on Heroku. We’ll see where it goes.