For at least a few years now at Kounta we have been using Github for all of our repositories. There is a lot to like about Github, unfortunately there is quite a bit that's not to like.
Our teams use code review amongst themselves to make sure at least two pairs of eyes have seen every line of code that makes it into the product. With more risky changes we can get even more people to review the code.
I won't go into the detail about why code review is so crucial for the quality of the product, however here's a great article from Atlasssian.
Isn't That What Pull Requests Are For?
The short answer is, Yes. The longer answer is, a team of a few people can quickly out grow the very minimalistic features of Github code review - specifically:
- Comments left on code that is subsequently updated hides the comments as "outdated". Github just assumes the issue is fixed or the comment is otherwise is no longer useful. However, if the comment disappears it's very difficult to do a proper follow up with the updated code.
- Github has ridiculously small limits on what they will even display in a pull request. If the diff exceeds this limits, which is not rare, code review is impossible.
- It's important to see just the changes since the last review. You don't need to see all of the code all of the time, that's even worse. We tried using tags for each person which will appear after a commit in the timeline. And yes, you could create a URL to reflect a range of commits since the last tag, but then you can't comment on the lines since your no longer inside the pull request. Huge pain in the butt.
- It's ultimately just an open discussion platform with no congruency on process or responsibility. You can simulate Approvals with tags but then those tags are not removed if there are future changes.
The more consistent process we tried to implement the more hacky and unreliable it actually became. We needed something better, we needed a real code review tool.
The Hunt for a Better Code Review Tool
A change in process for a team is seldom easy, even when everyone is keen for improvement. In my experience, any process that isn't totally integrated, unavoidable and easy will never work consistently. That's why CI is so important as part of our code review.
That said, we needed a tool that integrated tightly with Github and Travis CI - narrowing it down to:
The one that looked like it might be the best for us was reviewable.io. The plan was for everyone to jump on and test it in our real process for 1 sprint (2 weeks) to really get a feel for it and discuss it again at the next retro. We didn't need to try any of the others because reviewable.io ticked all the boxes for us, and we still love it!
Reviewable.io. Just Try It.
There is a demo review you can do. However, in my opinion it doesn't really give you a great feel for the product. Why? Code review is heavily dependent on communication between team members and you can't feel that in the demo.
Reviewable.io is so tightly integrated with Github (a major advantage) and is free for open source repositories (fantastic) so you can open any existing pull request on any repository (not just your own). I really recommend that you try a real review (or participate in another) where you can get that feedback from another engineer.
I want to highlight some bits and pieces that took us some time to clarify, that should hopefully help you (or you team) as well.
If you've never used a dedicated code review tool the first thing you will notice with reviewable.io is that it can be a bit overwhelming. There are a lot of features packed into a small space. Fear not! Once you learn how to harness them it's a very powerful and even more efficient tool.
The most important thing is the checks circle at top. It displays the overall progress. Once it has gone completely green the pull request will be ready to merge in.
This is great because it also shows the status of Travis CI, Scrutinizer CI or anything else that reports its status to Github pull requests.
Depending on how your team works this is a very important checkbox and button. We had a few hiccups until we worked it out.
When entering a review you may want to see all the changes since you last reviewed it, or see all the changes since anybody else last reviewed it.
When you leave a comment on a line it will create a discussion thread for the replies. By default if the original engineer replies with any comment, then you come back and click Acknowledge the discussion thread will be Resolved. This works really well in the majority of cases.
If it's something that's really important you can alter your status in the bottom right (the light grey circle).
By selecting Blocking the issue cannot be resolved by simply acknowledging the response. You will have to manually change your status to Satisfied.
Also, if you would like to leave a note that does not require a code change you can set the status to Satisfied when you create the initial comment and it will be immediately resolved. The original engineer will still be notified and be able to click Acknowledge.
Since reviewable.io runs in the browser it can be extremely heavy when it comes to reviewing large pull requests, or pull requests that have undergone many revisions.
At the top of the page is the ability to show or hide the file matrix. If you are having performance issues I would recommend you turn off the file matrix and your review will be snappy again!
Press ? to see all the key bindings. Once you learn the hot keys you can proceed though a review with awesome speed! Well, hopeful not too fast…
Another great thing about reviewable.io is that all of your changes are saved real time with Firebase.
As you progress through a review all the comments you make and the files you mark off as reviewed will be saved and accumulated automatically. However, none of your changes will be visible to others until you click Publish in the top right.
This is especially useful when doing large reviews where you might not complete the entire review in one sitting, or even 5 sittings.
The original engineer(s) won't have to worry about replying to comments and fixing up code until your whole code review is finished.
Reviewable.io really is a great tool with a lot of features. If your currently using Github for code review or any other tool your feeling isn't giving you the best process or experience - I really recommend you give it a try.