“Is Code Review Important and Effective?”

There is a consensus across the industry that code review is an effective and practical way to collar code inconsistency and possible defects early in the software development life cycle. Among others some of the advantages of code reviews are,

• Bugs are found faster
• Forces developers to write readable code (code that can be read without explanation or introduction!)
• Optimization methods/tricks/productive programs spread faster
• Programmers as specialists "evolve" faster
• It's fun

“Code review is systematic examination (often known as peer review) of computer source code. It is intended to find and fix mistakes overlooked in the initial development phase, improving both the overall quality of software and the developers' skills. Reviews are done in various forms such as pair programming, informal walkthroughs, and formal inspections.” Wikipedia

No where does the definition mention whether its better to review code before the code has been committed to version control or after the commit has been performed. No matter which side you favour, Visual Studio 2012 allows you to request for a code review both before check in and also request for a review after check in. Let’s weigh the pros and cons of the approaches independently.

Code Review Before Check In or Code Review After Check In?

Approach 1 – Code Review before Check in

Developer completes the code and feels the code quality is appropriate for check in to TFS. The developer raises a code review request to have a second pair of eyes validate if the code abides to the recommended best practices, will not result in any defects due to common coding mistakes and whether any optimizations can be made to improve the code quality.

                                            Image 1 – code review before check in

Pros

1. Everything that gets committed to source control is reviewed.
2. Minimizes the chances of smelly code making its way into the code base.
3. Decreases the cost of fixing bugs, remember, the earlier you find them, the lesser the pain in fixing them.

Cons

1. Development Code Freeze – Since the changes aren’t in the source control yet. Further development can only be done off-line.
2. The changes have not been through a CI build, hard to say whether the code abides to all build quality standards.
3. Inconsistent! Cumbersome to track the actual code review process. 
4. Not every change to the code base is worth reviewing, a lot of effort is invested for very little gain.

Approach 2 – Code Review after Check in

Developer checks in, random code reviews are performed on the checked in code. 

                                                    Image 2 – Code review after check in

Pros

1. The code has already passed the CI build and run through any code analysis plug ins you may have running on the build server.
2. Instruct the developer to ensure ZERO fx cop, style cop and static code analysis before check in. Code is cleaner and smell free even before the code review.
3. No Offline development, developers can continue to develop against the source control.

Cons

1. Bad code can easily make its way into the code base.
2. Since the review take place much later in the cycle, the cost of fixing issues can prove to be much higher.

Approach 3 – Hybrid Approach

The community advocates a more hybrid approach, a blend of tooling and human accountability quotient.

                                                              Image 3 – Hybrid Approach

1. Code review high impact check ins. It is not possible to review everything, by setting up code review check in policies you can end up slowing your team. More over, the code that you are reviewing before check in hasn't even been through a green CI build either.

2. Tooling. Let the tooling work for you. By running static analysis, fx cop, style cop and other plug ins on the build agent, you can identify the real issues that in my opinion can't possibly be identified using human reviews. Configure the tooling to report back top 10 issues every day. Mandate the manual code review of individuals who keep making it to this list of shame more often.

3. During Merge. I would prefer eliminating some of the other code issues during merge from Main branch to the release branch. In a scrum project this is still easier because cheery picking the merges is a possibility and the size of code being reviewed is still limited.

Let the tooling work for you, if some one breaks the CI build often, put them on a gated check in build course until you see improvement. If some one appears on the top 10 list of shame generated via the build then ensure that all their code is reviewed till you see improvement. At the end of the day, the goal is to ensure that the code being delivered is top quality. By enforcing a code review before any check in, you force the developer to work offline or stay put till the review is complete.

What do the experts say?

So I asked a few expects what they thought of “Code Review quality gate before Checking in code?"

Terje Sandstrom | Microsoft ALM MVP

You mean a review quality gate BEFORE checking in code?????

That would mean a lot of code staying either local or in shelvesets, and not even been through a CI build, and a green CI build being the main criteria for going further, f.e. to the review state.

I would not like code laying around with no checkin’s.

Having a requirement that code is checked in small pieces, 4-8 hours work max, and AT LEAST daily checkins, a manual code review comes second down the lane.

I would expect review quality gates to happen before merging back to main, or before merging to release.  But that would all be on checked-in code. 

Branching is absolutely one way to ease the pain.
 
Another way we are using is automatic quality builds, running metrics, coverage, static code analysis.  Unfortunately it takes some time, would be great to be on CI’s – but…., so it’s done scheduled every night. Based on this we get, among other stuff,  top 10 lists of suspicious code, which is then subjected to reviews.  If a person seems to be very popular on these top 10 lists, we subject every check in from that person to a review for a period. That normally helps.
 
None of the clients I have can afford to have every checkin reviewed, so we need to find ways around it. I don’t disagree with the nicety of having all the code reviewed, but I find it hard to find those resources in today’s enterprises.

David V. Corbin | Visual Studio ALM Ranger

I tend to agree with both sides. I hate having code that is not checked in, but at the same time hate having “bad” code in the repository. I have found that branching is one approach to solving this dilemma. Code is checked into the private/feature branch before the review, but is not merged over to the “official” branch until after the review.

I advocate both, depending on circumstance (especially team dynamics)
 
- The “pre-checkin” is usually for elements that may impact the project as a whole. Think of it as another “gate” along with passing unit tests.

- The “post-checkin” may very well not be at the changeset level, but correlates to a review at the “user story” level.
 
Again, this depends on team dynamics in play….

Robert MacLean | Microsoft ALM MVP

I do not think there is no right answer for the industry as a whole.

In short the question is why do you do reviews? Your question implies risk mitigation, so in low risk areas you can get away with it after check in while in high risk you need to do it before check in. An example is those new to a team or juniors need it much earlier (maybe that is before checkin, maybe that is soon after) than seniors who have shipped twenty sprints on the team.

Abhimanyu Singhal | Visual Studio ALM Ranger

Depends on per scenario basis.

We recommend post check-in reviews when:

1. We don't want to block other checks and processes on manual code reviews. Manual reviews take time, and some pieces may not require manual reviews at all.
2. We need to trace all changes and track history.
3. We have a code promotion strategy/process in place. For risk mitigation, post checkin code can be promoted to Accepted branches. Or can be rejected.

Pre Checkin Reviews are used when

1. There is a high risk factor associated
2. Reviewers are generally (most of times) have immediate availability.
3. Team does not have strict tracking needs.

Simply speaking, no single process fits all scenarios. You need to select what works best for your team/project.

Thomas Schissler | Visual Studio ALM Ranger

This is an interesting discussion, I’m right now discussing details about executing code reviews with my teams. I see and understand the aspects you brought in, but there is another side as well, I’d like to point out.

1.) If you do reviews per check in this is not very practical as a hard rule because this will disturb the flow of the team very often or it will lead to reduce the checkin frequency of the devs which I would not accept.

2.) If you do later reviews, for example if you review PBIs, it is not easy to find out which code you should review. Either you review all changesets associate with the PBI, but then you might review code which has been changed with a later checkin and the dev maybe has already fixed the issue. Or you review the diff of the latest changeset of the PBI with the first but then you might also review changes of other PBIs.

Jakob Leander | Sr. Director, Avanade

In my experience, manual code review:

1. Does not get done and at the very least does not get redone after changes (regardless of intentions at start of project)

2. When a project actually do it, they often do not do it right away = errors pile up

3. Requires a lot of time discussing/defining the standard and for the team to learn it

However code review is very important since e.g. even small memory leaks in a high volume web solution have big consequences

In the last years I have advocated following approach for code review

- Architects up front do “at least one best practice example” of each type of component and tell the team. Copy from this one. This should include error handling, logging, security etc.

- Dev lead on project continuously browse code to validate that the best practices are used. Especially that patterns etc. are not broken. You can do this formally after each sprint/iteration if you want. Once this is validated it is unlikely to “go bad” even during later code changes

Agree with customer to rely on static code analysis from Visual Studio as the one and only coding standard. This has HUUGE benefits

- You can easily tweak to reach the level you desire together with customer

- It is easy to measure for both developers/management

- It is 100% consistent across code base

- It gets validated all the time so you never end up getting hammered by a customer review in the end

- It is easy to tell the developer that you do not want code back unless it has zero errors = minimize communication

You need to track this at least during nightly builds and make sure team sees total # issues. Do not allow #issues it to grow uncontrolled.

On the project I run I require code analysis to have run on code before checkin (checkin rule). This means

-  You have to have clean compile (or CA wont run) so this is extra benefit = very few broken builds

- You can change a few of the rules to compile as errors instead of warnings. I often do this for “missing dispose” issues which you REALLY do not want in your app

Tip: Place your custom CA rules files as part of solution. That  way it works when you do branching etc. (path to CA file is relative in VS)

Some may argue that CA is not as good as manual inspection. But since manual inspection in reality suffers from the 3 issues in start it is IMO a MUCH better (and much cheaper) approach from helicopter perspective

Tirthankar Dutta | Director, Avanade

I think code review should be run both before and after check ins.

There are some code metrics that are meant to be run on the entire codebase …

Also, especially on multi-site projects, one should strive to architect in a way that lets men manage the framework while boys write the repetitive code… scales very well with the need to review less by containment and imposing architectural restrictions to emphasise the design.

Bruno Capuano | Microsoft ALM MVP

For code reviews (means peer reviews) in distributed team I use http://www.vsanywhere.com/default.aspx 

David Jobling | Global Sr. Director, Avanade

Peer review is the only way to scale and its a great practice for all in the team to learn to perform and accept. In my experience you soon learn who's code to watch more than others and tune the attention.

Mikkel Toudal Kristiansen | Manager, Avanade

If you have several branches in your code base, you will need to merge often. This requires manual merging, when a file has been changed in both branches. It offers a good opportunity to actually review to changed code. So my advice is: Merging between branches should be done as often as possible, it should be done by a senior developer, and he/she should perform a full code review of the code being merged.
As for detecting architectural smells and code smells creeping into the code base, one really good third party tools exist: Ndepend (http://www.ndepend.com/, for static code analysis of the current state of the code base). You could also consider adding StyleCop to the solution.

Jesse Houwing | Visual Studio ALM Ranger

I gave a presentation on this subject on the TechDays conference in NL last year. See my presentation and slides here (talk in Dutch, but English presentation):

http://blog.jessehouwing.nl/2012/03/did-you-miss-my-techdaysnl-talk-on-code.html 

I’d like to add a few more points:

- Before/After checking is mostly a trust issue. If you have a team that does diligent peer reviews and regularly talk/sit together or peer review, there’s no need to enforce a before-checkin policy. The peer peer-programming and regular feedback during development can take care of most of the review requirements as long as the team isn’t under stress.

- Under stress, enforce pre-checkin reviews, it might sound strange, if you’re already under time or budgetary constraints, but it is under such conditions most real issues start to be created or pile up.

- Use tools to catch most common errors, Code Analysis/FxCop was already mentioned. HP Fortify, Resharper, Coderush etc can help you there. There are also a lot of 3^rd party rules you can add to Code Analysis. I’ve written a few myself (http://fccopcontrib.codeplex.com) and various teams from Microsoft have added their own rules (MSOCAF for SharePoint, WSSF for WCF). For common errors that keep cropping up, see if you can define a rule. It’s much easier. But more importantly make sure you have a good help page explaining *WHY* it's wrong.

If you have small feature or developer branches/shelvesets, you might want to review pre-merge. It’s still better to do peer reviews and peer programming, but the most important thing is that bad quality code doesn’t make it into the important branch.

So my philosophy:

- Use tooling as much as possible.

- Make sure the team understands the tooling and the importance of the things it flags. It’s too easy to just click suppress all to ignore the warnings.

- Under stress, tighten process, it’s under stress that the problems of late reviews will really surface

- Most importantly if you do reviews do them as early as possible, but never later than needed. In other words, pre-checkin/post checking doesn’t really matter, as long as the review is done before the code is released. It’ll just be much more expensive to fix any review outcomes the later you find them.

---

I would love to hear what you think!