LGTM or TL;DR? The Problem of Cop-out Code Reviews

At CodeSandbox, we run your code in our cloud infrastructure, configure the environment for you and keep your code always ready, behind a shareable URL. Give it a try with this Next.js example or import your GitHub repo!

---

You are completely focused on writing some code, in a perfect state of flow, when that Slack message comes in:

"Hey! Can you have a look at this PR?"

If this sounds familiar, chances are you have shared the pain of countless other developers—having to switch context to handle a code review, especially when it’s urgent.

But that’s just the tip of the iceberg. The concept of “code review” itself seems to be very polarizing in the development community. There’s a reason “LGTM” became a meme for cop-out code reviews. And as a developer myself, I can understand the frustration with some parts of this process.

This got me wondering if this may be a symptom of something fundamentally broken in how we do code reviews.

So, earlier this month, we ran a survey focusing on code reviews. And today, we published the full report “The State of Code Reviews (2023)”, which you can download for free. I’m very excited to dive into the data to answer this main question—are most developers eyeballing code diffs while reviewing PRs, or actually running the code to visually inspect changes?

Code Diffs and Running Previews

I’ll start with a hot take— in 95% of PR reviews, you should be running the branch to visually inspect changes.

But I’ll take a wild guess that is probably not something most developers enjoy.

You have to stash the changes of your current work, checkout into the PR branch, make sure your setup is up to date (and possibly deal with troubleshooting it from time to time), wait for the dev server to start, and finally open the preview to visually inspect the changes.

So, what did we hear from developers when we asked about this?

Within our sample of 1638 developers, the majority say they most often only review a PR by looking at the code diff. When we asked a similar question on “I locally run the branch under review to see the running code”, 36% said they never or very rarely do it.

I find this to be very curious. In a way, it matches this hypothesis that perhaps many developers are relying a lot on code diffs and/or just trying to get reviews out of the way as fast as possible. After reading some of the additional feedback we received in our survey, I’m pretty sure this is the case for many teams:

“Unfortunately many companies pretend they do serious code reviews while in fact devs are approving everything after 2 minutes spent reading it and leaving no feedback in the vast majority of PRs.”

At first glance, it seems that the lack of time and guidelines might be two key reasons behind hasty PR reviews. Plus, a lot of the feedback we received seems to indicate that this could lead to a decline in code quality. So let’s dig deeper!

Impact on Code Quality

While the original scope of our survey was to simply get a picture of how the different stakeholders in code reviews feel about and address this process, the results got us wondering if there’s any sort of correlation between code quality and running the branch locally.

While there’s no clear-cut correlation, the data suggests a pattern where developers that more frequently run the branch locally to see the running code are also more satisfied with the quality of the code being released.

This is all very fascinating and might indicate that implementing robust guidelines and proper tooling for the code review process could have a significant impact on code quality, as mentioned by a few respondents:

“I think currently tooling for code review is bad, you just have a diff and then it depends on your own process you established based on your experience and your attentiveness and time you currently have.”

And here’s where it gets better: because we also sent out our survey to Engineering Managers, we got their unique insight on this matter. But let’s leave that for a follow-up post! If you don’t want to wait, download the full report.

Code Reviews in CodeSandbox

At CodeSandbox, we have been thinking long and hard about improving the PR review workflow.

As our report shows, there are three fundamental problems with this process:

1. Context switching breaks the coding flow and siphons energy from developers.
2. Setting up and running a branch locally takes too much time and is prone to errors and inconsistencies.
3. The lack of guidelines and tooling for code reviews results in ambiguity, cop-out reviews, and lower code quality.

I have good reason to believe we have fixed all of these problems with CodeSandbox.

The “new” version of CodeSandbox we started building in 2022 provides instant cloud development environments powered by our unique microVM architecture. And I say “unique” because we made the opinionated choice of giving a dedicated microVM to every branch (and every PR) while making sure that it resumes in 1 second.

In practice, this means that, if you’re using GitHub as your git provider, CodeSandbox gives you a live-running environment for every PR that is one click away. Inside, you will find the PR’s diff, a pre-configured running preview, all the review tooling you need (comments, suggestions, commits, approvals, etc.), plus a built-in AI assistant.

This makes reviewing PRs much faster, as it removes context switching (the PR opens on a browser tab and you can actually open as many as you want in parallel!) and only takes 1 second for the dev server to resume.

We are thrilled to hear how teams that have been using CodeSandbox for reviews have reported massive time savings and increased contributions from team members.

So, if you or your team have been looking for a better way to do code reviews, give CodeSandbox a try by importing a repo! We have a 14-day free trial you can use to test all our features.

We truly believe we’re solving some of the main pains development teams have been facing in code reviews. And, if nothing else, we will help everyone “LGTM” with confidence again.