Sunday, September 26, 2021

Code reviews (Synchronous and Asynchronous)

There are different types of Code Reviews with different objectives. This article only refers to those Code Reviews that another team member does before incorporating the change in the shared code repository.

The objective of these reviews is to improve the quality, avoid errors introduced in the shared code repository, and prevent issues from going out to production.

Therefore the code reviews we are talking about:

  • Avoid errors/bugs by acting as a safeguard.
  • Occur within the flow of making changes to production.

In this blog post, we will not discuss other types of Code Reviews that are done outside of the normal development flow.

Most common: Asynchronous code reviews.

In this context, a very common (and for some, recommended) way of performing code reviews is that a developer (A) works individually on a task (d) and upon completion of the general change, creates a Pull Request (PR)/ Merge Request (MR) that another person in the team must review asynchronously.

We can see an example of this way of working in the following diagram. In it, we can see how developer A generates PR/MRs that developer B reviews, generating proposals for changes that developer A should incorporate.

In the example, the feature/change is composed of increments d1, d2, d3, and we see that we deploy it to production at the end of the code review of increment d3.

We can see that A's development flow is constantly interrupted to wait for the corresponding feedback for each code review. This way of working delays the release to production, increasing the time it takes to receive real feedback from customers.

Because each individual code review takes so long, there is an unconscious and natural tendency for developers to work in increasing larger increments. At least, this is what I found in many teams I have worked with.

It is a curious situation because, on the one hand, the industry says that working in small safe steps is a recommended practice. On the other hand, many companies use async reviews that tend to discourage them.

So the most common way of working looks like the following diagram:

Moreover, in this case, by generating large PRs/MRs, the code review loses meaning and, in many cases, becomes a pantomime (LGTM). (See: how we write/review code in big tech companies)
With this last way of working, the lead time to production improves a little, but at the cost of taking bigger steps and losing the benefits of detailed code reviews.

When working with asynchronous code reviews, the reality is usually more complex than what we have seen in the previous diagrams. The reality usually includes many developers, simultaneous code reviews and cross flows in which there is a strong tendency for individuals to work alone and do continuous context switching.

In fact, there is no incentive for the team to focus and collaborate on a single user story at the same time since everyone would be busy with their tasks and doing asynchronous code reviews for each other.

Another problem that often occurs is that code reviews do not have sufficient priority and generate long queues of pending reviews. These queues cause the lead time to grow and create deployments with larger batches of changes (more risk).

Is there a better way?

Fortunately, there are synchronous code reviews and continuous code reviews.

Synchronous, high priority code reviews

The first step is to give the highest priority to code reviews and to do them immediately and synchronously between the person who developed the increment and the person in charge of doing the review.

This way of working reduces the total lead time of each increment and encourages working in small increments.

As a side effect, the code reviews should be more straightforward because the developer themself explains them and can express in detail why they have taken each of the decisions.

This way of working requires a lot of coordination between team members, and it is not easy to make them always synchronous, but of course, it is a simple step if we are already doing asynchronous code reviews.

It is as simple as making PRs/MRs top priority, and as soon as you have one ready, coordinate with someone else to review it together on a single computer.

Pairing/Ensemble programming: Continuous Code Review

Pair programming: "All code to be sent into production is created by two people working together at a single computer. Pair programming increases software quality without impacting time to deliver. It is counter intuitive, but 2 people working at a single computer will add as much functionality as two working separately except that it will be much higher in quality. With increased quality comes big savings later in the project.".
When we work using pair programming, as recommended in extreme programming, two people design and develop the code. This way of working generates a continuous review of the code, so it no longer requires a final review phase. 

Of course, from the point of view of coordination, it is the simplest system, since once the pairs have been formed, each pair organizes itself to make increments, which are implicitly reviewed continuously.

In this case, the diagram representing this way of working would be: 

A new way of working known as mob/ensemble programming has appeared as an evolution of the pair programming practice. In this modality, the whole team works simultaneously on a single task and using a single computer. 

The whole team collaborates in the design and development, limiting the context switching and minimizing the lead time for the task. As in pair programming, mob/ensemble programming performs a continuous and synchronous review of the code.

Of the ways of working that we have seen, pair or group programming has the shortest lead time for each change, fewer context switching interruptions, and requires less coordination.

Even so, pair and group programming is not a simple practice and requires effort and a lot of practice. IMHO it is clear that this way of working takes the quality benefits usually associated with code reviews to the extreme.

The industry vision, my personal vision

The industry seems to agree that code reviews are a good practice and should be implemented by all development teams.
On the other hand, it is also recommended in most cases to work in small increments so that the complexity and risk of each change can be better controlled.

With these two points in mind, it seems a good idea to use Code Reviews and make them as synchronous and continuous as possible to work on those small increments of low complexity and low risk.

In my case, I have been successfully pushing pair or mob programming for a few years now, creating a stream of small increments that are reviewed continuously.

In both Alea Soluciones and TheMotion, we used pair programming by default and mob programming occasionally. Both companies' deployment frequency was daily, and we could make considerable changes in small and safe steps. In addition to this constant and sustainable development flow, we got attractive additional benefits such as the ease of spreading domain and technical knowledge or make fast onboarding of new members. In the case of Alea Soluciones, we hired people who already had experience in pairing and XP practices, so it was easier. In the case of TheMotion, however, we had to introduce some of the agile development practices (pairing, TDD, etc.), so we hired technical coaching help.
In Nextail there was a mix of teams doing pairing and continuous reviews and others using asynchronous code reviews but prioritizing to avoid queues.
At Clarity AI, some teams use asynchronous Code Reviews but prioritize them to minimize the lead time. There are already two teams doing continuous code reviews (using pairing or mob programming) and we are gradually expanding this way of working to other teams.

Of course, the statistical value of my experience is zero, and it is much more interesting to analyze the results of more rigorous analyses such as DORA ( In them, we can see how the following practices are associated with high-performance teams:


Sunday, September 12, 2021

Good talks/podcasts (September 2021 II)

These are the best podcast/talks I've seen/listen to recently:

  • Chapter Two with Christina Wodtke, Author of Radical Focus (Christina Wodtke, Barry O'Reilly) [Company Culture, Culture, Engineering Culture, Inspirational] [Duration: 0:31:00] A great discussion to understand how to correctly introduce and use OKRs.
  • Day Two Cloud 111: Infrastructure As Software With Kris Nóva (Kris Nóva) [Cloud, Devops, Infrastructure, Platform, Platform as a product] [Duration: 0:53:00] thought-provoking podcast episode. Kris Nóva, claims that managing infrastructure using tools like Terraform isn’t that far away from just writing your own code to do the job yourself.
  • Rails Conf 2013 The Magic Tricks of Testing (Sandi Metz) [Agile, Technical Practices, XP, testing] [Duration: 0:32:00] (⭐⭐⭐⭐⭐) This talk strips away the veil and offers simple, practical guidelines for choosing what to test and how to test it. Finding the right testing balance isn't magic, it's a magic trick; come and learn the secret of writing stable tests that protect your application at the lowest possible cost.
  • Honeycomb's secret to high-performing teams (Charity Majors) [Continuous Delivery, Engineering Culture, Operations, Technical leadership] [Duration: 0:51:00] (⭐⭐⭐⭐⭐) Interesting conversation about high performance engineering organizations and how the pace of production is the heartbeat of the organization. They also discuss some very interesting ideas about how great teams generate great engineers, rather than the other way around.
  • Why There are Now So Many Shortages (It's Not COVID) [General, Lean] [Duration: 0:20:00] Interesting documentary on the impact on supply chains of the pandemic and how in many cases trying to use (without following the principles) lean ideas has generated fragility and lack of adaptability.
  • Software Engineering podcast: Episode 473: Mike Del Balso on Feature Stores (Mike Del Balso) [Data Engineering, Data Science, Machine Learning] [Duration: 0:55:00] (⭐⭐⭐⭐⭐) Mike Del Balso co-founder of Tecton discusses Feature Stores and how it helps operationalize Machine Learning.
  • WTF is Platform as a Product? (Matthew Skelton) [Devex, Platform, Platform as a product, team topologies] [Duration: 1:28:00] Matthew explains why the platform-as-product approach can be a game-changer for organisations building and running software-enabled products and services. Using ideas and patterns from Team Topologies—including Thinnest Viable Platform, team cognitive load, and the evolutionary team interaction modes—he explains how organisations like Adidas and Uswitch have successfully used the platform-as-product model to accelerate and simplify the delivery of software at scale.
Reminder, All these talks are interesting even just listening to them, without seeing them.


Sunday, September 05, 2021

Good talks/podcasts (September 2021 I)


These are the best podcast/talks I've seen/listen to recently:

  • Platform Engineering as a (Community) Service (Nicki Watt) [Devex, Platform, Platform as a product] [Duration: 0:39:00] Interesting talk about what is platform engineering, how to create a great developer experience.
  • Principles Of Microservices (Sam Newman) [Architecture, Architecture patterns, DDD, Microservices] [Duration: 0:56:00] This talk describe the microservices principles (modeled around business domain, a culture of automation, hide Implementation details, decentralize, deploy Independently, isolate failure, highly observable, etc).
  • Platform as a Product: How to Delight Your Developers and Deliver Value for Your Customers (Paula Kennedy, Colin Humphreys) [Architecture, Devex, Platform, Platform as a product] [Duration: 0:57:00] In this talk, Paula and Colin will provide detailed, practical steps on how to form a successful platform team, how to apply the Build-Measure-Learn feedback loop to your internal platform, and how to engage your application developers and delight them. By creating and operating a platform that meets their needs, your developers can focus on what's most important for them: delivering their products, delighting their customers, and driving business value.
  • Real Example of a Deployment Pipeline in the Fintech Industry (Dave Farley) [Continuous Delivery, Devex, Platform] [Duration: 0:18:00] Dave Farley shows a real deployment pipeline in-use in a Fintech company called TransFICC based in London. TransFICC adopted Continuous Delivery from the start and practice it at an advanced level. Delivering frequent, changes into production in a complex system, working in a regulated industry.
  • How To Build Quality Software Fast (Dave Farley) [Continuous Delivery, Engineering Culture] [Duration: 0:15:00] (⭐⭐⭐⭐⭐) In this episode, Dave Farley explores how we can move fast with high quality and how one reinforces the other. Speed and quality are both hallmarks of a Continuous Delivery approach and best practices for building great software.
  • Making developer effectiveness a reality (Rebecca Parsons, Tim Cochran, Keyur Govande, Pia Nilsson) [Devex, Platform as a product] [Duration: 0:41:00] There’s often lots of talk about how companies can make their developers more productive. But it may be more useful to think about developer effectiveness: how to ensure they’re building the most useful products for their customers. This is about providing an environment that allows developers to be effective. Our special guests from Spotify and Etsy give us their unique perspectives.
  • Small batches podcast: The Toyota Way (Adam Hawkins) [Lean] [Duration: 0:08:00] (⭐⭐⭐⭐⭐) A summary of 14 principles described in Jeffery Liker's 2021 book "The Toyota Way".
  • WorkLife podcast: Why it Pays to Raise Pay (Adam Grant) [Company Culture, Culture, Inspirational, Management] [Duration: 0:44:00] When employees are paid more, they give more. Going above market pay might sound like a fantasy, but in a growing number of companies it’s becoming a profitable reality. Peek inside workplaces that have reinvented their pay structures to give employees their worth and more—and explore the science of how it can pay off for everyone in the long run.
Reminder, All these talks are interesting even just listening to them, without seeing them.