Saturday, July 03, 2021

Good talks/podcasts (Jul 2021 I)


These are the best podcast/talks I've seen/listen to recently:

  • Traditional vs Modern Internal Platforms: Find the Differences! (Manuel Pais) [Devex, Devops, Platform as a product, team topologies] [Duration: 1:04:00] A very interesting discussion about the key differences between the traditional approach to internal platforms and tooling vs the modern approach.
  • ProductTank New Zealand - The What & Why of Continuous Discovery (Teresa Torres) [Lean Product Management, Product, Product Discovery] [Duration: 1:00:00] the best product teams are shifting from a project mindset to a continuous mindset. In this talk, Teresa explores the key differences between project-based discovery and continuous discovery and give your team a clear benchmark to aspire to.
  • How to Build Products Users Love with Kevin Hale (How to Start a Startup 2014: Lecture 7) (Kevin Hale) [Product, Product Discovery, Product Strategy, startup] [Duration: 0:48:00] Kevin Hale, Founder of Wufoo and Partner at Y Combinator, explains how to build products that create a passionate user base invested in your startup's success.
  • A Guide To Managing Technical Teams (Dave Farley) [Engineering Culture, Management, Technical leadership] [Duration: 0:17:00] In this episode, Dave Farley of Continuous Delivery offers his advice on leading software development teams. How do you make the step from developer to manager? How do you deal with difficult situations and conversations? What are some of the commonest mistakes that people starting out in team leadership often make?
  • Are you ready for production? (Jaana Dogan) [Devops, Engineering Culture, Observability, Operations, Platform] [Duration: 0:35:00] This talk examines practices from design & development, configuration management, observability, release management, capacity planning, security, and compliance.
  • Foundations of Modern Product Organizations (Gerard Chiva) [Company Culture, Lean Product Management, Product, leadership] [Duration: 0:41:00] (⭐⭐⭐⭐⭐) This talk explains the keys to the success of digital product organizations. Technology is an essential part of any business today, not just a cost center.
  • In Search of the Perfect Cloud Native Developer Experience (Daniel Bryant) [Cloud, Devex, Devops, Engineering Culture, Platform, Platform as a product] [Duration: 0:45:00] Interesting talk to understand different approaches to achieve an effective development flow in a cloud-based environment. X explains interesting ideas to create an internal self-service development platform to reduce friction for stream-aligned teams.
  • Testing Strategy for DevOps (Dave Farley) [Devops, Engineering Culture, Testing in production, testing] [Duration: 0:14:00] (⭐⭐⭐⭐⭐) Dave Farley describes his take on what a holistic test-strategy looks like, and gives us examples from the real world to describe it.
Reminder, All these talks are interesting even just listening to them, without seeing them.


Monday, June 14, 2021

Good talks/podcasts (Jun 2021 I)

These are the best podcast/talks I've seen/listen to recently:

  • Test Driven Development vs Behavior Driven Development (Dave Farley) [Technical Practices, XP, testing] (⭐⭐⭐⭐⭐) In this episode on the Continuous Delivery channel, Dave Farley, who was involved in the creation of BDD, explains his slightly more unusual take on the real value of this approach to creating automated tests. Dave believes that TDD and BDD are at the foundations of a sound software engineering approach to software development and explores these ideas with some real-world examples of good and bad automated tests.
  • Make the Right Thing the Easy Thing: Designing Processes Teams Will Actually Follow (Jason Lengstorf) [Devex, Engineering Culture, Platform] In this talk, we’ll learn how IBM is working to get huge, distributed teams to not only agree on a set of best practices but to actually follow them
  • Making developer experience a reality (Tim Cochran, Keyur Govande, Pia Nilsson, Rebecca Parsons) [Devex, Engineering Culture, Platform, Platform as a product] There’s often lots of talk about how companies can make their developers more productive. But it may be more useful to think about developer effectiveness: how to ensure they’re building the most useful products for their customers. This is about providing an environment that allows developers to be effective. Our special guests from Spotify and Etsy give us their unique perspectives.
  • Observability in the SSC: Seeing Into Your Build System (Ben Hartshorne) [Devex, Devops, Observability, Platform, Platform as a product] Ben Hartshorne describes the transformation that Honeycomb went through, when they dropped build times by 40% and gave themselves the ability to track build times and asset sizes over time. Hartshorne covers the techniques one can use to accomplish the same goals in different environments.

Reminder, All these talks are interesting even just listening to them, without seeing them.


Sunday, May 16, 2021

Good talks/podcasts (May 2021 I)


These are the best podcast/talks I've seen/listen to recently:

  • 36 - Building High-Performing Teams with Observability and CI/CD (Charity Majors) [Agile, Continuous Delivery, Engineering Culture, Inspirational, Technical Practices, Technical leadership] A high-performing team is one that gets to spend almost all of their time solving interesting problems that move the business forward. Not doing a lot of toil. Not working on things they have to do in order to get to the things they want to do.
  • What Are Reactive Systems? (Dave Farley) [Architecture, Design, Microservices] Dave describes the use of Asynchronous messaging as the foundation of this approach and how a simple system built this way would be be fast, responsive, resilient
  • Improving Oncall Support with Meaningful Alarms (Brian Scanlan) [Architecture, Architecture patterns, Engineering Culture, Operations] Robby speaks with Brian Scanlan, Principal Systems Engineer at Intercom. They discuss the benefits of having consistent software patterns across an organization, how run books are a must-have (and why they should be tested), and optimizing for developer productivity. Brian also talks about approaches to reducing deployment time and why it is so vital to be responsive to their customers.
  • Ep. #4, Everything is an Experiment with Charity Majors and Liz Fong-Jones of Honeycomb (Charity Majors, Liz Fong-Jones) [Continuous Delivery, Engineering Culture, Operations] In episode 4 of Unintended Consequences, Yoz and Kim speak with Charity Majors and Liz Fong-Jones of Honeycomb about scaling teams and company values. They share how their team makes space for curiosity and learning as a group, and why this has been so important for the Honeycomb platform.

Reminder, All these talks are interesting even just listening to them, without seeing them.


How big is the backpack your organisation is holding

Author: Jorge Jardines

Every experiment, invalidated product feature or unused line of code you haven't deleted is adding weight to your backpack.

Every morning when you wake up and need to walk the next X miles you feel the weight in your back and you feel slower than yesterday. It is like adding an extra bottle of water on your backpack, water you aren't gonna drink but still needs to hold on your back.

Your team or organisation is holding this huge backpacks all the time. Do you ever wonder why adding a new experiment gets more and more slower and costly for your teams, do ever check how much code isn't being used, how many features are not being used I your product, how many "brilliant" ideas were transform into code that nobody used. Do you ever wonder how big is the backpack your organisation is holding.


Monday, May 03, 2021

API contract & Expand. (Small Safe Steps)

In the latest Toughtworks tech radar (Vol 24 April 2021), API Contract & Expand appears as a technique to be adopted. I couldn't agree more.
We often use complex versioning mechanisms even if a more straightforward API expand and contract strategy works perfectly for most cases.

In general, API expand and contract should be enough for internal APIs. For external APIs, expand and contract should work for small changes, and API versioning is a better option for more significant changes.

If you want to practice this and other strategies to reduce risk and slice technical changes, please check out this Small Safe Steps workshop.


Saturday, May 01, 2021

It's not only waste; it's a burden

Talking about software systems, let's stop to talk about waste when we refer to something not necessary or created without a clear understanding of the problem to solve. Let's call it a burden (since you are going to have it holding you back forever)...

If you implement it to experiment, learn and later throw it away, perfect... If you are keeping it, you are fooling yourself. 

This code is a burden, worse than the sunk cost, and it will generate a recurrent cost forever.

There are two types of complexity, the inherent and the accidental/incidental.

The inherent complexity can't be avoided because it is intrinsic to the problem we want to solve. But the accidental/incidental is all the complexity we introduce by not fully understanding the problem, not knowing how to solve it, or by our lack of mastery on the subject, etc.

This accidental/incidental complexity is also a burden. The inherent one is not. :)

Do yourself a favor, reduce the burden. reduce the basal cost. improve your life and the life of your team.

Develop product in an agile way:
  1. Work in small steps
  2. Try to really understand the real problem and the meaningful impact you want to make
  3. Validate each increment and its impact
  4. Always work with quality-in and with attention to technical excellence
  5. Refactor continuously (to reduce the accidental/incidental complexity)
  6. And for God's sake, delete everything that is not necessary. Ruthlessly remove; code, dependencies, features, complexity… Minimize basal cost and make the evolution of the system easier.
And follow the principles:
  • Continuous attention to technical excellence and good design enhances agility.
  • Simplicity --the art of maximizing the amount of work not done-- is essential.


Saturday, April 24, 2021

Good talks/podcasts (April 2021 I)



These are the best podcast/talks I've seen/listen to recently:

  • Inside-Out TDDDD (Amitai Schleier) [Inspirational, Technical Practices, XP, testing] [Duration: 2 days, 12:51:00] (⭐⭐⭐⭐⭐) Talk about joy and humanity in software development. The talk was primarily concerned with the effects of software craft on our emotional states and working relationships. Along the way, Amitai touched on Theory of Constraints, stable and unstable equilibria in work environments, a significant and deliberate omission from Scrum, my take on “legacy code”, applied empathy (never mentioned explicitly, merely woven in), and what agility looks and feels like when you’ve got it.
  • Extreme Programming 20 years later (Kent Beck) [Technical Practices, XP] [Duration: 1 day, 17:49:00] Kent reviewed what he got right, what he got wrong, what he's still working on and how things work at Facebook.
  • Looking Back at Working Effectively with Legacy Code (Michael Feathers) [Design, Technical Practices, XP] [Duration: 1 day, 5:57:00] Wes Reisz and Michael Feathers go back and review the book. The two spend some time reviewing key concepts from the book and then discuss how the techniques can be applied today. The two wrap with a discussion on what might change in a new version of the book.
  • Project to Product: How Value Stream Networks Will Transform IT & Business (Mik Kersten) [Engineering Culture, Product, Technology Strategy] [Duration: 1 day, 7:17:00] How to Survive and Thrive in the Age of Digital Disruption using Product mindset (instead of project mindset) and the flow framework.
  • The Technical Debt Trap (Doc Norton) [Engineering Culture, Technical Practices] [Duration: 2 days, 5:13:00] (⭐⭐⭐⭐⭐) What is technical debt? What is not technical debt? Why should we care? What is the cost of misunderstanding? What do we do about it? Doc discusses the origins of the metaphor, what it means today, and how we properly identify and manage technical debt.
  • When TDD is Difficult - Try This! (Dave Farley) [Technical Practices, XP] [Duration: 12:09:00] In this episode Dave Farley explores the more complex cases in TDD. Testing at the edges of the systems that we build. How do we test code that talks to Databases, Disks or Displays or other edges of our system? How do we apply this software engineering discipline to the creation of our software systems to greater effect?
  • When to Use Microservices (And When Not To!) (Martin Fowler, Sam Newman) [Architecture, Microservices] [Duration: 1 day, 14:44:00] The two touch upon the main reasons for using or not using microservices, and, if you decide to do use microservices, what else you should change along the way to fully benefit from the switch, plus much more.
  • E3: Marty Cagan, Founder & Partner @ Silicon Valley Product Group (Marty Cagan) [Lean Product Management, Product, Product Team] [Duration: 1 day, 16:58:00] (⭐⭐⭐⭐⭐) Very interesting conversation about Empowered Product Teams, Product Culture.

Reminder, All these talks are interesting even just listening to them, without seeing them.


Sunday, March 21, 2021

Good talks/podcasts (March 2021 II)



These are the best podcast/talks I've seen/listen to recently:

  • The resource utilization trap (Henrik Kniberg) [Flow, Lean, WIP] [Duration: 0:05:33] (⭐⭐⭐⭐⭐) Henrik Kniberg explains the resource utilization trap, how it impacts our ability to deliver and what we can do about it.
  • Henrik Kniberg : Multiple WIP vs One Piece Flow Example (Henrik Kniberg) [Flow, Lean, WIP] [Duration: 0:07:05] (⭐⭐⭐⭐⭐) Brilliant explanation of the concept of WIP and how limiting it improves delivery flow.
  • Jason Yip : Stop starting and start finishing (Jason Yip) [Flow, Lean, TOC, WIP] [Duration: 0:05:24] (⭐⭐⭐⭐⭐) Great explanation about Lean concepts (Limit WIP, class of services, Root cause analysis...)
  • Lightning Talk: Certified Really Agile Practitioner (CRAP) (Jon Smart) [Agile, Engineering Culture] [Duration: 0:05:10] Ironic (and sad) vision about digital transformation and the misunderstood agility that is unfortunately common these days.
  • The CIO Who Mistook a Value Stream for a Hat (Mik Kersten) [Devops, Engineering Culture, Management, Technical leadership] [Duration: 0:23:30] Interesting talk on development flow with interesting examples from large companies.
  • Being Outcomes Driven (Gabrielle Benefield) [Lean Product Management, Product Discovery] [Duration: 0:08:26] Interesting point of view on how to improve delivery by adding descovery and avoiding much of the waste.

Reminder, All these talks are interesting even just listening to them, without seeing them.


Sunday, March 07, 2021

Good talks/podcasts (March 2021 I)


These are the best podcast/talks I've seen/listen to recently:

  • Ep. #29, Testing in Production with Glen Mailer of CircleCI (Glen Mailer, Charity Majors) [Operations, Testing in production, testing] An interesting discussion about testing in production and rethinking socio-technical systems from the ground up.
  • GOTO 2020 • Talking With Tech Leads (Patrick Kua) [Engineering Culture, Technical leadership, leadership] (⭐⭐⭐⭐⭐) Practical tips to be a successful Tech Lead. Very interesting for anyone interested in a leadership role for an engineering team.
  • GOTO 2020 • Where We’re Going, We Don’t Need Servers! (Sam Newman) [Architecture, Architecture patterns, Microservices, Serverless] Interesting talk about: why serverless, especially FAAS, technology is so powerful, how serverless technology and microservices fit together, and some tips for adopting serverless incrementally.
  • The Limited Red Society (Joshua Kerievsky) [Agile, Continuous Delivery, Technical Practices, XP] (⭐⭐⭐⭐⭐) Joshua Kerievsky discusses the need to reduce “red” periods of time while developing software. One is in the red when he spends too much time designing, or having compilation errors or the tests do not pass. Kerievsky demonstrates a method (Parallel Change) of reducing the red while refactoring code, and discusses another approach called Narrowed Change, and answers refactoring related questions.
  • Chris Matts on BDD, Real Options, Risk Management and the Impact of Culture for Effective Outcomes (chris Matts) [Agile, Company Culture] Lots of topics in this podcasts: Real options, last responsible moment, a community of needs vs community of practices.
  • Reliable Messaging Without Distributed Transactions (Udi Dahan) [Architecture patterns, Design] (⭐⭐⭐⭐⭐) Particular CEO, Udi Dahan, describes how you can still do reliable messaging without using distributed transactions
  • How To Build Big Software With Small Agile Teams (Dave Farley) [Agile, Continuous Delivery, Engineering Culture] In this episode, Dave Farley explores the trade-off at the heart of scaling-up and describes useful techniques to allow you to scale your big software projects.
  • How Fast is Your Computer? | DESIGNING FOR HIGH PERFORMANCE (Mechanical Sympathy) (Dave Farley) [Inspirational, Performance, Scalability] Mechanical Sympathy is a term that Dave Farley's team at LMAX started to use to describe how they took the constraints of their hardware into consideration when designing their software to maximise its performance in the high-performance computing world of financial trading systems.
  • Acceptance Testing | Webinar (Dave Farley) [Continuous Delivery, Technical Practices, XP, testing] In this talk Dave Farley will describe approaches to Acceptance Testing that allow teams to: work quickly and effectively; build excellent functional coverage for complex enterprise-scale systems; manage and maintain those tests in the face of change, and of evolution in both the codebase and the understanding of the business problem.

Reminder, All these talks are interesting even just listening to them, without seeing them.


Wednesday, March 03, 2021

Talk: (3s) Small Safe Steps: The secret to agility

Ayer tuve el gran honor de compartir cartel con dos grandisimas profesionales, María Berenguer y Angélica Lozano en el Software Development Summit organizado  por el grupo Voxel y presentar la charla  "(3s) Small Safe Steps: The secret to agility".
Muchas gracias a @vgaltes por la invitación. 




Resumen Visual

Resumen Visual by @Aitortxu


He añadido a la presentación una slide inicial para explicar el contexto que aplica en la misma y que uso para los ejemplos. En la presentación hablamos sobre todo de entornos en los que es posible hacer ciclos de feedback cortos.

Documento Original (Con Notas)


Friday, February 12, 2021

Good talks/podcasts (Feb 2021 I)


 These are the best podcast/talks I've seen/listen to recently:

  • Where do great architectures come from? (Mary Poppendieck) [Architecture, Architecture patterns, Inspirational] Mary Poppendieck walks you through a few of the more dramatic architectural changes, looking at what triggered them and how well they worked out. Interesting keynote to understand the fundamental architectural characteristics of software systems (redundancy, isolation, locality, security).
  • The elephant in the architecture (Martin Fowler) [Architecture, Inspirational, Technology Strategy] A short but interesting keynote about why business value should be treated as an architectural attribute.
  • Going deeper with maps (Simon Wardley) [Product Strategy, Technology Strategy] As always, inspiring, and thought-provoking....
  • Modern Agile (Joshua Kerievsky) [Agile, Company Culture, Engineering Culture, Lean] Josh Kerievsky explains what he means by modern agility, shares real-world modern agile stories, shows how modern agile addresses key risks while targeting results over rituals, and reveals how the 2001 agile manifesto can be updated to reflect modern agile's four guiding principles.
  • Matthew Skelton and Manuel Pais on Software Architecture, Team Topologies, and Platforms (Manuel Pais, Matthew Skelton) [Devops, Engineering Culture, Platform, Platform as a product, Teams] Topics discussed included: the role of a modern software architect, how team design impacts software architecture, creating “team APIs” in order to reduce cognitive load, and the benefits of building a “thinnest viable platform”.

Reminder, All these talks are interesting even just listening to them, without seeing them.


Monday, February 01, 2021

Basal Cost of software

In my opinion, the development cost of each feature in a product can be divided as follows:

  • The direct development cost of the feature or initial cost.
  • A weekly or monthly cost only associated with the existence of this feature in our system. Making a comparison with the Basal Metabolic Rate of a human body, we can call this second cost the Basal Metabolic Rate of a feature or Basal Cost of a feature.

The Basal Cost of a feature is composed of two different parts:

  • The direct impact in the team capacity of the added complexity of this feature (new dependencies, more code to understand, more possibilities for bugs to hide, etc).
  • The impact on the cost of the development or evolution of other features due to potential incompatibilities, coupling, etc.

The Initial cost of a feature

This is the cost/expense incurred by the team during the initial development of the functionality. It includes the cost since the team starts working on the feature until the customer has it available and starts using it. Of course, this process should consist of multiple deployments and partial releases to obtain feedback and make the necessary adjustments…

The Basal Cost of a feature

After the initial development cost, we had this continuous cost that reduces the team capacity for new features development (innovation).
This cost, which continues over time and only ends with the removal of the feature or the end of life of the product, is the cost incurred by the team for the existence of that code and that feature in the product.
It is important to keep in mind that this cost does not refer to the cost of making modifications to the feature or fixing bugs, it refers to the cost of simply having the code there...

Why does this cost happen? Why is there an ongoing cost for a feature that is not evolving?

  • The team has to know that code (where it is, what dependencies it has, who interacts with it, how it is designed...).
  • The team's knowledge and techniques are continuously evolving. When the team improves in any of these aspects, the team should update the feature's code.
  • When the team designs a new feature, the code should be design in a way that is compatible with all the previous features and doesn't generate problems or regressions. And of course, this cost is proportional to all the features we had in the system.
  • Something similar happens when the team has a new member. This member should learn about all the features, so the cost is proportional to the number of features.

And worst of all, this overhead is continuous until the "death" of the feature, i.e., the end of life of the product, until no user uses it, or until the end of the world (whichever comes first).

Evolution of the Cost of a feature

As we saw, the Basal Cost is more or less constant during the life of the feature. But each language, dependency, or technology can reach a point that is no longer usable for whatever reason (obsolete dependencies, security problems, normal evolution that deprecate the language version we used, etc).  From this moment, the cost can skyrocket because we are forced to update it even if we don't want to evolve the feature.

So in summary, the real cost of a feature can be represented by the following graph:

The problem

A common mistake is to neglect the Basal Cost and consider that if no changes are made to a feature, the feature's ongoing cost is zero.
I guess this comes from using metaphors from other professions such as construction that are not suitable for software development.

Non-infinite capacity

Although the capacity changes (positively or negatively) over time due to different factors (knowledge of the business, techniques, stress...), the team capacity is finite.
The accumulated Basal Cost of all the features that the teams own reduces the capacity available to develop new features (innovation).

Over time we can see how the team capacity for innovation shrinks very fast.

To a point where capacity is exhausted, and the team finds itself in a situation where it is not possible to innovate and spends all its time "maintaining" the functionalities it already owns.


The above sections highlight several principles that we must consider to improve our product teams' efficiency.

  • We should minimize the Basal Cost as much as possible, achieving the desired impact with as little code as possible.
    • If possible, even achieving the impact without the need to develop code.
    • Iterating the functionality to adapt it as much as possible to the user's needs and making the solution minimal.
    • Make the software/design as simple as possible. Easy to evolve, but without over-engineering (YAGNI).
  • Eliminate or remove any feature from the product that does not have the desired impact, thus eliminating its corresponding Basal Cost.
  • Monitor continuously the team’s code to detect obsolescence of dependencies, languages, and technologies and avoid the Basal cost to skyrocket.

Remember: “Simplicity --the art of maximizing the amount of work not done-- is essential.

Related & References


Saturday, January 30, 2021

Good talks/podcasts (Jan 2021 I)


 These are the best podcast/talks I've seen/listen to recently:

  • Beyond Developer (Dan North) [Agile, Company Culture, Engineering Culture, Inspirational] (⭐⭐⭐⭐⭐) The modern developer needs to be constantly reinventing themselves, learning, and helping others to do the same. In this session, Dan explores some of the skills and characteristics of the modern developer, and suggests some ways you can grow them for yourself.
  • Continuous Integration vs Feature Branch Workflow (Dave Farley) [Agile, Continuous Delivery, Technical Practices, XP] (⭐⭐⭐⭐⭐) Essential 5-minute video. In this video Dave Farley explains the difference and why the two are largely mutually exclusive, and then explains how to live in the CI world by describing three different approaches to keeping the software working as it evolves.
  • Product Roadmaps with Bruce McCarthy (Bruce McCarthy) [Lean Product Management, Product, Product Discovery, Product Strategy] A talk about Product Roadmaps and how changing how businesses see roadmaps will help teams build more impactful software for their customers.
  • Second Generation Lean Product Development Flow (Donald Reinertsen) [Lean, Lean Product Management, Mental models, Product, Product Strategy] (⭐⭐⭐⭐⭐) An introduction to Lean Product Development Flow given by Don Reinertsen at Adventures with Agile in London. This talk is a must to understand modern product development (Flow, uncertainty, Little's law, cost of delay, the value of feedback, queues, batch size, etc).
  • The Principles and Practices behind Team of Teams (Jessica Reif, David Silverman) [Agile, Inspirational, Management, leadership] In Part 1 of the interview, Gene and his guests discuss the structure and dynamics of the transformation described in Team of Teams and how these leadership characteristics are needed today in the new ways of working. This leadership framework reinforces the concepts of common purpose, shared consciousness, empowerment, and trust within organizations to help teams work together more effectively in complex environments, particularly when they have to continuously adapt to change.
  • Codurance Talks 25 - The challenges and opportunities of Platform engineering and DevOps (Phil Taprogge, Markus Seebacher, José Rodríguez Huerta) [Devops, Platform, Platform as a product] The three of them shared thoughts about common mistakes such as not knowing who your clients are, coupling to services, coupling to custom implementations or team silos, and how you can identify them rapidly.
  • Fundamentals of Lean Software Delivery: (01) Introduction to Lean Product Development (Jeff Koch) [Lean, Lean Product Management, Lean Software Development] An interesting course about Lean product and software development
  • How To Build Big Software With Small Agile Teams (Dave Farley) [Agile, Product Team, Technical Practices, XP] Dave Farley explores the trade-off at the heart of scaling-up and describes useful techniques to allow you to scale your big software projects.

Reminder, All these talks are interesting even just listening to them, without seeing them.


Thursday, January 07, 2021

Small batches for the win / Continuous Delivery

In software product development the batch size deeply affects flow efficiency and resource utilization. In this post, we will try to explain the underlying principles that make a good strategy to use small batches when we are developing software.

Let’s start with some basic concepts:

  • Batch: A group of items that move together to the next step in the process. In this case, is a group of changes that are deployed to production at the same time.
  • Item/change: Each one of the individual units of work composes a Batch. In our case any kind of change that affects our system, including new features, code improvement, configuration changes, bug fixes, and experiments.
  • Holding cost: The sum of the cost associated with delaying the deployment of each one of the items. In summary, the cost of delaying the feedback or the value delivered by each item. For example the cost of delay of a new feature or the cost associated with not putting in production a bugfix, etc.
  • Transaction cost: the cost associated with the deployment. The cost of executing a deployment (cost of people, cost of infrastructure, etc).

Batch size / Direct cost

If we only take into account the transaction cost the optimal solution is to make huge batches, as we only pay the transaction cost when we deploy. For example, deploying once a year.

If we only take into account the holding cost, the optimal solution is to use batches with only one item, to avoid delaying any kind of holding cost.

The reality is that if we try to optimize the two variables at the same time we have a U-Curve optimization problem.

U-curve graph

We can see that from the optimal batch size the total cost only grows and that before the optimal batch size we have a penalty due to our transaction cost. So a good strategy is always to minimize the batch size until the transaction cost makes smaller batches inefficient.

Batch size / Risk management

When we develop software each component is coupled with other components of the system. That is, that any part of the code has relations with other parts, such as static or runtime dependencies, common messages, data model dependencies, etc. If we invest in  good internal quality, we will minimize the unneeded coupling, but in the worse scenario, each part of the system can be potentially coupled with another part.

When we create a batch with a size of N changes to be deployed at the same time the potential interactions that can happen are:

  • Each change with the current version of the system. N
  • Each change with the rest of the changes in the batch. That is all the 1-to-1 relations between each change. (N*(N-1))/2

Potential Interactions (I)
Batch size (N)

I = N + (N*(N-1))/2

Whereas this formula describes the number of potential interactions, in general not all of those combinations are possible.

Iterations graph

The basic problem with the batch size for software development is that the universe of potential interactions (I) grows very fast. In fact, it is a quadratic function.

We can quickly conclude that the following problems grow depending on the size of the universe (quadratic):

  • The probability of an error or a negative impact (functional, performance, cost, etc).
  • The cost of detecting/troubleshooting an error.

At the same time we can see that the size of the batch (the number of changes) affects (linearly) the following:

  • The number of teams/people required to coordinate/synchronize the deployment (code freeze, communication, testing, etc).
  • The possibility of having a change difficult to revert (data model changes, high volume migrations, etc).

Let's illustrate how fast the problem grows with an example
If we have an error in 1 of 100 interactions we can see how fast grows the possibility of having an error:

Probability error. Batch size 5

Probability error. Batch size 25

Probability error. Batch size 50

With 25 changes, we already have 88% chance of having at least an error, and with 50 is near sure (99%). And we previously saw that these errors are more difficult to diagnose, and have more possibilities of not being easy to revert.

So clearly, for software development, increasing the size of the deployments (batch size) greatly increases (much more than linearly) the risk associated (risk of having an outage, loose availability, and frustrating our customers).

Batch size / Indirect cost and consequences

In the previous sections, we have seen the direct impact of the batch size on a product development process;
  • An increase in the total cost for a batch size greater than the optimal size.
  • A near quadratic growth of the risk associated with a deployment. 
  • An increasing number of production outages.

In addition to those direct costs, these are other indirect costs and consequences when batch size is large:
  • Lots of multitasking and the corresponding productivity and focus lost. The normal flow of work is frequently interrupted with problems that come from previous deployments.
  • A tendency to disconnect from the operation and impact of our changes in production. i.e.: when deploying something that you did several weeks ago.
  • Low psychological safety, because of the amount of risk and the probability of outages associated with the way of working.
  • Worse product decisions because there are fewer options to get fast feedback or to design new experiments to get more feedback.
  • Lack of ownership is a derived consequence of the previous points.


As we have shown, the size of the batch has many effects:
  • Risk and outage probability are proportional (and worse than linear) to the number of changes included in a deployment.
  • Our batch size should be as small as our transaction cost (deployment cost) allow us.
  • Large batches generate important indirect costs and consequences (lack of ownership, multitasking, low psychological safety, etc).

Consequently, we should invest as much as possible to try to work in small batches. If we detect that we already reached the optimal batch size for our current transaction cost but we are still having the same problems that we had before, perhaps the next step is to invest heavily in lowering the transaction cost (deployment automation, independent deployments per team, deployment time, etc).

In short:

Small batches -> faster feedback
Small batches -> Customer Value sooner
Small batches -> Reduce direct cost
Small batches -> Reduce deployment risk
Small batches -> Reduce errors
Small batches -> Reduce mean time to recover
Small batches -> Improve psychological safety

Small batches for the win!

The good news is that there is already an engineering capability focused on working in small batches. Continuous Delivery!

"The goal of continuous delivery is to make it safe and economic to work in small batches. This in turn leads to shorter lead times, higher quality, and lower costs." 

The importance of working in small batches has been validated statistically in the studies conducted by DevOps Research and Assessment (DORA) since 2014. You can see the details of these studies in the book Accelerate (Nicole Forsgren, Jez Humble, Gene Kim).


Saturday, January 02, 2021

Books I've read lately 2020

These are the books that I read lately: