Wednesday, October 27, 2021

"It depends" / The development Mix (Product, Engineering, Hygiene)

We already know that is everything about context. I read a lot of blog posts talking about how much time a team should use for decoupling components, introducing a new cache system, or improving the scalability of their systems. When reading this type of content, I always think that they are completely right and completely wrong. Everything in our profession depends a lot on the context (the moment of the company, the business strategy, the market traction, etc.).

I use a mental model that helps me classify the work we do, which allows me to communicate and make decisions. I call this mental model "The Mix".

In "The Mix", I classify the work we do as product engineers in:
  • Normal product development.
  • Implementing the Engineering Roadmap.
  • Basic hygiene work.

Normal product development

Normal product development should be the most common type of work for a Stream Aligned team. It should help to fulfill the mission of the team. It can be composed of new feature development, discovery experiments, feature evolution, etc. I prefer a very lean approach for this work, following agile development methods such as XP or Lean Software Development. It is essential to generate the expected outcomes with the minimal amount of code possible and a good internal quality that minimizes the maintenance cost. Following YAGNI, KISS, Simple design is the perfect approach for this kind of work. We don't know the future. The most efficient way to work is to have the most simple solution that covers our customer's needs without making any "speculative" design that generates tons of accidental complexity in 99% of the cases.

  • Focus on outcomes for the customer within the business constraints.
  • Evolutionary design uses Simple design and avoids creating anything for future "expected/invented" needs.
  • Use a Lean approach (working in small safe steps).
  • Avoid solving problems that we don't have.
  • High-speed feedback loop.
  • Aligned with the Product Roadmap.

Implementing the Engineering Roadmap

In parallel to the product work, it is very common to identify engineering needs derived from the company's engineering strategy. This strategy should prepare and maintain the current and future engineering capability. Examples of this type of work are:
  • Designing the system for fast expected growth (the number of customers, engineering team size, etc.).
  • A technology stack change.
  • A change in the delivery strategy (from On-Prem to SaaS, from Web to mobile, etc.).
  • Prepare the architecture to enable work in autonomous teams.
  • This kind of work usually affects several Stream Aligned teams simultaneously and requires coordination at the engineering organization level.
  • These initiatives require a lot of investment and should be coordinated with the product roadmap and aligned with the company's general strategy.

  • Focus on outcomes for the internal architecture and engineering processes.
  • Require more upfront effort to design the solution.
  • It can be implemented with an agile approach but based on the initial design.
  • Low-speed feedback loop.
  • By definition, try to solve problems that we don't have (yet).
  • It is aligned with the Engineering Roadmap (coordinated with the Product Roadmap).

Basic hygiene work

To develop any nontrivial product, we need to have some practices and development infrastructure that I consider basic hygiene. I'm talking about having a reasonable test strategy, zero-downtime releases, good internal code quality, basic security practices, etc.
In the middle of 2021, not considering these points above seems simply a lack of professionalism. 
So the Basic hygiene work includes any effort we make to implement or improve these minimal practices.

Of course, I am a big fan of product discovery with prototypes, and these, for example, do not have to have the same test strategy. But remember, a prototype that ends up in production, staying in front of our customers for months, is not a prototype. It is a trap.

Using The Mix

Thinking about these three types of work and separating them helps me be more explicit about the context and the appropriate trade-offs in each situation. For example, suppose we are in a Normal Product Development initiative. In that case, we cannot expect big architecture change decisions to emerge, and it is better to focus on small safe steps that add value. At the same time, we take notes to consider some initiatives to introduce in the engineering roadmap.

A mature product organization will introduce performance, scalability, and availability initiatives into the product roadmap. In a less mature organization, those needs are likely to be missing from the product roadmap, and it is up to engineering to fight to get them into the engineering roadmap.

We can summarize the different dimensions in this table:
Product development Engineering Roadmap Hygiene
Team (+Engineering)
Small Safe Steps
Upfront Planning + Small Safe Steps
Small Safe Steps
Evolutionary Architecture, Load Testing, Testing in Production, migration planning...
Clean Code, CI, CD, Zero downtime, Observability...
Type of needs
Current needs
Future needs
Value Delivery
Very Fast
Very Fast
Coordination Needs
The team should be autonomous
Coordination with other teams
Coordination with other teams

If we analyze the current trends in technology using this mental model, some questions arise:
  • How do technologies like PaaS or Serverless influence this Mix?
  • How does working in Cloud vs. working on-prem affect the engineering roadmap?
  • Does it make sense to consider ourselves good professionals if we don't have strong knowledge about hygiene factors?
  • How does the mix change in the different phases of the company (startup pre-product-market fit, scale-up, big tech)? And with the life cycle of the product?
The interesting thing about mental models is that they help us think. I hope this model is as valuable for you as it is to me.

Related / Other mental models

Sunday, October 17, 2021

Good talks/podcasts (October 2021 II)

These are the best podcast/talks I've seen/listen to recently:

  • How conscious investors can turn up the heat and make companies change (Vinay Shandal) [Inspirational] [Duration: 0:13:00] In a talk that's equal parts funny and urgent, consultant Vinay Shandal shares stories of the world's top activist investors, showing how individuals and institutions can take a page from their playbook and put pressure on companies to drive positive change. "It's your right to have your money managed in line with your values," Shandal says. "Use your voice, and trust that it matters."
  • Software at Scale 13 - Emma Tang: ex Data Infrastructure Lead, Stripe (Emma Tang) [Big Data, Data Engineering, Operations, Platform, Technical Practices] [Duration: 0:41:00] (⭐⭐⭐⭐⭐) Effective Management of Big Data Platforms. Very interesting discussion about the technological and organizational challenges of maintaining big data platforms.
  • Improving Software Flow (Randy Shoup) [Agile, Continuous Delivery, Engineering Culture, Inspirational, Technical leadership] [Duration: 0:46:00] (⭐⭐⭐⭐⭐) Great presentation, in which Randy, starts from the 5 ideals of the Unicorn project (Locality and Simplicity, Focus, Flow, and Joy, Improvement of Daily Work, Psychological Safety, Customer Focus) to describe what we can do as technical leaders and as engineers to improve our ability to build and deliver software.
  • Developer Productivity with Utsav Shah (Utsav Shah) [Devex, Devops, Platform, Platform as a product] [Duration: 0:41:00] In this episode of Software Engineering Daily podcast, Utsav Shah talk about developer productivity in the context of the monolith, CI/CD, and best practices for growing teams.
  • Simplifying The Inventory Management Systems at the World’s Largest Retailer Using Functional Programming Principles (Scott Havens, Gene Kim) [Architecture, Architecture patterns, Functional, Technical leadership, Technology Strategy] [Duration: 2:02:00] (⭐⭐⭐⭐⭐) Havens shares his views on what makes great architecture great. He details what happened when an API call required 23 other synchronous procedures calls to return a correct answer. He discusses the challenges of managing inventory at Walmart, how one implements event sourcing patterns on that scale, and the functional programming principles that it depends upon. Lastly, he talks about how much category theory you need to know to do functional programming and considerations when creating code in complex systems. It is recommended to first watch the talk or listen to the podcast
Reminder, All these talks are interesting even just listening to them.


Sunday, October 03, 2021

Good talks/podcasts (October 2021 I)


These are the best podcast/talks I've seen/listen to recently:

  • You're Testing WHAT? (Gojko Adzic) [Technical Practices, Testing in production, testing] [Duration: 0:38:00] Gojko presents five universal rules for test automation, that will help you bring continuous integration and testing to the darkest corners of your system. Learn how to wrestle large test suites into something easy to understand, maintain and evolve, at the same time increasing the value from your automated tests.
  • Using Observability to improve the Developer Experience (Borja Burgos) [Devex, Devops, Platform, Platform as a product] [Duration: 0:15:00] Observability is often associated with production and live environments, but it shouldn't be! In this talk we'll explore innovative ways in which modern observability tools and best practices can be leveraged during development to: improve developer productivity, identify regressions earlier in the SDLC, and increase the performance and reliability of our CI/CD workflows.
  • Continuous Delivery (Jez Humble) [Agile, Continuous Delivery, Engineering Culture, Lean Software Development] [Duration: 0:47:00] (⭐⭐⭐⭐⭐) Great 2012 presentation on Continuous Delivery. Jez discusses the value of CD to the business. He presents the principles and related practices, including value stream mapping, deployment pipelines, acceptance test-driven development, zero-downtime releases, etc. This talk is a while old, but still as relevant as the first day.
  • Test Driven Development with Geepaw Hill (Clare Sudbery, GeePaw Hill) [Agile, Technical Practices, tdd, testing] [Duration: 0:50:00] Clare talks to Geepaw Hill about why he loves TDD so much and how he spreads that love to software teams all over the world.
  • Software Delivery Glossary (Adam Hawkins) [Continuous Delivery, Lean, Lean Software Development] [Duration: 0:09:00] This podcast describes a few software delivery concepts (Lead time, Deployment frequency, MTTR, Change Failure Rate, Jidoka, Kaizen).
  • Churn FM. EP131 How the best leaders empower their product teams and set them up for success (Marty Cagan) [Inspirational, Lean Product Management, Product, Product Team] [Duration: 0:36:00] An interesting conversation about the tremendous gap between how the best companies operate and how the rest work, what they do differently, and why.  They also discussed how the best leaders empower their teams, how real product discovery and product work happens and then talked about how alignment is a consequence of a good product strategy.
Reminder, All these talks are interesting even just listening to them.