Monday, September 19, 2022

Good talks/podcasts (Sept 2022 I)

These are the best podcast/talks I've seen/listen to recently:
  • 20 Questions To Ask Your Next Employer (Dave Farley) [Engineering Career, Engineering Culture] [Duration: 0:17:00] In this episode, Dave Farley offers his advice on how to pick great employers. He describes the things you should explore, when looking for a new job, to decide if a potential employer is only talking a good game, rather than playing a good game.
  • Agile at Tesla with Joe Justice Agile Munich (Joe Justice) [Agile, Lean] [Duration: 0:58:00] interesting description of the agile principles and practices applied at tesla.
  • FYAC: The 3 Levels of User Story Slicing (Neil Killick) [Agile, Lean Product Management, Product Discovery, slicing] [Duration: 0:37:00] (⭐⭐⭐⭐⭐) Very good ideas on how to approach product slicing, option generation and agile product development.
  • The Sociotechnical Path to High-Performing Teams (Charity Majors) [Continuous Delivery, Devops, Engineering Culture, Teams, team topologies] [Duration: 0:41:00] (⭐⭐⭐⭐⭐) This talk describe the social and technical strategies that great teams all of the world are using to be happier, more productive and make their users happy too.
  • Tips For Building Successful Platform Teams (Dave Farley) [Platform, Platform as a product, Platform teams] [Duration: 0:22:00] (⭐⭐⭐⭐⭐) In this episode, Dave describes the pitfalls common to platform teams, the principles that are important for platform engineering, the strategic goals that platform teams can use to guide their decisions, and some tips for platform teams that will help them protect their users from changes in services.
  • Resilience In Complex Adaptive Systems (Richard Cook) [Inspirational, Resilience] [Duration: 0:20:00] This talk introduces resilience and a model of system dynamics useful in analyzing failed and successful event management and offer an explanation for why our systems run at the edge of failure.
  • Operational simplicity is a gift to you (Gary Bernhardt) [Architecture, Design, Inspirational, Operations] [Duration: 0:57:00] Very entertaining episode with great examples of how to make very operable systems by systematically searching for simplicity.
Reminder, All these talks are interesting even just listening to them.

Related:

Wednesday, August 24, 2022

Lean Software development: The art of postponing decisions

One of the seven principles of Lean Software Development is Defer Commitment. This principle tries to maintain options open (options have value) and avoid waste by only solving the current problem (avoiding speculative design). Following this principle allows us to generate easy to change, minimal systems with low accidental complexity and minimal features (as we have a low lock-in).

Keep in mind that software is only a means to an end, and that we are trying to generate impact, not software.

Over the years, I have found that postponing decisions and working in small, safe steps have been the key to creating sustainable solutions.

My experience tells me this is an excellent way to work, at least for product development. 

This article is based on my experience over the past 12-13 years. All of the companies that I talk about are product companies, some of them startups (TheMotion, Nextail, ClarityAI) and others more classic companies (Alea Soluciones).

What we mean

Postponing all technical decisions until the last responsible moment. This practice implies scheduling irreversible decisions for the last responsible moment (LRM) and trying to make as many reversible decisions as possible.

What we do not postpone

For the last 12-13 years, I have been trying to work in an agile way, which implies that there are certain decisions in the way I work that I do not postpone, Agile, continuous delivery, eXtreme Programming, etc.

Some decisions are difficult to postpone. I think the language (ecosystem) and the approach to development (objects vs. functional) should not be postponed and must be clear from the beginning. 

Some of these decisions may evolve over time, but we must have a solid starting point.

Working with technical excellence is another decision we do not postpone. It is implicit in my decision to work with Agile, but I think it is important to make it explicit.

Why we want to postpone / Benefits achieved

As we will see in the article, postponing decisions is not easy and requires practice. But as I mentioned in the introduction, it allows us to make sustainable and easy to evolve products.

  • A later decision will give us a greater understanding of business and technology.
  • Simpler and smaller solutions
  • Less work :-) (less basal cost  and less time implementing things we don’t need)
  • More likely to not do anything that doesn't bring real value now
  • More likely not to over-engineer
  • Less effort in redoing work if necessary
  • Our goal is to have a lot of real options... react well and fast to anything... and without fear...
  • The more we postpone, the less we fill the backpack, and the less unnecessary stuff we add (less waste).
  • We all know that it’s easier to move with fewer things in the backpack.
  • Good architecture allows us to postpone important decisions, but this does not mean we are obliged to do so. However, being able to postpone them gives us a lot of flexibility.

This approach is aligned with the lean principles of eliminating waste and the agile principles of postponing until the last responsible moment (LRM) and is one of the approaches that can help the creation of good architecture. In turn, good architecture makes it easier to postpone other decisions.

How we do it

It is very difficult to explain everything that needs to be taken into account in order to postpone decisions/commitments without risk and achieve the benefits mentioned above. Anyhow, here is what has worked for us:

  • We postpone decisions in a conscious and meditated manner.
  • We consider a decision to be a good one when:
    • It compromises us as little as possible
    • Allows us to postpone other decisions
    • It is easy to change/revert
    • Attacks a current (not future) problem
    • Good enough (no over-engineering, neither ours nor other people's)
    • No Megaconstructions
  • We reuse libraries but not Frameworks (we don't let them use us)
  • We consider that everything can change (code/design/process)
  • We work in a very iterative manner (See https://productdeveloper.net/iterative-incremental-development)
  • When we start a new feature, we focus on identifying what we need to change in our architecture and our understanding of the ubiquitous domain/language.
  • We assume that what we decided was the right thing to do at that time and in that context, and if it needs to be changed, that's fine.
  • We put a lot of effort into good vertical slicing for the features (with the Hamburger method for example) and even technical slicing to be capable of working in small safe steps.
  • We strive for technical excellence (Clean Code, Simple Design, TDD, Evolutionary Design, etc).
  • We use DDD techniques and Hexagonal architecture.

The points I mention are not necessary, but they can serve as inspiration for any team that wants to gain a deeper understanding of agility.

What we need to make it easy to postpone

  • Safety/Confidence: about our system and code and at the team level (psychological safety)
  • Fast and continuous feedback: Customer feedback, System feedback (monitoring, observability), Development feedback (automatic tests, continuous code review with pairing, etc)
  • Maintainable/Evolvable product/code base: We use XP, Devops practices and lean mindset
    • Simple design
    • TDD
    • Continuous refactor
  • Accept/Assume that everything can be changed/removed/evolved (Features, Code, Design, etc)

We split medium/large changes into small changes, even duplicating code and duplicating data if necessary... all of this in order to be able to postpone decisions, not impede deployment and make incremental small, and safe changes.

Some examples

In this section, I will describe some examples of systems in which we have had a lean approach trying to postpone all possible decisions until the last responsible moment. This has allowed us to have a very low time to market and to adapt the solutions to the feedback we were getting all the time.

Software for small telecommunications networks (up to 5000 subscribers). New fiber optic provision software with a very aggressive time to market:

  • First rollout version with python serialization to file as persistence 
  • After one month, we added a python text search to the persistence layer
  • After two months, we migrated the persistence layer to Redis to improve performance
  • 5 or 6 months later, we migrated the persistence to MySQL with Full-Text Search once we saw the growth trends
This approach allowed us to roll out a new product line in less than two months and evolve the product during the last ten years.

At TheMotion, we decided to create a command line application to deploy, check status, create new services, and other common operations. The challenge was to make this application available as soon as possible and evolve it while changing TheMotion's system environment/infrastructure. Another important requirement was the security of access to the application.

  • The first version was implemented as a shell script running in a server and executed by the developers via ssh (authorization, encryption at transit) using an alias.
  • In less than a month, this shell script was migrated to python.
  • And after several months, we removed the ssh call with a python client.

The developers' command executions remained largely unchanged during these months. In the background, the deployment, the infrastructure, and the running platform were rapidly evolving.


Also at TheMotion, we were creating a SaaS b2b product from scratch, and in the beginning, we didn’t had customers and the access was using an API. So we took a lean approach to user management, access, and role management.

  • The first version only was usable by a few concrete companies, and we didn’t need to bill anything, so we only restricted access via IP address.
  • The next step was to generate an ID and API Key for each customer and differentiate them by the API Key.
  • After three months, we manually created users (name, email, etc), but without password, only the API Key.
  • The next step was to allow some users to impersonate others (for example, an agency using the SaaS for several final customers).
  • I don’t remember when, but later we created an admin website, and at this moment, we included passwords for the users.

You can see more examples and strategies in this blog post System: Control its evolution / or be its slave  

I am also working on a workshop to practice how to defer commitment that will have more concrete examples.


Problems/Sensations generated by postponing decisions…

Certainly, postponing decisions is difficult since it generates feelings that we do not usually like:

  • Uncertainty
  • Anxiety
  • Conflict (as engineers)

Studies show that uncertainty generates fear and anxiety, so it's not surprising that we try to avoid it. Additionally, as software engineers, we have been trained to solve problems, which is why postponing decisions generates conflict.

Once you get used to working in small steps, with high quality and continuously iterating solutions, much of that anxiety and uncertainty disappears, and development becomes a continuous flow of change.

Conclusions

Postponing decisions and commitments is not easy, it’s difficult, and sometimes it can be confused with laziness or unprofessionalism. It couldn't be further from the truth. Professionalism is about solving current problems (YAGNI) with the most straightforward/simple possible solutions (Simple Design, pragmatism) rather than speculating about the future.

As we have seen, although difficult, postponing decisions/commitment until the last responsible moment has great benefits.

So, Postpone... to infinity and beyond…



Until the last responsible moment!!!!

References and related resources:

Real options 

Related blogposts

Examples

Practices & Techniques


Thanks

The post has been improved based on feedback from:

Thank you very much to all of you





Monday, August 15, 2022

Good talks/podcasts (Aug 2022 I)

 


These are the best podcast/talks I've seen/listen to recently:
  • Hey CTO - S01E01 - with Hass Chapman (Hass Chapman) [Engineering Culture, Engineering Scalability, Technical leadership, leadership] [Duration: 0:42:00] Interesting interview with Hass Chapman, Head of Product & Engineering at Proto Energy
  • Shared Mental Models Part 1 Shared Mental Models Part 2 (Jessica Kerr) [Engineering Culture, Inspirational, Mental models] [Duration: 0:33:00] (⭐⭐⭐⭐⭐) This was an excellent talk, full of insights that prompted reflection. In this talk Jessica looks at how the shared mental models created while mob programming work throughout the team even when they are not actively mobbing. She also explores the other practices she’s found complementary in creating a high functioning team and how looking at your contribution from a generative (helping other create) vs a productive (what I created, myself) frame can lead to a happier, better and more productive team.
  • Scaling Personalization: DevOps at Stitch Fix (Randy Shoup) [Continuous Delivery, Devops, Engineering Culture, Technical leadership] [Duration: 0:32:00] This talk discusses the technology approach and modern development practices we have put in place to make this model succeed.
  • Investing in Innovation: How Situational Awareness Can Put Your Business on the Map (Mike Lamb) [Wardley maps] [Duration: 0:06:00] Pill on the importance of situational awareness and wardley maps.
  • Scaling from 5 to 250 engineers (Eiso Kant) [Engineering Culture, Engineering Scalability, Technical leadership] [Duration: 1:03:00] Talk on scaling your team from 5 to 250 engineers — lessons learned, mental models, metrics, and everything that is useful to know.
  • The Dark Side of Events (Vladik Khononov) [Architecture patterns, Evolutionary Design, Software Design] [Duration: 0:44:00] In this talk, you will learn heuristics for identifying problems based on events and eliminating event-driven complexities.
Reminder, All these talks are interesting even just listening to them.

Related:

Monday, July 25, 2022

Good talks/podcasts (Jul 2022 I)

These are the best podcast/talks I've seen/listen to recently:
  • Why Your Software Team CAN’T Scale (Dave Farley) [Engineering Culture, Teams, team topologies] [Duration: 0:19:00] In this episode, Dave Farley, author of the best selling books Continuous Delivery, CD Pipelines and Modern Software Engineering, looks into how to organise software at scale, and how the Team Topologies model works and helps us to design organisations that can produce better software faster.
  • The Flywheel Effect Creates Space for Innovation (Europe 2022) (David Anderson, Michael O'Reailly) [Engineering Culture, Platform, Technology Strategy] [Duration: 0:26:00] The flywheel effect occurs when we balance business strategy (via Wardley mapping) with technology strategy (via Modern cloud). As a preview of the upcoming IT Revolution book, Dave creates a Wardley Map to show how to use the value flywheel to build confidence, improve your cloud stance and create space for innovation.
  • An Inside Look At eBay’s Software Practices (Dave Farley) [Continuous Delivery, Engineering Culture] [Duration: 0:21:00] In this episode, Dave Farley, explores the changes that eBay have made, their impact, and what they have learned from the experience so far.
  • Backstage: Restoring Order To Your Chaos (Dave Zolotusky) [Devex, Engineering productivity, Platform, Platform as a product] [Duration: 0:35:00] good introduction to Backstage, the environment, the main plugins, etc.
  • Code for Ukraine #2: Tidy First? Daily Empirical Software Design & Why It Works (Kent Beck) [Software Design, XP] [Duration: 0:58:00] (⭐⭐⭐⭐⭐) Interesting presentation about software design and trade-offs and techniques to refactor the code before applying a change. The talk is full of interesting insights. It is also very interesting the Q&A part that is not only focused on the content of the talk.
  • How to scale platforms in high-growth companies? (Jelmer Borst) [Platform, Platform as a product] [Duration: 0:13:00] This is a story about why Picnic favours boring tech with consolidated tech stacks to actually move very, very fast.
  • Continuous Delivery - Sounds Great But It Won't Work Here (Jez Humble) [Continuous Delivery, Engineering Culture] [Duration: 0:49:00] (⭐⭐⭐⭐⭐) Since the Continuous Delivery book came out in 2010, it’s gone from being a controversial idea to a commonplace… until you consider that many people who say they are doing it aren’t really, and there are still plenty of places that consider it crazy talk. In this session Jez will present some of the highlights and lowlights of the past six years listening to people explain why continuous delivery won’t work, and what he learned in the process.
Reminder, All these talks are interesting even just listening to them.

Related:

Sunday, June 26, 2022

Good talks/podcasts (Jun 2022 II)


 

These are the best podcast/talks I've seen/listen to recently:
  • LOTE #7: Kelsey Hightower on Developer Experience, PaaS, and Testing in Production (Kelsey Hightower) [Cloud, Developer Productivity, Devex, Platform as a product] [Duration: 0:41:00] In the seventh episode of the Ambassador Livin’ on the Edge podcast, Kelsey Hightower, technologist at Google, discusses his thought on cloud developer experience, modern Platform-as-a-Service (PaaS), and explores the reality that every organisation is testing in production
  • Should Computers Run the World? (Hannah Fry) [AI, Data Science, General, Inspirational] [Duration: 0:36:00] Hannah Fry takes us on a tour of the good, the bad and the downright ugly of the algorithms that surround us. She lifts the lid on their inner workings, to demonstrate their power, expose their limitations, and examine whether they really are an improvement on the humans they are replacing.
  • Small Batches - PDCA (Plan-Do-Check-Act) (Adam Hawkins) [Lean] [Duration: 0:06:00] Adam present's Dr. Deming's PDCA cycle and how it applies to the daily work of delivering software.
  • Steve Jobs on programmer productivity (Steve Jobs) [Lean Product Management, Lean Software Development] [Duration: 0:01:00] Excerpt (1m) of Steve Jobs presenting the software as a liability to minimize. You know, make impact, not software. "The way you get programmer productivity is not by increasing the lines of code per programmer per day. That doesn’t work. The way you get programmer productivity is by eliminating lines of code you have to write."
  • From initial request to software in production in 3 weeks (Christin Gorman) [Inspirational, Lean Software Development] [Duration: 0:22:00] (⭐⭐⭐⭐⭐) Simplicity--the art of maximizing the amount of work not done--is essential. Great talk on how to focus on the essentials and make simple solutions.
  • The Future of Mars Exploration (Anita Sengupta) [Inspirational] [Duration: 0:54:00] In this talk, you will learn about the motivation for Mars exploration and how computational modeling, high-tech solutions, and out-of-the-box thinking can be used to overcome engineering challenges.
Reminder, All these talks are interesting even just listening to them.

Related:

Sunday, June 12, 2022

Good talks/podcasts (Jun 2022 I)


 


These are the best podcast/talks I've seen/listen to recently:
  • Scaling Organizations and Design with James Shore (James Shore) [Agile, Evolutionary Design, Software Design, XP] [Duration: 0:48:00] (⭐⭐⭐⭐⭐) Great interview with great ideas (mocking-free testing, scalability of horizontal organization, FAST framework, code evolution with evolutionary design...). Very worthwhile.
  • Feature Branches and Toggles in a Post-GitHub World (Sam Newman) [Continuous Delivery, Technical Practices, Trunk Based Development] [Duration: 0:50:00] In this presentation Sam explains how working in feature branches creates friction when our goal is to achieve continuous delivery. Sam explains how to use Feature Toggles to improve delivery frequency.
  • From Kubernetes to PaaS to ... Err, What's Next? (Daniel Bryant) [Developer Productivity, Devex, Platform, Platform as a product] [Duration: 0:31:00] (⭐⭐⭐⭐⭐) In this talk Daniel reviews his experience in building platforms, both as an end-user and now as part of an organization that helps our clients do the same. She discusses topics such as DevEx, UX, workflows, available tools, etc.
  • Make Impacts Not Software (Gojko Adzic) [Lean Product Management, Lean Software Development, Product, Product Strategy] [Duration: 0:51:00] (⭐⭐⭐⭐⭐) An essential talk to understand how to get the most impact with the least amount of software (and thereby reduce basal cost and time to market). Highly recommended.
  • The Difference between Software Engineering and Manufacturing (Donald Reinertsen) [Lean Product Management, Product] [Duration: 0:09:00] In this interview, Don explains the basic difference between applying lean to a manufacturing process and to a digital/software product creation process.
  • the deep synergy between testability and good design (Michael Feathers) [Software Design, Technical Practices, XP, testing] [Duration: 0:50:00] Interesting talk about how to do good software design and the relationship between this design and the ease of testing.
  • Less - The Path to Better Design (Sandi Metz) [OOP, Software Design] [Duration: 0:50:00] (⭐⭐⭐⭐⭐) This talk strips away the well-known design principles and exposes the hidden, underlying goals of design. It reveals programming techniques that allow you to write less code while creating beautiful, flexible applications.
Reminder, All these talks are interesting even just listening to them.

Related: