Monday, December 26, 2022

My premises about software development

It is very important that, as a software development team, we share certain premises about software and software development. This allows us to work more efficiently and effectively, as we are all aligned in terms of our vision and goals.

That is why I have tried to do an exercise in introspection to extract which are the basic premises that I almost unconsciously apply when I think about software and the process of developing it.



These are my premises for software system development:

  • Software development is a team activity: We cannot work in isolation but must collaborate with other developers, designers, and other professionals to create a high-quality product.
  • There is no real trade-off between quality and speed: The way to go fast is to have good quality. Low quality in the long run only ensures that we will go slower, as we will have to fix errors and problems.
  • Software is a means to achieve an impact: We are not working just for the software itself but to achieve an impact on the "business". We must keep this in mind when making decisions and prioritizing tasks.
  • Only one-third of the business ideas contribute effectively to the outcome we aim (Online Experimentation at Microsoft): We have to accept this and optimize for learning and distinguish between product development, prototyping, and experiments.
  • When developing software-based systems, there is always a high degree of uncertainty, so we must optimize for quick feedback and work iteratively and incrementally, always taking small, safe steps (https://www.eferro.net/p/small-safe-steps-3s-workshop.html).
  • Software has a basal cost (https://www.eferro.net/2021/02/basal-cost-of-software.html) that must be taken into account. When combined with the understanding that software is only a means to achieve an impact, it becomes clear that software can be considered a liability.
  • In software development, maintaining options and flexibility is valuable. To do this, it is essential to defer commitment and try to use reversible decisions whenever possible.

These are my personal premises with respect to software development, and they are not necessarily universal principles. However, they are essential for working efficiently and effectively in software development. These premises may not apply in the same way in all contexts, but I perceive them as accurate in the contexts in which I have worked.

Some of these ideas have been confirmed in the industry through data, but for others, I only have more support than my own experience. In any case, I would love to share and contrast ideas with more colleagues.

Do you share these premises? Do you have others that are similar? Do any of these premises seem contradictory or contradict your own premises?



Related:

Monday, December 12, 2022

Mini TejonesConf 2022 / Honey Badger team gathering

It's been almost three years since we last gathered as a small group of friends for a (reduced) TejonesConf. This time, with the challenges posed by the COVID-19 pandemic, we decided to prioritize reconnecting with the original Honey Badger team and some special guests. We wanted to enjoy each other's company and be able to have intimate, in-person conversations.


As in previous TejonesConfs, we decided to use the open space format for this edition. Due to the small size of our group, we opted for a single track to allow for more focused discussions and deeper conversations. As always, the open space format proved to be a success, providing us with a minimal structure that allowed us to share experiences and spend a lot of time enjoying each other's company.


During the conference, we still managed to share some interesting experiences and insights. We discussed topics like observability and the transition from unstructured logs to structured events, lean software development and the importance of deferring commitment, the challenges of Kubernetes packaging, and the differences between clean architecture and vertical slice architecture. We also touched on product development through experimentation.


As always, the format allowed for some great conversations during lunch and breaks. We discussed issues like power dynamics in pair programming, the toxicity of certain corporate cultures, and the challenges facing our profession.


It was an absolute pleasure to reconnect with old friends and catch up on each other's lives. While we couldn't get the entire group together due to logistical limitations, it was wonderful to spend time with those who could make it. Some of us even continued the conversation over dinner.


Until next time, friends!


BTW Thanks to Nextail for allowing us to use their offices and to Fran for managing it.


Related:



Tuesday, December 06, 2022

Good talks/podcasts (Dec 2022 I)

 


These are the best podcasts/talks I've seen/listened to recently:
  • Keynote: Why web tech is like this (Steve Sanderson) [Inspirational, Technology] [Duration: 1:00:00] Interesting talk about the evolution of web technology. The talk includes interesting demos.
  • #12 e-Meetup | Sam Newman - Monolith Decomposition patterns (Sam Newman) [Architecture, Architecture patterns, Evolutionary Architecture] [Duration: 1:30:00] Great presentation on how to approach the division of a monolith and the patterns to evolve an architecture towards microservices. In addition to the presentation, the subsequent questions are also very interesting.
  • What makes a good developer (Christin Gorman) [Engineering Career, Inspirational] [Duration: 0:11:00] (⭐⭐⭐⭐⭐) Inspiring lightning talk about the importance of humanities, empathy for users and information management as the basis of our profession (beyond pure technology and knowledge about logic or mathematics).
  • Stop Writing Dead Programs (Jack Rusher) [Inspirational] [Duration: 0:43:00] (⭐⭐⭐⭐⭐) This talk argues that it would be better to focus on building new live programming environments that can help us solve the problems of the future.
  • SEDaily: Serverless Clickhouse for Developers with Jorge Sancha (Jorge Sancha) [Big Data, Data Engineering, Product, startup] [Duration: 0:40:00] Interesting conversation about tinybird with one of its founders. They talk about the company, the problem they solve and other interesting topics related to real-time data analysis.
  • Improving Observability and Testing In Production (Dave Farley) [Observability, Testing in production, testing] [Duration: 0:19:00] In this episode, Dave explores what Testing in Production is, and how to Test in Production, and how this is related to working more experimentally.
Reminder, All these talks are interesting even just listening to them.

Related:

Sunday, November 20, 2022

Good talks/podcasts (Nov 2022 II)

 


These are the best podcasts/talks I've seen/listened to recently: Reminder, All these talks are interesting, even just listening to them.

Related:

Sunday, November 06, 2022

Good talks/podcasts (Nov 2022 I)




These are the best podcasts/talks I've seen/listened to recently:
  • Jessica Kerr on how Software keeps Evolving and is never Done (Jessica Kerr) [Inspirational, Mental models, Systems Thinking] [Duration: 0:44:00] Interesting conversation in which Jessica explains how software behaves in a very similar way to biology. Lots of interesting points to learn from. Very aligned with this dynamic view of systems and their constant evolution.
  • Agile Book Club: Evolutionary Design (with Kent Beck) (James Shore, Kent Beck) [Agile, Evolutionary Design, Software Design, Technical Practices] [Duration: 0:42:00] (⭐⭐⭐⭐⭐) This was one of the best talks I've heard on software design in a long time. Excellent points on evolutionary design, testing, technical practices, etc. A full three quarters of an hour of wisdom.
  • Agile Book Club: Test-Driven Development (with Mike “GeePaw” Hill and J.B. Rainsberger) (GeePaw Hill, J.B. Rainsberger, James Shore) [Evolutionary Design, Software Design, Technical Practices, tdd] [Duration: 0:43:00] (⭐⭐⭐⭐⭐) Very interesting conversation with great TDD experts. Some interesting ideas, techniques and tricks.
  • Adaptive Planning Beyond User Stories (Gojko Adzic) [Lean Product Management, Lean Software Development, Product, Product Discovery] [Duration: 0:55:00] (⭐⭐⭐⭐⭐) Gojko gives a masterclass on how to focus on impacts, understand the need, and improve product decisions. Good ideas to postpone decisions, focus on behavioral changes, and be cautious with metrics. An essential talk for any product developer.
  • The Most Common Test Driven Development Mistakes (Dave Farley) [Software Design, Technical Practices, tdd] [Duration: 0:14:00] In this episode, Dave Farley describes some of the common problems that people face, and gives his top tips for avoiding them.
Reminder, All these talks are interesting even just listening to them.

Related:

Sunday, October 23, 2022

SCBCN22: Taller Desarrollo Lean: posponer decisiones

Este fin de semana hemos facilitado por primera vez el taller que hemos preparado @islomar y yo sobre Desarrollo Lean: Posponer decisiones. Muchisimas gracias a la organizaciĆ³n de esta magnifica conferencia Softwarecrafters Barcelona por confiar en nosotros y dejarnos probar el taller por prinmera vez.
Actualizaremos este post con mĆ”s informaciĆ³n y prepararemos una guia de facilitaciĆ³n al estilo de S3 Workshop, pero de momento os dejamos la presentaciĆ³n por aquĆ­ para que los que habeĆ­s estado en el taller podais reforzar los conceptos.

PresentaciĆ³n

PresentaciĆ³n Original

Sunday, October 16, 2022

Good talks/podcasts (Oct 2022 II)

 


These are the best podcasts/talks I've seen/listened to recently:
  • The nature of product (Marty Cagan) [Product, Product Team] [Duration: 0:59:00] (⭐⭐⭐⭐⭐) A very interesting interview to Marty Cagan. Very good insights about empowered teams, product management, and team organization.
  • What Does Tech Excellence Look Like? (Martin Fowler) [Engineering Culture] [Duration: 0:31:00] Martin explores how culture, structures, talent, and technology choices help to build organisations that have technical excellence ingrained in them.
  • Legacy Code, OOP vs Functional Programming & MORE (Michael Feathers, Dave Farley) [Engineering Culture, Legacy code, Software Design, Technical Practices] [Duration: 1:14:00] Michael and Dave talk broadly about automated testing, software architecture and design principles for quality code, and Michael claims that “OO, when it's done right, looks a lot like FP”.
  • Integrated Tests Are A Scam (J.B. Rainsberger) [Technical Practices, testing] [Duration: 1:04:00] (⭐⭐⭐⭐⭐) Integrated tests are a scam. You’re probably writing 2-5% of the integrated tests you need to test thoroughly. You’re probably duplicating unit tests all over the place. Your integrated tests probably duplicate each other all over the place. When an integrated test fails, who knows what’s broken? Integrated tests probably do you more harm than good. Learn the two-pronged attack that solves the problem: collaboration tests and contract tests.
  • Hibernate should be to programmers what cake mixes are to bakers: beneath their dignity. (Christin Gorman) [Engineering Culture, Inspirational] [Duration: 0:08:00] (⭐⭐⭐⭐⭐) Great lightning talk, provocative and inspiring.
Reminder, All these talks are interesting even just listening to them.

Related:

Friday, October 14, 2022

Good talks/podcasts (Oct 2022 I)

 


These are the best podcasts/talks I've seen/listened to recently:
  • Building Evolutionary Architectures: Principles and Practices (Rebecca Parsons) [Architecture, Architecture patterns, Evolutionary Design] [Duration: 0:42:00] This talk presents some principles of evolutionary architecture that allow systems to respond to change without needing to predict the future. We then briefly describe approaches that realize these principles and discuss how these approaches support the adaptability of systems in an evolutionary way.
  • Wired for Success - Business Modelling for Startups (Carlos Miragall) [Business models, startup] [Duration: 1:06:00] Interesting talk about how startups make money and spend less than they earn. This a good talk to understand core metrics for several common business models.
  • How To Avoid Designing A Big Ball Of Mud (YAGNI) (Dave Farley) [Agile, Continuous Delivery, Technical Practices] [Duration: 0:19:00] (⭐⭐⭐⭐⭐) In this episode, Dave Farley explores how to manage complexity in software systems, and how to adopt an evolutionary approach to design that ensures that we can change our software as we learn more.
  • Sufficiently Advanced Monitoring is Indistinguishable from Testing (Ed Keyes) [Technical Practices, Testing in production, testing] [Duration: 0:05:00] (⭐⭐⭐⭐⭐) Interesting ideas about testing in production.
  • How Well Designed Is Your Microservice? (Dave Farley) [Architecture, Architecture patterns, Microservices, Technical Practices] [Duration: 0:20:00] Dave describes what it takes to design and test a variable microservice, and explores the use of contract testing, and loose-coupled interface design, to retain their deployment independence.
  • Fast-track from Idea to Impact (Gojko Adzic) [Duration: 0:54:00] (⭐⭐⭐⭐⭐) Gojko presents very interesting examples of developments that have achieved very good impacts (by minifying the developed software). He also explains the use of the impact mapping technique, which is very useful for focusing on the impact.
Reminder, All these talks are interesting, even just listening to them.

Related:

Monday, September 19, 2022

Good talks/podcasts (Sept 2022 I)

These are the best podcast/talks I've seen/listen to recently:
  • 20 Questions To Ask Your Next Employer (Dave Farley) [Engineering Career, Engineering Culture] [Duration: 0:17:00] In this episode, Dave Farley offers his advice on how to pick great employers. He describes the things you should explore, when looking for a new job, to decide if a potential employer is only talking a good game, rather than playing a good game.
  • Agile at Tesla with Joe Justice Agile Munich (Joe Justice) [Agile, Lean] [Duration: 0:58:00] interesting description of the agile principles and practices applied at tesla.
  • FYAC: The 3 Levels of User Story Slicing (Neil Killick) [Agile, Lean Product Management, Product Discovery, slicing] [Duration: 0:37:00] (⭐⭐⭐⭐⭐) Very good ideas on how to approach product slicing, option generation and agile product development.
  • The Sociotechnical Path to High-Performing Teams (Charity Majors) [Continuous Delivery, Devops, Engineering Culture, Teams, team topologies] [Duration: 0:41:00] (⭐⭐⭐⭐⭐) This talk describe the social and technical strategies that great teams all of the world are using to be happier, more productive and make their users happy too.
  • Tips For Building Successful Platform Teams (Dave Farley) [Platform, Platform as a product, Platform teams] [Duration: 0:22:00] (⭐⭐⭐⭐⭐) In this episode, Dave describes the pitfalls common to platform teams, the principles that are important for platform engineering, the strategic goals that platform teams can use to guide their decisions, and some tips for platform teams that will help them protect their users from changes in services.
  • Resilience In Complex Adaptive Systems (Richard Cook) [Inspirational, Resilience] [Duration: 0:20:00] This talk introduces resilience and a model of system dynamics useful in analyzing failed and successful event management and offer an explanation for why our systems run at the edge of failure.
  • Operational simplicity is a gift to you (Gary Bernhardt) [Architecture, Design, Inspirational, Operations] [Duration: 0:57:00] Very entertaining episode with great examples of how to make very operable systems by systematically searching for simplicity.
Reminder, All these talks are interesting even just listening to them.

Related:

Wednesday, August 24, 2022

Lean Software development: The art of postponing decisions

One of the seven principles of Lean Software Development is Defer Commitment. This principle tries to maintain options open (options have value) and avoid waste by only solving the current problem (avoiding speculative design). Following this principle allows us to generate easy to change, minimal systems with low accidental complexity and minimal features (as we have a low lock-in).

Keep in mind that software is only a means to an end, and that we are trying to generate impact, not software.

Over the years, I have found that postponing decisions and working in small, safe steps have been the key to creating sustainable solutions.

My experience tells me this is an excellent way to work, at least for product development. 

This article is based on my experience over the past 12-13 years. All of the companies that I talk about are product companies, some of them startups (TheMotion, Nextail, ClarityAI) and others more classic companies (Alea Soluciones).

What we mean

Postponing all technical decisions until the last responsible moment. This practice implies scheduling irreversible decisions for the last responsible moment (LRM) and trying to make as many reversible decisions as possible.

What we do not postpone

For the last 12-13 years, I have been trying to work in an agile way, which implies that there are certain decisions in the way I work that I do not postpone, Agile, continuous delivery, eXtreme Programming, etc.

Some decisions are difficult to postpone. I think the language (ecosystem) and the approach to development (objects vs. functional) should not be postponed and must be clear from the beginning. 

Some of these decisions may evolve over time, but we must have a solid starting point.

Working with technical excellence is another decision we do not postpone. It is implicit in my decision to work with Agile, but I think it is important to make it explicit.

Why we want to postpone / Benefits achieved

As we will see in the article, postponing decisions is not easy and requires practice. But as I mentioned in the introduction, it allows us to make sustainable and easy to evolve products.

  • A later decision will give us a greater understanding of business and technology.
  • Simpler and smaller solutions
  • Less work :-) (less basal cost  and less time implementing things we don’t need)
  • More likely to not do anything that doesn't bring real value now
  • More likely not to over-engineer
  • Less effort in redoing work if necessary
  • Our goal is to have a lot of real options... react well and fast to anything... and without fear...
  • The more we postpone, the less we fill the backpack, and the less unnecessary stuff we add (less waste).
  • We all know that it’s easier to move with fewer things in the backpack.
  • Good architecture allows us to postpone important decisions, but this does not mean we are obliged to do so. However, being able to postpone them gives us a lot of flexibility.

This approach is aligned with the lean principles of eliminating waste and the agile principles of postponing until the last responsible moment (LRM) and is one of the approaches that can help the creation of good architecture. In turn, good architecture makes it easier to postpone other decisions.

How we do it

It is very difficult to explain everything that needs to be taken into account in order to postpone decisions/commitments without risk and achieve the benefits mentioned above. Anyhow, here is what has worked for us:

  • We postpone decisions in a conscious and meditated manner.
  • We consider a decision to be a good one when:
    • It compromises us as little as possible
    • Allows us to postpone other decisions
    • It is easy to change/revert
    • Attacks a current (not future) problem
    • Good enough (no over-engineering, neither ours nor other people's)
    • No Megaconstructions
  • We reuse libraries but not Frameworks (we don't let them use us)
  • We consider that everything can change (code/design/process)
  • We work in a very iterative manner (See https://productdeveloper.net/iterative-incremental-development)
  • When we start a new feature, we focus on identifying what we need to change in our architecture and our understanding of the ubiquitous domain/language.
  • We assume that what we decided was the right thing to do at that time and in that context, and if it needs to be changed, that's fine.
  • We put a lot of effort into good vertical slicing for the features (with the Hamburger method for example) and even technical slicing to be capable of working in small safe steps.
  • We strive for technical excellence (Clean Code, Simple Design, TDD, Evolutionary Design, etc).
  • We use DDD techniques and Hexagonal architecture.

The points I mention are not necessary, but they can serve as inspiration for any team that wants to gain a deeper understanding of agility.

What we need to make it easy to postpone

  • Safety/Confidence: about our system and code and at the team level (psychological safety)
  • Fast and continuous feedback: Customer feedback, System feedback (monitoring, observability), Development feedback (automatic tests, continuous code review with pairing, etc)
  • Maintainable/Evolvable product/code base: We use XP, Devops practices and lean mindset
    • Simple design
    • TDD
    • Continuous refactor
  • Accept/Assume that everything can be changed/removed/evolved (Features, Code, Design, etc)

We split medium/large changes into small changes, even duplicating code and duplicating data if necessary... all of this in order to be able to postpone decisions, not impede deployment and make incremental small, and safe changes.

Some examples

In this section, I will describe some examples of systems in which we have had a lean approach trying to postpone all possible decisions until the last responsible moment. This has allowed us to have a very low time to market and to adapt the solutions to the feedback we were getting all the time.

Software for small telecommunications networks (up to 5000 subscribers). New fiber optic provision software with a very aggressive time to market:

  • First rollout version with python serialization to file as persistence 
  • After one month, we added a python text search to the persistence layer
  • After two months, we migrated the persistence layer to Redis to improve performance
  • 5 or 6 months later, we migrated the persistence to MySQL with Full-Text Search once we saw the growth trends
This approach allowed us to roll out a new product line in less than two months and evolve the product during the last ten years.

At TheMotion, we decided to create a command line application to deploy, check status, create new services, and other common operations. The challenge was to make this application available as soon as possible and evolve it while changing TheMotion's system environment/infrastructure. Another important requirement was the security of access to the application.

  • The first version was implemented as a shell script running in a server and executed by the developers via ssh (authorization, encryption at transit) using an alias.
  • In less than a month, this shell script was migrated to python.
  • And after several months, we removed the ssh call with a python client.

The developers' command executions remained largely unchanged during these months. In the background, the deployment, the infrastructure, and the running platform were rapidly evolving.


Also at TheMotion, we were creating a SaaS b2b product from scratch, and in the beginning, we didn’t had customers and the access was using an API. So we took a lean approach to user management, access, and role management.

  • The first version only was usable by a few concrete companies, and we didn’t need to bill anything, so we only restricted access via IP address.
  • The next step was to generate an ID and API Key for each customer and differentiate them by the API Key.
  • After three months, we manually created users (name, email, etc), but without password, only the API Key.
  • The next step was to allow some users to impersonate others (for example, an agency using the SaaS for several final customers).
  • I don’t remember when, but later we created an admin website, and at this moment, we included passwords for the users.

You can see more examples and strategies in this blog post System: Control its evolution / or be its slave  

I am also working on a workshop to practice how to defer commitment that will have more concrete examples.


Problems/Sensations generated by postponing decisions…

Certainly, postponing decisions is difficult since it generates feelings that we do not usually like:

  • Uncertainty
  • Anxiety
  • Conflict (as engineers)

Studies show that uncertainty generates fear and anxiety, so it's not surprising that we try to avoid it. Additionally, as software engineers, we have been trained to solve problems, which is why postponing decisions generates conflict.

Once you get used to working in small steps, with high quality and continuously iterating solutions, much of that anxiety and uncertainty disappears, and development becomes a continuous flow of change.

Conclusions

Postponing decisions and commitments is not easy, it’s difficult, and sometimes it can be confused with laziness or unprofessionalism. It couldn't be further from the truth. Professionalism is about solving current problems (YAGNI) with the most straightforward/simple possible solutions (Simple Design, pragmatism) rather than speculating about the future.

As we have seen, although difficult, postponing decisions/commitment until the last responsible moment has great benefits.

So, Postpone... to infinity and beyond…



Until the last responsible moment!!!!

References and related resources:

Real options 

Related blogposts

Examples

Practices & Techniques


Thanks

The post has been improved based on feedback from:

Thank you very much to all of you





Monday, August 15, 2022

Good talks/podcasts (Aug 2022 I)

 


These are the best podcast/talks I've seen/listen to recently:
  • Hey CTO - S01E01 - with Hass Chapman (Hass Chapman) [Engineering Culture, Engineering Scalability, Technical leadership, leadership] [Duration: 0:42:00] Interesting interview with Hass Chapman, Head of Product & Engineering at Proto Energy
  • Shared Mental Models Part 1 Shared Mental Models Part 2 (Jessica Kerr) [Engineering Culture, Inspirational, Mental models] [Duration: 0:33:00] (⭐⭐⭐⭐⭐) This was an excellent talk, full of insights that prompted reflection. In this talk Jessica looks at how the shared mental models created while mob programming work throughout the team even when they are not actively mobbing. She also explores the other practices she’s found complementary in creating a high functioning team and how looking at your contribution from a generative (helping other create) vs a productive (what I created, myself) frame can lead to a happier, better and more productive team.
  • Scaling Personalization: DevOps at Stitch Fix (Randy Shoup) [Continuous Delivery, Devops, Engineering Culture, Technical leadership] [Duration: 0:32:00] This talk discusses the technology approach and modern development practices we have put in place to make this model succeed.
  • Investing in Innovation: How Situational Awareness Can Put Your Business on the Map (Mike Lamb) [Wardley maps] [Duration: 0:06:00] Pill on the importance of situational awareness and wardley maps.
  • Scaling from 5 to 250 engineers (Eiso Kant) [Engineering Culture, Engineering Scalability, Technical leadership] [Duration: 1:03:00] Talk on scaling your team from 5 to 250 engineers — lessons learned, mental models, metrics, and everything that is useful to know.
  • The Dark Side of Events (Vladik Khononov) [Architecture patterns, Evolutionary Design, Software Design] [Duration: 0:44:00] In this talk, you will learn heuristics for identifying problems based on events and eliminating event-driven complexities.
Reminder, All these talks are interesting even just listening to them.

Related:

Monday, July 25, 2022

Good talks/podcasts (Jul 2022 I)

These are the best podcast/talks I've seen/listen to recently:
  • Why Your Software Team CAN’T Scale (Dave Farley) [Engineering Culture, Teams, team topologies] [Duration: 0:19:00] In this episode, Dave Farley, author of the best selling books Continuous Delivery, CD Pipelines and Modern Software Engineering, looks into how to organise software at scale, and how the Team Topologies model works and helps us to design organisations that can produce better software faster.
  • The Flywheel Effect Creates Space for Innovation (Europe 2022) (David Anderson, Michael O'Reailly) [Engineering Culture, Platform, Technology Strategy] [Duration: 0:26:00] The flywheel effect occurs when we balance business strategy (via Wardley mapping) with technology strategy (via Modern cloud). As a preview of the upcoming IT Revolution book, Dave creates a Wardley Map to show how to use the value flywheel to build confidence, improve your cloud stance and create space for innovation.
  • An Inside Look At eBay’s Software Practices (Dave Farley) [Continuous Delivery, Engineering Culture] [Duration: 0:21:00] In this episode, Dave Farley, explores the changes that eBay have made, their impact, and what they have learned from the experience so far.
  • Backstage: Restoring Order To Your Chaos (Dave Zolotusky) [Devex, Engineering productivity, Platform, Platform as a product] [Duration: 0:35:00] good introduction to Backstage, the environment, the main plugins, etc.
  • Code for Ukraine #2: Tidy First? Daily Empirical Software Design & Why It Works (Kent Beck) [Software Design, XP] [Duration: 0:58:00] (⭐⭐⭐⭐⭐) Interesting presentation about software design and trade-offs and techniques to refactor the code before applying a change. The talk is full of interesting insights. It is also very interesting the Q&A part that is not only focused on the content of the talk.
  • How to scale platforms in high-growth companies? (Jelmer Borst) [Platform, Platform as a product] [Duration: 0:13:00] This is a story about why Picnic favours boring tech with consolidated tech stacks to actually move very, very fast.
  • Continuous Delivery - Sounds Great But It Won't Work Here (Jez Humble) [Continuous Delivery, Engineering Culture] [Duration: 0:49:00] (⭐⭐⭐⭐⭐) Since the Continuous Delivery book came out in 2010, it’s gone from being a controversial idea to a commonplace… until you consider that many people who say they are doing it aren’t really, and there are still plenty of places that consider it crazy talk. In this session Jez will present some of the highlights and lowlights of the past six years listening to people explain why continuous delivery won’t work, and what he learned in the process.
Reminder, All these talks are interesting even just listening to them.

Related:

Sunday, June 26, 2022

Good talks/podcasts (Jun 2022 II)


 

These are the best podcast/talks I've seen/listen to recently:
  • LOTE #7: Kelsey Hightower on Developer Experience, PaaS, and Testing in Production (Kelsey Hightower) [Cloud, Developer Productivity, Devex, Platform as a product] [Duration: 0:41:00] In the seventh episode of the Ambassador Livin’ on the Edge podcast, Kelsey Hightower, technologist at Google, discusses his thought on cloud developer experience, modern Platform-as-a-Service (PaaS), and explores the reality that every organisation is testing in production
  • Should Computers Run the World? (Hannah Fry) [AI, Data Science, General, Inspirational] [Duration: 0:36:00] Hannah Fry takes us on a tour of the good, the bad and the downright ugly of the algorithms that surround us. She lifts the lid on their inner workings, to demonstrate their power, expose their limitations, and examine whether they really are an improvement on the humans they are replacing.
  • Small Batches - PDCA (Plan-Do-Check-Act) (Adam Hawkins) [Lean] [Duration: 0:06:00] Adam present's Dr. Deming's PDCA cycle and how it applies to the daily work of delivering software.
  • Steve Jobs on programmer productivity (Steve Jobs) [Lean Product Management, Lean Software Development] [Duration: 0:01:00] Excerpt (1m) of Steve Jobs presenting the software as a liability to minimize. You know, make impact, not software. "The way you get programmer productivity is not by increasing the lines of code per programmer per day. That doesn’t work. The way you get programmer productivity is by eliminating lines of code you have to write."
  • From initial request to software in production in 3 weeks (Christin Gorman) [Inspirational, Lean Software Development] [Duration: 0:22:00] (⭐⭐⭐⭐⭐) Simplicity--the art of maximizing the amount of work not done--is essential. Great talk on how to focus on the essentials and make simple solutions.
  • The Future of Mars Exploration (Anita Sengupta) [Inspirational] [Duration: 0:54:00] In this talk, you will learn about the motivation for Mars exploration and how computational modeling, high-tech solutions, and out-of-the-box thinking can be used to overcome engineering challenges.
Reminder, All these talks are interesting even just listening to them.

Related:

Sunday, June 12, 2022

Good talks/podcasts (Jun 2022 I)


 


These are the best podcast/talks I've seen/listen to recently:
  • Scaling Organizations and Design with James Shore (James Shore) [Agile, Evolutionary Design, Software Design, XP] [Duration: 0:48:00] (⭐⭐⭐⭐⭐) Great interview with great ideas (mocking-free testing, scalability of horizontal organization, FAST framework, code evolution with evolutionary design...). Very worthwhile.
  • Feature Branches and Toggles in a Post-GitHub World (Sam Newman) [Continuous Delivery, Technical Practices, Trunk Based Development] [Duration: 0:50:00] In this presentation Sam explains how working in feature branches creates friction when our goal is to achieve continuous delivery. Sam explains how to use Feature Toggles to improve delivery frequency.
  • From Kubernetes to PaaS to ... Err, What's Next? (Daniel Bryant) [Developer Productivity, Devex, Platform, Platform as a product] [Duration: 0:31:00] (⭐⭐⭐⭐⭐) In this talk Daniel reviews his experience in building platforms, both as an end-user and now as part of an organization that helps our clients do the same. She discusses topics such as DevEx, UX, workflows, available tools, etc.
  • Make Impacts Not Software (Gojko Adzic) [Lean Product Management, Lean Software Development, Product, Product Strategy] [Duration: 0:51:00] (⭐⭐⭐⭐⭐) An essential talk to understand how to get the most impact with the least amount of software (and thereby reduce basal cost and time to market). Highly recommended.
  • The Difference between Software Engineering and Manufacturing (Donald Reinertsen) [Lean Product Management, Product] [Duration: 0:09:00] In this interview, Don explains the basic difference between applying lean to a manufacturing process and to a digital/software product creation process.
  • the deep synergy between testability and good design (Michael Feathers) [Software Design, Technical Practices, XP, testing] [Duration: 0:50:00] Interesting talk about how to do good software design and the relationship between this design and the ease of testing.
  • Less - The Path to Better Design (Sandi Metz) [OOP, Software Design] [Duration: 0:50:00] (⭐⭐⭐⭐⭐) This talk strips away the well-known design principles and exposes the hidden, underlying goals of design. It reveals programming techniques that allow you to write less code while creating beautiful, flexible applications.
Reminder, All these talks are interesting even just listening to them.

Related:

Sunday, May 15, 2022

Good talks/podcasts (May 2022 I)

 


These are the best podcast/talks I've seen/listen to recently:
  • Full Cycle Developers at Netflix (Greg Burrell) [Architecture, Devops, Operations] [Duration: 0:48:00] (⭐⭐⭐⭐⭐) This talk presents Netflix' journey from siloed teams to our Full Cycle Developer model for building and operating our services at Netflix. Greg discusses the various approaches they’ve tried, the motivations that pushed them to keep evolving, and the lessons learned along the way.
  • Building DevX Teams, my story (Cirpo Cinelli) [Developer Productivity, Devex, Teams] [Duration: 0:42:00] (⭐⭐⭐⭐⭐) In this presentation, Cirpo talks about his past 4 years of experience setting up a DevX team from scratch, the main challenges, the pain, the gain, and the lessons learned.
  • How To Avoid Big Upfront Design (Dave Farley) [Architecture, Continuous Delivery, Evolutionary Design] [Duration: 0:18:00] In this episode, Dave Farley, describes how to avoid big up-front design, but also how to do enough design to make progress safely and allow us the freedom to learn and adapt our designs as we learn.
  • The Root Causes of Product Failure (Marty Cagan) [Product, Product Strategy, Product Team] [Duration: 0:49:00] Interesting talk on how the best companies and product teams work.
  • TDD Discipline - Thinking Ahead Without Coding Ahead (Jason Gorman) [Technical Practices, XP, tdd] [Duration: 0:11:00] Jason Gorman demonstrates a very common mistake TDD newbies make - writing the code you \*believe\* is required, instead of just the code needed to pass your tests.
Reminder, All these talks are interesting even just listening to them.

Related:

Monday, April 18, 2022

Systems: Control its evolution / or be its slave

 When we develop software (digital products), we usually deal with complex systems. We are not prepared (by definition) for emergent behaviors of these types of systems. As a result, it is hard to predict its future evolution and very easy to lose control over it over time.


If we don't control complexity, it will become increasingly difficult to adapt the system to what we need, and we will move from developing and evolving the system to spending more time reacting to it as it evolves. We will become slaves to the system rather than controlling its evolution. We will be spending our time working on production incidents, scalability improvements under pressure, and being in a firefighting mode instead of adding value to the product in a sustainable and continuous manner. Needless to say, the business impact is terrible (poor efficiency, unhappy customers, difficulty in defining strategy, etc).


In my experience, we can keep the system under control by using appropriate tactics.

Tactics to control system evolution

I have used the following tactics to control the evolution of systems I have worked on:

  • Good basic hygiene: automated tests, minimum observability, product instrumentation, etc.
  • Simple design based on current needs: Using incremental design and lean product development we can develop simple solutions that fulfill our current needs but allow us to evolve the system in the future.
  • Evolutionary architectures: Invest in an architecture that can evolve and that allows us to postpone decisions (reversible decisions, fitness functions, observability, etc.).
  • Acceptance criteria: Make the acceptance criteria explicit, so that we all know what we consider sufficient hygiene (testing, observability, performance, etc.).
  • Define product limits: Define product limits that allow us to convert problems commonly considered "technical" to business/product problems. For example, instead of talking about scalability as an abstract concept, concrete it at the product level with clear limits (number of concurrent users/requests, maximum response times, etc).
  • Look for boundaries and use them to assign clear ownership for each identified part. The concepts of bounded contexts of DDD and value stream can assist in identifying such limits within the system. Having clear ownership of each part of the system is fundamental since we must remember that we work on complex socio-technological systems.
  • Work in small safe steps: Controlling the evolution of the system is complex and requires us to constantly rethink and adapt. We can only accomplish this if we work in small safe increments of functionality that allow us to change direction without leaving features half-integrated.

The use of these tactics is usually adequate, but as with everything in our profession, it depends, there are times when a little upfront planning and attention to future needs would be beneficial. See "It depends" / The development Mix (Product, Engineering, Hygiene.


Below I include several examples of how I have utilized some of these tactics to control the evolution of the system and not be at its mercy.

Examples

System Load / Scalability evolution



Having performance or stability/availability problems when a system is under high load is very common. The most common approaches for this evolution dimension are:

  • Doing nothing and reacting when we discover the first problems (potentially damaging the relationship with our users due to production incidents).
  • Solving “imaginary” scalability problems by preparing/designing our systems for a load that is not real and generating a high waste. Sometimes this is the default option chosen in engineering simply for the pleasure of solving an interesting problem.

I think that there is a more sensible approach that allows us to control evolution without generating waste (See Lean software development). In this case, we need to have some metrics about the current load and a rough idea of how much maximum load our system can support (without degrading).

With this information, we can define a soft limit (Product Limit) for the load, as the maximum supported load minus a threshold that gives us time to evolve the system to support more load.

As we can see in the example diagram, we receive a notification at t1 informing us that we reach the moment to improve the scalability of the system. Between t1 and t2, we improve the scalability to support a new load maximum and redefine the new soft limit and repeat the process when the limit is reached (t3).

Storage volume growth



When we store information it is very common that the associated growth of the information volume reaches a point when the technology selected is no longer valid or that we can start to have some problems (high cost, backup problems, etc). 

As in the previous example, we can do nothing and react after we start to have problems or use a tactic to control the evolution/growth.

In this case, if we need to maintain all the historical information, we can follow a similar approach as in the previous example about scalability. But if we don’t need to store the historical information we can create a soft limit to detect when we should start to develop an automated data purge process that periodically removes obsolete information. This approach is very lean because it allows us to postpone the implementation of the data purge process until the last responsible moment.


Controlled errors and quotas


One of the easiest ways to lose control of a system is to have no controlled way of handling errors. When we don’t control errors, we are in risk of having cascade errors and affecting more users. This is one of the most common examples of losing control of the system.

One of the first tactics that help us to maintain control of a system is to detect unexpected runtime errors, avoid crashing and at least send a message to the user and avoid as much sa possible that this error impacts other users.

Another good tactic is to define quotas per user (Product Limits) and use these quotas to throttle requests and show a warning message to the user. In the message, we can inform the user to contact support and use this opportunity to get more information about how the user is using the system or even offer a better SLA to the user.


Committed capacity


When we work in a multitenant SaaS environment is very common that the resources used by the sum of all of our customers increase very fast. One interesting tactic to maintain the control of the evolution of the system in this context is to define how much capacity (of a resource) we commit to each type of user (Product limits). Even knowing that customers will not fully utilize their full capacity, having these definitions allows us to identify the worst-case load scenarios we could have.

In the example diagram, we define the different quotas for each type of user (basic, professional, enterprise). With these definitions and the number of users of each type, we can calculate the maximum committed load at any moment and use this information to make decisions.

This tactic is used frequently by AWS. For each type of resource, they define default limits, and to extend these limits you need to contact support. AWS can use the information from support requests to make very accurate capacity plans and make better product decisions.


Modular monolith


Building a system from scratch is not an easy task, and it is very easy to lose control of the system as it grows (more developers / more functionality). The two most common failures are:

  • Growing a monolith organically without committing effort to modularization and ending up with a big ball of mud (http://www.laputan.org/mud/).
  • Incorporating a microservices architecture when it is not yet time to scale greatly increases complexity without obtaining any of the benefits of this architecture.

In general, better results can be achieved by developing a monolith, but organizing the code and data into internal modules (modular monolith). As we become familiar with the system and domain, we organize it internally into modules (in alignment with bounded contexts), so we can maintain control of the architecture throughout its evolution. 

When we see the need, we can extract some of these modules into independent services.

It is essential to create mechanisms that help us maintain this modularity during evolution (to perform architecture tests to avoid not allowed dependencies between modules, to use different schemas in the DB, to analyze dependencies periodically).

In addition, we can use these modules to assign ownership of each to a team, allowing each team to work independently.

Conclusions

When we lose control over the evolution of the system:

  • we become inefficient and fail to create value.
  • we lose customer trust (incidents, errors, etc).
  • we react to problems instead of being able to follow our product strategy.

The business impact is huge. Therefore, it is essential to manage complexity and keep evolution under our control.


The tactics I discuss in this article are quite simple and focus on detecting needs early enough so that we can react in a planned way (rather than reacting under pressure). Moreover, they do not require large investments, in fact, they allow us to have a lean approach and avoid over-engineering. But they do require good hygiene practices (testing, observability, reasonable quality, etc). 


References and related content:

Thanks

The post has been improved based on feedback from:

Thank you very much to all of you





Saturday, April 09, 2022

Good talks/podcasts (April 2022 I)




These are the best podcast/talks I've seen/listen to recently:
  • Autonomy, mastery, purpose (Drive summary) (Dan Pink) [Inspirational, Management] [Duration: 0:03:00] (⭐⭐⭐⭐⭐) Daniel Pink shares a study about what truly motivates employees. Excerpted from his talk on "motivation."
  • The puzzle of motivation (Dan Pink) [Inspirational, Management] [Duration: 0:18:00] (⭐⭐⭐⭐⭐) Dan Pink examines the puzzle of motivation, starting with a fact that social scientists know but most managers don't: Traditional rewards aren't always as effective as we think.
  • The secret to giving great feedback (LeeAnn Renninger) [Management] [Duration: 0:05:00] Cognitive psychologist LeeAnn Renniger shares a scientifically proven method for giving effective feedback.
  • Making Badass Developers (Kathy Sierra) [Developer Productivity, Devex, Inspirational] [Duration: 0:23:00] Interesting talk to understand how cognitive load works, how we learn and how we should take into account these concepts to improve our experience as developers.
  • Responsible Engineers and Outcomes with Mary and Tom Poppendieck (Mary Poppendieck, Tom Poppendieck) [Agile, Engineering Culture, Lean Product Management, Lean Software Development] [Duration: 0:49:00] Mary and Tom Poppendieck discuss about "Responsible Engineers and Outcomes". They walkthrough concrete examples of what it looks like for lean teams to maximize outcomes and minimize outputs. While exploring these examples, they touch on customer interaction, feedback, the responsible engineer, and the single threaded leader.
  • Production - Designing for Testability (Michael Bryzek) [Continuous Delivery, Devops, Engineering Culture, Testing in production, testing] [Duration: 0:50:00] (⭐⭐⭐⭐⭐) Michael Bryzek explores what it’s like to build quality software with no development, QA, or staging environments. He includes a deep dive into “verification in production” and what it really takes to build software that can safely be tested continuously in production.
  • Keynote: Creating a Holistic Developer Experience (Jasmine James) [Developer Productivity, Devex] [Duration: 0:15:00] (⭐⭐⭐⭐⭐) Great talk to understand what is development experience.
  • OOP is Dead! Long Live OODD! (David West) [OOP, Software Design] [Duration: 1:07:00] Interesting talk about object orientation, the history and how in many cases we don't understand it correctly. It's a bit ranty, but very interesting and inspiring in any case.
Reminder, All these talks are interesting even just listening to them.

Related:

Thursday, March 31, 2022

Mesa Redonda: ColaboraciĆ³n entre perfiles de producto y desarrollo (Code Sherpas)



El pasado 24 de Marzo tuve el placer de participar en una mesa redonda sobre "ColaboraciĆ³n entre perfiles de producto y desarrollo" donde pude aprender y compartir con Marta Manso, MarĆ­a Granadino, Isabel Garrido y Cristina Verdi.

Gracias a Code Sherpas por la organizaciĆ³n y dinamizaciĆ³n del evento.

Monday, March 28, 2022

Our DevOps Journey @ ClarityAI [2022 03 DevOps Lisbon]

Few days ago (2022-03-14), I had the pleasure of talk at the DevOps Lisbon about the last two year evolution of the engineering organization at ClarityAI. In my presentation, I examined the evolving DevOps culture as well as strategies for structuring teams according to Team Topologies' ideas.
Thanks to the Devos Lisbon for the invitation and to all the participants for the attention.

Video



Slide deck

Original Doc (with notes)



References