Saturday, December 25, 2021

Good talks/podcasts (December 2021 I)


 


These are the best podcast/talks I've seen/listen to recently:

  • What It Takes To Be A Software Engineer (Dave Farley) [Engineering Culture, Inspirational, Software Design] [Duration: 0:18:00] (⭐⭐⭐⭐⭐) Great and concise description of what software engineering is and the forces that apply to our profession.
  • MMMSS – The Intrinsic Benefit of Steps (GeePaw Hill) [Agile, Flow, XP] [Duration: 0:12:00] (⭐⭐⭐⭐⭐) Why we should work in small safe steps (3s).
  • Wardley Maps for concious strategy definition (Ismael Castillo, Enrique Caballero) [Architecture, Product Strategy, Wardley maps] [Duration: 0:30:00] Share ideas on how to align strategy and create situational awareness using Wardley Maps, DDD, and Team Topologies among others tools and frameworks.
  • The New Faces of Continuous Improvement | Dev Interrupted Engineering Panel (Charity Majors, Kathryn Koehler, Dana Lawson) [Continuous Delivery, Engineering Culture, Teams] [Duration: 0:55:00] A very interesting panel on high performance teams, engineering culture, continuous delivery, etc.
  • Keynote: Systems Thinking (Jessica Kerr, Kent Beck) [Inspirational, Systems Thinking] [Duration: 0:59:00] A great introduction to systems thinking.
Reminder, All these talks are interesting even just listening to them.

Related: 

Sunday, November 14, 2021

Improve or Die

A software development/product development organization should always be learning and improving.

When the organization is not learning or improving means that it is going backward, software development is a complex socio-technical system formed by several interrelated reinforcing loops. Some of the loops are positive (virtuous cycles) and some negative (vicious cycles), but the problem is that in such a complex system is difficult to find any balance, so in general, we are always moving. 

So the question is, in which direction? Are we learning and improving as a team, or are we dying or falling behind

Even if we managed to maintain a continuous flow of development with stable quality and speed (which is impossible), the whole ecosystem around us continues to improve and advance, so even in that case, we would be losing ground.

In general, the reinforcing loops are generated by things that compound with time or volume. 

For example, these are some things with negative compound effects: 

  • Complexity and basal cost of the product (cumulative features)
  • Quality problems.
  • Technical debt (if not managed).

Virtuous cycles examples:

  • Continuous delivery requires quality, requires small batches, removes silos, improves ownership, etc.
  • Product Team ownership requires autonomy, requires product instrumentation, requires learning from customers, generating more value, etc. 
  • etc

Vicious cycles examples:

  • Unmanaged technical debt, remove capacity from the team, generate more pressure, generate more technical debt, etc.
  • Accidental complexity makes difficult to understand the code, so generate more bugs, generate more pressure for the team, generate poor solutions with more accidental complexity, etc.
  • A bad deployment process generates frustration, so we tend to larger batches that are riskier, so have more problems, generate even more frustration, etc.
  • etc.

When we have several of these vicious cycles, it is easier than it seems to fall into a downward spiral from which we cannot get out.

So, are you investing in breaking the vicious cycles of poor quality, high resource usage, and unmanaged technical debt? Or are you investing in improving your virtuous cycles of working in small batches, with ownership and high quality?

Are you improving, or are you dying and falling behind?


And if the problem is that you don't know what a high-performance technology organization should look like, you are lucky; we now have information on how it should be (Accelerate book).


Related:


Sunday, November 07, 2021

Good talks/podcasts (November 2021 I)



These are the best podcast/talks I've seen/listen to recently:

  • Debt Metaphor (Ward Cunningham) [Inspirational, Technical Practices, Technology Strategy, XP] [Duration: 0:05:00] (⭐⭐⭐⭐⭐) Ward Cunningham reflects on the history, motivation and common misunderstanding of the "debt metaphor" as motivation for refactoring.
  • EP 47: How to scale engineering processes w/ Twitter's VP of Engineering (Maria Gutierrez) [Engineering Career, Engineering Culture, leadership] A very interesting interview with Maria Gutierrez. Great lessons about team management, building a company culture, hiring, and mentorship.
  • Getting Started With Microservices (Dave Farley) [Architecture, Architecture patterns, Continuous Delivery] In this episode, a microservices tutorial, Dave Farley describes the microservices basics that help you to do a better job. He describes three different levels that we need to think about when designing a service and offers his advice on how to focus on the right parts of the problem to allow you to create better, more independent, services, based on Dave’s software engineering approach.
  • Industry Keynote: The DevOps Transformation (Jez Humble) [Agile, Continuous Delivery, Devops, Engineering Culture, leadership] (⭐⭐⭐⭐⭐) In this talk Jez will describe how to implement devops principles and practices, how to overcome typical obstacles, and the outcomes DevOps enables. A must-see talk.
  • Lunch & Learn How to Misuse DORA DevOps Metrics (Bryan Finster) [Devops, Engineering Culture, leadership] Interesting presentation in which bryan describes an agile/devops transformation, telling us about mistakes and successes. Interesting learnings, tips, and ideas.
Reminder, All these talks are interesting even just listening to them.

Related: 

    Wednesday, October 27, 2021

    "It depends" / The development Mix (Product, Engineering, Hygiene)

    We already know that is everything about context. I read a lot of blog posts talking about how much time a team should use for decoupling components, introducing a new cache system, or improving the scalability of their systems. When reading this type of content, I always think that they are completely right and completely wrong. Everything in our profession depends a lot on the context (the moment of the company, the business strategy, the market traction, etc.).

    I use a mental model that helps me classify the work we do, which allows me to communicate and make decisions. I call this mental model "The Mix".

    In "The Mix", I classify the work we do as product engineers in:
    • Normal product development.
    • Implementing the Engineering Roadmap.
    • Basic hygiene work.

    Normal product development

    Normal product development should be the most common type of work for a Stream Aligned team. It should help to fulfill the mission of the team. It can be composed of new feature development, discovery experiments, feature evolution, etc. I prefer a very lean approach for this work, following agile development methods such as XP or Lean Software Development. It is essential to generate the expected outcomes with the minimal amount of code possible and a good internal quality that minimizes the maintenance cost. Following YAGNI, KISS, Simple design is the perfect approach for this kind of work. We don't know the future. The most efficient way to work is to have the most simple solution that covers our customer's needs without making any "speculative" design that generates tons of accidental complexity in 99% of the cases.

    Summary:
    • Focus on outcomes for the customer within the business constraints.
    • Evolutionary design uses Simple design and avoids creating anything for future "expected/invented" needs.
    • Use a Lean approach (working in small safe steps).
    • Avoid solving problems that we don't have.
    • High-speed feedback loop.
    • Aligned with the Product Roadmap.

    Implementing the Engineering Roadmap

    In parallel to the product work, it is very common to identify engineering needs derived from the company's engineering strategy. This strategy should prepare and maintain the current and future engineering capability. Examples of this type of work are:
    • Designing the system for fast expected growth (the number of customers, engineering team size, etc.).
    • A technology stack change.
    • A change in the delivery strategy (from On-Prem to SaaS, from Web to mobile, etc.).
    • Prepare the architecture to enable work in autonomous teams.
    • This kind of work usually affects several Stream Aligned teams simultaneously and requires coordination at the engineering organization level.
    • These initiatives require a lot of investment and should be coordinated with the product roadmap and aligned with the company's general strategy.

    Summary:
    • Focus on outcomes for the internal architecture and engineering processes.
    • Require more upfront effort to design the solution.
    • It can be implemented with an agile approach but based on the initial design.
    • Low-speed feedback loop.
    • By definition, try to solve problems that we don't have (yet).
    • It is aligned with the Engineering Roadmap (coordinated with the Product Roadmap).

    Basic hygiene work

    To develop any nontrivial product, we need to have some practices and development infrastructure that I consider basic hygiene. I'm talking about having a reasonable test strategy, zero-downtime releases, good internal code quality, basic security practices, etc.
    In the middle of 2021, not considering these points above seems simply a lack of professionalism. 
    So the Basic hygiene work includes any effort we make to implement or improve these minimal practices.

    Of course, I am a big fan of product discovery with prototypes, and these, for example, do not have to have the same test strategy. But remember, a prototype that ends up in production, staying in front of our customers for months, is not a prototype. It is a trap.
     


    Using The Mix

    Thinking about these three types of work and separating them helps me be more explicit about the context and the appropriate trade-offs in each situation. For example, suppose we are in a Normal Product Development initiative. In that case, we cannot expect big architecture change decisions to emerge, and it is better to focus on small safe steps that add value. At the same time, we take notes to consider some initiatives to introduce in the engineering roadmap.

    A mature product organization will introduce performance, scalability, and availability initiatives into the product roadmap. In a less mature organization, those needs are likely to be missing from the product roadmap, and it is up to engineering to fight to get them into the engineering roadmap.

    We can summarize the different dimensions in this table:
     
    Product development Engineering Roadmap Hygiene
    Source
    Product
    Engineering
    Team (+Engineering)
    Development
    Small Safe Steps
    Upfront Planning + Small Safe Steps
    Small Safe Steps
    Practices
    YAGNI, KISS, TDD...
    Evolutionary Architecture, Load Testing, Testing in Production, migration planning...
    Clean Code, CI, CD, Zero downtime, Observability...
    Type of needs
    Current needs
    Future needs
    Prerequisite
    Value Delivery
    Very Fast
    Slow
    Very Fast
    Coordination Needs
    The team should be autonomous
    Coordination with other teams
    Coordination with other teams


    If we analyze the current trends in technology using this mental model, some questions arise:
    • How do technologies like PaaS or Serverless influence this Mix?
    • How does working in Cloud vs. working on-prem affect the engineering roadmap?
    • Does it make sense to consider ourselves good professionals if we don't have strong knowledge about hygiene factors?
    • How does the mix change in the different phases of the company (startup pre-product-market fit, scale-up, big tech)? And with the life cycle of the product?
    The interesting thing about mental models is that they help us think. I hope this model is as valuable for you as it is to me.

    Related / Other mental models

    Sunday, October 17, 2021

    Good talks/podcasts (October 2021 II)



    These are the best podcast/talks I've seen/listen to recently:

    • How conscious investors can turn up the heat and make companies change (Vinay Shandal) [Inspirational] [Duration: 0:13:00] In a talk that's equal parts funny and urgent, consultant Vinay Shandal shares stories of the world's top activist investors, showing how individuals and institutions can take a page from their playbook and put pressure on companies to drive positive change. "It's your right to have your money managed in line with your values," Shandal says. "Use your voice, and trust that it matters."
    • Software at Scale 13 - Emma Tang: ex Data Infrastructure Lead, Stripe (Emma Tang) [Big Data, Data Engineering, Operations, Platform, Technical Practices] [Duration: 0:41:00] (⭐⭐⭐⭐⭐) Effective Management of Big Data Platforms. Very interesting discussion about the technological and organizational challenges of maintaining big data platforms.
    • Improving Software Flow (Randy Shoup) [Agile, Continuous Delivery, Engineering Culture, Inspirational, Technical leadership] [Duration: 0:46:00] (⭐⭐⭐⭐⭐) Great presentation, in which Randy, starts from the 5 ideals of the Unicorn project (Locality and Simplicity, Focus, Flow, and Joy, Improvement of Daily Work, Psychological Safety, Customer Focus) to describe what we can do as technical leaders and as engineers to improve our ability to build and deliver software.
    • Developer Productivity with Utsav Shah (Utsav Shah) [Devex, Devops, Platform, Platform as a product] [Duration: 0:41:00] In this episode of Software Engineering Daily podcast, Utsav Shah talk about developer productivity in the context of the monolith, CI/CD, and best practices for growing teams.
    • Simplifying The Inventory Management Systems at the World’s Largest Retailer Using Functional Programming Principles (Scott Havens, Gene Kim) [Architecture, Architecture patterns, Functional, Technical leadership, Technology Strategy] [Duration: 2:02:00] (⭐⭐⭐⭐⭐) Havens shares his views on what makes great architecture great. He details what happened when an API call required 23 other synchronous procedures calls to return a correct answer. He discusses the challenges of managing inventory at Walmart, how one implements event sourcing patterns on that scale, and the functional programming principles that it depends upon. Lastly, he talks about how much category theory you need to know to do functional programming and considerations when creating code in complex systems. It is recommended to first watch the talk https://www.youtube.com/watch?v=n5S3hScE6dU or listen to the podcast https://itrevolution.com/the-idealcast-episode-22/
    Reminder, All these talks are interesting even just listening to them.

    Related: 

    Sunday, October 03, 2021

    Good talks/podcasts (October 2021 I)

     


    These are the best podcast/talks I've seen/listen to recently:

    • You're Testing WHAT? (Gojko Adzic) [Technical Practices, Testing in production, testing] [Duration: 0:38:00] Gojko presents five universal rules for test automation, that will help you bring continuous integration and testing to the darkest corners of your system. Learn how to wrestle large test suites into something easy to understand, maintain and evolve, at the same time increasing the value from your automated tests.
    • Using Observability to improve the Developer Experience (Borja Burgos) [Devex, Devops, Platform, Platform as a product] [Duration: 0:15:00] Observability is often associated with production and live environments, but it shouldn't be! In this talk we'll explore innovative ways in which modern observability tools and best practices can be leveraged during development to: improve developer productivity, identify regressions earlier in the SDLC, and increase the performance and reliability of our CI/CD workflows.
    • Continuous Delivery (Jez Humble) [Agile, Continuous Delivery, Engineering Culture, Lean Software Development] [Duration: 0:47:00] (⭐⭐⭐⭐⭐) Great 2012 presentation on Continuous Delivery. Jez discusses the value of CD to the business. He presents the principles and related practices, including value stream mapping, deployment pipelines, acceptance test-driven development, zero-downtime releases, etc. This talk is a while old, but still as relevant as the first day.
    • Test Driven Development with Geepaw Hill (Clare Sudbery, GeePaw Hill) [Agile, Technical Practices, tdd, testing] [Duration: 0:50:00] Clare talks to Geepaw Hill about why he loves TDD so much and how he spreads that love to software teams all over the world.
    • Software Delivery Glossary (Adam Hawkins) [Continuous Delivery, Lean, Lean Software Development] [Duration: 0:09:00] This podcast describes a few software delivery concepts (Lead time, Deployment frequency, MTTR, Change Failure Rate, Jidoka, Kaizen).
    • Churn FM. EP131 How the best leaders empower their product teams and set them up for success (Marty Cagan) [Inspirational, Lean Product Management, Product, Product Team] [Duration: 0:36:00] An interesting conversation about the tremendous gap between how the best companies operate and how the rest work, what they do differently, and why.  They also discussed how the best leaders empower their teams, how real product discovery and product work happens and then talked about how alignment is a consequence of a good product strategy.
    Reminder, All these talks are interesting even just listening to them.

    Related: 

    Sunday, September 26, 2021

    Code reviews (Synchronous and Asynchronous)

    There are different types of Code Reviews with different objectives. This article only refers to those Code Reviews that another team member does before incorporating the change in the shared code repository.

    The objective of these reviews is to improve the quality, avoid errors introduced in the shared code repository, and prevent issues from going out to production.

    Therefore the code reviews we are talking about:

    • Avoid errors/bugs by acting as a safeguard.
    • Occur within the flow of making changes to production.

    In this blog post, we will not discuss other types of Code Reviews that are done outside of the normal development flow.

    Most common: Asynchronous code reviews.

    In this context, a very common (and for some, recommended) way of performing code reviews is that a developer (A) works individually on a task (d) and upon completion of the general change, creates a Pull Request (PR)/ Merge Request (MR) that another person in the team must review asynchronously.

    We can see an example of this way of working in the following diagram. In it, we can see how developer A generates PR/MRs that developer B reviews, generating proposals for changes that developer A should incorporate.

    In the example, the feature/change is composed of increments d1, d2, d3, and we see that we deploy it to production at the end of the code review of increment d3.



    We can see that A's development flow is constantly interrupted to wait for the corresponding feedback for each code review. This way of working delays the release to production, increasing the time it takes to receive real feedback from customers.

    Because each individual code review takes so long, there is an unconscious and natural tendency for developers to work in increasing larger increments. At least, this is what I found in many teams I have worked with.

    It is a curious situation because, on the one hand, the industry says that working in small safe steps is a recommended practice. On the other hand, many companies use async reviews that tend to discourage them.

    So the most common way of working looks like the following diagram:


    Moreover, in this case, by generating large PRs/MRs, the code review loses meaning and, in many cases, becomes a pantomime (LGTM). (See: how we write/review code in big tech companies)
    With this last way of working, the lead time to production improves a little, but at the cost of taking bigger steps and losing the benefits of detailed code reviews.

    When working with asynchronous code reviews, the reality is usually more complex than what we have seen in the previous diagrams. The reality usually includes many developers, simultaneous code reviews and cross flows in which there is a strong tendency for individuals to work alone and do continuous context switching.




    In fact, there is no incentive for the team to focus and collaborate on a single user story at the same time since everyone would be busy with their tasks and doing asynchronous code reviews for each other.

    Another problem that often occurs is that code reviews do not have sufficient priority and generate long queues of pending reviews. These queues cause the lead time to grow and create deployments with larger batches of changes (more risk).

    Is there a better way?

    Fortunately, there are synchronous code reviews and continuous code reviews.

    Synchronous, high priority code reviews

    The first step is to give the highest priority to code reviews and to do them immediately and synchronously between the person who developed the increment and the person in charge of doing the review.

    This way of working reduces the total lead time of each increment and encourages working in small increments.

    As a side effect, the code reviews should be more straightforward because the developer themself explains them and can express in detail why they have taken each of the decisions.


    This way of working requires a lot of coordination between team members, and it is not easy to make them always synchronous, but of course, it is a simple step if we are already doing asynchronous code reviews.

    It is as simple as making PRs/MRs top priority, and as soon as you have one ready, coordinate with someone else to review it together on a single computer.

    Pairing/Ensemble programming: Continuous Code Review

    Pair programming: "All code to be sent into production is created by two people working together at a single computer. Pair programming increases software quality without impacting time to deliver. It is counter intuitive, but 2 people working at a single computer will add as much functionality as two working separately except that it will be much higher in quality. With increased quality comes big savings later in the project.". http://www.extremeprogramming.org/rules/pair.html
     
    When we work using pair programming, as recommended in extreme programming, two people design and develop the code. This way of working generates a continuous review of the code, so it no longer requires a final review phase. 

    Of course, from the point of view of coordination, it is the simplest system, since once the pairs have been formed, each pair organizes itself to make increments, which are implicitly reviewed continuously.

    In this case, the diagram representing this way of working would be: 



    A new way of working known as mob/ensemble programming has appeared as an evolution of the pair programming practice. In this modality, the whole team works simultaneously on a single task and using a single computer. 

    The whole team collaborates in the design and development, limiting the context switching and minimizing the lead time for the task. As in pair programming, mob/ensemble programming performs a continuous and synchronous review of the code.

    Of the ways of working that we have seen, pair or group programming has the shortest lead time for each change, fewer context switching interruptions, and requires less coordination.

    Even so, pair and group programming is not a simple practice and requires effort and a lot of practice. IMHO it is clear that this way of working takes the quality benefits usually associated with code reviews to the extreme.

    The industry vision, my personal vision

    The industry seems to agree that code reviews are a good practice and should be implemented by all development teams.
    On the other hand, it is also recommended in most cases to work in small increments so that the complexity and risk of each change can be better controlled.

    With these two points in mind, it seems a good idea to use Code Reviews and make them as synchronous and continuous as possible to work on those small increments of low complexity and low risk.

    In my case, I have been successfully pushing pair or mob programming for a few years now, creating a stream of small increments that are reviewed continuously.

    In both Alea Soluciones and TheMotion, we used pair programming by default and mob programming occasionally. Both companies' deployment frequency was daily, and we could make considerable changes in small and safe steps. In addition to this constant and sustainable development flow, we got attractive additional benefits such as the ease of spreading domain and technical knowledge or make fast onboarding of new members. In the case of Alea Soluciones, we hired people who already had experience in pairing and XP practices, so it was easier. In the case of TheMotion, however, we had to introduce some of the agile development practices (pairing, TDD, etc.), so we hired technical coaching help.
    In Nextail there was a mix of teams doing pairing and continuous reviews and others using asynchronous code reviews but prioritizing to avoid queues.
    At Clarity AI, some teams use asynchronous Code Reviews but prioritize them to minimize the lead time. There are already two teams doing continuous code reviews (using pairing or mob programming) and we are gradually expanding this way of working to other teams.

    Of course, the statistical value of my experience is zero, and it is much more interesting to analyze the results of more rigorous analyses such as DORA (https://www.devops-research.com/research.html). In them, we can see how the following practices are associated with high-performance teams:

    Related:

    Sunday, September 12, 2021

    Good talks/podcasts (September 2021 II)



    These are the best podcast/talks I've seen/listen to recently:

    • Chapter Two with Christina Wodtke, Author of Radical Focus (Christina Wodtke, Barry O'Reilly) [Company Culture, Culture, Engineering Culture, Inspirational] [Duration: 0:31:00] A great discussion to understand how to correctly introduce and use OKRs.
    • Day Two Cloud 111: Infrastructure As Software With Kris Nóva (Kris Nóva) [Cloud, Devops, Infrastructure, Platform, Platform as a product] [Duration: 0:53:00] thought-provoking podcast episode. Kris Nóva, claims that managing infrastructure using tools like Terraform isn’t that far away from just writing your own code to do the job yourself.
    • Rails Conf 2013 The Magic Tricks of Testing (Sandi Metz) [Agile, Technical Practices, XP, testing] [Duration: 0:32:00] (⭐⭐⭐⭐⭐) This talk strips away the veil and offers simple, practical guidelines for choosing what to test and how to test it. Finding the right testing balance isn't magic, it's a magic trick; come and learn the secret of writing stable tests that protect your application at the lowest possible cost.
    • Honeycomb's secret to high-performing teams (Charity Majors) [Continuous Delivery, Engineering Culture, Operations, Technical leadership] [Duration: 0:51:00] (⭐⭐⭐⭐⭐) Interesting conversation about high performance engineering organizations and how the pace of production is the heartbeat of the organization. They also discuss some very interesting ideas about how great teams generate great engineers, rather than the other way around.
    • Why There are Now So Many Shortages (It's Not COVID) [General, Lean] [Duration: 0:20:00] Interesting documentary on the impact on supply chains of the pandemic and how in many cases trying to use (without following the principles) lean ideas has generated fragility and lack of adaptability.
    • Software Engineering podcast: Episode 473: Mike Del Balso on Feature Stores (Mike Del Balso) [Data Engineering, Data Science, Machine Learning] [Duration: 0:55:00] (⭐⭐⭐⭐⭐) Mike Del Balso co-founder of Tecton discusses Feature Stores and how it helps operationalize Machine Learning.
    • WTF is Platform as a Product? (Matthew Skelton) [Devex, Platform, Platform as a product, team topologies] [Duration: 1:28:00] Matthew explains why the platform-as-product approach can be a game-changer for organisations building and running software-enabled products and services. Using ideas and patterns from Team Topologies—including Thinnest Viable Platform, team cognitive load, and the evolutionary team interaction modes—he explains how organisations like Adidas and Uswitch have successfully used the platform-as-product model to accelerate and simplify the delivery of software at scale.
    Reminder, All these talks are interesting even just listening to them, without seeing them.

    Related: 

    Sunday, September 05, 2021

    Good talks/podcasts (September 2021 I)

     


    These are the best podcast/talks I've seen/listen to recently:

    • Platform Engineering as a (Community) Service (Nicki Watt) [Devex, Platform, Platform as a product] [Duration: 0:39:00] Interesting talk about what is platform engineering, how to create a great developer experience.
    • Principles Of Microservices (Sam Newman) [Architecture, Architecture patterns, DDD, Microservices] [Duration: 0:56:00] This talk describe the microservices principles (modeled around business domain, a culture of automation, hide Implementation details, decentralize, deploy Independently, isolate failure, highly observable, etc).
    • Platform as a Product: How to Delight Your Developers and Deliver Value for Your Customers (Paula Kennedy, Colin Humphreys) [Architecture, Devex, Platform, Platform as a product] [Duration: 0:57:00] In this talk, Paula and Colin will provide detailed, practical steps on how to form a successful platform team, how to apply the Build-Measure-Learn feedback loop to your internal platform, and how to engage your application developers and delight them. By creating and operating a platform that meets their needs, your developers can focus on what's most important for them: delivering their products, delighting their customers, and driving business value.
    • Real Example of a Deployment Pipeline in the Fintech Industry (Dave Farley) [Continuous Delivery, Devex, Platform] [Duration: 0:18:00] Dave Farley shows a real deployment pipeline in-use in a Fintech company called TransFICC based in London. TransFICC adopted Continuous Delivery from the start and practice it at an advanced level. Delivering frequent, changes into production in a complex system, working in a regulated industry.
    • How To Build Quality Software Fast (Dave Farley) [Continuous Delivery, Engineering Culture] [Duration: 0:15:00] (⭐⭐⭐⭐⭐) In this episode, Dave Farley explores how we can move fast with high quality and how one reinforces the other. Speed and quality are both hallmarks of a Continuous Delivery approach and best practices for building great software.
    • Making developer effectiveness a reality (Rebecca Parsons, Tim Cochran, Keyur Govande, Pia Nilsson) [Devex, Platform as a product] [Duration: 0:41:00] There’s often lots of talk about how companies can make their developers more productive. But it may be more useful to think about developer effectiveness: how to ensure they’re building the most useful products for their customers. This is about providing an environment that allows developers to be effective. Our special guests from Spotify and Etsy give us their unique perspectives.
    • Small batches podcast: The Toyota Way (Adam Hawkins) [Lean] [Duration: 0:08:00] (⭐⭐⭐⭐⭐) A summary of 14 principles described in Jeffery Liker's 2021 book "The Toyota Way".
    • WorkLife podcast: Why it Pays to Raise Pay (Adam Grant) [Company Culture, Culture, Inspirational, Management] [Duration: 0:44:00] When employees are paid more, they give more. Going above market pay might sound like a fantasy, but in a growing number of companies it’s becoming a profitable reality. Peek inside workplaces that have reinvented their pay structures to give employees their worth and more—and explore the science of how it can pay off for everyone in the long run.
    Reminder, All these talks are interesting even just listening to them, without seeing them.

    Related: 

    Saturday, August 14, 2021

    Good talks/podcasts (August 2021 I)


     

    These are the best podcast/talks I've seen/listen to recently:

    • Top Software Engineering Interview Tips (Dave Farley) [Engineering Culture, professionalism] [Duration: 0:15:00] In this episode, Dave Farley, of Continuous Delivery, talks about his experience of technical interviews. Dave has conducted hundreds of software engineer job interviews and has some thoughts on how we could do better as both interviewers and as interviewees.
    • The reality of testing in an artificial world (Angie Jones) [AI, testing] [Duration: 0:27:00] Inspirational (and funny) talk about how to approach testing in a system using AI.
    • Theory of Constraints 3 Bottle Demo to improve Flow (Arrie van Niekerk) [Lean, TOC] [Duration: 0:06:00] This demo shows the strategies for dramatically improving flow using Theory of Constraints, using an experiment with a glass bottle to show the potential improvements that can be unlocked if the right strategy is applied at the bottleneck.
    • Output vs. Outcome & Impact (Jeff Patton) [Lean Product Management, Product, Product Discovery] [Duration: 0:10:00] Product thinking moves our focus from fast output to outcome and impact. This video is a great summary of this thinking.
    • One Step At A Time (Jason Gorman) [Technical Practices, XP, tdd] [Duration: 0:12:00] (⭐⭐⭐⭐⭐) Jason Gorman explains why focusing on one problem at a time is such a valuable habit for a software developer, and why the quickest route is usually the safest one
    • Reduce Alerting Noise with One Weird Trick (Liz Fong-Jones) [Observability, Technical Practices] [Duration: 0:10:00] (⭐⭐⭐⭐⭐) A great and concise description of SLI/SLOs and how to use them to improve our lives.
    • Domain Driven Design with BDD (Dave Farley) [DDD, Technical Practices, XP, tdd] [Duration: 0:16:00] Dave Farley explores how we can use this behavioural focus as a tool to better structure our software development and software engineering approach. How it can enhance our understanding, bridge gaps between different groups of people in the development process, better structure our development activities to focus more on the outcomes that we are aiming for and make the whole thing more testable.
    • GeePaw Hill "More Smaller Steps" (GeePaw Hill) [Agile, Lean Software Development, Small Safe Steps (3s), XP] [Duration: 1:20:00] (⭐⭐⭐⭐⭐) A great talk for anyone trying to do lean/agile software development. Explain why we need to give small safe steps (3s). Very interesting Q&A session at the end.
    Reminder, All these talks are interesting even just listening to them, without seeing them.

    Related: 

    Saturday, August 07, 2021

    Revisiones de código (Síncronas y Asíncronas)

    Actualización 30/8/2021: 
    Ampliada seccion "Visión de la industria y personal" y añadida referencias adicionales.


    ¿Qué es una Code Review?

    Aunque existen distintos tipos y con distintos objetivos en este articulo nos vamos a referir como Code Review a aquellas revisiones de un cambio de codigo/configuracion/esquema BD que solicitamos que nos haga otro compañero antes de incorporar el cambio en el repositorio común de código. El objetivo de estás revisiones es controlar la calidad e intentar evitar que se introduccan errores en el repositorio común de código y por tanto evitar que el cambio salga a producción.

    Por tanto las code review de las que hablamos:

    • Intentan evitar errores/bugs actuando como salvaguarda.
    • Se producen dentro del flujo para llevar cualquier cambio a producción.

    Existen otros tipos de code review que no se hacen en el flujo normal de desarrollo y que suelen usarse para aprendizaje compartido del equipo. En este articulo no las vamos a tratar.

    Lo más común: las Code reviews asíncronas

    En este contexto, una forma muy común (y para algunos recomendada) de realizar las code reviews, consiste en que un desarrollador (A) trabaja de forma individual en una tarea (d) y al terminar el cambio general una Pull Request / Merge Request que otra persona del equipo debe revisar asíncronamente.

    Podemos ver un ejemplo de esta forma de trabajar en el siguiente diagrama. En él se puede ver como el desarrollador A genera PR/MRs que el desarrollador B revisa, generando propuestas de cambios que el desarrollador A deberá incorporar.

    En el ejemplo, el desarrollo a realizar está compuesto por los incrementos d1, d2, d3, y vemos como es desplegado a producción al finalizar el la code review del incremento d3.


    Podemos ver que el flujo de desarrollo de A se interrumpe constantemente para esperar el feedback correspondiente a cada code review. Esto hace que el tiempo total que pasa hasta que hacemos una release a producción y conseguir feedback de verdad es muy largo.

    Con esta forma de trabajar, es probable que inconscientemente exista la tendencia a trabajar en incrementos más grandes, para evitar la espera por la code review. De hecho, esta tendencia a hacer pasos más grandes es lo que me he encontrado en muchos equipos con los que he trabajado. Es una situación curiosa, por que por un lado se transmite la necesidad de trabajar en pasos pequeños y por otro lado se usan prácticas que tienden a desincentivarlos.

    Por lo que la forma de trabajo más común se parece al siguiente diagrama:


    Además, en este caso, al generar PRs/MRs grandes, la code review, pierde sentido y en muchos casos se convierte en una pantomima (LGTM). (Ver: how we write/review code in big tech companies)

    Con esta última forma de trabajar se mejora un poco el lead time hasta poner en producción, pero a costa de dar pasos más grandes, y de perder los beneficios de las code reviews serias.

    Cuando se trabaja con code reviews asíncronas la realidad no suele ser tan sencilla como lo que hemos visto en los diagramas. La realidad suele incluir muchos flujos simultáneos y cruzados en el que existe una tendencia fuerte al trabajo individual y al cambio de contexto continuo.

    De hecho, no se genera ningún incentivo para que el equipo trabaje con foco en una sola historia de usuario al mismo tiempo, puesto que cada uno estaría ocupado con sus propias tareas y haciendo code reviews asíncronas para los demás.

    Otro problema que suele suceder es que las code reviews no tengan suficiente prioridad y se generen colas. Esto hace que el lead time crezca y que se desplieguen a producción cambios más grandes con el incremento de riesgos que ello conlleva.

    ¿No hay otras opciones?

    Por suerte sí que las hay, las code reviews síncronas y las code reviews continuas.

    Code review síncronas y con prioridad

    El primer paso puede ser dar la máxima prioridad a las code reviews y hacerlas de forma inmediata y síncrona, entre el que ha desarrollado el incremento y el encargado de hacer la review.

    Con esta forma de trabajar se consigue que el lead time total de cada incremento sea menor y se incentiva trabajar en pasos pequeños.

    En este caso, como efecto colateral, las code reviews deberían ser más fáciles, puesto que la explica el propio desarrollador y puede expresar con detalle por qué ha tomado cada una de las decisiones.

    Esta forma de trabajar requiere mucha coordinación entre la los miembros del equipo y no es sencillo hacerlas siempre síncronas, pero desde luego es un paso sencillo si ya estamos realizando code reviews asíncronas.

    Realmente es tan simple como que las PRs/MRs tengan la máxima prioridad y que en cuanto tengas una preparada, te coordines con alguien para revisarla los dos juntos en un único ordenador.

    Pairing/Ensemble programming: Code review continua

    Programación en parejas: "Todo el código que se envía a producción es creado por dos personas que trabajan juntas en un mismo ordenador. La programación en parejas aumenta la calidad del software sin que ello afecte al tiempo de entrega. Es contraintuitivo, pero dos personas trabajando en un mismo ordenador añadirán tanta funcionalidad como dos trabajando por separado, salvo que será de mucha mayor calidad. El aumento de la calidad supone un gran ahorro en la fase posterior del proyecto". http://www.extremeprogramming.org/rules/pair.html 


    Cuando trabajamos usando programación en parejas tal y como se recomienda en programación extrema, el código se diseña y desarrolla de forma conjunta por dos personas. Esta forma de trabajar genera una revisión constante del código por lo que no requiere una fase final de revisión. 

    Desde luego, desde el punto de vista de coordinación, es el sistema más sencillo, puesto que una vez que se han formado las parejas, cada pareja se organiza para ir haciendo incrementos, que de forma implícita van siendo revisados de forma continua.

    En este caso, el diagrama que representa esta forma de trabajo sería: 


    A partir de la programación en parejas y viendo sus beneficios, ha aparecido una nueva forma de trabajar conocida como programación en grupo (mob/ensemble programming). En esta modalidad todo el equipo trabaja al mismo tiempo en un único cambio y usando un único ordenador. De esta forma todo el equipo colabora en el diseño de cada pieza de código limitando al máximo los cambios de contexto y minimizando el lead time de cada uno de los cambios. Al igual que en la programación en parejas, la programación en grupo realiza una revisión continua y sincrona del código.

    De las formas de trabajar que hemos visto, la programación en pareja o en grupo, es la que tiene un menor lead time para cada cambio, menos cambios de contexto e interrupciones y requiere menos coordinación.

    Aun así la programación en parejas y en grupo no es una práctica sencilla y requiere esfuerzo y mucha práctica. Lo que sí tengo claro es que lleva al extremo los beneficios de calidad que se suelen asociar con las revisiones de código.

    Visión de la industria y personal

    Si en una cosa parece estar de acuerdo la industria es que hacer code reviews es una buena práctica y que deberían hacer todos los equipos de desarrollo. Por otro lado también se recomienda en la mayoría de los casos trabajar en incrementos pequeños, de forma que se pueda controlar mejor la complejidad y el riesgo de cada uno de los cambios.

    Teniendo en cuenta estos dos puntos, parece una buena idea usar Code Reviews y que estás sean lo más sincronas y continuas posible, de forma que podamos trabajar en esos incrementos pequeños de baja complejidad y bajo riesgo.

    En mi caso, llevo unos cuantos año empujando la programación en parejas o en grupo, creando un flujo de incrementos pequeños que se revisan de forma continua.

    Tanto en Alea Soluciones como en TheMotion, utilizábamos programación en parejas por defecto y programación en grupo ocasionalmente. La frecuencia de despliegue de ambas empresas era diaria, y podíamos hacer cambios considerables en pasos pequeños y seguros (3s). Además de este flujo de desarrollo constante y sostenible, obtuvimos atractivas ventajas adicionales, como la facilidad para difundir el dominio y el conocimiento técnico o hacer una rápida incorporación de nuevos miembros. En el caso de Alea Soluciones, contratamos a personas que ya tenían experician en programación en parejas y en prácticas XP, por lo que fue más fácil. Sin embargo, en el caso de TheMotion, tuvimos que introducir las prácticas XP (TDD, programación en parejas, etc.), por lo que contratamos ayuda de coaching técnico.

    En Nextail había una mezcla de equipos que hacían programación en pareja y revisión continua y otros que utilizaban revisiones de código asíncronas pero las priorizando para evitar las colas.

    En Clarity AI, algunos equipos utilizan revisiones de código asíncronas pero las priorizan para minimizar el lead time. Ya hay dos equipos que hacen revisiones de código continuas (utilizando la programación en parejas o en grupo) y estamos ampliando gradualmente esta forma de trabajar a otros equipos.

    Por supuesto el valor estadistico de mi experiencia es nulo y es mucho más interesante analizar los resultados de analisis más serios como el DORA (https://www.devops-research.com/research.html). En ellos podremos ver como se asocian a equipos de alto rendimiento las siguientes prácticas:


    Relacionado:

    Sunday, July 25, 2021

    Good talks/podcasts (Jul 2021 II)



    These are the best podcast/talks I've seen/listen to recently:

    • Benefits Of Going Beyond The Make-It-Work Phase (Francisco Climent) [Agile, Technical Practices, XP, tdd, testing] [Duration: 0:40:00] Interesting talk about the benefits of using TDD in embedded systems. The talk is full of useful tips, patterns, and strategies very useful in both embedded and non-embedded environments. Special mention to how Francisco uses mutation testing in the development process. There is a new version of the same talk at: ‘Unembedding’ embedded systems with TDD
    • DRY Software Patterns & Microservices (Dave Farley) [Software Design, Technical Practices] [Duration: 0:16:00] In this episode, Dave Farley of Continuous Delivery explores DRY, Coupling and Microservices and how they interact. Dave takes a pragmatic software engineering approach to exploring the pros and cons of DRY on different scales and describes why the microservices example may be more complicated than it looks.
    • Want More Value Faster? Take Many More Much Smaller Steps (GeePaw Hill) [Agile, Lean Software Development, Technical Practices, XP, tdd] [Duration: 0:55:00] (⭐⭐⭐⭐⭐) A must-talk for anyone trying to do lean/agile software development. The talk delves into the reasons why the strategy of using small, safe steps is the right one to steadily evolve a software system. I am very much aligned with this approach to software development.
    • The Trimodal Nature of Software Engineer Compensation: Why Positions Pay a (Very) Different Salar (Gergely Orosz) [Engineering Career, General, Management, Technical leadership] [Duration: 0:13:00] Interesting information about compensation /salary for software engineering positions in Europe mainly.
    • KEYNOTE Designing change (Jessica Kerr, Avdi Grimm) [Agile, Architecture, Evolutionary Design, Inspirational, Software Design] [Duration: 0:48:00] (⭐⭐⭐⭐⭐) The journey of a software developer is a climb through abstraction: algorithms, patterns, architecture.... How do we keep expanding scope, without losing focus on the real work? Join us for a journey into the fourth dimension, where we don't just change code; we design change.
    Reminder, All these talks are interesting even just listening to them, without seeing them.

    Related: 

    Saturday, July 03, 2021

    Good talks/podcasts (Jul 2021 I)

     

    These are the best podcast/talks I've seen/listen to recently:

    • Traditional vs Modern Internal Platforms: Find the Differences! (Manuel Pais) [Devex, Devops, Platform as a product, team topologies] [Duration: 1:04:00] A very interesting discussion about the key differences between the traditional approach to internal platforms and tooling vs the modern approach.
    • ProductTank New Zealand - The What & Why of Continuous Discovery (Teresa Torres) [Lean Product Management, Product, Product Discovery] [Duration: 1:00:00] the best product teams are shifting from a project mindset to a continuous mindset. In this talk, Teresa explores the key differences between project-based discovery and continuous discovery and give your team a clear benchmark to aspire to.
    • How to Build Products Users Love with Kevin Hale (How to Start a Startup 2014: Lecture 7) (Kevin Hale) [Product, Product Discovery, Product Strategy, startup] [Duration: 0:48:00] Kevin Hale, Founder of Wufoo and Partner at Y Combinator, explains how to build products that create a passionate user base invested in your startup's success.
    • A Guide To Managing Technical Teams (Dave Farley) [Engineering Culture, Management, Technical leadership] [Duration: 0:17:00] In this episode, Dave Farley of Continuous Delivery offers his advice on leading software development teams. How do you make the step from developer to manager? How do you deal with difficult situations and conversations? What are some of the commonest mistakes that people starting out in team leadership often make?
    • Are you ready for production? (Jaana Dogan) [Devops, Engineering Culture, Observability, Operations, Platform] [Duration: 0:35:00] This talk examines practices from design & development, configuration management, observability, release management, capacity planning, security, and compliance.
    • Foundations of Modern Product Organizations (Gerard Chiva) [Company Culture, Lean Product Management, Product, leadership] [Duration: 0:41:00] (⭐⭐⭐⭐⭐) This talk explains the keys to the success of digital product organizations. Technology is an essential part of any business today, not just a cost center.
    • In Search of the Perfect Cloud Native Developer Experience (Daniel Bryant) [Cloud, Devex, Devops, Engineering Culture, Platform, Platform as a product] [Duration: 0:45:00] Interesting talk to understand different approaches to achieve an effective development flow in a cloud-based environment. X explains interesting ideas to create an internal self-service development platform to reduce friction for stream-aligned teams.
    • Testing Strategy for DevOps (Dave Farley) [Devops, Engineering Culture, Testing in production, testing] [Duration: 0:14:00] (⭐⭐⭐⭐⭐) Dave Farley describes his take on what a holistic test-strategy looks like, and gives us examples from the real world to describe it.
    Reminder, All these talks are interesting even just listening to them, without seeing them.

    Related: 

    Monday, June 14, 2021

    Good talks/podcasts (Jun 2021 I)


    These are the best podcast/talks I've seen/listen to recently:

    • Test Driven Development vs Behavior Driven Development (Dave Farley) [Technical Practices, XP, testing] (⭐⭐⭐⭐⭐) In this episode on the Continuous Delivery channel, Dave Farley, who was involved in the creation of BDD, explains his slightly more unusual take on the real value of this approach to creating automated tests. Dave believes that TDD and BDD are at the foundations of a sound software engineering approach to software development and explores these ideas with some real-world examples of good and bad automated tests.
    • Make the Right Thing the Easy Thing: Designing Processes Teams Will Actually Follow (Jason Lengstorf) [Devex, Engineering Culture, Platform] In this talk, we’ll learn how IBM is working to get huge, distributed teams to not only agree on a set of best practices but to actually follow them
    • Making developer experience a reality (Tim Cochran, Keyur Govande, Pia Nilsson, Rebecca Parsons) [Devex, Engineering Culture, Platform, Platform as a product] There’s often lots of talk about how companies can make their developers more productive. But it may be more useful to think about developer effectiveness: how to ensure they’re building the most useful products for their customers. This is about providing an environment that allows developers to be effective. Our special guests from Spotify and Etsy give us their unique perspectives.
    • Observability in the SSC: Seeing Into Your Build System (Ben Hartshorne) [Devex, Devops, Observability, Platform, Platform as a product] Ben Hartshorne describes the transformation that Honeycomb went through, when they dropped build times by 40% and gave themselves the ability to track build times and asset sizes over time. Hartshorne covers the techniques one can use to accomplish the same goals in different environments.


    Reminder, All these talks are interesting even just listening to them, without seeing them.

    Related: 

    Sunday, May 16, 2021

    Good talks/podcasts (May 2021 I)

     

    These are the best podcast/talks I've seen/listen to recently:

    • 36 - Building High-Performing Teams with Observability and CI/CD (Charity Majors) [Agile, Continuous Delivery, Engineering Culture, Inspirational, Technical Practices, Technical leadership] A high-performing team is one that gets to spend almost all of their time solving interesting problems that move the business forward. Not doing a lot of toil. Not working on things they have to do in order to get to the things they want to do.
    • What Are Reactive Systems? (Dave Farley) [Architecture, Design, Microservices] Dave describes the use of Asynchronous messaging as the foundation of this approach and how a simple system built this way would be be fast, responsive, resilient
    • Improving Oncall Support with Meaningful Alarms (Brian Scanlan) [Architecture, Architecture patterns, Engineering Culture, Operations] Robby speaks with Brian Scanlan, Principal Systems Engineer at Intercom. They discuss the benefits of having consistent software patterns across an organization, how run books are a must-have (and why they should be tested), and optimizing for developer productivity. Brian also talks about approaches to reducing deployment time and why it is so vital to be responsive to their customers.
    • Ep. #4, Everything is an Experiment with Charity Majors and Liz Fong-Jones of Honeycomb (Charity Majors, Liz Fong-Jones) [Continuous Delivery, Engineering Culture, Operations] In episode 4 of Unintended Consequences, Yoz and Kim speak with Charity Majors and Liz Fong-Jones of Honeycomb about scaling teams and company values. They share how their team makes space for curiosity and learning as a group, and why this has been so important for the Honeycomb platform.


    Reminder, All these talks are interesting even just listening to them, without seeing them.

    Related: 

    How big is the backpack your organisation is holding

    Author: Jorge Jardines https://www.gardenunez.net/

    Every experiment, invalidated product feature or unused line of code you haven't deleted is adding weight to your backpack.

    Every morning when you wake up and need to walk the next X miles you feel the weight in your back and you feel slower than yesterday. It is like adding an extra bottle of water on your backpack, water you aren't gonna drink but still needs to hold on your back.

    Your team or organisation is holding this huge backpacks all the time. Do you ever wonder why adding a new experiment gets more and more slower and costly for your teams, do ever check how much code isn't being used, how many features are not being used I your product, how many "brilliant" ideas were transform into code that nobody used. Do you ever wonder how big is the backpack your organisation is holding.

    References:

    Monday, May 03, 2021

    API contract & Expand. (Small Safe Steps)

    In the latest Toughtworks tech radar (Vol 24 April 2021), API Contract & Expand appears as a technique to be adopted. I couldn't agree more.
    We often use complex versioning mechanisms even if a more straightforward API expand and contract strategy works perfectly for most cases.

    In general, API expand and contract should be enough for internal APIs. For external APIs, expand and contract should work for small changes, and API versioning is a better option for more significant changes.

    If you want to practice this and other strategies to reduce risk and slice technical changes, please check out this Small Safe Steps workshop.

    References:

    Saturday, May 01, 2021

    It's not only a waste; it's a burden

    Talking about software systems, let's stop to talk about waste when we refer to something not necessary or created without a clear understanding of the problem to solve. Let's call it a burden (since you are going to have it holding you back forever)...

    If you implement it to experiment, learn and later throw it away, perfect... If you are keeping it, you are fooling yourself. 

    This code is a burden, worse than the sunk cost, and it will generate a recurrent cost forever.


    There are two types of complexity, the inherent and the accidental/incidental.


    The inherent complexity can't be avoided because it is intrinsic to the problem we want to solve. But the accidental/incidental is all the complexity we introduce by not fully understanding the problem, not knowing how to solve it, or by our lack of mastery on the subject, etc.
     

    This accidental/incidental complexity is also a burden. The inherent one is not. :)

    Do yourself a favor, reduce the burden. reduce the basal cost. improve your life and the life of your team.

    Develop product in an agile way:
    1. Work in small steps
    2. Try to really understand the real problem and the meaningful impact you want to make
    3. Validate each increment and its impact
    4. Always work with quality-in and with attention to technical excellence
    5. Refactor continuously (to reduce the accidental/incidental complexity)
    6. And for God's sake, delete everything that is not necessary. Ruthlessly remove; code, dependencies, features, complexity… Minimize basal cost and make the evolution of the system easier.
    And follow the principles:
    • Continuous attention to technical excellence and good design enhances agility.
    • Simplicity --the art of maximizing the amount of work not done-- is essential.

    References: