Sunday, December 31, 2023

Low friction cli tools using docker

Although I last wrote about technical topics here a while ago, I would like to share some ideas that are helping me reduce friction in creating tools for my colleagues at ClarityAI.
As the Platform Lead, among other things, I am involved in the development experience, which means that my team creates tooling (as part of our platform) for the rest of the engineering and data science teams in the company. This tooling includes command-line tools that help users use our platform with minimal friction and autonomously (without requiring support).
In our case, this means we want command-line tools with the following characteristics:

  • Usable on Linux and Mac
  • Low friction for usage (easy installation and usage)
  • Easy to evolve
  • Easy to distribute to all our users
  • Few external dependencies so that our users don't have to install additional tools

In this context, and considering that we are already heavy users of Docker, it is a good idea to implement these types of tools as Docker images that our users can use.
This would allow us to include any external tools our users may need in the Docker images and eliminate friction by not requiring them to install those tools.

At the same time, using Docker presented its own challenges:

  • Difficulty in distributing a Docker image from a private repository
  • Difficulty in usage for execution requiring many parameters (volumes, environment variables, execution parameters and arguments, etc.)

Below, I will explain how we have solved (or are in the process of solving) each of the mentioned challenges.

Controlled Environment

Any modern development platform is implemented using different technologies and solutions.
This means the user must use different clients for each of these technologies (kubectl, AWS CLI, Git, Teleport, etc.).
To eliminate friction, avoid installation problems, and reduce the number of potential issues due to misuse or using incorrect versions, what has worked very well for us is to include all these tools (with controlled versions and environment) in a Docker image along with code to control their usage and configuration.
This allows the platform team to forget about problems caused by using the wrong version of kubectl or incorrect AWS client configuration.

Installation

If the installation process of a tool is complicated, you can be sure that a significant percentage of users will encounter problems and require support.
It is often assumed that good documentation is sufficient, but no matter how good the documentation is, if there are many steps to follow and many tools to install, there will be problems.
A better solution is to create a specific installer without configuration and with the fewest possible dependencies.
In our case, the tool to be distributed was built with Docker and stored in a private repository (AWS ECR).
As a lean team, we have iteratively improved the solution, reducing friction at each step:

  1. Installation instructions are documented in the engineering handbook. It depends on Docker, properly configured AWS CLI, and knowledge of ECR.
  2. Installer script in the code repository. It improved the experience and generated a helper script for future usage. It depends on Docker and properly configured AWS CLI but does not require knowledge of ECR.
  3. Cross-platform installer implemented with Golang. This is the version currently in development. It only requires Docker, and the user has an AWS Key. (Example: ECR Docker image puller / installer)

Execution

When we create an open-source solution or a product that will be used in different environments, the ability to configure and adapt it to different environments is crucial.
But when we develop a tool for our colleagues and our goal is to minimize friction and reduce cognitive load, we need to provide the minimum necessary flexibility.
We can even avoid the need for any parameters. 
This becomes a challenge when you have a tool built with Docker that requires multiple credentials and the ability to modify files on the user's machine, as it requires running Docker with many parameters and environment variables.
Once again, iteration has allowed us to improve the solution. The steps we have taken are:
  1. Document all the parameters and environment variables for tool execution in the README. This serves us well for development, but we never intended for our users to use the tool following this documentation.
  2. Using an execution script that hides all the parameters and variables. In our case, this execution script was generated by the installer.
  3. Cross-platform wrapper implemented in Golang. This allows us to distribute a binary that only depends on having Docker and hides the tool's complexity. This is the version we will distribute soon. (Example: Simple wrapper to execute (interactive/non interactive) docker image)

Continuous Updating

In addition to facilitating installation, it is essential that the platform team can quickly push new versions. In this case, having Docker already facilitates this because simply uploading new versions to the Docker repository makes them available to our users.
On the other hand, to streamline the update process, what has worked well for us is to include an option in the execution script/wrapper to check and download new versions.
Initially, we made it so that each execution would check for new versions, but we found that it caused too much delay during startup, so now we prefer to let the user decide when to update.


Conclusions:

  • Docker allows packaging applications with specific versions in a controlled environment.
  • Creating a cross-platform wrapper in Go is an excellent solution to eliminate friction in the distribution and execution of tools implemented using Docker.
  • Investing in making installation and updating simple allows tools to evolve rapidly without generating much friction.


Wednesday, November 15, 2023

Updated My Personal Mission. Same Essence, Different Words

 

Eduardo Ferro's Personal Mission

Cultivate Happiness: Foster personal joy and spread happiness to those around me, starting with family and friends, while continuously expanding my circle of influence.

Methods of Achievement

  • Responsibility and Respect: Uphold personal accountability and respect for others in all interactions.
  • Balance Seeking: Strive for a harmonious balance in personal life and contribute towards global equilibrium.
  • Authenticity: Remain true to myself in every aspect of life, including professional pursuits.
  • Value Generation: Utilize my abilities to create both intrinsic and economic value, mindful of social and ethical considerations.
  • Win-Win Scenarios: Continually seek outcomes that are mutually beneficial.
  • Continuous Learning: Embrace lifelong learning, both personally and professionally.
  • Conscious Decision-Making: Choose my reactions and actions thoughtfully.

Ethical Stance

Selective Engagement: Refrain from involvement with businesses that conflict with my values, including:
  • Financial enterprises that prioritize profit over social and environmental responsibility.
  • Weapon manufacturers and the military industry.
  • Casinos and gambling enterprises.
  • Entities involved in the abuse or exploitation of humans, animals, or the environment.
  • Businesses that negatively impact the lives of stakeholders.

Vision for the Future

I believe we are at a pivotal moment where our actions can significantly impact the planet, both positively and negatively. It's crucial to remember this responsibility at all times. We are amidst a revolution that blends collaboration, knowledge, and a federation of power. Networks are becoming the structures of the future, and their growth should be both sustained and sustainable.

Professional Alignment

My focus is on engaging with green and teal organizations, where people are valued as the primary asset. I am drawn to environments that prioritize sustainable and ethical practices, recognizing the importance of human capital in driving positive change.

Personal Commitment

I engage with the world passionately and responsibly, yet always at a measured pace, ensuring a balanced approach in all my endeavors.

Sunday, November 12, 2023

Learning lean/Agile concepts / short videos

 


Yesterday, my colleague Cristina Verdi asked for interesting resources to help introduce Lean Software / Product Development concepts.
I passed her a series of short videos that I always use to illustrate certain related concepts. I'm posting them here in case they can help someone.

Resource Efficiency vs Flow Efficiency:

Wednesday, November 01, 2023

Harnessing Efficiency: The Theory of Constraints in Software Development



This micro post was previously published at linkedin

Navigating the multi-faceted domain of software development often presents a series of bottlenecks that could hinder project momentum and delivery timelines. The Theory of Constraints (TOC) serves as a beacon, guiding teams to identify, address, and overcome these bottlenecks, thereby unlocking a pathway to streamlined processes and enhanced productivity.

Here's a snapshot of how TOC unfolds in software development:
  1.  Identify the most important Constraint: Pinpoint the process, resource, or technology bottleneck obstructing progress (lack of quality, knowledge silos, convoluted deployment process, individual ownership about parts of the code or processes, etc.).
  2. Exploit the Constraint: Maximize the efficiency of the identified constraint without additional resources (reducing the WIP, redirecting people to help with the bottleneck, etc.).
  3. Subordinate Everything Else to the Constraint: Ensure all other processes are aligned to support and work around the constraint.
  4. Elevate the Constraint: Allocate necessary resources to alleviate or eliminate the constraint, promoting better throughput (investing in automation, test automation, pair/ensemble programming to spread knowledge, feature toggles to reduce risk, etc.).
  5. Repeat the Process: Embark on a cycle of continuous identification and resolution of constraints to foster a culture of ongoing improvement.

For a deeper dive into TOC and its application in IT landscapes, 'The Goal' by Eliyahu M. Goldratt and 'The Phoenix Project' by Gene KimKevin Behr, and George. Spafford are essential reads.

Using the Theory of Constraints (TOC) can help software development professionals and teams deal with problems smartly. It helps turn these problems into opportunities for improvement, growth and faster delivery.

Indeed, the Theory of Constraints relates to Lean Software Development and agile software development methodologies.

The lightning talk "Stop Starting and Start Finishing" by Jason Yip is one of the best explanations I know for understanding how to use TOC and other Lean ideas to optimize the sustained delivery flow of a software development team.

#TheoryOfConstraints #SoftwareDevelopment #ContinuousImprovement #TheGoal #LeanSoftwareDevelopment

The virtuous loop of software development



This micro post was previously published at linkedin

The ultimate aim is Continuous Delivery (CD), a goal that enables fast flow with rapid iterations and continuous feedback. At the same time, this goal promotes technical excellence and good design.

Continuous Integration (CI) is required (integrating at least once a day, also known as Trunk Based Development), along with a strong focus on Test-Driven Development (TDD) or other similar practices to ensure high confidence and emphasis on excellent and simple design. This approach is closely linked to Extreme Programming and the DevOps mindset, which emphasizes collaboration and continuous improvement. By following these principles, software development teams can enhance their efficiency and deliver high-quality products to customers.


Here are some related resources that you might find interesting:


Monday, October 16, 2023

Good talks/podcasts (Oct 2023 I)


These are the best podcasts/talks I've seen/listened to recently:
  • The Elegant Solution: Toyota's Formula for Mastering Innovation (Matthew May) [Lean, Lean Manufacturing] [Duration: 1:15:00] Interesting presentation on how Toyota incorporates continuous learning and innovation into the company's daily operations.
  • "Simple Made Easy" (12-minute redux) by Rich Hickey (2011) (Rich Hickey) [Architecture, Inspirational, Scalability, Software Design] [Duration: 0:12:00] (⭐⭐⭐⭐⭐) This is a 12-minute redux of the 1-hour talk by Rich Hickey, for really impatient people. Original: https://www.youtube.com/watch?v=SxdOUGdseq4
  • Infrastructure as actual Code • YOW! 2022 (Gregor Hohpe) [Architecture, Architecture patterns, Infrastructure, Serverless] [Duration: 0:59:00] This talk shares recent trends in infrastructure automation, debunks some common misconceptions, and shows you how you can combine AWS’ serverless ecosystem and AWS CDK to rethink application development, deployment, and integration.
  • Lean Turned Up to 10 (Chris Lucian, Austin Chadwick) [Lean Software Development, MobProgramming, Technical Practices] [Duration: 0:58:00] (⭐⭐⭐⭐⭐) Chris and Austin explain why mob programming could be a way to implement lean software development in current teams. Very interesting.
  • LKUK13: Cynefin in Action - Liz Keogh (Liz Keogh) [Culture, Engineering Culture, Inspirational] [Duration: 0:51:00] In this talk Liz explains the Cynefin framework and other concepts as Real Options, Deliberate Discovery, Feature Injection, etc.
Reminder: All of these talks are interesting, even just listening to them.

Related:

Sunday, October 15, 2023

The Art of Small Steps in Software Development: A Lean Vision

Have you ever played Tetris? If so, you know that every decision, every piece, and every move counts. The same applies to software development. Every choice, every line of code, and every interaction influences the final outcome. This post will explore how Lean software development and the "safe small steps" approach can simplify complexity and create efficient systems.

Small Steps: The Simplicity of Tetris

Just like in Tetris, where we choose where to place small pieces to complete lines, in software development, we continuously make decisions about how to build our product. These decisions, when made in small increments, allow us to:

  • Iterate quickly: Fixes and adjustments are simpler.
  • Reduce risks: We limit potential errors at each stage.
  • Understand better: Each step is understood and analyzed more clearly.

Principles of Lean Software Development

Based on Lean philosophy, Lean software development focuses on maximizing customer value and minimizing waste. Some of the key principles include:

  • Reduce waste: Eliminate the unnecessary and focus on the essentials. Every step must add value or increase the internal or external quality of the product.
  • Decide as late as possible: Be flexible and adapt to changes. See Lean Software development: The art of postponing decisions.
  • Deliver quickly: Allows for feedback and adaptation to customer needs.
  • Build with integrity and quality: Ensures a reliable and long-lasting product. This is key to maintaining speed sustainably over time.
  • Proactively simplify: Avoid unnecessary complexity and keep the system manageable. Simplification also adds value, protecting the current value and avoiding future costs.

Small Steps: The Simplicity and Continuity of Tetris

Tetris, one of the most iconic video games in history, presents us with a continuous and infinite challenge. In it, pieces of different shapes fall from the top of the screen at an increasing speed. Our task is to accommodate these pieces efficiently at the bottom, trying to fit them so that they complete horizontal lines. Completing these lines eliminates them, granting us points and, more importantly, space to continue playing.

In Tetris, value accumulates by adding points, piece by piece, line by line. However, the real challenge and essence of the game lie in balancing the accumulation of that value (points) while managing increasing complexity (the accumulated pieces). If we allow the pieces to pile up and reach the top of the screen, we lose the game.

Similarly, in software development, we continuously decide how to build our product. Like in Tetris, we must accumulate value (features and characteristics) while managing and keeping complexity at bay. These decisions, when made in small increments or "steps," allow us to:

  • Iterate quickly: Fixes and adjustments are simpler.
  • Reduce risks: We limit potential errors at each stage.
  • Understand better: Each step is understood and analyzed more clearly.

Tetris / Larger vs Smaller pieces

Just like playing with large pieces in Tetris, working with large steps in software development makes the game/work much more difficult and stressful.

Conclusion

Software development is not just about writing code; it's about making informed and continuous decisions to create valuable products. Just like in Tetris, every piece, every step, counts. Embrace the Lean philosophy and focus on safe small steps to simplify, adapt, and deliver with quality.

If this approach has intrigued you, I invite you to delve deeper into Lean software development and experience how it can transform your development process.

Related:

Friday, October 06, 2023

DevOpsDays Madrid: The Complexity Trap Reevaluating our incentives

 I've had the pleasure of being invited to give a Keynote at the DevOpsDays Madrid event.


I'm leaving the slides and some interesting related references here.

The Complexity Trap Reevaluating our incentives


Video (Spanish):

Youtube video

Thanks to Autentia for the recording of the session.
You can watch the sessions of the conference and other conferences on their channel https://www.youtube.com/@AutentiaMedia 


Slides:

Original document

Related Quotes:

  • "Art is the elimination of the unnecessary." Pablo Picasso
  • "Code is cost, and production code is by far the higher cost.  If you can write more support code (tests, automation, developer tooling) in order to write less&better production code more smoothly, Win!" @jessitron
  • "Code is liability. To build something is to be responsible for its existence, forever. Write more software only as a last resort." @mipsytipsy
  • "Complexity is what makes software hard to change. That, and duplication." Ralph Johnson
  • "Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it." - Alan Perlis
  • "If you don’t actively fight for simplicity in software, complexity will win…and it will suck." Henrik Joreteg
  • "irreversibility was one of the prime drivers of complexity" Enrico Zaninotto
  • "Keep it simple, make it valuable, build it piece by piece" Ron Jeffries
  • "My position is that code, pretty much exclusively, is cost. It’s all liability. The asset is what it does. So, if I can get simpler code that does the same thing, I’ll take the simpler code. If I can have no code and get the same capability, I’ll take no code. No code is better than some code, some code is better than lots of code!" Dan North @tastapod
  • "Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better." Edsger W. Dijkstra
  • "Simplicity is the ultimate sophistication"  Leonardo da Vinci
  • "Simplicity--the art of maximizing the amount  of work not done--is essential." Principles behind the Agile Manifesto
  • "Software maintenance is not "keep it working like before." It is "keep being useful in a changing world"" Jessitron
  • "The art of programming is the art of organizing complexity." Edsger W. Dijkstra
  • "The biggest cause of failure in software-intensive systems is not technical failure; it’s building the wrong thing." Mary Poppendieck
  • "The faster you build crap the more crap you get" Jeff Patton
  • "The old truths: Keep it simple, Make it small, Make it correct, Fight complexity" Joe Armstrong

Related / References:

  • The art of destroying software (Greg Young)
  • Simplicidad para desarrolladores (Spanish)
  • Basal Cost of Software
  • 8 lines of code
  • Complexity Is Outside the Code 
  • Simplicity, The Way of the Unusual Architect 
  • Simple-Made-Easy/

  • Saturday, August 26, 2023

    Good talks/podcasts (Ago 2023 I)

     

    These are the best podcasts/talks I've seen/listened to recently:
    • Simon Sinek Performance vs Trust (Simon Sinek) [Company Culture, Culture, Inspirational] [Duration: 2:21] (⭐⭐⭐⭐⭐) Great description of the impact of trust on team members and leaders.
    • CONSTANT Changes To User Requirements Drive Me CRAZY (Dave Farley) [Agile, Continuous Delivery, Inspirational, Lean Product Management, Lean Software Development] [Duration: 0:13:00] (⭐⭐⭐⭐⭐) This presentation by Dave Farley shows that software development is not just about translating perfect requirements into code, but rather a process of discovery and exploration. It acknowledges that the nature of the problems being solved has changed and that it is impossible to have all the answers. It emphasizes that successful software products must be able to adapt and evolve over time, and that the key to success is embracing change and making it easy, safe, and low-cost.
    • Tips For Technical Startup Founders | Startup School (Diana Hu) [Inspirational, Lean Software Development, Lean Startup, startup] [Duration: 0:28:00] (⭐⭐⭐⭐⭐) Diana Hu shares her advice for being a technical founder at the earliest stages - including topics like how to ship an MVP fast, how to deal with technology choices and technical debt, and how and when to hire an engineering team.
    • Scaling Shape Up Beyond Bootstrapped Companies (Ryan Singer) [Product, Product Leadership, Product Strategy] [Duration: 0:47:00] Interesting talk about the ShapeUp product development methodology and how it can be used in different contexts. Very interesting ideas even if you don't use ShapeUp.
    • AI Impacts in Development Practice (Michael Feathers) [AI, Devex] [Duration: 0:33:00] Interesting reflections on how the profession of software developer might evolve with the introduction of AI (LLMs, GenAI). Among other topics, Michael talks about the importance of quality in this environment, the adaptation of practices like TDD, and the need to specify precise requirements, etc.
    • FAST '23 - Building and Operating a Pretty Big Storage System (My Adventures in Amazon S3) (Andy Warfield) [Engineering Culture, Engineering Scalability, Scalability] [Duration: 0:53:00] he perspective of evolution, design, and team organization in the context of rapid growth and at a scale hard to imagine.
    • Minimum Viable Architecture (Randy Shoup) [Architecture, Architecture patterns, Evolutionary Architecture] [Duration: 0:47:00] Presentation on 'Minimal Viable Architecture', discussing its evolution alongside a product's lifecycle phases: idea, start, scale, and optimize. Each phase examines business goals, constraints, and suggests corresponding software architecture.
    • Artificial Intelligence seen from the software development lifecycle perspective (Nerea Luis) [AI, MLOps] [Duration: 0:54:00] (⭐⭐⭐⭐⭐) Great introduction to the differences between traditional software development and the development cycle with AI models. Nerea introduces concepts such as Continuous training, model deployment, MLOps, and collaboration between data scientists and software engineers. Highly recommended for software engineers looking to delve into these topics and collaborate more closely on AI-based feature development.
    Reminder: All of these talks are interesting, even just listening to them.

    Related:

    Tuesday, August 15, 2023

    Why Lean Product Development is Essential for Effective Product Teams

     In today's fast-paced and competitive business landscape, it is crucial for companies to have effective product teams that can deliver high-impact solutions. However, many product teams often struggle to achieve this goal due to various challenges and constraints. In this blog post, we will explore the importance of lean product development from the perspective of engineering and how it can enable effective product teams. We will also discuss the role of system architecture in facilitating product delivery&discovery and highlight some common pitfalls that hinder the success of product teams.

    The Risks of Product Development

    Before diving into the details, let's first understand the risks associated with product development. According to a blog post by Marty Cagan, there are four major risks: value, usability, feasibility, and business viability.

    • Value: Will customers buy or use our software? This risk pertains to whether the customer sees value in the product and is willing to adopt it.
    • Usability: Can users figure out how to use the software? This risk focuses on the user experience and whether the software is intuitive and easy to use.
    • Feasibility: Can our engineers build the software within the given constraints? This risk considers the technical feasibility of implementing the desired features.
    • Business Viability: Does the solution align with the overall business strategy and goals? This risk examines whether the solution is viable from a business perspective.

    It is important to address these risks to ensure the success of a product. However, many product teams often overlook or underestimate these risks, leading to failures and wasted resources.

    Learning from Industry Leaders

    To emphasize the significance of lean product development, let's take a look at some insights from industry leaders. Microsoft conducted research on their own product development process and found that only one-third of their ideas actually improved metrics. Similarly, Amazon's success rate was below half of their ideas. These findings highlight the importance of validating ideas and the need to filter out ineffective ones.

    It is crucial to recognize that even highly successful companies like Microsoft and Amazon face challenges in generating impactful ideas. Therefore, assuming that all ideas are great without proper validation can be detrimental to a product team's success.

    Not only are many of the ideas not positive, but also the ones that are, require several iterations (with real feedback) to begin generating positive impact.




    The Vicious Cycle of Software Development

    To understand why lean product development is essential, we need to examine the vicious cycle of software development. In this cycle, a product team has a fixed capacity to develop software. While some features may generate positive impact, others may go unused or fail to meet user expectations. Additionally, the software incurs maintenance costs, such as debugging, monitoring, and infrastructure support (See: Basal Cost of Software).

    Over time, the accumulation of software and its associated maintenance costs can deplete the team's capacity for innovation. This cycle hinders the team's ability to focus on high-impact initiatives and leads to a decrease in overall productivity.

    Furthermore, the cost of maintaining software is not linear with its size. Doubling the size of an application does not double the maintenance cost; it increases it even more. This exponential increase in maintenance costs further limits the team's capacity for innovation.

    Viewing Software as a Means to an End

    To break free from the vicious cycle of software development, it is crucial to view software as a means to an end rather than the end itself. Software should be seen as a liability and an expensive asset that should be minimized. The focus should be on maximizing outcomes, impact, and return on investment while minimizing functionality, requirements, and maintenance costs.

    By adopting this mindset, product teams can prioritize high-impact initiatives and avoid wasting resources on unnecessary features. This shift in perspective enables teams to make better decisions and deliver more value to customers.

    This way of thinking also helps protect against the Feature creep that many products suffer from.

    Effective Product Teams and Lean Product Development

    To achieve high performance, product teams need to embrace effective lean product practices. These practices involve building the right thing and making the thing right.
    Building the right thing involves minimizing the amount of software while maximizing the impact. It requires filtering out unnecessary ideas and focusing on initiatives that align with the overall business strategy. This approach ensures that the team's efforts are directed towards high-impact solutions.
    Making the thing right involves continuous learning, hypothesis experiments, and user stories. It requires a collaborative team effort, with everyone involved in the product discovery&delivery process. By working together, teams can validate assumptions, reduce risks, and deliver elegant and impactful solutions.

    This process of continuous learning, filtering ideas, continuously experimenting, and delivering in small, safe steps that allow us to learn (through feedback), minimizing waste as much as possible, is what we would call Lean Product Development.



    The Role of System Architecture

    System architecture plays a crucial role in facilitating lean product delivery. It should be designed to optimize learning, experimentation, and evolution. Here are some key considerations for system architecture:
    • Decoupling Release and Deployment: To enable business-driven decisions, release should be decoupled from deployment. Feature flags can be used to control the release of new features and enable experimentation. This ability to release to a group of users independently enables the team to progressively validate hypotheses with different customer groups, thus managing the risk assumed in each case.
    • Product Instrumentation: The system should be instrumented to gather relevant metrics and operational data. This data provides valuable insights into user behavior and helps in making informed decisions. Assuming that a team can be autonomous and make good decisions when flying blind and without instruments makes no sense. Product-level instrumentation and observability are not a luxury, but an essential part of any effective product team.
    • Operational Data Exploration: It is essential to have systems in place to explore operational data effectively. This ensures that the collected data is utilized to its full potential and helps in identifying patterns and trends. This point complements the previous one; having good instrumentation is useless if it is not easy to exploit for making product decisions or learning about our users' behavior.
    • Safe-to-Learn Environment: The system should provide a safe environment for experimentation and learning. A blameless culture should be fostered, where failures are seen as opportunities for improvement rather than as a cause for blame.
    • Modular/Decoupled Architecture: A modular architecture enables multiple product teams to work autonomously on different parts of the system. It reduces the blast radius of failures and allows for more efficient development and maintenance. This architecture should also allow the team to easily compose new applications, proof of concepts, and demos based on the available modules and components. When the design quality is good, it is much easier to reuse components for experiments and as a basis for new use cases.
    By incorporating these architectural considerations, product teams can create an environment that enables continuous evolution, learning, and experimentation. Ultimately, it all comes down to having an adaptable system that we can trust and that is optimized for making small, low-risk changes at high speed.




    Depending on the type of system, we can even include architecture decisions that specifically help us experiment and learn more quickly. For example, if it is a product that can be considered a platform, having all capabilities and use cases accessible through APIs can allow us to experiment and create prototypes very quickly using those endpoints from Low Code or No Code systems.

    Overcoming Challenges and Maximizing Outcomes

    To build effective product teams, it is essential to address common challenges and pitfalls. Here are some key points to consider:
    • Autonomy and Decision-Making: Product teams should have the autonomy to make decisions and generate ideas. Stakeholders should trust the expertise of engineers and involve them in the product discovery process.
    • Technical Debt and Stability: Addressing technical debt and ensuring system stability are crucial for successful product delivery. Without a stable foundation, teams will struggle to innovate and deliver high-impact solutions.
    • Focus on Outcomes: Engineers should shift their focus from technology to outcomes. The value and impact delivered to customers should be the primary concern, rather than the features or technology used.
    • Continuous Learning and Improvement: Product teams should embrace a culture of continuous learning and improvement. This involves validating assumptions, running experiments, and constantly seeking feedback from customers and the system.
    • Collaboration and Ownership: Effective product teams require collaboration and shared ownership. Product managers, designers, and engineers should work together as a cohesive unit, leveraging their respective expertise to deliver high-impact solutions.
    • Praise Continuous Impact and Learning: To generate the appropriate dynamics within the team, we must celebrate the impacts we achieve (changes in user behavior, metric improvements, cost reductions, etc.) and not so much the functionalities we make available (which would only be the means). It is also essential that we celebrate the learnings since they allow us to make better decisions and increase future impact.





    Conclusion

    In conclusion, lean product development is a critical component of effective product teams. By prioritizing product discovery&delivery and incorporating it into system architecture and practices, teams can maximize outcomes, minimize waste, and deliver high-impact solutions. It is essential for engineers to embrace a product mindset and actively participate in all aspect of the product development process (discovery, delivery). By doing so, they can contribute to innovation and drive the success of their product teams.

    If you want to learn more about product discovery&delivery and effective product teams, I recommend reading the following books:
    Or if you prefer talks, I recommend these ones:
    Remember, product development and continuous delivery go hand in hand. To achieve success, product teams must embrace both and strive for continuous learning and improvement.

    References and related content



    Wednesday, August 09, 2023

    Engineering & Product relationship: random notes

    These notes come from my preparation for a panel on technology and product to which I was invited (https://lu.ma/tech-sherpas-madrid). I am publishing them here in case they contribute or help to initiate some conversation. Of course, the panel was great, and it included responses from Isaura Fontcuberta and Rebeca Francisco Najarro that contributed more than my own, but for obvious reasons this post only includes mine since they are the ones I have notes on. The panel was very well conducted by Pedro Pablo Aparicio



    Q&A Notes

    Let's start, should a product company have two Product and Technology Roadmaps, or should it have a single roadmap?

    Ideally a single balanced roadmap. The problem is that in my experience this has many implications. Assuming that there are two different reporting lines (something that we should be able to question), there tends to be an imbalance. By instilling product sensitivity within engineering, this imbalance can be mitigated, making engineering attuned to these sensitivities without seeing them as antagonistic. On a small scale, what has worked best for me is that this division does not exist at all. :) If, out of necessity, there are two roadmaps, there must be clear rules for prioritizing (percentages, for example). (See: https://www.eferro.net/2021/10/it-depends-development-mix-product.html)


    Continuing with the topic, taking advantage of the fact that we are talking about RoadMap, who is responsible in the case that there is 1 or 2 roadmaps….

    If two reporting lines exist, it becomes a matter of negotiation, although I question the necessity for dual lines in all situations. The final responsibility can vary depending on several factors such as the nature of the product (be it business digitalization, information systems, or deep tech), the business domain's complexity, or our strategic approach. More often than not, Product assumes the final responsibility. Regardless, my fundamental belief is that product development is a cooperative team endeavor. Thus, if two lines exist, the process mandates not just continuous negotiation but, even more importantly, continuous collaboration.


    Do these product minded engineers exist? Is it a creation, what would be its definition?

    They exist, and I know quite a few. I would also consider myself in that group. Actually, seeing software as a means to achieve an impact (product) is one of my fundamental premises of this profession. Obviously you need to recruit with this mindset in mind, fostering and developing this orientation among your team members. My experience at Alea Solutions and The Motion has been like this, and then at Nextail and ClarityAI. I strive to ensure that my teams adopt this mentality, and my hiring practices reflect it. (See: https://productdeveloper.net/from-software-engineer-to-product-engineer/ https://svpg.com/empowered-product-teams/ )


    Do all engineers have to have this predisposition or what would be the percentage within the team or squad to carry it out?

    At least the senior people on the team. In any case, I think this mindset can be generated and trained. In some cases where the product is very technical or deep tech, perhaps there wouldn't be as much difference between people who have this predisposition and people who don't.


    Do you think it is good that there are two people as leaders of product and technology? Does it depend on the type of product or company? 

    I believe that it's not necessary and that it's a problem that has arisen due to the speed of innovation and the immense amount of technologies and the cognitive load that this generates. 

    Nonetheless, I firmly believe we need to actively address this issue. It is essential to guide engineers towards understanding that their work's ultimate goal is creating impact, while also helping Product people to grasp the necessity of sustainable engineering processes and practices. From my perspective, the solution is intimately linked with lean methodologies, specifically Lean Software Development and Lean Product Development.


    The alignment between Product Manager (PM) and Tech Lead (TL), what is in your experience the best way to achieve this alignment?

    Promote cooperation understanding that the result and impact is always at the team level. In that context, the PM, has to learn about sustainable software development and the TL must learn about product/impact and how to work with customers. Both must also learn techniques to maximize impact and develop product in very small steps, with continuous feedback, postponing decisions until the last responsible moment, minimizing as much as possible the Basal Cost of the solution. (See: https://www.eferro.net/2021/02/basal-cost-of-software.html )


    Closing / Conclusions

    As a closing, can you give your 2-3 tips, based on your experience, on how to have a great relationship between Product and Technology?

    1. 1 Hire/grow product (minded) engineers. It's easier to learn about product/business as a developer than the other way around (plus it's necessary).
    2. 2 Many of the tensions are alleviated by using the same language (centered on business), defining product boundaries, clear acceptance criteria and clearly defining what is a prototype, an MVP, and a normal product (so that it's defined how to make each one).
    3. 3 If performance is incentivized and evaluated at the team level with common objectives, I think it's much easier to align. Of course this implies that the objectives include sustainability and the ability to continue the evolution of the system."



    References: