Showing posts with label product management. Show all posts
Showing posts with label product management. Show all posts

Wednesday, August 09, 2023

Engineering & Product relationship: random notes

These notes come from my preparation for a panel on technology and product to which I was invited (https://lu.ma/tech-sherpas-madrid). I am publishing them here in case they contribute or help to initiate some conversation. Of course, the panel was great, and it included responses from Isaura Fontcuberta and Rebeca Francisco Najarro that contributed more than my own, but for obvious reasons this post only includes mine since they are the ones I have notes on. The panel was very well conducted by Pedro Pablo Aparicio



Q&A Notes

Let's start, should a product company have two Product and Technology Roadmaps, or should it have a single roadmap?

Ideally a single balanced roadmap. The problem is that in my experience this has many implications. Assuming that there are two different reporting lines (something that we should be able to question), there tends to be an imbalance. By instilling product sensitivity within engineering, this imbalance can be mitigated, making engineering attuned to these sensitivities without seeing them as antagonistic. On a small scale, what has worked best for me is that this division does not exist at all. :) If, out of necessity, there are two roadmaps, there must be clear rules for prioritizing (percentages, for example). (See: https://www.eferro.net/2021/10/it-depends-development-mix-product.html)


Continuing with the topic, taking advantage of the fact that we are talking about RoadMap, who is responsible in the case that there is 1 or 2 roadmaps….

If two reporting lines exist, it becomes a matter of negotiation, although I question the necessity for dual lines in all situations. The final responsibility can vary depending on several factors such as the nature of the product (be it business digitalization, information systems, or deep tech), the business domain's complexity, or our strategic approach. More often than not, Product assumes the final responsibility. Regardless, my fundamental belief is that product development is a cooperative team endeavor. Thus, if two lines exist, the process mandates not just continuous negotiation but, even more importantly, continuous collaboration.


Do these product minded engineers exist? Is it a creation, what would be its definition?

They exist, and I know quite a few. I would also consider myself in that group. Actually, seeing software as a means to achieve an impact (product) is one of my fundamental premises of this profession. Obviously you need to recruit with this mindset in mind, fostering and developing this orientation among your team members. My experience at Alea Solutions and The Motion has been like this, and then at Nextail and ClarityAI. I strive to ensure that my teams adopt this mentality, and my hiring practices reflect it. (See: https://productdeveloper.net/from-software-engineer-to-product-engineer/ https://svpg.com/empowered-product-teams/ )


Do all engineers have to have this predisposition or what would be the percentage within the team or squad to carry it out?

At least the senior people on the team. In any case, I think this mindset can be generated and trained. In some cases where the product is very technical or deep tech, perhaps there wouldn't be as much difference between people who have this predisposition and people who don't.


Do you think it is good that there are two people as leaders of product and technology? Does it depend on the type of product or company? 

I believe that it's not necessary and that it's a problem that has arisen due to the speed of innovation and the immense amount of technologies and the cognitive load that this generates. 

Nonetheless, I firmly believe we need to actively address this issue. It is essential to guide engineers towards understanding that their work's ultimate goal is creating impact, while also helping Product people to grasp the necessity of sustainable engineering processes and practices. From my perspective, the solution is intimately linked with lean methodologies, specifically Lean Software Development and Lean Product Development.


The alignment between Product Manager (PM) and Tech Lead (TL), what is in your experience the best way to achieve this alignment?

Promote cooperation understanding that the result and impact is always at the team level. In that context, the PM, has to learn about sustainable software development and the TL must learn about product/impact and how to work with customers. Both must also learn techniques to maximize impact and develop product in very small steps, with continuous feedback, postponing decisions until the last responsible moment, minimizing as much as possible the Basal Cost of the solution. (See: https://www.eferro.net/2021/02/basal-cost-of-software.html )


Closing / Conclusions

As a closing, can you give your 2-3 tips, based on your experience, on how to have a great relationship between Product and Technology?

  1. 1 Hire/grow product (minded) engineers. It's easier to learn about product/business as a developer than the other way around (plus it's necessary).
  2. 2 Many of the tensions are alleviated by using the same language (centered on business), defining product boundaries, clear acceptance criteria and clearly defining what is a prototype, an MVP, and a normal product (so that it's defined how to make each one).
  3. 3 If performance is incentivized and evaluated at the team level with common objectives, I think it's much easier to align. Of course this implies that the objectives include sustainability and the ability to continue the evolution of the system."



References:

Monday, April 18, 2022

Systems: Control its evolution / or be its slave

 When we develop software (digital products), we usually deal with complex systems. We are not prepared (by definition) for emergent behaviors of these types of systems. As a result, it is hard to predict its future evolution and very easy to lose control over it over time.


If we don't control complexity, it will become increasingly difficult to adapt the system to what we need, and we will move from developing and evolving the system to spending more time reacting to it as it evolves. We will become slaves to the system rather than controlling its evolution. We will be spending our time working on production incidents, scalability improvements under pressure, and being in a firefighting mode instead of adding value to the product in a sustainable and continuous manner. Needless to say, the business impact is terrible (poor efficiency, unhappy customers, difficulty in defining strategy, etc).


In my experience, we can keep the system under control by using appropriate tactics.

Tactics to control system evolution

I have used the following tactics to control the evolution of systems I have worked on:

  • Good basic hygiene: automated tests, minimum observability, product instrumentation, etc.
  • Simple design based on current needs: Using incremental design and lean product development we can develop simple solutions that fulfill our current needs but allow us to evolve the system in the future.
  • Evolutionary architectures: Invest in an architecture that can evolve and that allows us to postpone decisions (reversible decisions, fitness functions, observability, etc.).
  • Acceptance criteria: Make the acceptance criteria explicit, so that we all know what we consider sufficient hygiene (testing, observability, performance, etc.).
  • Define product limits: Define product limits that allow us to convert problems commonly considered "technical" to business/product problems. For example, instead of talking about scalability as an abstract concept, concrete it at the product level with clear limits (number of concurrent users/requests, maximum response times, etc).
  • Look for boundaries and use them to assign clear ownership for each identified part. The concepts of bounded contexts of DDD and value stream can assist in identifying such limits within the system. Having clear ownership of each part of the system is fundamental since we must remember that we work on complex socio-technological systems.
  • Work in small safe steps: Controlling the evolution of the system is complex and requires us to constantly rethink and adapt. We can only accomplish this if we work in small safe increments of functionality that allow us to change direction without leaving features half-integrated.

The use of these tactics is usually adequate, but as with everything in our profession, it depends, there are times when a little upfront planning and attention to future needs would be beneficial. See "It depends" / The development Mix (Product, Engineering, Hygiene.


Below I include several examples of how I have utilized some of these tactics to control the evolution of the system and not be at its mercy.

Examples

System Load / Scalability evolution



Having performance or stability/availability problems when a system is under high load is very common. The most common approaches for this evolution dimension are:

  • Doing nothing and reacting when we discover the first problems (potentially damaging the relationship with our users due to production incidents).
  • Solving “imaginary” scalability problems by preparing/designing our systems for a load that is not real and generating a high waste. Sometimes this is the default option chosen in engineering simply for the pleasure of solving an interesting problem.

I think that there is a more sensible approach that allows us to control evolution without generating waste (See Lean software development). In this case, we need to have some metrics about the current load and a rough idea of how much maximum load our system can support (without degrading).

With this information, we can define a soft limit (Product Limit) for the load, as the maximum supported load minus a threshold that gives us time to evolve the system to support more load.

As we can see in the example diagram, we receive a notification at t1 informing us that we reach the moment to improve the scalability of the system. Between t1 and t2, we improve the scalability to support a new load maximum and redefine the new soft limit and repeat the process when the limit is reached (t3).

Storage volume growth



When we store information it is very common that the associated growth of the information volume reaches a point when the technology selected is no longer valid or that we can start to have some problems (high cost, backup problems, etc). 

As in the previous example, we can do nothing and react after we start to have problems or use a tactic to control the evolution/growth.

In this case, if we need to maintain all the historical information, we can follow a similar approach as in the previous example about scalability. But if we don’t need to store the historical information we can create a soft limit to detect when we should start to develop an automated data purge process that periodically removes obsolete information. This approach is very lean because it allows us to postpone the implementation of the data purge process until the last responsible moment.


Controlled errors and quotas


One of the easiest ways to lose control of a system is to have no controlled way of handling errors. When we don’t control errors, we are in risk of having cascade errors and affecting more users. This is one of the most common examples of losing control of the system.

One of the first tactics that help us to maintain control of a system is to detect unexpected runtime errors, avoid crashing and at least send a message to the user and avoid as much sa possible that this error impacts other users.

Another good tactic is to define quotas per user (Product Limits) and use these quotas to throttle requests and show a warning message to the user. In the message, we can inform the user to contact support and use this opportunity to get more information about how the user is using the system or even offer a better SLA to the user.


Committed capacity


When we work in a multitenant SaaS environment is very common that the resources used by the sum of all of our customers increase very fast. One interesting tactic to maintain the control of the evolution of the system in this context is to define how much capacity (of a resource) we commit to each type of user (Product limits). Even knowing that customers will not fully utilize their full capacity, having these definitions allows us to identify the worst-case load scenarios we could have.

In the example diagram, we define the different quotas for each type of user (basic, professional, enterprise). With these definitions and the number of users of each type, we can calculate the maximum committed load at any moment and use this information to make decisions.

This tactic is used frequently by AWS. For each type of resource, they define default limits, and to extend these limits you need to contact support. AWS can use the information from support requests to make very accurate capacity plans and make better product decisions.


Modular monolith


Building a system from scratch is not an easy task, and it is very easy to lose control of the system as it grows (more developers / more functionality). The two most common failures are:

  • Growing a monolith organically without committing effort to modularization and ending up with a big ball of mud (http://www.laputan.org/mud/).
  • Incorporating a microservices architecture when it is not yet time to scale greatly increases complexity without obtaining any of the benefits of this architecture.

In general, better results can be achieved by developing a monolith, but organizing the code and data into internal modules (modular monolith). As we become familiar with the system and domain, we organize it internally into modules (in alignment with bounded contexts), so we can maintain control of the architecture throughout its evolution. 

When we see the need, we can extract some of these modules into independent services.

It is essential to create mechanisms that help us maintain this modularity during evolution (to perform architecture tests to avoid not allowed dependencies between modules, to use different schemas in the DB, to analyze dependencies periodically).

In addition, we can use these modules to assign ownership of each to a team, allowing each team to work independently.

Conclusions

When we lose control over the evolution of the system:

  • we become inefficient and fail to create value.
  • we lose customer trust (incidents, errors, etc).
  • we react to problems instead of being able to follow our product strategy.

The business impact is huge. Therefore, it is essential to manage complexity and keep evolution under our control.


The tactics I discuss in this article are quite simple and focus on detecting needs early enough so that we can react in a planned way (rather than reacting under pressure). Moreover, they do not require large investments, in fact, they allow us to have a lean approach and avoid over-engineering. But they do require good hygiene practices (testing, observability, reasonable quality, etc). 


References and related content:

Thanks

The post has been improved based on feedback from:

Thank you very much to all of you





Thursday, March 31, 2022

Mesa Redonda: Colaboración entre perfiles de producto y desarrollo (Code Sherpas)



El pasado 24 de Marzo tuve el placer de participar en una mesa redonda sobre "Colaboración entre perfiles de producto y desarrollo" donde pude aprender y compartir con Marta Manso, María Granadino, Isabel Garrido y Cristina Verdi.

Gracias a Code Sherpas por la organización y dinamización del evento.

Tuesday, October 06, 2020

The Big Branch Theory - Product Teams podcast

El pasado 1 de octubre se publico el episodio de Big Branch Theory en el que pude conversar con mi amigo Ronny sobre los equipos de Producto, los equipos de Plataforma y algún que otro tema relacionado.

La verdad es que como siempre disfrute de la conversación con Ronny y creo que salieron temas bastante interesantes.


Espero que nuestra coversación le pueda aportar algo a alguien, y estaría encantado de conocer otras opiniones sobre los equipos de producto, las plataformas como producto y sobre el resto de los temas que tratamos.

Podeis descargar el episodio desde:
https://bigbranchtheory.dev/post/product-teams-edu-ferro/

Y no dejeis de serguir este interesantisimo podcast https://x.com/BigBranchTheory

Monday, August 17, 2020

Resources for Product Management for Internal Platform

"The purpose of a platform team is to enable stream-aligned teams to deliver work with substantial autonomy. The stream-aligned team maintains full ownership of building, running, and fixing their application in production. The platform team provides internal services to reduce the cognitive load that would be required from stream-aligned teams to develop these underlying services."

The last year I have been leading the Platform Team in a startup during the scale phase. First at Nextail and currently at Clarity.ai. Although it is the first time I focus exclusively on that team, the truth is that throughout my career, I have always got very involved in any initiative or organization focused on improving internal tools or provide a platform for the rest of the engineering team.


I have always liked these teams, for the transversal impact they have on the whole product organization and the capacity they have to enable the DevOps culture.


But I've always seen that they usually focus too much on technology and not so much in the customer, our colleagues.  Happily, it seems that these teams are starting to be treated as product teams and are beginning to use modern product management techniques.

There are still not many resources dedicated to how to treat the Platform as an internal product. So I have decided to compile in this post the most interesting resources I have found about it.
 

Interesting presentations

Projects

Articles and blog posts

Books

I didn't find any specific book about Product management techniques for Platform teams, but we can always adapt the techniques described in modern product management books.

Other resources

I'd love to share resources with anyone who's interested in the subject. Any suggestions/resources/ideas are welcome.


Updates:

Sunday, July 12, 2020

Continuous learning. What / How


“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
“Working software is the primary measure of progress.”

But making software is complex and requires a good understanding of the customer's needs, the market (WHAT), and the reactions of the system we are developing (HOW). Let's remember that current software systems are complex systems, and that implies that they have emerging behaviors that we don't know when we create them.
“The main idea behind complex systems is that the ensemble behaves in way not predicted by the components. The interactions matter more than the nature of the units. ... This is called an “emergent” property of the whole, by which parts and whole differ because what matters is the interactions between such parts. And interactions can obey very simple rules.” 
Nassim Nicholas Taleb

Therefore the bottleneck to create software that meets customer needs is continuous learning. But learning is useless if we bring the system down or piss off our customers.

In a software system each of the deployments/changes we make, allows us to learn and make decisions for the next steps. Therefore we are interested in shortening the learning and decision cycles as much as possible.

What is Agile delivery's strategy for doing so?

Agile Delivery's strategy is to deliver increments in small safe steps (3S), each of which must attempt to generate learning about the WHAT (user needs) or the HOW (technology/system).

Learning WHAT / User needs / Valuable software

If we have a lean product development process, we will try to validate and learn this aspect by generating as little software as possible. This implies generating hypotheses and trying to achieve validated learning using everything we have at our disposal: data analysis, prototypes with or without code, or software increments. For the latter, we will have to divide using Vertical Slicing in increments so small as possible.
Of course, to learn about user behavior, it is also essential that we instrument the product to learn continuously from the use that the customer makes of the system.

Learning HOW / Technology/System / Working software

Software is a complex system. This means that we can't foresee the final behavior when we develop each of its parts independently. That is why it is essential to make small safe changes (technical Slicing) that allow us to learn from the behavior of the system in production without affecting the user. Of course, in order to get these learnings, we need to decouple the deployments (putting into production a change) from the releases (activating a change for the end-user).


Summary

In summary, the sure way to achieve this continuous learning of the WHAT and HOW that agile delivery offers is the Continuous Delivery of Small Safe Steps (both purely technical increases and increases in value for the user).
It seems that the data confirm that this is the right approach for making digital products.



References:

Small Safe Steps

Vertical Slicing

Continuous Delivery