Introduction
"Optimize the Whole" is one of the fundamental principles of Lean and Lean Software Development. It means setting aside local improvements to look at the entire system, from the idea to the user—people, processes, and technology—and aligning everything toward a common goal: delivering value to the user quickly and sustainably.
In my experience, many teams believe they are being efficient because no one ever stops working, but in the end, the user waits weeks (or months) to see real value. I’ve learned that true improvement comes from a systemic vision, working together so that the flow of value moves without friction from end to end.
Fragmentation: The Legacy of Taylorism
The paradigm of software development has often—consciously or unconsciously—inherited principles rooted in Taylorism and Fordism, conceiving the creation of digital products as a fragmented "assembly line." Under this view, each phase (analysis, design, development, QA, operations) becomes a functional silo, where teams specialize in specific tasks and focus on optimizing their local efficiency.
However, what in physical goods manufacturing could generate economies of scale for mass production has over time also shown its limits by sacrificing flexibility and the ability to quickly adapt to changes in demand or user needs.
In software, this translates into chronic bottlenecks, costly handoffs, and a fundamental disconnect between those who define needs and those who implement them. This fragmentation breaks the flow of value, fosters the accumulation of "inventory" in the form of work in progress, and hinders rapid adaptation—resulting in features that don’t solve real problems or that take months to reach the user, undermining the promise of agility and continuous value.
|
| Comparison of Taylorism, Fordism, and Lean Thinking |
What Happens When We Don’t Optimize the Whole?
Over the years working with different teams, I’ve observed that when we don’t optimize the whole, we fall into the trap of optimizing locally—almost always with good intentions but with unintended consequences. Teams may become very “efficient” in their own internal metrics, but if they are not aligned with user value, the overall flow slows down. Bottlenecks appear, handoffs multiply, and work gets stuck somewhere waiting.
I’ve seen this especially when engineering or development is seen as a feature factory that “executes” or “implements” what others decide. The team just implements, without understanding the problem, the priority, or the real impact on the user—and without contributing their technical knowledge to decisions. The result: solutions disconnected from real needs, slow feedback, and features that don’t solve the problem.
In my experience, the root cause is usually a functional and fragmented organizational structure inherited from “assembly line” thinking. But I’ve learned that software doesn’t behave like a linear factory. Software needs product teams with end-to-end responsibility, without silos (backend, frontend, QA, ops, design, etc.), and with real autonomy to make decisions and operate what they build.
I’ve found that this is not just a matter of motivation: it’s the only way to optimize the total flow and deliver real value to the user quickly and sustainably.
Bottlenecks and the Theory of Constraints
The Theory of Constraints (TOC) reminds us that in any system there is always at least one constraint that determines the maximum delivery capacity. Identifying and managing that bottleneck is essential to improving the overall flow.
For example, at ClarityAI, in the beginning, features were released to production but could remain behind a feature toggle for weeks, waiting for product to decide when to expose them to users. Even though they were technically ready, value wasn’t flowing.
Another example: when workflows were separated by functions (frontend, backend, data engineering), any change could take weeks because each group optimized its own flow or backlog instead of thinking about the overall flow from the user’s perspective.
(Fortunately, these specific problems at Clarity AI were solved long ago, but they serve as an example of what can happen when we don’t optimize the whole.)
In my experience working with different teams, I’ve learned that speeding everything up indiscriminately only makes the backlog pile up and creates frustration.
A necessary condition to identify real constraints is that all work is visible—not only development tasks, but also testing, operations, support, automation, documentation, analysis, coordination, etc. If a significant part of the team’s effort is hidden (because it’s not recorded, visualized, or considered “technical work”), it’s very easy for the real constraints to go unnoticed. As Dominica DeGrandis points out in
Making Work Visible, what you can’t see, you can’t manage or improve. Making all work visible is a key step for informed decision-making, reducing work in progress, and better focusing continuous improvement efforts.
The key steps are:
- Identify the constraint. Make it visible and prioritize it.
- Exploit the constraint. Keep it focused, avoid distractions, and ensure it’s always working on the highest-value items.
- Subordinate the rest of the system. Adjust rhythms and priorities so as not to overload the constraint.
- Elevate the constraint. Improve its capacity through automation, training, or process redesign.
- Review and repeat. There will always be a new constraint after each improvement.
Over the years, I’ve noticed that the more separate stages with queues there are between the idea and the value delivered to the user, the greater the chances of bottlenecks forming. Furthermore, if each stage belongs to a different group (organizationally speaking) that may even have its own agenda, there is likely little interest in optimizing value for the user. In these cases, each group may focus solely on improving its part of the process—or on avoiding being perceived as the bottleneck.
End-to-End Teams and Real Optimization
In every team I’ve built, I’ve insisted that they be product teams with end-to-end responsibility, without silos for QA, operations, security, or design. The reason is simple: if the team doesn’t control or understand the entire flow, it can’t optimize the whole, and it also risks not taking full responsibility for the complete product.
When the same team is in charge of conceiving, building, deploying, and operating, it eliminates the waste that arises in each handoff and accelerates learning. Every member understands the impact of their work on the end user and the value actually being delivered. End-to-end responsibility not only improves technical quality but also strengthens the focus on user value, avoiding unnecessary investments in local optimizations that don’t contribute to the overall goal.
In this work environment, more generalist profiles—often referred to as "T" (deep in one area and broad in others) or "M" (with several deep specializations and wide versatility)—are especially valued. These professionals, being able to contribute in multiple phases of development and having a more holistic view, are key in contexts where flow efficiency is a priority.
While pure specialists, also known as "I" profiles, are still needed in very specific domains, their role changes: they become enablers and educators, helping to scale their knowledge and train the rest of the team in various facets of the product.
In addition, keeping options open, working in short cycles, and making small deliveries allows for better adaptation to what the business and data reveal. Simplicity and vertical slicing reduce risk and facilitate rapid learning, freeing the team from the mental load of large, fixed plans and fostering autonomy.
This way of working is unfeasible when the product must pass through different silos before reaching the end user.
End-to-End Teams in Practice: Real Experiences
When I formed the team at Alea Soluciones, from the very beginning I pushed for us to take on all possible tasks and functions. I remember both the CEO and the CTO offering me the option to take on fewer responsibilities “to have less pressure and less work.” But I refused that proposal: I knew that if we only kept part of the scope, we would lose the global vision and the ability to optimize the entire system.
By taking on all areas—development, support, product ideas, operations—we always maintained a holistic view of the product. This allowed us to identify at any given moment where the real bottleneck was and act where we could have the greatest impact, without depending on other teams or functional silos. This meant we could decide whether, at a given moment, it made more sense to focus on support, to speed up key developments, or to think about new features—always optimizing the total flow of value to the user.
At Nextail and Clarity AI, together with others, I worked to evolve teams toward an end-to-end model, avoiding QA, operations, or product silos. In these cases, we applied ideas from Team Topologies to transform the structure: we moved from a function-based organization (infrastructure/operations, frontend, backend, data engineering, product) to a matrix of autonomous product teams.
The goal was always the same: to shorten the lead time of any change, from the idea to when the user can actually use it. With autonomous, end-to-end responsible teams, we could deliver value faster, learn continuously, and improve the product in very short cycles.
In all these contexts, I’ve found that the end-to-end approach has not only been a technical or organizational improvement—it has been the key to maintaining a user-centered mindset, reducing waste, and optimizing the whole, every day.
How Optimizing the Whole Helps Eliminate Waste
One of the fundamental principles of Lean is
eliminating waste. When we focus only on optimizing local parts, it’s very easy to accumulate work that doesn’t add value, create unnecessary features, or generate invisible delays. In contrast, by optimizing the whole and looking at the complete system, we naturally reduce multiple types of waste.
In my work with different teams, I’ve observed that when we don’t think about the global flow and focus on optimizing “my part,” incentives appear to produce more features even if they don’t provide real value to the user. The priority becomes “staying busy” and completing tasks, rather than questioning whether they are necessary.
I’ve seen how, without early feedback and working in long cycles, teams advance without validating and end up building features nobody asked for or that don’t solve any important problem.
Moreover, when decisions are fragmented (product defines large packages, engineering executes without questioning, and QA validates afterward), the vision of impact is lost, and the backlog tends to swell with features that remain blocked behind a feature toggle or are never released at all.
By optimizing the whole and aligning each step with the complete flow of value to the user, each feature is reviewed with key questions:
- Does it solve a real, high-priority problem?
- Can we release it in a small version to learn quickly?
- How will we know if it truly adds value?
In this way, we build only what’s necessary, learn early, and avoid turning effort into waste.
- Avoid unnecessary features (overproduction): By prioritizing real user value and working in small deliveries, hypotheses are validated quickly. This avoids building large features that nobody uses or that the business decides not to launch.
- Reduce waits and blockages (waiting times): By eliminating silos and working end-to-end, work doesn’t get stuck waiting for another team or function to pick it up. This speeds up flow and eliminates idle time.
- Less rework and late fixes: Delivering in short cycles and validating early allows problems to be detected quickly and corrected at low cost. Otherwise, small local decisions can lead to large refactorings later.
- Avoid useless local optimizations (unnecessary movement): Optimizing your own “department” or backlog can create a false sense of efficiency, but doesn’t generate value if it doesn’t move the complete flow forward. Looking at the global system avoids this kind of waste.
- Reduce hidden inventory: Limiting work in progress and prioritizing constant flow minimizes the inventory of half-built or unreleased features, which consume energy and create confusion.
- Lower opportunity waste: By having a clear view of the whole and being aligned with the business, we avoid investing in the wrong directions and respond quickly to new opportunities. This reduces the risk of missing the right moment to impact the user.
In my experience, when we optimize the complete system, every decision is made with the flow of value to the end user in mind. This way, every line of code, every validation, and every deployment helps reduce waste and maximize impact.
When each component of the system focuses on its own optimization, bottlenecks, waiting queues, and handoffs arise, breaking the continuity of flow. Paradoxically, when “everyone is very busy,” it’s often a clear sign that there is a serious problem with the flow of value to the user. Work piles up, deliveries are delayed, and the organization is investing significant effort in activities that don’t quickly translate into real value.
No comments:
Post a Comment