Cloud mindset (tech pill)

To create a significant software system using the cloud we need to think differently than when we are building a system to run in our data center or using VPS from a classic hosting provider.

Outside the Cloud (data centers and VPS),  the underlying computing resources are:

• Effectively fixed because obtaining new resources requires months.
• They are scarce because we try to buy the minimum resources needed.
• Expensive since we have to pay them entirely and amortize them.

But in the Cloud, the underlying computing resources are abundant and fast to provision and this change the rules of the game and deserve a mindset change.

In the Cloud we should:

• Consider the servers/disks/databases as disposable resources, treated as "just one more". This concept is called Pets vs Cattle and even if I don't like the analogy too much, it is true that it is interesting to read about it: The History of Pets vs Cattle and How to Use the Analogy Properly 
• Always assume that anything will fail and design for it. The Cloud is based on commodity hardware that is not under our control and allows us to create massive scale systems. Due to the scale, we experience partial failures all the time (machines go down, disks broke, network fail...). As we can not predict at this scale, the best approach is to design for failure and improve the resilience of our system continuously. 
• Define the infrastructure as code to allow rapid iteration. You can learn more about this concept in this previous blog post  Immutable infrastructure (tech pill).
• Design to take advance of the elasticity of the Cloud. This includes architect for horizontal scalability so you can scale during peak hours without paying for this capacity during low activity hours.
• Pay per use and rent not buy. When we design for the Cloud, we should try to use the resources as much as possible and deal with the variability in the load using the elasticity. The mindset should be, rent the most suited resources for the task and release them as soon as possible.

In summary, understanding the Cloud providers APIs and services is only a small part of your journey to the Cloud. The most critical step is to change your mindset and accept new underlying assumptions.

A new engineering challenge 

Construct a highly agile and highly available service from ephemeral and assumed broken components. (Adrian Cockcroft)

Related:

• Immutable infrastructure
• Infrastructure as code 
• The History of Pets vs Cattle and How to Use the Analogy Properly
• Technology pills

Good talks/podcasts (Aug 2018 I)

Some good talks that I show lately:

• What is Happening: Attempting to Understand Our Systems Beau Lyddon. A good talk about the need for observability for our systems.
• RailsConf 2015 - Nothing is Something Sandi Metz. Good insights about Objects design. (Via Fran Ortiz)
• Lambda Architectures: a Snapshot, a Stream, and a Bunch of Deltas Adrian Trenaman
• The Journey to Continuous Delivery Dan North. Modern agile and how to introduce it. (good ideas about, flow, value, and teams).
• This Is Water David Foster Wallace Full Speech

Honey Badgers, from Team to Tribe

A team is a group of individuals working together to achieve a goal. However, a tribe (in anthropology) is a human social group. That is: a group of people who interact with one another, share similar characteristics and have a collective sense of unity, and identity.

As I already explained in several blog posts, all the people that worked in the tech/dev department at The Motion formed a cohesive team, called the Honey Badger Team, with shared goals and a common mindset.

The Motion was an early stage startup. The platform itself was working well, we didn’t reach the product market fit, at least with the initial idea. The company, the product, and the business pivoted several times looking for monetization. The current model seems to work better than the initial one, but during all these changes, part of the team, including myself, gradually left. Motivation for leaving was different for each engineer, but in my case, I had already fostered a great team and had helped building a scalable platform.
So I looked for a new challenge in another startup. I found a great opportunity at Nextatil who is leading the agile retail revolution, but this is another story for a different blog post.

The important thing was that we were not a team anymore. The Honey Badger Team was no more, and at first, this made me sad. It was one of the cons about changing job.
However, (and I am not sure about how), we created a social group. With our mindset. With our shared history. With our way of working.
We evolved from a team to a tribe. A tribe with our own communication channels, our jokes, our standard way of solving problems. With a grown/agile mindset, aligned with the XP/DevOps values and practices.

It is incredible, but we are now spread over more than five different startups, and still all of us are thinking about spreading our values, our way of thinking, our way of solving problems. Sharing knowledge, sharing values, sharing solutions. I didn't lose a great team; I got a tribe instead. :)

Currently, we have two telegram channels, which we use to communicate or to obtain great solutions to any problem that we share.
Moreover, we are thinking about creating our invitational conference: the “Tejones Conf", an event for ourselves, our families and friends of us. If we aren't a tribe, I don’t know what we are.

References

The history of the Honey Badger team:

• honey badger team visual history
• honey badger team visual history II
• honey badger team

Blogs from other other Honey Badger tribe members:

• http://blog.fortizabril.com/
• http://cesarob.com/
• https://www.gardenunez.net/
• https://blog.kartones.net/
• http://ronnyancorini.es/

Good talks/podcasts (Jul 2018 II)

Some good talks. In this case, focused on some DevOps practices:

• How to run a successful Continuous Delivery enablement team Robin Weston. A good explanation about how to introduce CD using a specialized team (not the only approach, but a good one).
• Observability
• O11ycast Ep. #2, Observable DevOps with Christina Noren of CloudBees
• How to Build Observable Distributed Systems

Agile is counterintuitive

A good Agile culture for a technology-based company is counterintuitive for most people, as we were educated to use cost accounting and resource efficiency instead of flow efficiency. Neither do we know how to collaborate and work in teams, as well as it being very common to see Engineering as a team to control: as a service provider, instead of as part of the business.

At the organization and team level, at least, the following practices are counterintuitive:

• Increase Focus and reduce the work in progress (WIP).
• Organize people around the work minimizing hands offs (requires generalists and collaboration), instead of in functional silos (need specialists) with central coordination.
• Self-organized autonomous cross-functional product teams that require trust, power, and team responsibility, instead of a culture of Command and Control. 
• The NoEstimates movement to avoid waste.
• A continuous flow of small increments (instead of large batches of work).
• Maximizing the amount of work not done, that can often be confused with laziness, without keeping in mind that this is an excellent strategy to avoid complexity, reduce waste and reduce maintenance costs.
• And related to the previous one, Maximize the outcomes and minimize the outputs.

To reach technical excellence, and with the focus on software development, a lot of practices are initially counterintuitive. For example:

• Test Driven Development (or for some people even automatic testing).
• Pair programming. This practice is a classic one. It's very easy to measure the cost of two developers. But is very difficult to estimate the cost of a bad design, the reduction of the bus factor, the improvement in the knowledge about the code, the system, or about a technical practice. So if we try to analyze from a cost efficiency perspective in a command and control culture that doesn't understand our profession, it is nearly impossible to understand the real benefits of this practice for the product/ the team/ the business.
• Continuous integration (the practice) that encourages trunk based development or working in short-lived branches (less than one day). 
• Continuous Delivery or even Continuous Deployment that requires technical excellence, self-testing code, and proper tooling for recovery in case of failure.
• Introduce operability, security, quality in the process instead of having specific teams and phases to validate these activities.
• Optimize for fast recovery (MTTR) instead of increasing the mean time between failures (MTBF).

As we can see, in general, Agile is counterintuitive... and this is one of the reasons for the vast amount of fake agile cultures that are doing a lot of harm to a lot of companies.

In summary, agile software development is counterintuitive because it requires a culture of respect for people, collaboration and is incompatible with the classic low trust Command and Control organization. The problem is that being adaptable to changes is no longer optional...
Embrace change, Embrace failure, Embrace uncertainty...

It is not necessary to change. Survival is not mandatory. W. Edwards Deming

Resources and related posts:

• Other related posts in this blog
• Agile is not a recipe for success...
• Don't blame the Manifesto for Agile Software Development
• The two pillars of agile software development
• Deadlines and commitments... the fallacy
• Related resources:
• Why Every Company Is A Technology Company
• Developers Should Abandon Agile
• Lean and Agile Adoption with the Laloux Culture Model
• Theory X and Theory Y

Thanks for the feedback and/or for the contributions to:

• The honey Badger team
• Vanesa Tejada
• Unai Roldan 
• Francisco Moreno
• Philippa Skillman

Good talks/podcasts (Jun/Jul 2018 I)

More interesting stuff for the holidays:

• Digital Transformation Adrian Cockcroft
• DevOps Patterns Distilled: A Fifteen Year Study of High Performing IT Organizations Gene Kim
• 3X Explore/Expand/Extract Kent Beck
• The Continuous Culture Kim van Wilgen