Saturday, September 09, 2017

Good talks/podcasts (August 2017)


These are some interesting talks/podcast that I've seen during August:

Tuesday, September 05, 2017

Pub-Sub the swiss army knife (tech pill)

Pub-Sub / Publish-Subscribe

"In software architecturepublish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorize published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are." wikipedia Publish-Subscribe pattern
As the Wikipedia explains, the pub-sub patterns allow decoupling the publishers (P) from the possible subscribers or consumers (C). So the publisher doesn't need to have any knowledge about the components interested in receiving the message.
To implement this pattern we need a central service in charge to receive the messages from the publisher and resend the message for each one of the interested subscribers. Usually, this central service is called broker (B).

Some common characteristics for a broker are:
  • Allow broadcasting message to several consumers.
  • Allow the consumers to configure in which messages are interested using some kind of consumer defined rule.
    • By topic
    • By regular expression over a topic 
    • By a query over some attributes of the message
  • Allow the publisher to send messages and add some meta info to it
    • Attributes
    • A destination topic
    • Other meta info as priority, TTL, etc
  • Allow several consumers to receive the same message
Another typical characteristic, and in fact, the most important one is that for each consumer the broker usually has a queue to maintain the pending messages.
And this queue has all the characteristics that we explain in the Queues vs Distributed log post... For example, the broker can load balance the messages between a group of consumers.



In summary, the main characteristics of a pub-sub system are:
  • Publishers:
    • They don't know the final destination of the messages (decoupling).
    • They send the messages to a topic/exchange/abstract destination.
    • They can add attributes to the messages.
  • Consumers/Subscribers:
    • Each one informs the broker the in which messages are interested (using the name of the topic, a regular expression over the topic or a combination of attributes of the message).
    • Each message can be consumed by several consumers (broadcasting).

Pros:

  • Great decoupling between publishers/consumers.
  • Allow easy creation of flexible communication topologies.
  • All the pros of Queues.
    • Easy to implement.
    • Unlimited/Easy horizontal scalability.
    • Fault tolerance is very easy to implement (time out and re-queue of the message).
    • Allow easy balance between latency (time at the queue + processing time) and the cost of the concurrent workers.

Cons (same as queues):

  • The order is not guaranteed.
  • Usually, we can have duplicates.
  • Requires more resources from the broker some in some scenarios it has worst scalability than other solutions (not important for the majority of the cases).

Use a Pub-Sub system when:


For any use case that requires flexibility, broadcasting of messages and doesn't require that order of the messages is guaranteed. In these scenarios, a pub-sub system is a great solution because it includes all the use cases of a queue and all the flexibility of sending the same message to several queues/consumers.

The real potential of this kind of solution is when you combine several brokers (using federation or replication)  to create flexible topologies that can communicate several systems and services.


Use cases / Examples:

  • Any goof scenario for queues.
  • Async processing of requests.
  • When we can allow losing some data (or having some delays), Pub-Sub systems are great for:
    • Log / Monitoring info distribution and processing.
    • Asynchronous processing of email requests.
    • Processing of independent batch jobs.
  • Using Federation
    • Info replication and distribution between data centers.
    • Global distribution of periodic information.
    • Global distribution of reference info.

Implementations:

Related content:

Monday, August 21, 2017

Good talks/podcasts (Purging the queue III)

Friday, August 18, 2017

Good talks/podcasts (Purging the queue II)



Sunday, August 13, 2017

Books I have read lately and Reads In Progress (RIP)

Continuing with my process of recovering my previous reading habit (books-i-have-read-in-last-12-months http://www.eferro.net/search/label/books) these are the books that I read lately:

  • "Nonviolent Communication: A Language of Life"  Marshall B. Rosenberg Review
  • "The Power of Habit: Why We Do What We Do in Life and Business" Charles Duhigg
  • "Los 7 Habitos de la Gente Altamente Efectiva" Stephen R. Covey
  • "The Toyota Way" Jeffrey Liker
  • "Algorithms to Live By: The Computer Science of Human Decisions" Brian Christian, Tom Griffiths
  • "El Arte de Vivir: Meditación Vipassana tal y como la enseña S.N. Goenka [The Art of Living]" William Hart
  • "The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good Life" Mark Manson
  • "Tribes: We Need You to Lead Us" Seth Godin
  • "The Five Dysfunctions of a Team: A Leadership Fable" Patrick M. Lencioni Review
  • "The Happiness Hypothesis: Finding Modern Truth in Ancient Wisdom" Jonathan Haidt
  • "Team of Teams: New Rules of Engagement for a Complex World" General Stanley McChrystal, Tantum Collins, David Silverman, Chris Fussell Review
  • "Focus: A Simplicity Manifesto in the Age of Distraction" Leo Babauta
  • "The Sales Bible: The Ultimate Sales Resource" Jeffrey Gitomer 
  • "The Ideal Team Player: How to Recognize and Cultivate the Three Essential Virtues: A Leadership Fable" Patrick M. Lencioni
  • "Ego Is the Enemy" Ryan Holiday
  • "Barking up the Wrong Tree: The Surprising Science Behind Why Everything You Know About Success Is (Mostly) Wrong" Eric Barker
  • "Scrum" Jeff Sutherland, JJ Sutherland


Read in progress (RIP) :)

  • "The Lean Product Playbook: How to Innovate with Minimum Viable Products and Rapid Customer Feedback" Dan Olsen
  • "This is Lean: Resolving the Efficiency Paradox" Niklas Modig, Par Ahlstrom
  • "Implementation Patterns" Kent Beck (2º reading)
  • "The real startup Book" v0.3 

Friday, August 11, 2017

Scalable design pattern sample (tech pill)


As a complement to the previous blog post Queues vs Distributed logs (tech pill) this blog post describes how to solve a concrete business problem using a distributed log and a queue.... I used this design several times in production with good results.

Any feedback or improvements of the design will be more than welcomed :)

The problem:

We have to implement a business process with the following characteristics:
  • The business process (Job1) can be divided into three different sequential phases (S1, S2, S3). For example, we can think about the generation and mailing of all the invoices for all the customer of one account manager.
  • The second phase (S2) can be divided in several (hundreds or thousands) individual and independent sub-jobs (S2.1, S2.2, ...). This sub-jobs can be executed/processed in any order. For example, generate and email one invoice. Each sub-job require one minute of process time.
  • The complete process should complete in less than 90 minutes without being affected by the number of sub-jobs of the second step (up to 10k sub-jobs).




We also need to:
  • Notify when each step starts and when each step finished.
  • Generate some statistics of each the process.
  • Access to the detailed status of the process at any moment.
The solution should have the following characteristics:
  • We can have several numbers of concurrent business processes of this type for each customer.
  • For the sub-jobs at step 2:
    • We have retries for the sub-job execution.
    • We can balance the time of the process and the cost.
    • We have horizontal scalability.

The Design:

After analyzing the requirements and make a fast web storming session we consider the following events:


If we need to include more information about the detailed state of the process we can also signal the starting point of each step using a StepXStarted event. But these Starting Events are not required because we already know when a step started (just when the previous step finished).

Implementation


As we want to implement several functionalities for the same events and we want to have each one completely separated, we can use a distributed log / stream that allows us to design a simple solution to manage the workflow, calculate statistics, compute a detailed status.
The distributed log / stream and the corresponding services can scale using as partition/sharding key the job id or the customer id (assuming that each customer can generate several jobs at the same time).



The Step2 require additional design.
It can be subdivided in several individual jobs (S2.2, S2.2, ...), this jobs can be processed in any order and in parallel so we can have a queue for dispatching this subjobs to a pool workers that can be scaled out if needed.
We can balance the number of workers (and the corresponding cost) with the duration of the Step2 that we want. Each worker get a job description from the queue, execute the job, and include the result in an event (Step2SubjobCompleted) published in the distributed log / stream.



The Workflow Manager should implement a response for each event and generate corresponding events to make the job progress.

The responses for each event are:

Job1Requested:

  • Execute Step1
  • Publish Step1Completed

Step1Completed:

  • Split the Step2 in several subjobs (S2.1, S2.2, ...)
  • Store the identifiers of the jobs created (S2.1, S2.2, ...)
  • Send the subjobs to the queue of S2 subjobs

Step2SubjobCompleted:

  • Mark the id of the job as no longer pending
  • Validate if we have jobs pending
  • if there is no more jobs pending, publish Step2Completed

Step2Completed:

  • Execute Step3
  • Publish Step3Completed

Step3Completed:

  • Do any garbage recollection needed
  • Publish Job1Completed

Job1Completed:

  • Nothing, Everything is already done  :)

For the same events, other functionalities as Statistics, reports or notifications, will define other reactions/implementation to process each event... Compute and store statistics, generate emails or push notification, create a view with detailed information on the status of the job, etc...


Notes and complementary designs:

  • The queue allows duplication so the workflow should be prepared to receive several events Step2SubjobCompleted for the same subjob.
  • We can have retries for the queue jobs, so we should include a mechanism to detect when we should stop waiting for Step2SubjobCompleted. For example, we can use a periodic tick event and use this event to decide if we should continue with the next step (for example marking a subjob as an error).
  • Is also possible to continue receiving Step2SubjobCompleted even if we are at Step3, the easy solution is to ignore this messages.
  • If we select the Job id as the sharding/partition key for the distributed log / stream we can easily scale out the number of workflow processors. We only need to add more stream shards and the corresponding new processes.
  • For the fault tolerance of the workflow we can store the events that we already processed and recover in case of failure from this events.

Related references (very recommended):

Sunday, July 30, 2017

Honey Badger Team - Visual History III - CD as driving force

After publishing Honey Badger Team visual history, @artolamola asked me on twitter if I used "continuous delivery" as the driving force for the transformation and why. The short answer is Yes, and in this blog post will try to explain the motivations behind.

For me, agile requires two things (see the-two-pillars-of-agile-software):
  • A healthy knowledge culture focused on people (collaboration, learning, respect, team work, creativity...) 
  • Looking for quality and technical excellence (for example XP practices are a great starting point).
To introducing the healthy knowledge culture, I introduce retrospectives and worked as a facilitator,  change agent and "Developer Whisperer".
In our profession knowledge is the bottleneck

For the technical excellence I think that we should follow this steps:
  • If there is a classic vicious cycle of bad quality or bad practices going on. We should stop and break this cycle immediately. The typical example consists in having technical debt, that generates bugs, that generate even more technical debt that ends in the complete collapse in few months.
  • Assume that we have the capacity and very good intentions.
  • Help to create a virtuous cycle to improve our technical quality day by day.
  • To create this virtuous cycle we need to define a clear goal to align all of our efforts.
Vicious Cycle of technical debt :)

In our case, and having in mind that we are a cloud native product, I thought that we can use Continuous Delivery as our team Goal

Why Continuous Delivery?

The answer is that I thought that we require a good practice that force us to do the thing right and doesn't allow us to hide our unprofessionalism or incompetence. And for this, and in the cloud, I think that in our case Continuous Delivery will put a lot of pressure on doing the things right.

Continuous Delivery implies:
  • Low cost and a smooth process for deployment that doesn't affect customers. This requires (or at least recommend) introducing DevOps, automation, rolling deploys and Continuous Integration.
  • Having good confidence in our product. This requires Continuous Integration, Automatic testing, and Monitoring for production.
  • Automatic testing requires integration tests and unit tests, and for sure, having unit tests put a lot of pressure on doing a good design (for example to have a hexagonal architecture or other architecture that decouple business logic from infrastructure).
So indirectly, trying to implement Continuous Delivery requires or recommend doing the following practices: DevOps, Infrastructure Automation, Zero down time deploys (for example rolling deploys), Continuous Integration (perhaps with trunk based development), Integration testing, Unit testing, Good Design (for example using hexagonal architecture).

Virtuous Cycle created when CD is the goal


Continuous Delivery doesn't allow you to hide bad technical quality and force you to introduce good technical practices and work very hard to have a healthy code base.

With this approach, we, as a team, improve a lot our technical knowledge, our code base and our practices. We have to improve many technical practices but we have already come a long way and I sincerely believe that we are going in the right direction. :)

Extraball:

  • If you use micro services, CD force you to think about independent deployability.
  • CD force you to learn how to make parallel changes (using feature toggles, database refactoring, etc.)
  • Some additional insights from @artolamola:
    • Above CD you can implement any agile method for team organization (scrum, kanban, etc.).
    • CD force you to think how to decompose each feature in vertical slices.

References:

Some of these ideas come from:

Some context about how I understand agile:


The history of the Honey Badger team:

Saturday, July 22, 2017

Queues vs Distributed logs (tech pill)

TL;DR A queue is a good choice when you have one kind of job to do that you can divide in independent smaller jobs that can execute in any order and a distributed log is a good choice when you have several kinds of jobs or functionalities for the same stream of data (logs, events, etc.).

Queues vs Distributed Logs

This blog post tries to explain the typical use for Queues and for Distributed Log, but of course, a system usually uses these solutions in combination or in other ways. But I will try to summarize the usual use because some times I see some confusion about how we can use these interesting solutions.

Queue



Use a Queue when:

  • You want to distribute workload between workers for the same feature. 
  • Each message can represent a job and don't require knowledge about other messages at the queue.

Pros:

  • Easy to implement.
  • Unlimited/Easy horizontal scalability.
  • Fault tolerance is very easy to implement (time out and re-queue of the message).
  • Allow easy balance between latency (time at the queue + processing time) and the cost of the concurrent workers.

Cons:

  • The order is not guaranteed.
  • Each message can only be consumed by one worker for one feature.
  • Usually, we can have duplicates.

Examples

  • Asynchronous processing of email requests.
  • Processing of independent batch jobs.
  • A supermarket queue :)

Alternatives and related variations

  • Queues with guaranteed order (Ex: SQS FIFO).
  • TTL for messages in the Queue.
  • Queues with a max amount of queued messages.

Implementations:

  • AWS SQS, RabbitMQ

Distributed Log


Use a Distributed Log when:

  • You want to execute several functionalities for the same stream of ordered data (log aggregation, statistics calculation, event sourcing, store streams of information, etc.)
  • You want that the data corresponding to the same partition key is processed in order by the same worker instance.

Pros:

  • Allow several functionalities for the same data. Each functionality knows what is his own offset.
  • If two or more functionalities are at the same offset, they can be sure that have seen the same messages in the same order.
  • Having a guaranteed order,  at least at the partition key level, allow design simple solution for calculating statistics, event sourcing, near real-time calculation, etc. 
  • You can add new functionalities that use/process the same data with any change in the actual system (Open/Closed principle: open for extension but closed for modification).

Cons:

  • Usually, the order of the messages is preserved for each partition key only.
  • Scalability using sharding/partitioning.
  • More difficult than queues.
  • Difficult to have balanced shards/partitions (hot partition problem).

Examples

  • Log aggregation / distribution / statistics calculation from the logs.
  • Event streaming to allow event sourcing.

Implementations:

  • AWS Kinesis, Kafka

Practical scalability example:

In this example we have the following:

  • One logical stream
  • Divided into two shards
  • Each partition key (pk) will always be associated with a shard or partition
  • We have one function that runs in two concurrent processes, so the speed of processing is x2. Each process always read events/messages for the same partition keys.

In this example, the first process reads events/messages from the partition key 1 and from the partition key 2.

If the shard s1 have too many event/messages per second, we should scale out dividing the shard s1 into two sub shards.


Now, our speed is x3, and we have one process for each partition key.
As we can see the scalability is not trivial because we should detect when a shard/partition should be divide and rebalance the shards/partition between the running processes.


References:

Saturday, July 15, 2017

Honey badger Team / The visual history II


Disclaimer: I am experimenting with different formats to create presentations, blog post, and other documents, mixing sketch noting and using index cards... I will appreciate your feedback.

In the previous post about the history of the Honey Badger Team, the focus was to define the context, explain how we evolve our practices, our culture and get our identity as a team. But in parallel, there is a history of tension and lot of work to make this change possible :)

As in any software product development effort, the path was not easy. Perhaps the first blog post doesn't reflect the amount of energy and discussions required to introduce quality and good practices in our team.

During this process (one year and a half), there was tension between different forces: time, scope, learning needs, maintainability, quality, short term value, long term value...

As usual, the main problem is that not all the company have the same experience about technology development so is very difficult to reach a complete alignment for a sustainable strategy...

This was my job or at least part of my job, and for sure, it wasn't easy and requires a lot of energy, patience, and continuous learning...  Learn about how to sell ideas, learn about how to get some margin, learn about how to generate trust...
I also tried to teach about agile concepts, lean, collaboration, etc.

During part of this journey we also had a CTO that helped us to defend the need for change, so even if the change process was already there, is true that he helped me to accelerate the process a little bit (for example he negotiated a percentage of time to improve our test network). He also helped me to translate technical concepts to the C level.


The main problem, as in any complex system that involves humans, was the communication. Seems that we use a completely different vocabulary and even language. I try to solve this miscommunication trying to learn about our business, gain trust and adapting our message to the actual understanding of the technology.


There was an inflection point in the process that could cause the collapse of the team. We started to work as a team, introduce the XP practices and improve day by day, but in parallel, there was some misalignments at the company level that was introducing a lot of structure, a lot of levels and an environment not very aligned with our spirit of collaboration...
This problem drained the motivation of the tech team, that feel very good improvements regarding our internal processes but feel the friction generated by this amount of structure and the misalignment.

This process ended with some deep changes at the company level, but during this period of time, we lost great engineers and great colleagues :(  Personally, this was a rough time for me.
Seems that we have overcome that problem and that we are in the right direction, adding new honey badgers to the team.

In summary, the first blog post explained the internal changes but this doesn't reflect the whole history... in parallel, there was a huge effort to improve the communication between each group in the company, deal with all the typical tensions, and a lot of small changes to introduce a more agile culture in times of huge uncertainty...

And I can assure that this path was not easy... required patience and energy...   and this are a scarce resource :)

Reference:



Monday, July 10, 2017

Charlas interesantes (Junio/Julio 2017)


Estas son algunas charlas muy interesante que he visto últimamente:

Saturday, July 08, 2017

Honey badger Team / The visual history (I)

Disclaimer: I am experimenting with different formats to create presentations, blog post, and other documents, mixing sketch noting and using index cards... I will appreciate your feedback.

I prepared these notes for the past AOS 2017, but finally, the session was not selected... The initial idea was to describe our process for creating an agile software development team and create a discussion to identify next steps to expand our agile culture to the rest of the organization


Honey Badger Team / The visual history



In the beginning, there was a bunch of developers at TheMotion starting to create the product. But there was no process and the "established" tendency was to try to optimize the resource usage. As we already know, optimizing the resource usage have some serious problems:
  1. People are not resources :) (so this not work very well outside some simple industrial processes/pipelines that don't involve humans)
  2. This process generates busy work but not necessarily good outcomes.
  3. To optimize the resource usage, we need:
    1. Push until all of the resources are 100% used
    2. To create buffers to have everybody busy
    3. Generate SILOs / Specialization for each skill (front end, backend, video)
The result of this process was:
  • An incredible high bus factor
  • Stress and bad quality code and practices
  • Tons of WIP/inventory and unneeded features as we develop based on how is available (frontend, backend...)
I arrived at this point to the company... There was a Funny party going on :)

One thing was clear (at least for me), using a Lean and Agile approach can help to create a product in a startup.... It also can help to improve the overall frustration :)

I am completely biased because I like the agile mindset... but I sincerely think that in a startup the agile approach is the right choice:  Fail (agile is not recipe for success) and learn as fast as possible :)

For me, Agile is a set of values and principles, not a methodology and two pillars:
  • A healthy culture focused on people (collaboration, respect, teamwork, creativity...) 
  • Looking for quality and technical excellence (for example XP practices are a great starting point).

So my first target was introducing quality and technical excellence to be capable of delivering software in small cycles (days instead of weeks)... My idea was to introduce the culture at the same time that we improve our technical capacity. Any software developer is more open to a new ideas if he shows improvements in the code and in his day to day work. :)

But how can we detect if we are in the right path?
I discovered that a good KPI for measure these improvements was how much anxiety was generated in each team member when we change the frequency of a development event (deploy, merge, etc...)
For example, the target was:
  • Deploys frequency, from 1 a month, to several per day (the deploy is not an event).
  • Integration frequency, from branches that last forever to trunk based development.
  • Feedback time:
    • For review, from code reviews and pull request to pairing and continuous review.
    • Subsecond for unit tests.
    • Seconds for integration tests.
    • Minutes for production monitoring.


But How can we improve the quality and our technical capacity???

For me, the response is agile development, XP practices and modern agile... In summary being professionals and pragmatic delivering software. And mentoring is the only way I know to introduce this technical practices....


Initialy, I was the only one that started to send the message about XP, Craftmanship, cloud native, etc... At the same time, we were trying to improve the day-to-day work improving the infrastructure and the base platform.

We, as a team, was making improvements, but learning TDD or pairing have an important entry barrier so we need external help with experience teaching this practices (by doing of course).


To improve the "mentoring power", the collaboration of Modesto San Juan and Alfredo Casado from Carlos Ble y Asociados (now codesai) was key. They help us to improve our TDD and to introduce other XP practices (shared ownership, pairing, etc). At this point, our internal alignment started to be evident.

The first day that I detected that the mentoring effort was having a profound effect was when one developer, talking aloud, explained what Modesto or Alfredo would say about his code, and immediately he started to improve the code :-)

Another key point was that from day 0, we assumed that everybody wants to make a good job so the smarter approach to improve is to "Make easy to make the right thing". Instead on focus in preaching how the things should be done and instead of forcing to do the things in a certain way, our focus was on creating an awesome tooling and platform that make very easy to create microservices, include instrumentation, metrics...  We pushed very hard to have an automated infrastructure and a very good tooling.... This includes rolling deploys, standard makefiles, a devbox (now deprecated), containers, etc.a great monitoring.

With this platform and this tooling, it was easy to introduce a DevOps culture, with microservices and improve our deployment cadence day by day.

An important note here.... during this process, the most important factor was to use an agile process and make small improvements, with low risk in parallel to the normal development...



Right now, our practices include, continuous delivery (several times a day), automatic tests (TDD in the way but not the default option yet), simple design, immutable infrastructure, infrastructure as code, collective ownership and we work as a high-performance team with our own identity and a high level of trust….
We have a lot of things to improve, including our quality at the front end (software design and practices), more solid understanding and usage of the XP practices, and so on... but I can say that the agile mindset, the learning culture, and the eager to improve, are already there. :)

We continue our journey, improving continuously...


At a company level, we passed for some periods with lack of alignment, but this is changing very fast and seems that in the next months we will have very clear goals at a company level. This is great and can help us with our next challenge... Expand the agile mindset to the rest of the organization...

A final note:

IMHO now that we are capable of creating/evolving a software product, is the right time to expand this culture to the rest of the organization... I see a lot of times that there is a huge effort trying to introduce the agile mindset at the company level but without having a solid technical capacity, and I think that this is a huge mistake for software/tech based companies. In such cases, the story usually goes as follows:  Huge effort to transmit the benefits of the agile mindset, everybody is very excited, the change begins, but the core of the systems is not capable of generating software at a sustainable pace, the technical debt begins to grow without control,  and the product collapse... The cultural change fails, and everybody thinks that Agile doesn't work...

The explanation is easier than that: You don't know how to make software. So first, learn how to make software and later you can introduce other parts of agile. (See: the two pillars of agile).

A culture of Agile software development with a good quality is a prerequisite for introducing Agile at a company level for software based companies.


References:


Thanks:

  • To Jorge Jardines for the fast feedback and some improvements
  • To the Honey Badger team :)


Sunday, July 02, 2017

AOS Agile Open Space 2017 Segovia

Disclaimer: I am experimenting with different formats to create presentations, blog post, and other documents, mixing sketch noting and using index cards... I will appreciate your feedback.




The past 23,24th of June I was at the Agile Open Space at Segovia (AOS)... This is my sixth consecutive year going to the AOS (you can see almost all of my past experiences at http://www.eferro.net/search/label/aos). I like this unconference because is a great excuse to see friends and enjoy great conversations with other "agilistas".

The Agile community in Spain organizes two events, the CAS that is a regular conference and the AOS that is an Open Space. The AOS instead or being the classic conference with a clear separation between speakers and attendees, is more like a meeting of community members... something like a community of practice of the members of Agile Spain.

As the community evolves, the AOS evolves also and reflect this change. Over the years I've been seeing how the community is shifting from a community of software developers to a community focused on organizational change and agile outside the software products.

I have come to the conclusion that in order to not feel frustration I should not go to AOS as a software developer. :)

I will write a blog post about this change and the relation between software development, agile and craftsmanship.

The Sessions


Sessions I attended or participated:
  • Parallel Changes: I facilitate this session. The idea was to explain why is important to be capable of making huge changes in software using a sequence of multiple small changes with low risk. The first 10min I explained why this is a good idea and showed some samples. The rest of the session was working in groups and presenting other examples or related problems/challenges to solve. I will create a specific blog post for this.
  • Cómo combatir el efecto Pokemon: Interesting session about a usual problem about having a lot of tasks that can be only done by some people of the team. A classical problem is for UX/design tasks or with some low level debugging stuff or DB optimization. Lot of people at that session had this problem, for all the tasks they had… :( The reason is simple, they allow working each member of the team individually and don't rotate… they allow rambos, heroes, and other kinds of “good developers” that can’t work in teams…. (I mean BAD developers). If we think that developing is a team effort, there is an easy solution to avoid this problem: XP, multidisciplinary teams, pairing, mob programming, share ownership… Is easy, hire people to work in teams and help them to work as a team.
  • Facilitación gráfica / Visual thinking: A good session about how to express drawing and how to take notes… This was an introductory talk and have three parts, an ice breaking dynamic, some tips to draw the most common shapes (faces, titles, people, etc) and a final dynamic to collaboratively make a visual dictionary with difficult words as transparency, value, and so on… very abstract words that are difficult to represent. This was interesting because each one tries to express these words drawing so you can take some of the ideas…. I am trying to improve my visual thinking and sketch noting skills so this session helps me in that path….
  • Disciplina positiva y Agile Kids: I was especially interested in this session.. because for me is so natural the agile mindset and continuous improvement that is easy to think that is also useful for personal life and kids education. I attended to this presentation with my wife to take notes for future family steps… The session was a presentation of the experience of two parents following the advice from the book: Agile Kids ( niños agiles )
  • Slow: The session of this session was about how we can use some techniques from the Slow movement to improve our day-to-day capacity and life. We experience some exercises, lot of them related with mindfulness. https://www.slideshare.net/twallet/pequeas-tcnicas-para-un-da-slow
  • Autoexigencia de los equipos: (https://twitter.com/artzis) The format of this session was a Fishbowl. The topics were, should we expect that the team has self-demand / self-discipline? why? how? and the individual? The conversation was very interesting and has some good insights about motivation, learning and hiring the right people…



Final conclusions and Notes

The sessions are important but for me the best of this AOS at Segovia was:

  • I went to Segovia with my family... my wife and my daughter... :)  this was great and have an important effect regarding our internal organization as a family... We even are testing using a kanban board.
  • A lot of my coworkers come and enjoyed the power of the self-organization and the agile mindset... I am quite sure that this will have a strong effect on the culture of our company. (this is part of my plan to conquer the world) :)
  • I enjoyed a lot talking with old friends, reinforcing relations and even creating new ones... Not bad for an introvert software developer.



Thanks


I have special thanks to the organization, Delia Estebaranz, Javier Garcia Garrido from agiletorrezno for making this conference a pleasure for going with the family. Having the opportunity of enjoying the conference as a family was GREAT.

Lot of thanks for the volunteers.

Also thanks to the sponsors of the conference for making it possible: Deiser, USTGlobal, Jerónimo Palacios, Thinking With You, Ticketea, Agilar, Biko2, Flywire, Look Forward Consulting, Runroom, Kairos, Colegio Profesional de Ingenieros en Informática de Castilla y León, Segovia Activa, Kleer...





Other interesting blog posts about this AOS edition:

Sunday, June 18, 2017

Pamplona Software Craftmanship 2017




Este pasado 2 y 3 de Junio tuvo lugar la Pamplona Software Craftmanship 2017... Este blog post es mi crónica de la mitad del evento, puesto que el colegio de mi hija decidió que el 2 de junio era buen día para hacer la graduación del ciclo de infantil... así que la primera sesión a la que pude llegar fue la última del primer día.



Bueno, vamos al tema...

Sesiones:

Estas son las sesiones a las que puede asistir:
  • Los programadores no tienen que ser malos diseñadores de interfaces (Sebastian Hermida) (Video). Me aporto mucho puesto me permitió entender las diferentes practicas relacionadas a la experiencia de usuario y al diseño y las explico contraponiendolas a conceptos de desarrollo.
  • Arquitecturas, arquitectos y otros animales mitológicos. (Vicenç García-Altés) (Video) De esta charla me he quedado con algunas herramientas, reuniones, y artefactos a investigar para identificar capacidades del sistema y conceptos del negocio que afectan a la arquitectura. 
  • Speed Dating (Sebastian Hermida) (Open Space) Interesante dinámica para conocer gente, que es uno de los puntos fuertes de una conferencia de craftmanship.... conocer otros profesionales interesantes e intercambiar conocimiento. 
  • Onboarding (Angélica). No pude estar en toda la sesión así que solo pude escuchar parte. Se genero una especie de mesa redonda explicando como hacían en diversas empresas el onboarding de nuevos compañeros... me quede con algunas ideas para implementar.
  • El arte del patadon pa'lante / Libro de recetas (artolamola & eferro (yo mismo)) (Video) En la charla intente dar ejemplos prácticos de cómo podemos posponer decisiones y evitar trabajo innecesario, teniendo así la oportunidad de atacar más problemas. Cualquier feedback en referencia a la charla será más que bienvenido.
  • Cuentame tu stack  (Vicenç García-Altés) (Open Space) Minipresentaciones de distintas arquitecturas y stacks tecnológicos. Un formato de sesión muy interesante y que invita a la conversación sobre arquitectura partiendo de ejemplos reales.
  • Power point karaoke (Laura Morillo). Sin duda una de las sesiones más divertidas. Para el que no lo conozca, el power point karaoke, consiste en que  se eligen presentaciones cortas y se saca a gente que no ha visto jamas esas presentaciones a que las presenten... Un descojone.
  • Building a transactional function pipeline (with TDD and Elixir) (https://twitter.com/jacegu) (Video) Interesante presentación de Javi, en la que mostró como usar de forma casi mecánica una practica como TDD le permitió avanzar en un contexto desconocido para el. Muchas veces nos olvidamos de como una practica nos puede ayudar dando estrategias para solventar nuevos problemas. Muy interesante.

Como siempre, además de las charlas en las que estuve, lo mejor de un open space son las conversaciones que se producen en todo momento... dentro y fuera de la conferencia... Poder compartir con compañeros profesionales y amigos nuestra pasión, es lo más valioso de una conferencia de este tipo. Por otro lado,  con cada conferencia que pasa, se amplia el número de compañeros a los que puedo considerar amigos :-)



Charlas grabadas que he visto

Por otro lado, y gracias a la inestimable ayuda de Autentia,  algunas de las charlas a las que no pude asistir fueron grabadas así que me las estoy viendo poco a poco.... De ellas, las que más me han impactado de momento son:



Hay otras que también están muy bien, pero que no me han impactado tanto (en algunos casos porque el tema ya lo conocía):



Tenéis el post de Autentia con todas las grabaciones en:
https://www.autentia.com/2017/06/13/pamplona-software-craftsmanship-2017/

Resumen y agradecimientos


Como resumen general, puedo decir que la organización de la conferencia es de lo mejor que he visto. La gente de 540, los organizadores, son unos cracks y no solo están atentos a todos los detalles y desarrollan como campeones sino que además son unos genios del marketing.
La verdad es que me extraño que no terminase por venir Martin Fowler a la conferencia, puesto que si alguien lo puede conseguir son ellos...

Me encanta el formato, me encanta el contenido, me encanta reencontrarme con amigos, me encanta que pongan Kalimotxo y me parecen que los amigos de 540 son lo más TOP!!!

Thursday, June 15, 2017

Kent Beck 3x (Great idea that deserves our time)




The life of a product, a tech company, and a software system is a continuous process of decision making... and the best answer for each of these decisions is always "it depends". It always depends on the context and the phase of the product/company/system...
Lately, Kent Beck is presenting a framework to better understand and define this "needed" context information. This great proposal, which he calls 3x (explore, expand, extract), helps us to understand the trade-offs and make better decisions.

We can read as an introduction:
And for the original notes:
And there are some talks about this matter at:
Any comment or discussion about this interesting concept is more than welcome :)


Saturday, June 10, 2017

Quotes about software architecture

Interesting quotes that can inspire you as software developer or at least make you think :)

"In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called ‘architecture.’ This understanding includes how the system is divided into components and how the components interact through interfaces. These components are usually composed of smaller components, but the architecture only includes the components and interfaces that are understood by all the developers."  Ralph Johnson
"A complex system that works is invariably found to have evolved from a simple system that worked." John Gall
 "Architecture is the decisions that you wish you could get right early in a project." Ralph Johnson
"So you might end up defining architecture as things that people perceive as hard to change." Martin Fowler
"I think that one of an architect's most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs." Martin Fowler
"Complexity is what makes software hard to change. That, and duplication." Ralph Johnson
"irreversibility was one of the prime drivers of complexity" Enrico Zaninotto

"If you don’t actively fight for simplicity in software, complexity will win…and it will suck." - @HenrikJoreteg

References about Evolutionary Architectures

After my talk El arte del patadon pa'lante / Libro de recetas (Spanish) at the Pamplona Software Craftmanship 2017, some colleagues ask me about references for evolutionary architectures. This post tries to give the best references I know about this concept...

To have context for the other references, I find the following resources essential:


As general resources:

You also need techniques to make changes to things that traditionally are "hard to change"... For this Martin Fowler bliki is a gold mine:

To document all the decisions you make to gain the sharing understanding of the design:



We also need a lot of tricks for our bag of tricks to have different approaches for different problems... Related with this, I think is important to have knowledge about: