Saturday, July 22, 2017

Queues vs Distributed logs (tech pill)

TL;DR A queue is a good choice when you have one kind of job to do that you can divide in independent smaller jobs that can execute in any order and a distributed log is a good choice when you have several kinds of jobs or functionalities for the same stream of data (logs, events, etc.).

Queues vs Distributed Logs

This blog post tries to explain the typical use for Queues and for Distributed Log, but of course, a system usually uses these solutions in combination or in other ways. But I will try to summarize the usual use because some times I see some confusion about how we can use these interesting solutions.

Queue



Use a Queue when:

  • You want to distribute workload between workers for the same feature. 
  • Each message can represent a job and don't require knowledge about other messages at the queue.

Pros:

  • Easy to implement.
  • Unlimited/Easy horizontal scalability.
  • Fault tolerance is very easy to implement (time out and re-queue of the message).
  • Allow easy balance between latency (time at the queue + processing time) and the cost of the concurrent workers.

Cons:

  • The order is not guaranteed.
  • Each message can only be consumed by one worker for one feature.
  • Usually, we can have duplicates.

Examples

  • Asynchronous processing of email requests.
  • Processing of independent batch jobs.
  • A supermarket queue :)

Alternatives and related variations

  • Queues with guaranteed order (Ex: SQS FIFO).
  • TTL for messages in the Queue.
  • Queues with a max amount of queued messages.

Implementations:

  • AWS SQS, RabbitMQ

Distributed Log


Use a Distributed Log when:

  • You want to execute several functionalities for the same stream of ordered data (log aggregation, statistics calculation, event sourcing, store streams of information, etc.)
  • You want that the data corresponding to the same partition key is processed in order by the same worker instance.

Pros:

  • Allow several functionalities for the same data. Each functionality knows what is his own offset.
  • If two or more functionalities are at the same offset, they can be sure that have seen the same messages in the same order.
  • Having a guaranteed order,  at least at the partition key level, allow design simple solution for calculating statistics, event sourcing, near real-time calculation, etc. 
  • You can add new functionalities that use/process the same data with any change in the actual system (Open/Closed principle: open for extension but closed for modification).

Cons:

  • Usually, the order of the messages is preserved for each partition key only.
  • Scalability using sharding/partitioning.
  • More difficult than queues.
  • Difficult to have balanced shards/partitions (hot partition problem).

Examples

  • Log aggregation / distribution / statistics calculation from the logs.
  • Event streaming to allow event sourcing.

Implementations:

  • AWS Kinesis, Kafka

Practical scalability example:

In this example we have the following:

  • One logical stream
  • Divided into two shards
  • Each partition key (pk) will always be associated with a shard or partition
  • We have one function that runs in two concurrent processes, so the speed of processing is x2. Each process always read events/messages for the same partition keys.

In this example, the first process reads events/messages from the partition key 1 and from the partition key 2.

If the shard s1 have too many event/messages per second, we should scale out dividing the shard s1 into two sub shards.


Now, our speed is x3, and we have one process for each partition key.
As we can see the scalability is not trivial because we should detect when a shard/partition should be divide and rebalance the shards/partition between the running processes.


References:

Saturday, July 15, 2017

Honey badger Team / The visual history II


Disclaimer: I am experimenting with different formats to create presentations, blog post, and other documents, mixing sketch noting and using index cards... I will appreciate your feedback.

In the previous post about the history of the Honey Badger Team, the focus was to define the context, explain how we evolve our practices, our culture and get our identity as a team. But in parallel, there is a history of tension and lot of work to make this change possible :)

As in any software product development effort, the path was not easy. Perhaps the first blog post doesn't reflect the amount of energy and discussions required to introduce quality and good practices in our team.

During this process (one year and a half), there was tension between different forces: time, scope, learning needs, maintainability, quality, short term value, long term value...

As usual, the main problem is that not all the company have the same experience about technology development so is very difficult to reach a complete alignment for a sustainable strategy...

This was my job or at least part of my job, and for sure, it wasn't easy and requires a lot of energy, patience, and continuous learning...  Learn about how to sell ideas, learn about how to get some margin, learn about how to generate trust...
I also tried to teach about agile concepts, lean, collaboration, etc.

During part of this journey we also had a CTO that helped us to defend the need for change, so even if the change process was already there, is true that he helped me to accelerate the process a little bit (for example he negotiated a percentage of time to improve our test network). He also helped me to translate technical concepts to the C level.


The main problem, as in any complex system that involves humans, was the communication. Seems that we use a completely different vocabulary and even language. I try to solve this miscommunication trying to learn about our business, gain trust and adapting our message to the actual understanding of the technology.


There was an inflection point in the process that could cause the collapse of the team. We started to work as a team, introduce the XP practices and improve day by day, but in parallel, there was some misalignments at the company level that was introducing a lot of structure, a lot of levels and an environment not very aligned with our spirit of collaboration...
This problem drained the motivation of the tech team, that feel very good improvements regarding our internal processes but feel the friction generated by this amount of structure and the misalignment.

This process ended with some deep changes at the company level, but during this period of time, we lost great engineers and great colleagues :(  Personally, this was a rough time for me.
Seems that we have overcome that problem and that we are in the right direction, adding new honey badgers to the team.

In summary, the first blog post explained the internal changes but this doesn't reflect the whole history... in parallel, there was a huge effort to improve the communication between each group in the company, deal with all the typical tensions, and a lot of small changes to introduce a more agile culture in times of huge uncertainty...

And I can assure that this path was not easy... required patience and energy...   and this are a scarce resource :)

Reference:



Monday, July 10, 2017

Charlas interesantes (Junio/Julio 2017)


Estas son algunas charlas muy interesante que he visto últimamente:

Saturday, July 08, 2017

Honey badger Team / The visual history (I)

Disclaimer: I am experimenting with different formats to create presentations, blog post, and other documents, mixing sketch noting and using index cards... I will appreciate your feedback.

I prepared these notes for the past AOS 2017, but finally, the session was not selected... The initial idea was to describe our process for creating an agile software development team and create a discussion to identify next steps to expand our agile culture to the rest of the organization


Honey Badger Team / The visual history



In the beginning, there was a bunch of developers at TheMotion starting to create the product. But there was no process and the "established" tendency was to try to optimize the resource usage. As we already know, optimizing the resource usage have some serious problems:
  1. People are not resources :) (so this not work very well outside some simple industrial processes/pipelines that don't involve humans)
  2. This process generates busy work but not necessarily good outcomes.
  3. To optimize the resource usage, we need:
    1. Push until all of the resources are 100% used
    2. To create buffers to have everybody busy
    3. Generate SILOs / Specialization for each skill (front end, backend, video)
The result of this process was:
  • An incredible high bus factor
  • Stress and bad quality code and practices
  • Tons of WIP/inventory and unneeded features as we develop based on how is available (frontend, backend...)
I arrived at this point to the company... There was a Funny party going on :)

One thing was clear (at least for me), using a Lean and Agile approach can help to create a product in a startup.... It also can help to improve the overall frustration :)

I am completely biased because I like the agile mindset... but I sincerely think that in a startup the agile approach is the right choice:  Fail (agile is not recipe for success) and learn as fast as possible :)

For me, Agile is a set of values and principles, not a methodology and two pillars:
  • A healthy culture focused on people (collaboration, respect, teamwork, creativity...) 
  • Looking for quality and technical excellence (for example XP practices are a great starting point).

So my first target was introducing quality and technical excellence to be capable of delivering software in small cycles (days instead of weeks)... My idea was to introduce the culture at the same time that we improve our technical capacity. Any software developer is more open to a new ideas if he shows improvements in the code and in his day to day work. :)

But how can we detect if we are in the right path?
I discovered that a good KPI for measure these improvements was how much anxiety was generated in each team member when we change the frequency of a development event (deploy, merge, etc...)
For example, the target was:
  • Deploys frequency, from 1 a month, to several per day (the deploy is not an event).
  • Integration frequency, from branches that last forever to trunk based development.
  • Feedback time:
    • For review, from code reviews and pull request to pairing and continuous review.
    • Subsecond for unit tests.
    • Seconds for integration tests.
    • Minutes for production monitoring.


But How can we improve the quality and our technical capacity???

For me, the response is agile development, XP practices and modern agile... In summary being professionals and pragmatic delivering software. And mentoring is the only way I know to introduce this technical practices....


Initialy, I was the only one that started to send the message about XP, Craftmanship, cloud native, etc... At the same time, we were trying to improve the day-to-day work improving the infrastructure and the base platform.

We, as a team, was making improvements, but learning TDD or pairing have an important entry barrier so we need external help with experience teaching this practices (by doing of course).


To improve the "mentoring power", the collaboration of Modesto San Juan and Alfredo Casado from Carlos Ble y Asociados (now codesai) was key. They help us to improve our TDD and to introduce other XP practices (shared ownership, pairing, etc). At this point, our internal alignment started to be evident.

The first day that I detected that the mentoring effort was having a profound effect was when one developer, talking aloud, explained what Modesto or Alfredo would say about his code, and immediately he started to improve the code :-)

Another key point was that from day 0, we assumed that everybody wants to make a good job so the smarter approach to improve is to "Make easy to make the right thing". Instead on focus in preaching how the things should be done and instead of forcing to do the things in a certain way, our focus was on creating an awesome tooling and platform that make very easy to create microservices, include instrumentation, metrics...  We pushed very hard to have an automated infrastructure and a very good tooling.... This includes rolling deploys, standard makefiles, a devbox (now deprecated), containers, etc.a great monitoring.

With this platform and this tooling, it was easy to introduce a DevOps culture, with microservices and improve our deployment cadence day by day.

An important note here.... during this process, the most important factor was to use an agile process and make small improvements, with low risk in parallel to the normal development...



Right now, our practices include, continuous delivery (several times a day), automatic tests (TDD in the way but not the default option yet), simple design, immutable infrastructure, infrastructure as code, collective ownership and we work as a high-performance team with our own identity and a high level of trust….
We have a lot of things to improve, including our quality at the front end (software design and practices), more solid understanding and usage of the XP practices, and so on... but I can say that the agile mindset, the learning culture, and the eager to improve, are already there. :)

We continue our journey, improving continuously...


At a company level, we passed for some periods with lack of alignment, but this is changing very fast and seems that in the next months we will have very clear goals at a company level. This is great and can help us with our next challenge... Expand the agile mindset to the rest of the organization...

A final note:

IMHO now that we are capable of creating/evolving a software product, is the right time to expand this culture to the rest of the organization... I see a lot of times that there is a huge effort trying to introduce the agile mindset at the company level but without having a solid technical capacity, and I think that this is a huge mistake for software/tech based companies. In such cases, the story usually goes as follows:  Huge effort to transmit the benefits of the agile mindset, everybody is very excited, the change begins, but the core of the systems is not capable of generating software at a sustainable pace, the technical debt begins to grow without control,  and the product collapse... The cultural change fails, and everybody thinks that Agile doesn't work...

The explanation is easier than that: You don't know how to make software. So first, learn how to make software and later you can introduce other parts of agile. (See: the two pillars of agile).

A culture of Agile software development with a good quality is a prerequisite for introducing Agile at a company level for software based companies.


References:


Thanks:

  • To Jorge Jardines for the fast feedback and some improvements
  • To the Honey Badger team :)


Sunday, July 02, 2017

AOS Agile Open Space 2017 Segovia

Disclaimer: I am experimenting with different formats to create presentations, blog post, and other documents, mixing sketch noting and using index cards... I will appreciate your feedback.




The past 23,24th of June I was at the Agile Open Space at Segovia (AOS)... This is my sixth consecutive year going to the AOS (you can see almost all of my past experiences at http://www.eferro.net/search/label/aos). I like this unconference because is a great excuse to see friends and enjoy great conversations with other "agilistas".

The Agile community in Spain organizes two events, the CAS that is a regular conference and the AOS that is an Open Space. The AOS instead or being the classic conference with a clear separation between speakers and attendees, is more like a meeting of community members... something like a community of practice of the members of Agile Spain.

As the community evolves, the AOS evolves also and reflect this change. Over the years I've been seeing how the community is shifting from a community of software developers to a community focused on organizational change and agile outside the software products.

I have come to the conclusion that in order to not feel frustration I should not go to AOS as a software developer. :)

I will write a blog post about this change and the relation between software development, agile and craftsmanship.

The Sessions


Sessions I attended or participated:
  • Parallel Changes: I facilitate this session. The idea was to explain why is important to be capable of making huge changes in software using a sequence of multiple small changes with low risk. The first 10min I explained why this is a good idea and showed some samples. The rest of the session was working in groups and presenting other examples or related problems/challenges to solve. I will create a specific blog post for this.
  • Cómo combatir el efecto Pokemon: Interesting session about a usual problem about having a lot of tasks that can be only done by some people of the team. A classical problem is for UX/design tasks or with some low level debugging stuff or DB optimization. Lot of people at that session had this problem, for all the tasks they had… :( The reason is simple, they allow working each member of the team individually and don't rotate… they allow rambos, heroes, and other kinds of “good developers” that can’t work in teams…. (I mean BAD developers). If we think that developing is a team effort, there is an easy solution to avoid this problem: XP, multidisciplinary teams, pairing, mob programming, share ownership… Is easy, hire people to work in teams and help them to work as a team.
  • Facilitación gráfica / Visual thinking: A good session about how to express drawing and how to take notes… This was an introductory talk and have three parts, an ice breaking dynamic, some tips to draw the most common shapes (faces, titles, people, etc) and a final dynamic to collaboratively make a visual dictionary with difficult words as transparency, value, and so on… very abstract words that are difficult to represent. This was interesting because each one tries to express these words drawing so you can take some of the ideas…. I am trying to improve my visual thinking and sketch noting skills so this session helps me in that path….
  • Disciplina positiva y Agile Kids: I was especially interested in this session.. because for me is so natural the agile mindset and continuous improvement that is easy to think that is also useful for personal life and kids education. I attended to this presentation with my wife to take notes for future family steps… The session was a presentation of the experience of two parents following the advice from the book: Agile Kids ( niños agiles )
  • Slow: The session of this session was about how we can use some techniques from the Slow movement to improve our day-to-day capacity and life. We experience some exercises, lot of them related with mindfulness. https://www.slideshare.net/twallet/pequeas-tcnicas-para-un-da-slow
  • Autoexigencia de los equipos: (https://twitter.com/artzis) The format of this session was a Fishbowl. The topics were, should we expect that the team has self-demand / self-discipline? why? how? and the individual? The conversation was very interesting and has some good insights about motivation, learning and hiring the right people…



Final conclusions and Notes

The sessions are important but for me the best of this AOS at Segovia was:

  • I went to Segovia with my family... my wife and my daughter... :)  this was great and have an important effect regarding our internal organization as a family... We even are testing using a kanban board.
  • A lot of my coworkers come and enjoyed the power of the self-organization and the agile mindset... I am quite sure that this will have a strong effect on the culture of our company. (this is part of my plan to conquer the world) :)
  • I enjoyed a lot talking with old friends, reinforcing relations and even creating new ones... Not bad for an introvert software developer.



Thanks


I have special thanks to the organization, Delia Estebaranz, Javier Garcia Garrido from agiletorrezno for making this conference a pleasure for going with the family. Having the opportunity of enjoying the conference as a family was GREAT.

Lot of thanks for the volunteers.

Also thanks to the sponsors of the conference for making it possible: Deiser, USTGlobal, Jerónimo Palacios, Thinking With You, Ticketea, Agilar, Biko2, Flywire, Look Forward Consulting, Runroom, Kairos, Colegio Profesional de Ingenieros en Informática de Castilla y León, Segovia Activa, Kleer...





Other interesting blog posts about this AOS edition:

Sunday, June 18, 2017

Pamplona Software Craftmanship 2017




Este pasado 2 y 3 de Junio tuvo lugar la Pamplona Software Craftmanship 2017... Este blog post es mi crónica de la mitad del evento, puesto que el colegio de mi hija decidió que el 2 de junio era buen día para hacer la graduación del ciclo de infantil... así que la primera sesión a la que pude llegar fue la última del primer día.



Bueno, vamos al tema...

Sesiones:

Estas son las sesiones a las que puede asistir:
  • Los programadores no tienen que ser malos diseñadores de interfaces (Sebastian Hermida) (Video). Me aporto mucho puesto me permitió entender las diferentes practicas relacionadas a la experiencia de usuario y al diseño y las explico contraponiendolas a conceptos de desarrollo.
  • Arquitecturas, arquitectos y otros animales mitológicos. (Vicenç García-Altés) (Video) De esta charla me he quedado con algunas herramientas, reuniones, y artefactos a investigar para identificar capacidades del sistema y conceptos del negocio que afectan a la arquitectura. 
  • Speed Dating (Sebastian Hermida) (Open Space) Interesante dinámica para conocer gente, que es uno de los puntos fuertes de una conferencia de craftmanship.... conocer otros profesionales interesantes e intercambiar conocimiento. 
  • Onboarding (Angélica). No pude estar en toda la sesión así que solo pude escuchar parte. Se genero una especie de mesa redonda explicando como hacían en diversas empresas el onboarding de nuevos compañeros... me quede con algunas ideas para implementar.
  • El arte del patadon pa'lante / Libro de recetas (artolamola & eferro (yo mismo)) (Video) En la charla intente dar ejemplos prácticos de cómo podemos posponer decisiones y evitar trabajo innecesario, teniendo así la oportunidad de atacar más problemas. Cualquier feedback en referencia a la charla será más que bienvenido.
  • Cuentame tu stack  (Vicenç García-Altés) (Open Space) Minipresentaciones de distintas arquitecturas y stacks tecnológicos. Un formato de sesión muy interesante y que invita a la conversación sobre arquitectura partiendo de ejemplos reales.
  • Power point karaoke (Laura Morillo). Sin duda una de las sesiones más divertidas. Para el que no lo conozca, el power point karaoke, consiste en que  se eligen presentaciones cortas y se saca a gente que no ha visto jamas esas presentaciones a que las presenten... Un descojone.
  • Building a transactional function pipeline (with TDD and Elixir) (https://twitter.com/jacegu) (Video) Interesante presentación de Javi, en la que mostró como usar de forma casi mecánica una practica como TDD le permitió avanzar en un contexto desconocido para el. Muchas veces nos olvidamos de como una practica nos puede ayudar dando estrategias para solventar nuevos problemas. Muy interesante.

Como siempre, además de las charlas en las que estuve, lo mejor de un open space son las conversaciones que se producen en todo momento... dentro y fuera de la conferencia... Poder compartir con compañeros profesionales y amigos nuestra pasión, es lo más valioso de una conferencia de este tipo. Por otro lado,  con cada conferencia que pasa, se amplia el número de compañeros a los que puedo considerar amigos :-)



Charlas grabadas que he visto

Por otro lado, y gracias a la inestimable ayuda de Autentia,  algunas de las charlas a las que no pude asistir fueron grabadas así que me las estoy viendo poco a poco.... De ellas, las que más me han impactado de momento son:



Hay otras que también están muy bien, pero que no me han impactado tanto (en algunos casos porque el tema ya lo conocía):



Tenéis el post de Autentia con todas las grabaciones en:
https://www.autentia.com/2017/06/13/pamplona-software-craftsmanship-2017/

Resumen y agradecimientos


Como resumen general, puedo decir que la organización de la conferencia es de lo mejor que he visto. La gente de 540, los organizadores, son unos cracks y no solo están atentos a todos los detalles y desarrollan como campeones sino que además son unos genios del marketing.
La verdad es que me extraño que no terminase por venir Martin Fowler a la conferencia, puesto que si alguien lo puede conseguir son ellos...

Me encanta el formato, me encanta el contenido, me encanta reencontrarme con amigos, me encanta que pongan Kalimotxo y me parecen que los amigos de 540 son lo más TOP!!!

Thursday, June 15, 2017

Kent Beck 3x (Great idea that deserves our time)




The life of a product, a tech company, and a software system is a continuous process of decision making... and the best answer for each of these decisions is always "it depends". It always depends on the context and the phase of the product/company/system...
Lately, Kent Beck is presenting a framework to better understand and define this "needed" context information. This great proposal, which he calls 3x (explore, expand, extract), helps us to understand the trade-offs and make better decisions.

We can read as an introduction:
And for the original notes:
And there are some talks about this matter at:
Any comment or discussion about this interesting concept is more than welcome :)


Saturday, June 10, 2017

Quotes about software architecture

Interesting quotes that can inspire you as software developer or at least make you think :)

"In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called ‘architecture.’ This understanding includes how the system is divided into components and how the components interact through interfaces. These components are usually composed of smaller components, but the architecture only includes the components and interfaces that are understood by all the developers."  Ralph Johnson
"A complex system that works is invariably found to have evolved from a simple system that worked." John Gall
 "Architecture is the decisions that you wish you could get right early in a project." Ralph Johnson
"So you might end up defining architecture as things that people perceive as hard to change." Martin Fowler
"I think that one of an architect's most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs." Martin Fowler
"Complexity is what makes software hard to change. That, and duplication." Ralph Johnson
"irreversibility was one of the prime drivers of complexity" Enrico Zaninotto

"If you don’t actively fight for simplicity in software, complexity will win…and it will suck." - @HenrikJoreteg

References about Evolutionary Architectures

After my talk El arte del patadon pa'lante / Libro de recetas (Spanish) at the Pamplona Software Craftmanship 2017, some colleagues ask me about references for evolutionary architectures. This post tries to give the best references I know about this concept...

To have context for the other references, I find the following resources essential:


As general resources:

You also need techniques to make changes to things that traditionally are "hard to change"... For this Martin Fowler bliki is a gold mine:

To document all the decisions you make to gain the sharing understanding of the design:



We also need a lot of tricks for our bag of tricks to have different approaches for different problems... Related with this, I think is important to have knowledge about:



Monday, May 29, 2017

Construction engineering is NOT a good metaphor for software

Metaphors are a great tool to explain other people complicated concepts. Often, they help us to drive interesting conversations and expand the boundaries of the knowledge we have by borrowing ideas from other fields.

In the Software industry, it seems like one of these metaphors is particularly strong, the one that tries to explain software as building houses. We even use it at our universities to introduce the topic to newcomers.

In my opinion, this metaphor is not only wrong but also very dangerous. Thinking about our profession as construction engineering makes us think that:
  • There is a clear construction phase separated from a maintenance phase.
  • Maintenance is cheaper and has a longer duration than the construction phase.
  • Once we have enough documentation and schemas, we can just follow the plan.
  • There is no incremental value. The construction goes from "not done" to "done".
Even if you think it might help in some way, please reconsider.



It does generate more problems and confusion than not using it at all.

In Software, features are done when the last customer stops using it. Maintenance is way more expensive than bootstrapping the project. There is no separation between construction and maintenance.

Software is Agile, Lean, it does change and evolve continuously. Software requires constant adaptation. Agile becoming mainstream in software projects is not a coincidence but the revelation on how Software works nowadays.

Software is not delivered in in CDs anymore. Systems are developed, grown and run as biological organisms evolving all the time.

It is true there are different phases within the roadmap of digital products and that there are different trade-offs to consider, but each of these phases are determined by evolutionary steps. Change is the only constant.


We can do better, we should strive for a better metaphor to help us being more effective explaining what do we do and how Software works.

Perhaps, a comparison with medicine might be more accurate

  • It is a knowledge-based profession
  • No one wants medicine or surgery if it can be avoided
  • Clean code is like eating healthy or making exercise
  • Pairing and code review is the second opinion
  • Making changes to the system running is like making surgery with the patient alive

Medicine might not be a great alternative but it's better than the original construction engineering proposition. Construction engineering will make us think the construction phase requires heavy investment when in reality the cost will always be in evolving the system.

Software is never done, it is always evolving. Software is not done until the last customer logs out and the servers are shut down.

Notes:

  • Some projects have very well defined phases, and that's okey. Hopefully, we'll have less and less of these projects each year.
  • Many thanks to @keyvanakbary for his help with the English writing.

Sunday, May 14, 2017

Interesting technical Talks/Podcasts (May 2017)

From the tech-related talks I've heard/seen lately, these are the most interesting:

Thursday, May 11, 2017

Focus on verbs, not on names



In my experience, a good strategy to understand deeply a system or a product is:
  • Focus the research and conversations on identifying behaviors (actions, verbs, business flows, and events).
  • For these behaviors, we should identify the dependencies, concurrency and parallelism requirements (what depends on what? what actions can happen at the same time? what actions don't depend on any other action? what are the sources/inputs for each action? etc).
Understanding the concurrent behaviors, allow us to design the system using small pieces that encapsulate state and communicate using messages. A good match for OOP, Actors, Microservices, etc.
Is easy to see that this strategy try to identify the interactions that the users have with our system and the corresponding actions (verbs). It does not focus on the entities used by the system (names).

However, the most used analysis strategy try to identify the entities first, losing all the context about the behavior (messages, flows, rules, etc). This strategy derives, not necessarily, but typically, on anemic and poor models. This is a common approach for people who have misunderstood OOP/OOA.


Conclusion:

I value the process of identifying the entities (names) of a system, but I value more identifying the behaviors (verbs) that define how is used our system and what actions provide to the user.

We should remember that the customer DON'T want software/systems. The software is only a liability, not an asset. The asset is the actions provided to the customer.

Additional notes:

  • This strategy can be used at different levels... It helps us to identify bounded contexts, domain events, microservices... And at the low level can also help us to see the concurrent behaviors of each application or service. This allows us to design more reactive and concurrent systems.
  • Focusing on identifying names first, make very hard to fulfill concurrency, parallelism and scalability requirements. 
  • In the real world, things happen concurrently.
  • Event storming seems to be a good exercise to identify the domain events, the dependencies, and the behaviors.
  • Any strategy for the analysis should be used continuously. For me, development is always iterative, I don't see it any other way.

Sunday, April 23, 2017

Interesting Talks/Podcasts (April 2017)




From the talks I've heard / seen lately, these are the most interesting:

Interesting articles and blog post I read lately...


These are two interesting articles or blog post I read recently:


Thursday, April 20, 2017

Remove the excuses for not being a professional



Do you want to remove the excuses for not being a professional software developer?

  1. Release as frequently as possible, it will give you tons of feedback about the product, the design, the code, the architecture and so on... And at the same time, it forces you to have confidence in the code, automatic testing, automatic deployment, good tooling and infrastructure and learn one of the most difficult but useful skills for a developer... the skill of making large changes in small chunks  (parallel changes, dark releases, refactoring (code and data), etc...)
  2. Use TDD to put pressure on the design and force you to think about it and decouple from infrastructure.
  3. Pair as much as possible to be forced to have focus, understand other developers, think as hard as possible in the problem at hand. Or mob programming as an alternative. No twitter, no Facebook, no notifications... only respect and hard work. Professionalism :) 
  4. Be a whole team for the product, including of course the customer, business, and so on... Learn about the problem to solve, the business language, propose new features and experiments, find alternative solutions, even find better problems... Be part of the business and the product.

So when some people said that agile is a excuse to avoid being professional, I don't longer know whether to laugh or cry. If I want to be unprofessional, I prefer a classic/waterfall approach, without any delivery in several months :)


Learn about business, learn your craft, be part of a team with a purpose...
Be a professional software developer

And remember: nothings add value, until it helps real customers to make something...

References and related posts:



Sunday, April 16, 2017

Book Review: Nonviolent Communication: A Language of Life


Nonviolent Communication: A Language of Life


This great book explains how to have real and deep communication among humans. No matter how difficult the subject or how difficult the situation, this method proposed by the Dr. Marshall Rosenberg can improve our relations and communications by removing judgment from the equation and helping us to express our real needs.

This is one of the books that can significantly change your life… This form of communication is in our nature, but in most cases, due to our education (and upbringing), we have lost this skill, so the book is a method and a guide to recovering and developing it. Dr. Marshall guides us by using real examples and conversations from his workshops or from his experience as a mediator in difficult conflicts. He uses very clear language, it is not only very easy to read, but also very instructive.

For sure, this book is only the first step to discovering this skill and how deeply it can change your relationships. There are communities for practice, workshops and other books with more examples or exercises. Any skill requires practice.

In summary, a great book with a great purpose: to improve our lives, and relationships, express our real needs and help us understand the needs of others.

Related resources:

Saturday, April 15, 2017

Socrates Canaries 2017, experience and notes

The past week I attended to my second Socrates Canaries (Software Craftsmanship and Testing Conference)... This posts summary the experience (things I have learned, ideas to explore, techniques to use...). The Socrates event is an open space, so whatever happens is the only thing that could have :)


Notes from the sessions:

Under the umbrella (elixir) @gemcfadyen

Interesting and very practical presentation about how to organize and structure elixir application using the umbrella concept (http://elixir-lang.org/getting-started/mix-otp/dependencies-and-umbrella-apps.html).  You have all the advantages of the microservices and you remove some of the cons because you bundle the services together with explicit dependencies. Georgine used https://github.com/gemcfadyen/umbrella_romanNumerals as example. She also commented https://github.com/elixirkoans/elixir-koans for learning elixir.

Craft thinking @mashooq

(Book Craftsman by Richard-Sennett)
Very interesting reflections on the crafts in general and the craft in the development of software. The ideas come from the book Craftsman (Richard-Sennett) and the interpretations from Mashooq. Great insights and very interesting discussion.
Some random notes:

  • Craftsmanship. Human Impulse. The desire to do a job well for it's own sake.
  • We should always consider "How" and "Why" we make something... 
  • From "problem solving" to a combination of "problem finding + problem solving". Finding problems is also a skill to learn.
  • Motivation matters more than talent
  • We need imagination to learn. We also need fail (a very important note for mentors...)
  • Learn to apply the minimal force needed (in software we should compare this idea with the use of frameworks... ;) )

Show your architecture @borillo @eferro (myself)

Ricardo and myself proposed this session... the idea was to show several architectures/designs and to discuss about the tradeoffs and decissions that generated each architecture. We discussed three designs, one from @borillo, another one from kuba and another one from myself (@eferro).
We discussed and explain why we create this design and what was the tradeoffs... I hope that this session was interesting for someone :)

Building learning teams

A round table about how we can introduce a learning culture in a software development team or company. This are some random ideas:

  • Do food
  • Lean coffee (initially with a meta topic or about a shared interests)
  • Skill matrix
  • Know your company
  • Introducing learning time in work hours (with the possibility to extend after work hours). A good idea to this kind of learning is Lunch & Learn.
  • The company pay pluralshight or similar but remove the service is not used. You don't use it, you lost it.
  • Some reserved time: Give a tech talk, and as fallback, show a recorded talk and as a fallback a refactoring session or a mob programming session.

Personal productivity. @gardenunez

Another round table about how we can organize ourselves, as obsessive people, to be productive and not going mad in the process :)
Some tips about focus, pomodoro technique to deal with procrastination, making a list of the things that really matter, and so on...
And also some comments about not pushing to hard to ourselves... :)


Non Violent Communication Miguel Cruz

A good explanation about the how to have better relations and real connection using non violent communication. This kind of communication is a difficult skill that can be learned using the book Non Violent Communication book or going to a workshop or participating in some of the related communities (like the new created group in Barcelona).

The overall process consists in the following steps:
  1. make the observation (without judgement)
  2. express your feeling (it's difficult to separate feelings and opinions)
  3. express your needs
  4. making a request (in a concrete way). Is a request, not a demand, so it can be rejected.
The method is divided in three steps/levels:
  1. Self empathy
  2. Empathetic listening
  3. Be able to express in an empathetic way

I already read the book, but it requires practice. This skill can change deeply your relationships and life... so it is no easy, but deserves the effort :)

Serverless with AWS Lambda @mashooq (I acted as host)

This was a session hosted by me to ask Mashooq about codurance's experience using AWS Lambda in production. The session was divided in two parts, the first one was a presentation about this codurance's project and the second one was a discussion to identify if in my use case, have sense or no to consider using lambda.

Some lessons:
  • Bad choice for constant load
  • Force you to focus on business code (not in infrastructure)
  • Good base for security, scalability, performance
  • 1º understand the lowlevel stuff using aws api or/and aws cli, before using any framework or abstraction (apex, claudia.js, etc)... 
  • Prepare for runing localy from the day 0. Developing any mock/glue code needed. 
  • For web servers:
    • Latency problem to serving http request with JVM based languages. The startup/bootstrap process have a huge latency. They implement a periodic ping to avoid the stop of the containers, but this is not an elegant solution.
    • If you serve the frontend (js, html, css) from CDN, is difficult to synchronize with backend code. They need to invalidate the CDN resources and this cost money.
Serverless:
cost model - request based (pay by request)
strong infrastructure abstraction.

Another alternative for a web application using a JVM, than also have a strong infrastructure abstraction is Elastic Beanstalk

The second part of the session was a conversation about the possibility to use lambda for my use case, but for this part I will generate another post when I make the initial testing... In summary, seems that Lambda can fit our problem, but we should make the calculations about cost.

Taming the monolith  @mashooq

This session gave some ideas about how we can divide a monolith and how we should have in mind where are the vector of changes, because there is no best approach, it always depend of the nature of the change/evolution request that our system have.
Two powerful ideas:

  • Have in mind, at all levels, coupling and cohesion (for modular decomposition, to identify bounded contexts, to deine service limits, etc.)
  • Include the user journey

Some random quotes from this session:

  • "We need everybody to be architects" Mash
  • "Architecture. Design that is difficult to change" Mash

Liquid modernity @jordianguela

The goal of this session was to discuss about how this concept of "liquidity" is affecting us. This concept was coined by the philosopher  Zygmunt Bauman that talked about globalization, modernity and postmodernity, consumerism and morality.
The process was, we show different phrases from Zygmunt and we discusses the meaning and the pros and cons. And interesting session.
The concept of minimalism also appeared as one way to deal with this amount of changes  (Lifestyle Minimalism - Renouncing Your Busy Stupid Life)


Vuex + IDDD @nestorsalceda, @gualison (from the codingstones)

As I have no experience at all with front-end I only attended for the first part of the session. They described how they approach the architecture for a front end application using Vuex and the concepts of IDDD... You can see the results at:
They show the code and seems to be very simple and clean... a good stuff to research further.

PostgreSQL @juanignaciosl

A great presentation of the advances features of PostgreSQL... it seems to be the most powerfull DB... is the "Swiss army knife" of the databases... (relational, document oriented, pub/sub, with GIS extensions, rich data types...)
These are the slides of the presentation POSTGRESQL The one tool to bring to a desert island
Very instructive.

Other sessions, lightning talks and conversations:

  • Mental health
  • Different organizations (codesai) (book Reinventing Organizations)
  • How the codingstones distribute the profits (codingstones)
  • pointers in go
  • gendered language
  • carbon offset / climate change
  • Mob programming sessions
  • ...

Resource Corner



Related links:


Personal feelings and notes:

  • Somehow I improved my English so I interacted more and with more people :)
  • Old friend, new friends... I like this event... 
  • Great idea to come with the family and use the event as excuse to have family time at Canaries.
  • I returned home full of energy.
  • Great to have more developers from TheMotion attending this kind of conferences... The Socrates is  a great introduction to the Software Craftmanship community and to the unconference format. Thank you @gardenunez for coming to this event.
  • I love the open space format.



Thanks to the organizers, mainly @suuigd (Dani), @mangelviera (Miguel), the @CraftsmanshipGC community and to codesai.  
And of course to @eidrien for being the facilitator.