Tuesday, September 26, 2017

Focusing my learning... (Js, Serverless, Aws, basic frontend)

I have some obsessive tendency to accumulate info/resources for learning (talks, presentations, blog post, books, podcast, etc).
From time to time I need to carry out an exercise to identify the things that I want to learn to complement my actual skills... This exercise helps me to narrow (temporarily) my scope for learning and gain speed, focus and motivation.

The process:

  1. I bring together a lot of topics that I am interested in (from languages, and technologies, to soft skills, and cultural topics).
  2. I classify each topic in several levels (my interest, my actual experience, level of alignment with my personal path/mission, etc.).
  3. I try to discard as many topics as possible... for example because I am already proficient or good enough. In these cases, I'd like to learn more, but I will not make any special effort.
  4. I select three or four topics.
  5. I remove mercilessly unrelated items from my list of resources (books, talks, blog posts, articles, etc...).  It is important to remove as much as possible because we don't have enough energy and if something is really important it will reappear in the future... :)

The results:

  1. Learning Javascript (the language). I will try to improve my Javascript knowledge up to a basic knowledge. Focused on the language itself and the most common execution environment (node and browser).
  2. Serverless. I see the serverless based architectures as the next step for a PaaS that force us to think in terms of events and create cloud-native designs. I think that this technology will be very important in the near future.
  3. Advanced AWS usage. In this case, I decided that even if I learned a lot about cloud technologies in my actual job I need to go deeper in AWS to really understand and gain more confidence.
  4. Frontend minimal stuff. I have developed from Linux kernel drivers up to cloud-native applications, but only a few times I needed to develop customer-facing apps (or at least in the actual frontend apps sense)... So my idea in this regard is to have the minimal experience with HTML5, CSS and the minimal amount of javascript needed to implement small SPAs.

I also take other actions to remove inputs for distraction:
  • Unfollow a lot of people on twitter (sorry, not enough time...).
  • Remove sync for email on my mobile.
  • And remove a lot of apps that generate push notifications.

This process also helps a lot with my impostor syndrome :)

And remember, when you buy a book, you don't buy the time to read it

Related content:

Saturday, September 09, 2017

Good talks/podcasts (August 2017)

These are some interesting talks/podcast that I've seen during August:

Tuesday, September 05, 2017

Pub-Sub the swiss army knife (tech pill)

Pub-Sub / Publish-Subscribe

"In software architecturepublish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorize published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are." wikipedia Publish-Subscribe pattern
As the Wikipedia explains, the pub-sub patterns allow decoupling the publishers (P) from the possible subscribers or consumers (C). So the publisher doesn't need to have any knowledge about the components interested in receiving the message.
To implement this pattern we need a central service in charge to receive the messages from the publisher and resend the message for each one of the interested subscribers. Usually, this central service is called broker (B).

Some common characteristics for a broker are:
  • Allow broadcasting message to several consumers.
  • Allow the consumers to configure in which messages are interested using some kind of consumer defined rule.
    • By topic
    • By regular expression over a topic 
    • By a query over some attributes of the message
  • Allow the publisher to send messages and add some meta info to it
    • Attributes
    • A destination topic
    • Other meta info as priority, TTL, etc
  • Allow several consumers to receive the same message
Another typical characteristic, and in fact, the most important one is that for each consumer the broker usually has a queue to maintain the pending messages.
And this queue has all the characteristics that we explain in the Queues vs Distributed log post... For example, the broker can load balance the messages between a group of consumers.

In summary, the main characteristics of a pub-sub system are:
  • Publishers:
    • They don't know the final destination of the messages (decoupling).
    • They send the messages to a topic/exchange/abstract destination.
    • They can add attributes to the messages.
  • Consumers/Subscribers:
    • Each one informs the broker the in which messages are interested (using the name of the topic, a regular expression over the topic or a combination of attributes of the message).
    • Each message can be consumed by several consumers (broadcasting).


  • Great decoupling between publishers/consumers.
  • Allow easy creation of flexible communication topologies.
  • All the pros of Queues.
    • Easy to implement.
    • Unlimited/Easy horizontal scalability.
    • Fault tolerance is very easy to implement (time out and re-queue of the message).
    • Allow easy balance between latency (time at the queue + processing time) and the cost of the concurrent workers.

Cons (same as queues):

  • The order is not guaranteed.
  • Usually, we can have duplicates.
  • Requires more resources from the broker some in some scenarios it has worst scalability than other solutions (not important for the majority of the cases).

Use a Pub-Sub system when:

For any use case that requires flexibility, broadcasting of messages and doesn't require that order of the messages is guaranteed. In these scenarios, a pub-sub system is a great solution because it includes all the use cases of a queue and all the flexibility of sending the same message to several queues/consumers.

The real potential of this kind of solution is when you combine several brokers (using federation or replication)  to create flexible topologies that can communicate several systems and services.

Use cases / Examples:

  • Any good scenario for queues.
  • Async processing of requests.
  • When we can allow losing some data (or having some delays), Pub-Sub systems are great for:
    • Log / Monitoring info distribution and processing.
    • Asynchronous processing of email requests.
    • Processing of independent batch jobs.
  • Using Federation
    • Info replication and distribution between data centers.
    • Global distribution of periodic information.
    • Global distribution of reference info.


Related content: