I decided to create this blog in 2019 with the aim of helping other IT enthusiasts. The blog will cover different aspects of my experience and interesting topics related to big data, functional programming, Spark, Kafka and Scala.
After more than 10 years working in IT, I found my passion in Scala and functional programming. After having worked so many years in Java, working in Scala has given me a new apetite for learning. Right now my passions are Functional programming, Cats and everything related to Big data: spark, akka, kafka, elastic search, airflow and the massive world of AWS.
This post is a good introduction to AWS ECS and more particularity to the Fargate deployment type. We will explain how Fargate works, differences with AWS ECS EC2 and how to migrate from ECS EC2 services to ECS Fargate.
This article is an introduction to the deployment service provided by AWS. After the introduction, we will discuss a real AWS CodeDeploy deployment issue and how to troubleshoot it.
In this post it will be explained how to deploy a new version of a Kafka cluster in a production environment using a BlueGreen deployment technique. This solution will allow producers to migrate seamlessly their traffic to the new cluster without any data loss.
We will explain the Spark Structured Streaming output mode and watermark features with a practical exercise based on Docker.
We will show what Spark Structured Streaming offers compared to its predecessor Spark Streaming. We will cover how to read JSON content from a Kafka Stream and how to aggregate data using spark windowing and watermarking.
During the lecture, the reader will discover the power of Akka Streams: how to manipulate streams combining the Akka Streams GraphDSL, windowing and different split aggregations.
Good introduction to Akka Streams explaining what it is windowing and how to implement it using Scala and Akka Streams.
Detailed steps about how to update and scale up a production Kafka cluster without losing any data. How to rebalance your Kafka cluster and update the broker instances.
Usage of Applicative and Validated Cats type classes to perform a multi level form validation. So much fun guaranteed...