Stream Processing

TL;DR Stream Processing is a big data paradigm. In Batch Processing, We need to have all data stored ahead of time. We process data in batches. We aggregate the results across all batches at the end. Batch processing tries to process all the data at once. In Stream Processing, Data come as a never-ending continuous stream of events. Stream processing naturally fits with time series data. Data are processed in real-time and we can respond to the events faster....

Mar 26, 2020 · 3 min · Milad Irannejad

Multi-Tenancy in Microservice Architecture

TL;DR Microservices architecture encompasses a few services to thousands of services that communicate with each other through APIs. Microservices should NOT introduce any breaking changes to their APIs. Every change in one microservice should be tested against other microservices that rely on it. There are two approaches for integration testing in a microservices architecture: Replica Environments (Parallel Testing) Creating a copy of the production environment for handling test traffic (integration or staging environment)....

Mar 23, 2020 · 4 min · Milad Irannejad

Developing Go Services For Kubernetes with Telepresence and konfig

The Problem As a developer when you are working on a Kubernetes application on your local machine, if you want to test or debug something, you have the following options: A full environment running using docker-compose. A full environment running in a local Kubernetes cluster (Minikube or Docker-for-Desktop) Pushing instrumented code, building, testing, and deploying to a dev Kubernetes cluster through CI/CD pipeline. The problem with the first two options is the environment you get is not close by any means to your actual final environment (staging and production)....

Mar 4, 2020 · 4 min · Milad Irannejad

Dynamic Configuration Management and Secret Injection with konfig

TL;DR Dynamic configuration management and secret injection refer to updating an application with new configurations and secrets in a non-disruptive way. Kubernetes ConfigMaps and Secrets mounted as files into containers will be updated with new values automatically. konfig makes dynamic configuration management and secret injection very easy to implement and use for Go applications. The Problem Dynamic configuration management and secret injection refer to a situation that your application can update its configurations and secrets without needing a restart....

Mar 4, 2020 · 5 min · Milad Irannejad

Zero-Config Configuration Management

konfig is a minimal and unopinionated configuration management library for Go applications. It is based on The 12-Factor App. I created this library as a response to repeating myself across almost every single service and application. It is a very minimal and lightweight library for reading configuration values either from command-line arguments, environment variables, or files. It uses reflection to automatically convert the input values to the desired types defined in Go....

Mar 4, 2020 · 8 min · Milad Irannejad