Mukesh Kumar

Part -2: Operators teach Kubernetes how to simplify stateful application…

I hope you have enjoyed my first article(link below) on Operator extension and Kubernetes Introduction. Now level up to this series another article where I’ll show case a Kafka operator and the Operator capabilities on Kubernetes to achieve a stateful behaviour. In case you miss first article here is the link. Now let’s jump on …

Part -2: Operators teach Kubernetes how to simplify stateful application… Read More »

Operators teach Kubernetes how to simplify the stateful application…

This is the first article to a series of articles to showcase how we use Operator that can leverage Kubernetes to create a stateful application such as Kafka Cluster. An Operator is a way to package, run, and maintain a Kubernetes application. An Operator builds on Kubernetes to automate the entire lifecycle of the software …

Operators teach Kubernetes how to simplify the stateful application… Read More »

LAMP stack in Cloud: Building a Scalable, Secure and Highly Available architecture using AWS

1. Requirement Overview The acronym LAMP (Linux, Apache, MySQL, PHP) refers to an open-source stack, used to run dynamic and static content of servers. A small startup organization uses the LAMP stack of software. The dynamic nature of demand and projected future growth in traffic drives the need for a massively scalable solution to enable …

LAMP stack in Cloud: Building a Scalable, Secure and Highly Available architecture using AWS Read More »

Reference architecture of bigdata solution in GCP and Azure…

This article is a showcase of a Reference architecture approach for the financial sector where stream and batch processing is a common part of its solution with other designs. Firstly the requirement analysis is the step to define the implementation of any use case. Therefore before moving to reference architecture we first need to understand …

Reference architecture of bigdata solution in GCP and Azure… Read More »

Error resolution of Zalando Research Flair NLP package installation on Centos 7, “Failed building wheel for regex…”​

I was working on an NLP tool for evaluation purposes and found an issue in creating the environment. They had set up everything on Ubuntu so they might not face this issue but I am replicating on Centos 7 and found an error. Hope this will help someone. The project is based on PyTorch 0.4+ …

Error resolution of Zalando Research Flair NLP package installation on Centos 7, “Failed building wheel for regex…”​ Read More »

How to create an Apache Beam data pipeline and deploy it using Cloud Dataflow in Java

Cloud Dataflow is a fully managed google service for executing data processing pipelines using Apache Beam. What do you mean by fully managed? Cloud dataflow like BigQuery dynamically provisions the optimal quantity and type of resource(i.e CPU or memory instances) based on volume and specific resource requirements for your job. Cloud dataflow is a server-less …

How to create an Apache Beam data pipeline and deploy it using Cloud Dataflow in Java Read More »

Google Dataflow Python ValueError: Unable to get the Filesystem for path gs://myprojetc/digport/ports.csv.gz

I am using google cloud to create an event on Cloud Storage to Big Query using Apache Beam pythons library. I was executing an ETL in the “DirectRunner” mode and found no issue. But later when I take everything on dataflow to execute found an error. Below command used to upload the file and I …

Google Dataflow Python ValueError: Unable to get the Filesystem for path gs://myprojetc/digport/ports.csv.gz Read More »