Part -2: Operators teach Kubernetes how to simplify stateful application…

Leave a Comment / Kubernetes / By

I hope you have enjoyed my first article(link below) on Operator extension and Kubernetes Introduction. Now level up to this series another article where I’ll show case a Kafka operator and the Operator capabilities on Kubernetes to achieve a stateful behaviour.

In case you miss first article here is the link.

Operators teach Kubernetes how to simplify the stateful application…

Now let’s jump on Operator, If we want to understand the Operator we first need to have an understanding of CR(Custom Resource) and CRD(Custom Resource Definition). Kubernetes API use store an object using API endpoints and a Custom resource(CR) allows you to create your own API object. Means CR allows you to extend Kubernetes capabilities by adding any kind of API object useful to your resource. CRD is just what we use to define our CR, for example a CRD.yaml. It’s analogous to a schema for the CR. CRs provide endpoints for reading and writing structured data. A cluster user can interact with CRs with kubectl or another Kubernetes client, just like any other API resource.

Making an Operator means creating a CRD and providing a program that runs in a loop watching CRs of that kind. The actions an Operator performs can include almost anything: scaling a complex app, application version upgrades, or even managing kernel modules for nodes in a computational cluster with specialized hardware.

A simple, “Hello World” application for Operator such as “etcd-operator” isn’t going to provide enough complexity(Figure below) to fully demonstrate what Operators can do. To really help you understand the capabilities of Operators, we need an application that requires multiple Kubernetes resources with configuration values that cross between them to use for demonstration.

Here I’ll introduce the Visitors Site application, which we will use as an example in the following article that cover writing Operators. We’ll take a look at the application architecture and how to run the site, as well as the process of installing it through traditional Kubernetes manifests. In the article that follow, we’ll create Operators to deploy this application using one of the approaches provided by the Operator SDK (Helm, Ansible, and Go), and explore some benefits and drawbacks of it.

Application Overview

The Visitors Site tracks information about each request to its home page. Each time the page is refreshed, an entry is stored with details about the client, backend server, and timestamp. The home page displays a list of the most recent visits. While the home page itself is fairly simple, the architecture is what makes this an interesting example for exploring Operators.

Each component in the Visitors Site requires two Kubernetes resources:

Deployment

Contains the information needed to create the containers, including the image name, exposed ports, and specific configuration for a single deployment.

Service

A network abstraction across all containers in a deployment. If a deployment is scaled up beyond one container, which we will do with the backend, the service sits in front and balances incoming requests across all of the replicas.

A third resource is used to store the authentication details for the database. The MySQL container uses this secret when it is started, and the backend containers use it to authenticate against the database when making requests.

Deploying MySQL

Below is the yaml file and deploy command:-

---
apiVersion: v1
kind: Secret
metadata:
  name: mysql-auth
type: Opaque
stringData:
  username: visitors-user
  password: visitors-pass
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: visitors
      tier: mysql
  template:
    metadata:
      labels:
        app: visitors
        tier: mysql
    spec:
      containers:
        - name: visitors-mysql
          image: "mysql:5.7"
          imagePullPolicy: Always
          ports:
            - name: mysql
              containerPort: 3306
              protocol: TCP
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: password
            - name: MYSQL_DATABASE
              value: visitors_db
            - name: MYSQL_USER
              valueFrom:
                secretKeyRef:
                  name: mysql-auth
                  key: username
            - name: MYSQL_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-auth
                  key: password
---
apiVersion: v1
kind: Service
metadata:
  name: mysql-service
  labels:
    app: visitors
    tier: mysql
spec:
  clusterIP: None
  ports:
    - port: 3306
  selector:
    app: visitors
    tier: mysql


$ kubectl apply -f database.yaml
secret/mysql-auth created
deployment.apps/mysql created
service/mysql-service created

Backend

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: visitors-backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: visitors
      tier: backend
  template:
    metadata:
      labels:
        app: visitors
        tier: backend
    spec:
      containers:
        - name: visitors-backend
          image: "jdob/visitors-service:1.0.0"
          imagePullPolicy: Always
          ports:
            - name: visitors
              containerPort: 8000
          env:
            - name: MYSQL_DATABASE
              value: visitors_db
            - name: MYSQL_SERVICE_HOST
              value: mysql-service
            - name: MYSQL_USERNAME
              valueFrom:
                secretKeyRef:
                  name: mysql-auth
                  key: username
            - name: MYSQL_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-auth
                  key: password
---
apiVersion: v1
kind: Service
metadata:
  name: visitors-backend-service
  labels:
    app: visitors
    tier: backend
spec:
  type: NodePort
  ports:
    - port: 8000
      targetPort: 8000
      nodePort: 30685
      protocol: TCP
  selector:
    app: visitors
    tier: backend


$ kubectl apply -f ch05/backend.yaml
deployment.apps/visitors-backend created
service/visitors-backend-service created

Frontend

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: visitors-frontend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: visitors
      tier: frontend
  template:
    metadata:
      labels:
        app: visitors
        tier: frontend
    spec:
      containers:
        - name: visitors-frontend
          image: "jdob/visitors-webui:1.0.0"
          imagePullPolicy: Always
          ports:
            - name: visitors
              containerPort: 3000
          env:
            - name: REACT_APP_TITLE
              value: "Visitors Dashboard"
---
apiVersion: v1
kind: Service
metadata:
  name: visitors-frontend-service
  labels:
    app: visitors
    tier: frontend
spec:
  type: NodePort
  ports:
    - port: 3000
      targetPort: 3000
      nodePort: 30686
      protocol: TCP
  selector:
    app: visitors
    tier: frontend


$ kubectl apply -f ch05/frontend.yaml
deployment.apps/visitors-frontend created
service/visitors-frontend-service created

The Visitors Site is a traditional, threetier application, consisting of:

  • A web frontend, implemented in React (https://reactjs.org/)
  • A REST API, implemented in Python (https://www.python.org/) using the Django framework (https://www.djangoproject.com/)
  • A database, using MySQL (https://www.mysql.com/)

Each component in the Visitors Site requires two Kubernetes resources:

Deployment

Contains the information needed to create the containers, including the image name, exposed ports, and specific configuration for a single deployment.

Service

A network abstraction across all containers in a deployment. If a deployment is scaled up beyond one container, which we will do with the backend, the service sits in front and balances incoming requests across all of the replicas.

PS C:\Users\mikem> kubectl get pods,service
NAME                                    READY   STATUS    RESTARTS   AGE
pod/mysql-645675f6c5-tfs98              1/1     Running   0          46m
pod/visitors-backend-c7798d94-7l47d     1/1     Running   0          44m
pod/visitors-frontend-dd7ff7777-7xplm   1/1     Running   0          41m
NAME                                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
service/kubernetes                  ClusterIP   10.96.0.1        <none>        443/TCP          10d
service/mysql-service               ClusterIP   None             <none>        3306/TCP         46m
service/visitors-backend-service    NodePort    10.109.161.168   <none>        8000:30685/TCP   44m
service/visitors-frontend-service   NodePort    10.102.91.126    <none>        3000:30686/TCP   41m

A third resource is used to store the authentication details for the database. The MySQL container uses this secret when it is started, and the backend containers use it to authenticate against the database when making requests.

Once your pods are up and running, you can access the Visitors Site by opening a browser and going to http://localhost:30686. Clicking refresh a few times will populate the table on that page with details of the internal cluster IP and the timestamp of each request.

We will use this sample visitor application in the following artcle to demonstrate a variety of technologies on which you can build Operators.

Cleaning Up

Similar to deploying the manifests, you delete the created resources using the kubectl command:

$ kubectl delete -f ch05/frontend.yaml
deployment.apps "visitors-frontend" deleted
service "visitors-frontend-service" deleted


$ kubectl delete -f ch05/backend.yaml
deployment.apps "visitors-backend" deleted
service "visitors-backend-service" deleted

$ kubectl delete -f ch05/database.yaml
deployment.apps "mysql" deleted
service "mysql-service" deleted

Stay tuned for next article to build Operator on this visitor sample application and Kafka Operator.

Leave a Comment