Kubernetes Handbook

https://github.com/kubernetes/website/tree/main/content/en/examples

https://k8s.io/examples/

sudo apt-get update sudo apt install docker.io -y sudo usermod -aG docker ubuntu logout

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256" echo "$(cat kubectl.sha256) kubectl" | sha256sum --check sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl kubectl version --client

# For AMD64 / x86_64 [ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.27.0/kind-linux-amd64 # For ARM64 [ $(uname -m) = aarch64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.27.0/kind-linux-arm64 chmod +x ./kind sudo mv ./kind /usr/local/bin/kind

config.yaml kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: - role: control-plane - role: worker - role: worker

---

kind create cluster --name k8s-playground --config kind-config.yaml

ds

wget https://k8s.io/examples/controllers/daemonset.yaml

kubectl get ds kubectl explain ds kubectl apply -f daemonset.yaml

-> DaemonSet are similar to ReplicaSet,But here we dont menioned any No. of pods to deploy. -> Instead pods will be deployed on every node in the cluster. This is useful for running system daemons on every node. -> For example, log collection, monitoring agents, etc. -> By default, DaemonSet will deploy pods on all nodes, but we can also specify nodeSelector to deploy pods on specific nodes. -> DaemonSet will not be deleted when nodes are deleted, but pods will be deleted. -> DaemonSet by default won't deploy any pods in control-plane node. You can use the name-space "kube-system" to deploy pods in control-plane node.

cj

wget https://k8s.io/examples/application/job/cronjob.yaml

kubectl get cj kubectl explain cj kubectl apply -f cronjob.yaml

-> CronJob is used to run jobs at scheduled times. -> You can't schedule a job to run at a specific time, but you can schedule a job to run at a specific interval. -> You can't schedule a cronJob to run below 1 minute interval. -> cronJob is useful for running jobs that need to run at a specific time, like backups, reports, etc. -> star representation of cronJob is as follows: * - minute (0 - 59) * - hour (0 - 23) * - day of month (1 - 31) * - month (1 - 12) * - day of week (0 - 6) (Sunday=0 or 7)

scheduling

We are going to see how to manually schedule a pod and how to use scheduler to schedule pods.

try to find the schedular pod kubectl get po -n kube-system | grep scheduler try to finding the node name of the schedular pod kubectl get po -n kube-system -o wide | grep schedule login into the container docker exec -it {container_id} bash try to find the kube-scheduler process # ps -ef | grep kube-scheduler find all the control plane components # cd /etc/kubernetes/manifests # ls -lrt Alter the kube-schedular.yaml file # vi kube-scheduler.yaml Now new pods will not be scheduled by the kube-scheduler. MANNUAL SCHEDULING create a pod with nodeName kubectl run nginx --image=nginx --dry-run=client -o yaml > pod.yaml vi pod.yaml nodeName: {node_name}

-> All the control-plane components are running as static pods. -> Static pods are managed by kubelet and are created while creating the cluster. -> Schedular is a control-plane component that is responsible for scheduling pods. -> We can manually schedule a pod by specifying the nodeName in the pod spec.

labels

selectors

we are going to see how to filter resources using labels and selectors.

try creating any resource wget https://k8s.io/examples/pods/pod-nginx.yaml find its labels cat pod-nginx.yaml kubectl apply -f pod-nginx.yaml try to filter it using its label kubectl get pods --show-labels kubectl get po --selector env=test

-> Labels are just used as filters, you can access those resources even without labels. -> Like namespaces they won't create logical isolation between resources.

taints

toleration

nodeSelector

-> we are going to see how to set taints on nodes and how to use tolerations to schedule pods on tainted nodes. -> then we are going to see how to use nodeSelector to schedule pods on specific nodes.

set taint on node k get nodes kubectl taint node {node_name} gpu=true:NoSchedule Check kubectl describe node {node_name} | grep -i taint create a pod with toleration kubectl run nginx --image=nginx --dry-run=client -o yaml > pod.yaml vi pod.yaml tolerations: - key: "gpu" operator: "Equal" value: "true" effect: "NoSchedule" run the pod kubectl apply -f pod.yaml check the pod run in the execpted node kubectl get po -o wide Now untain the node kubectl taint node {node_name} gpu=true:NoSchedule- create a pod with nodeSelector kubectl run nginx --image=nginx --dry-run=client -o yaml > pod.yaml vi pod.yaml nodeSelector: gpu: "true" run the pod kubectl apply -f pod.yaml check the pod run in the execpted node kubectl get po -o wide

-> Taints are used to mark NODES so that pods can be scheduled on them. -> Tolerations are used to allow PODS to be scheduled on nodes with taints.
-> Effects of taints are: 1. NoSchedule - Pods will not be scheduled on the nodes. (Effects on new pods) 2. PreferNoSchedule - Pods will be scheduled on the node, but if there are no other nodes available, they will be scheduled on the node with taint. (Effects on new pods) 3. NoExecute - Pods will be evicted from the node if they are running on it. (Effects on running & newer pods)
-> Node Selector is used to schedule pods on specific nodes. ->Differnece between taints and nodeSelector is that taints are used to mark nodes and tolerations are used to allow pods to be scheduled on them. Whereas nodeSelector is used to schedule pods on specific nodes. -> In NodeSelector, PODS choose the node based on the labels of the node. -> But in Taints and Tolerations NODES choose the pods based on the tolerations of the pods.

Node Affinity

Pod Affinity

Pod Anti-Affinity

First Create the pod with nodeAffinity. Then check the pod is not scheduled on the node. Then create the node with the label. Then check the pod is scheduled on the node.

try creating any resource wget https://k8s.io/examples/pods/pod-nginx.yaml find its node-affinity cat pod-with-node-affinity.yaml affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: topology.kubernetes.io/zone operator: In values: - antarctica-east1 - antarctica-west1 preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: another-node-label-key operator: In values: - another-node-label-value kubectl apply -f pod-with-node-affinity.yaml check pod kubectl get pods create label for the node& check the pod is scheduled kubectl label node {node_name} topology.kubernetes.io/zone=antarctica-east1 kubectl get pods

→ Node Affinity is used to schedule pods on specific nodes. Here, we use nodeSelectorTerms to control which nodes the pods are scheduled on. → Pod Affinity is used to schedule pods on nodes based on the labels of other existing pods (podAffinityTerms). This ensures that new pods are scheduled on the same node as those matching pods. → Pod Anti-Affinity is used to avoid scheduling pods on the same node as certain existing pods. Here, we use podAffinityTerms to make sure new pods are placed on different nodes. -> 2 types of node affinity: 1. requiredDuringSchedulingIgnoredDuringExecution: - This is used to schedule pods on specific nodes. If the node is not available, the pod will not be scheduled. - nodeaffinity is a hard requirement for the pod to be scheduled on the node. 2. preferredDuringSchedulingIgnoredDuringExecution: - This is used to schedule pods on specific nodes. If the node is not available, the pod will be scheduled on any node. - weight & preference is used to schedule pods on specific nodes. If the node is not available, the pod will be scheduled on any node.

requests

limits

1. First we will create a pod with requests and limits using the polinux/stress image. This image is used to stress test the CPU and memory. 2. Then we will increase the load on the pod over the request & limits and see the effect on the CPU and memory. 3. Then we will reduce the load on the pod and see the effect on the CPU and memory. 4. to track the CPU and memory usage we will use the metrics server.

Create a Metric Server vim metric-server.yaml apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: labels: k8s-app: metrics-server rbac.authorization.k8s.io/aggregate-to-admin: "true" rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-view: "true" name: system:aggregated-metrics-reader rules: - apiGroups: - metrics.k8s.io resources: - pods - nodes verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: labels: k8s-app: metrics-server name: system:metrics-server rules: - apiGroups: - "" resources: - nodes/metrics verbs: - get - apiGroups: - "" resources: - pods - nodes verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: k8s-app: metrics-server name: metrics-server-auth-reader namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: extension-apiserver-authentication-reader subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: labels: k8s-app: metrics-server name: metrics-server:system:auth-delegator roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:auth-delegator subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: labels: k8s-app: metrics-server name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: v1 kind: Service metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system spec: ports: - name: https port: 443 protocol: TCP targetPort: https selector: k8s-app: metrics-server --- apiVersion: apps/v1 kind: Deployment metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system spec: selector: matchLabels: k8s-app: metrics-server strategy: rollingUpdate: maxUnavailable: 0 template: metadata: labels: k8s-app: metrics-server spec: containers: - args: - --cert-dir=/tmp - --secure-port=10250 - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname - --kubelet-use-node-status-port - --kubelet-insecure-tls - --metric-resolution=15s image: registry.k8s.io/metrics-server/metrics-server:v0.7.1 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: /livez port: https scheme: HTTPS periodSeconds: 10 name: metrics-server ports: - containerPort: 10250 name: https protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /readyz port: https scheme: HTTPS initialDelaySeconds: 20 periodSeconds: 10 resources: requests: cpu: 100m memory: 200Mi securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 seccompProfile: type: RuntimeDefault volumeMounts: - mountPath: /tmp name: tmp-dir nodeSelector: kubernetes.io/os: linux priorityClassName: system-cluster-critical serviceAccountName: metrics-server volumes: - emptyDir: {} name: tmp-dir --- apiVersion: apiregistration.k8s.io/v1 kind: APIService metadata: labels: k8s-app: metrics-server name: v1beta1.metrics.k8s.io spec: group: metrics.k8s.io groupPriorityMinimum: 100 insecureSkipTLSVerify: true service: name: metrics-server namespace: kube-system version: v1beta1 versionPriority: 100 kubectl apply -f metric-server.yaml kubectl top node kubectl create namespace mem-example Create a pod with polinux/stress image wget https://k8s.io/examples/pods/resource/memory-request-limit.yaml apiVersion: v1 kind: Pod metadata: name: memory-demo namespace: mem-example spec: containers: - name: memory-demo-ctr image: polinux/stress

resources: limits: memory: "200Mi" requests: memory: "100Mi" command: ["stress"]

args: ["--vm", "1", "--vm-bytes", "150M", "--vm-hang", "1"] try increasing the stress args: ["--vm", "1", "--vm-bytes", "250M", "--vm-hang", "1"] kubectl apply -f memory-request-limit.yaml - You will see the pod is in pending state. - kubectl describe pod memory-demo -n mem-example - You will see the reason is OOM Killed. OUT OF MEMORY. try reducing the stress args: ["--vm", "1", "--vm-bytes", "150M", "--vm-hang", "1"] kubectl apply -f memory-request-limit.yaml - You will see the pod is in running state.

-> Requests are used to reserve resources for the pod. If the node does not have enough resources, the pod will not be scheduled. -> Limits are used to limit the resources that the pod can use. If the pod tries to use more resources than the limit, it will be killed.

HPA (default) - Horizontal Pod AutoScaling VPA - Vertical Pod AutoScaling

Cluster AutoScaler Node AutoScaler

1. Create a deployment using php-apache image. 2. autoscale the deployment using HPA. 3. Increase the load on the deployment by sending requests to the service. 4. Check the number of pods in the deployment. 5. Check the metrics of the deployment using metrics server.

Create metric-server follow the above day-16 to create metric-server. Create php-apache deployment with service wget https://k8s.io/examples/application/php-apache.yaml kubectl apply -f php-apache.yaml kubectl get deploy kubectl get svc kubectl get pods Create an HPA imperatively kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10 Check the HPA kubectl get hpa Now increase the request to the php server (run the below command in another terminal) kubectl run -i --tty load-generator --image=busybox /bin/sh -c "while true; do wget -q -O- http://php-apache; done" (The above command will send requests to the php server and increase the load on the server.) Check kubectl get hpa --watch kubectl get pods

HPA -> HPA is best for applications with high availability and low latency requirements, especially when the load varies and horizontal scaling (more pods) is needed. -> where you need to quickly add/remove pods to handle increased traffic or load. -> Common use case: Stateless services, microservices, APIs. -> Helps maintain high availability and low latency under dynamic traffic. VPA -> VPA is best for applications with Adjusts CPU and memory requests/limits for containers automatically (vertical scaling). -> VPA Restarts the pod when applying new resource settings, which can introduce brief downtime. -> VPA is Best suited for: Batch jobs, long-running workloads, or applications with steady load but unpredictable resource usage.

Create a http-liveness probe wget https://k8s.io/examples/pods/probe/exec-liveness.yaml Create a grpc-liveness probe wget https://k8s.io/examples/pods/probe/grpc-liveness.yaml Create a http-liveness probe wget https://k8s.io/examples/pods/probe/http-liveness.yaml Create a pod-with-tcp-socket-healthcheck probe wget https://k8s.io/examples/pods/probe/pod-with-tcp-socket-healthcheck.yaml

Health Probes -> Probes are system or process that monitor our system and take necessary actions if the system is not working as expected, run a script or command to check the health of the system. -> Monitor health of the application, Literally "Checkups". -> There are three types of health probes: 1. Liveness Probe (Ensure the application is running & restart the application when failed to respond, If not rectified throw error & kill the pod) ) 2. Readiness Probe (Ensure the application is ready to serve traffic) 3. Startup Probe (for slow legacy apps) -> Types of Health checks: 1. HTTP GET 2. TCP Socket 3. Exec 4. GRPC

cm

ConfigMap creation via Imperative way kubectl create cm app-cm --from-literal=key1=value1 --from-literal=key2=value2 Check kubectl describe cm app-cm Call the configmap in the spec block of pod ... .... spec: -name: my-app env: - name: FIRSTNAME value - configMapRef: name: app-cm key: key1 Exec into the pod and check the env variable kubectl apply -f pod.yaml kubectl exec -it my-app -- sh echo $FIRSTNAME kubectl describe pod my-app Call key-values from a file kubectl create cm app-cm --from-file=app.config app.config file --------------- key1=value1 key2=value2 ConfigMap creation via Declarative way wget https://k8s.io/examples/configmap/configmap-multikeys.yaml

ConfigMaps -> If we need to use same config file & Environment Values in multiple pods, we can use configmaps. -> ConfigMaps are used to store configuration data in key-value pairs. -> Secrets are used to store sensitive data like passwords, tokens, etc.

Genearte a Private Key openssl genrsa -out adam.key 2048 Creating a CSR req file openssl req -new -key adam.key -out adam.csr - subj "/CN=adam" Decrypt the code of .csr file cat adam.csr | base64 | tr -d '\n' (copy the output) Copy the output and paste in the request block in spec of 'csr.yaml' file csr.yaml file apiVersion: certificates.k8s.io/v1 kind: CertificateSigningRequest metadata: name: adam spec: request: {paste the output here} signerName: kubernetes.io/kube-apiserver-client expirationSeconds: 86400 usages: - client auth Apply the csr.yaml file kubectl apply -f csr.yaml Approve the csr kubectl certificate approve adam (here we are getting approval of an internal CA instead of authoirzed CA) Create a file to get the public cert kubectl get csr adam -o yaml > issuecert.yaml Open issuecert.yaml and copy the certificate vim issuecert.yaml (copy the certificate) Decoding the encoded cert echo {copied_cert} | base64 -d (we will add this cert to kubeconfig file using which user can access the cluster)

File Identification tip -> If you find 'cert'/'pem' extension in the file, it is a public key certificate. -> If you find 'key'/'pem' extension in the file, it is a private key certificate. Openssl -> Openssl is a toolkit for the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols. -> It is used to generate public key & private key certificate. CSR -> Certificate Signing Request is a request to a CA to sign a certificate. -> CSR contains the public key and is used to request a certificate from a CA. It does not generate keys

find the kubeconfig file cd $HOME ls -ltr cd .kube ls -lrt Login into controlplane docker ps docker exec -it {container_id} bash cd /etc/kubernetes/manifests ls -lrt cat kube-apiserver.yaml ( under command block you will find the authentication & authorization methods used by kube-apiserver. --authorization-mode=Node,RBAC where Node -> Priority 1 RBAC -> Priority 2 ) To find the certs and keys used by kube-apiserver cd /etc/kubernetes/pki ls -lrt ( you will find the certs and keys used by kube-apiserver)

-> Everytime you create a any resources, an additional values like client certificate are internally passed by kubeconfig file to authenticate the user with K8s API server. kubeconfig file -> "kubeconfig" is the file that contains the configuration for the kubelet to connect to the API server. Which takes care of authentication & authorization. -> A kubeconfig file is a YAML file that contains: - clusters info - users info - contexts info context -> A context is a combination of a cluster, a user, and a namespace. It is used to group the information about how to connect to a cluster. Types of Authorization 1. Symmetric key encryption 2. Asymmetric key encryption 3. Certificates based Types of Authorization 1. Node Authorization 2. ABAC (Attribute Based Access Control ) - restart the API server to apply changes 3. RBAC (Role Based Access Control) 4. Webhook (External Authorization)