CKA & important operation commands

Passed CKA exam last Sat, would like to summarize the useful commands for future reference.

Posted by Jamie Zhang on Monday, May 29, 2023

Complains about the CKA exam:
The CKA exam environment was changed from Terminal to Remote Desktop(VNC) from Last Jun, this is an absolutely shit
decision. The huge lagging causes that large amount of candidate’s time-wasting on always-waiting, and also Mouse is nearly useless - very difficult to locate what you want on Firefox browser, and eventually it highlights the good user experience on the k8s built-in documentation - Kubectl explain, how ridiculous it is !!!

Important commands

Key file path

Path Usage Remark
/etc/systemd/system/kubelet.service.d \n /lib/systemd/system/kubelet.service kubelet service
/lib/systemd/system/containerd.service containerd service
/var/lib/kubelet ephemeral storage path it will cause cluster HasDiskPressure if 85% disk of it is being used and cannot being re-cycled
/var/lib/containerd Image Fs it will cause cluster HasDiskPressure if 85% disk of it is being used and cannot being re-cycled
/etc/kubernetes K8s cluster config, certs, static pods path very important, be careful with each file under this path.
/etc/containerd/config.toml containerd config file
/etc/crictl.yaml crictl config file

Custom output - custom column

kubectl get pod -n book-store -o json | jq -c 'paths' | grep nodeName

kubectl get pod -n kube-system -o='custom-columns=PodName:.metadata.name,Memory-Limit:.spec.containers[].resources.limits.memory,Memory-Requested:.spec.containers[].resources.requests.memory, CPU-Limit:.spec.containers[].resources.limits.cpu, CPU-Requested:.spec.containers[].resources.requests.cpu'

List all the context

k config get-contexts –no-headers=true | awk ‘{print $2}’

Check cert info & renew certs

openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt | grep Validity -A2
kubeadm certs check-expiration
Kubeadm certs renew all

Join command & token

Worker node:

kubeadm token create --print-join-command

Network policy

The matchLabel support wildcard function, e.g. ‘app: user-service’, it can work on pods whose has label of app=user-service*.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: book-store
spec:
  podSelector:
    matchLabels:
      app: nginx
  policyTypes:
    - Ingress
    - Egress
  # allowed only 10.244.46.9 from book-store to access app=busybox on protocol ICMP or TCP 22.
  # allow app=busybox to access user-service on TCP 8080
  ingress:
    - from:
        # independent section for IP CIDR
        #- ipBlock:
        #    cidr: 10.244.0.0/16
        #    except:
        #      - 10.244.164.0/24
        # independent section for namespace
        #- namespaceSelector:
        #    matchLabels:
        #      kubernetes.io/metadata.name: test
        # independent section for current namespace(this is very important for understanding) + pod selector
        #- podSelector:
        #    matchLabels:
        #      app: nginx-client
        # namespace + pod selector as a combination
        #- namespaceSelector:
        #    matchLabels:
        #      kubernetes.io/metadata.name: test
        #  podSelector:
        #    matchLabels:
        #      app: nginx-client
        - podSelector:
            matchExpressions:
              - {key: app, operator: In, values: [nginx-client]}
      ports:
        - protocol: TCP
          port: 22
        - protocol: TCP
          port: 80
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: user-service
      ports:
        - protocol: TCP
          port: 8080

Resource allocatable

Node Allocatable Resource = Node Capacity - Kube-reserved - system-reserved - eviction-threshold

kubectl describe nodes | grep 'Name:\|  cpu\|  memory'

kubectl top nodes
kubectl top pod -A --containers=true

Role operations

"get", "list", "watch", "create", "update", "patch", "delete"

Check logs

/var/log/containers /var/log/pods/kube-system_kube-apiserver-controlplane_feb98415aef8299aa43e41041294290f/kube-apiserver/4.log /var/log/syslog == journalctl -xue kubelet crictl logs kubectl logs -f deploy/xx -c xxx -n xxx

Get ingress class name

k get ingressclass

Check authorization

kubectl auth can-i get deployment --as system:serviceaccount:ns2:pipeline -n default
kubectl auth can-i create deployment --as system:serviceaccount:ns1:pipeline -n ns1
k auth can-i create pods --as smoke -n applications

Check priority class

k get PriorityClass -- value is bigger, the priority is higher.

Service/pod DNS entry

<service-name>.<namespace-name>.svc.cluster.local
<pod-ip-replace-dot-with-hyphen>.<namespace>.pod.cluster.local

/ # nslookup redis-service.default.svc.it-meta.space
Server:         10.96.0.10
Address:        10.96.0.10#53

Name:   redis-service.default.svc.it-meta.space
Address: 10.103.53.98

/ # nslookup  10-244-46-7.default.pod.it-meta.space
Server:         10.96.0.10
Address:        10.96.0.10#53

Name:   10-244-46-7.default.pod.it-meta.space
Address: 10.244.46.7

/var/lib/kubelet disk is almost full(85%), which caused pod eviction

By default empty dir(ephemeral storage) path for kubernetes is /var/lib/kubelet.

# du -sh ./pods/*

-- remove images whose tag=none
crictl images ls | awk '{ if ($2 == "<none>") { print "crictl rmi " $3} }' | xargs -0 bash -c
-- remote images whose are not in use
crictl rmi --prune

Etcd backup & restore

# backup
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/peer.crt \
--key=/etc/kubernetes/pki/etcd/peer.key snapshot save \
/opt/k8s/etcd/backup/etcd-snapshot-`date +%Y%m%d`.db

# check status
ETCDCTL_API=3 etcdctl --write-out=table snapshot status /opt/k8s/etcd/backup/etcd-snapshot-20230525.db

# restore etcd
systemctl stop kubelet.service
cd /var/lib/etcd/
rm -rf /var/lib/etcd
ETCDCTL_API=3 etcdctl snapshot restore /opt/k8s/etcd/backup/etcd-snapshot-20230529.db \
 --data-dir="/var/lib/etcd" \
 --name=master.it-meta.space --skip-hash-check \
 --initial-advertise-peer-urls=https://192.168.179.128:2380 \
 --initial-cluster=master.it-meta.space=https://192.168.179.128:2380

Notes: the etcd restore info passed in command should match with the configs in static pod /etc/kubernetes/manifests/etcd.yaml containers:

  • command:
    • etcd
    • –advertise-client-urls=https://192.168.179.128:2379
    • –cert-file=/etc/kubernetes/pki/etcd/server.crt
    • –client-cert-auth=true
    • –data-dir=/var/lib/etcd
    • –experimental-initial-corrupt-check=true
    • –experimental-watch-progress-notify-interval=5s
    • –initial-advertise-peer-urls=https://192.168.179.128:2380
    • –initial-cluster=master.it-meta.space=https://192.168.179.128:2380
    • –key-file=/etc/kubernetes/pki/etcd/server.key
    • –listen-client-urls=https://127.0.0.1:2379,https://192.168.179.128:2379
    • –listen-metrics-urls=http://127.0.0.1:2381
    • –listen-peer-urls=https://192.168.179.128:2380
    • –name=master.it-meta.space

Referred docs

https://kubernetes.io/docs/reference/kubectl/cheatsheet/

「真诚赞赏,手留余香」

Jamie's Blog

真诚赞赏,手留余香

使用微信扫描二维码完成支付