关于Pod中进程在节点中的研究-Toy模板网

这篇具有很好参考价值的文章主要介绍了关于Pod中进程在节点中的研究。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

最近研究OpenShift virtulization, 各种Pod对KVM进程的封装，引发了Pod中进程到底在Node中是什么表现形势的好奇，因为对基础知识的不扎实，还是希望找个环境能仔细看看，建立起openshift 4.12的环境后，首先列出某个节点上的所有的Pod

[lab-user@bastion ~]$ oc get pods -A --field-selector=spec.host=ip-10-0-148-57.us-east-2.compute.internal
NAMESPACE                                NAME                                                     READY   STATUS      RESTARTS      AGE
openshift-cluster-csi-drivers            aws-ebs-csi-driver-node-c4ddr                            3/3     Running     0             46m
openshift-cluster-node-tuning-operator   tuned-5lrjg                                              1/1     Running     0             46m
openshift-console                        downloads-595667555c-477b2                               1/1     Running     0             39m
openshift-debug-b7f7z                    ip-10-0-148-57.us-east-2.compute.internal-debug          1/1     Running     0             22m
openshift-dns                            dns-default-cxr55                                        2/2     Running     0             45m
openshift-dns                            node-resolver-szmvw                                      1/1     Running     0             46m
openshift-image-registry                 image-registry-6945b97db4-gnjfl                          1/1     Running     0             44m
openshift-image-registry                 node-ca-56rgw                                            1/1     Running     0             44m
openshift-ingress-canary                 ingress-canary-7twkk                                     1/1     Running     0             44m
openshift-ingress                        router-default-74f6d94f48-cwczb                          1/1     Running     0             30m
openshift-machine-config-operator        machine-config-daemon-fgchf                              2/2     Running     0             46m
openshift-monitoring                     alertmanager-main-1                                      6/6     Running     1 (39m ago)   39m
openshift-monitoring                     node-exporter-w8f5k                                      2/2     Running     0             44m
openshift-monitoring                     prometheus-adapter-b87c6d546-7jlzw                       1/1     Running     0             39m
openshift-monitoring                     prometheus-k8s-1                                         6/6     Running     0             39m
openshift-monitoring                     prometheus-operator-admission-webhook-566d9cc57c-mfwc8   1/1     Running     0             52m
openshift-monitoring                     thanos-querier-7dcb8856d-d8g8t                           6/6     Running     0             39m
openshift-multus                         multus-8cgln                                             1/1     Running     0             46m
openshift-multus                         multus-additional-cni-plugins-cpdbf                      1/1     Running     0             46m
openshift-multus                         network-metrics-daemon-5rjqw                             2/2     Running     0             46m
openshift-network-diagnostics            network-check-source-6655898df9-p9qjz                    1/1     Running     0             54m
openshift-network-diagnostics            network-check-target-5qlvj                               1/1     Running     0             46m
openshift-operator-lifecycle-manager     collect-profiles-28103550-nrvlz                          0/1     Completed   0             40m
openshift-operator-lifecycle-manager     collect-profiles-28103565-lt2ct                          0/1     Completed   0             28m
openshift-operator-lifecycle-manager     collect-profiles-28103580-nwh94                          0/1     Completed   0             13m
openshift-ovn-kubernetes                 ovnkube-node-p4dld                                       5/5     Running     0             46m

以prometheus-adapter-xxxx为例，登录进去查看一下Pod里面运行的进程

[lab-user@bastion ~]$ oc project openshift-monitoring 
Already on project "openshift-monitoring" on server "https://api.cluster-gmwqp.gmwqp.sandbox1496.opentlc.com:6443".
[lab-user@bastion ~]$ oc rsh prometheus-adapter-b87c6d546-7jlzw 
sh-4.4$ ps -ef
UID          PID    PPID  C STIME TTY          TIME CMD
1000430+       1       0  0 08:34 ?        00:00:13 /usr/bin/adapter --prometheus-auth-config=/etc/prometheus-config/prometheus-config.yaml --config=/etc/adapter/config.yaml --logtostderr=
1000430+      37       0  1 09:15 pts/0    00:00:00 /bin/sh
1000430+      43      37  0 09:15 pts/0    00:00:00 ps -ef

比较简单，只有一个，然后登陆到节点查看相关进程

sh-4.4# ps -ef | grep prometheus-adapter
root       11677       1  0 08:34 ?        00:00:00 /usr/bin/conmon -b /run/containers/storage/overlay-containers/1961af83ec2bf15ae89371edd5ebb534886c9db6bfaa385ac4fb53b4232df966/userdata -c 1961af83ec2bf15ae89371edd5ebb534886c9db6bfaa385ac4fb53b4232df966 --exit-dir /var/run/crio/exits -l /var/log/pods/openshift-monitoring_prometheus-adapter-b87c6d546-7jlzw_18617999-d875-40e2-b29f-c40743f8314b/prometheus-adapter/0.log --log-level info -n k8s_prometheus-adapter_prometheus-adapter-b87c6d546-7jlzw_openshift-monitoring_18617999-d875-40e2-b29f-c40743f8314b_0 -P /run/containers/storage/overlay-containers/1961af83ec2bf15ae89371edd5ebb534886c9db6bfaa385ac4fb53b4232df966/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/1961af83ec2bf15ae89371edd5ebb534886c9db6bfaa385ac4fb53b4232df966/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/1961af83ec2bf15ae89371edd5ebb534886c9db6bfaa385ac4fb53b4232df966/userdata -r /usr/bin/runc --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio --syslog -u 1961af83ec2bf15ae89371edd5ebb534886c9db6bfaa385ac4fb53b4232df966 -s
root       64634   35507  0 09:16 pts/1    00:00:00 grep prometheus-adapter

可见Pod里面的进程和Nodes上的进程是一对一的关系，我们再找一个多进程的，比如prometheus-k8s-1

因为rsh进入pod以后发现没有ps,那么我们登录节点看看

sh-4.4# crictl ps | grep prometheus-k8s-1
1b1d29d044d4f       8dd80e25afda5c9466a26ef90346b419a8cc9840bd8b553257a0c6df2c5bac6c                                                         42 minutes ago      Running             kube-rbac-proxy-thanos                  0                   9d74de5a22175       prometheus-k8s-1
bacbcddd17d2c       8dd80e25afda5c9466a26ef90346b419a8cc9840bd8b553257a0c6df2c5bac6c                                                         42 minutes ago      Running             kube-rbac-proxy                         0                   9d74de5a22175       prometheus-k8s-1
22d9f341cb2b9       e1f27569f2e6745ac23c33e89b8e569069970ab56a54114e93f6a31570b86334                                                         42 minutes ago      Running             prometheus-proxy                        0                   9d74de5a22175       prometheus-k8s-1
a36c49e125c9a       4efdbf6a731d2feb217ecd0fb0d792dcadf854e826d4db235cafc0268b2621d4                                                         42 minutes ago      Running             thanos-sidecar                          0                   9d74de5a22175       prometheus-k8s-1
76140a26677b1       c9868e04bec41eda58502d70bacc3f2cb5084f5d643b76fb018e3076f7d38014                                                         42 minutes ago      Running             config-reloader                         0                   9d74de5a22175       prometheus-k8s-1
2b0aa6f218f55       quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:97ffae63353c694fe66615679429351a96711cbdeb400555181dea8688a388ea   42 minutes ago      Running             prometheus   

sh-4.4# crictl ps | grep prometheus-k8s-1 | wc -l
6

然后通过ps查看

sh-4.4# ps -ef | grep prometheus-k8s-1        
root       12918       1  0 08:34 ?        00:00:00 /usr/bin/conmon -b /run/containers/storage/overlay-containers/2b0aa6f218f552f0d1009b07a80d23d9d47c8b63c23777fb51dcb1aa648d0316/userdata -c 2b0aa6f218f552f0d1009b07a80d23d9d47c8b63c23777fb51dcb1aa648d0316 --exit-dir /var/run/crio/exits -l /var/log/pods/openshift-monitoring_prometheus-k8s-1_0a15cd78-757d-4322-9f35-94f7c42860b4/prometheus/0.log --log-level info -n k8s_prometheus_prometheus-k8s-1_openshift-monitoring_0a15cd78-757d-4322-9f35-94f7c42860b4_0 -P /run/containers/storage/overlay-containers/2b0aa6f218f552f0d1009b07a80d23d9d47c8b63c23777fb51dcb1aa648d0316/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/2b0aa6f218f552f0d1009b07a80d23d9d47c8b63c23777fb51dcb1aa648d0316/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/2b0aa6f218f552f0d1009b07a80d23d9d47c8b63c23777fb51dcb1aa648d0316/userdata -r /usr/bin/runc --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio --syslog -u 2b0aa6f218f552f0d1009b07a80d23d9d47c8b63c23777fb51dcb1aa648d0316 -s
root       12978       1  0 08:34 ?        00:00:00 /usr/bin/conmon -b /run/containers/storage/overlay-containers/76140a26677b1512e18864b5c1bfb31a2847d581721a64cd7f7633fb2a62ab66/userdata -c 76140a26677b1512e18864b5c1bfb31a2847d581721a64cd7f7633fb2a62ab66 --exit-dir /var/run/crio/exits -l /var/log/pods/openshift-monitoring_prometheus-k8s-1_0a15cd78-757d-4322-9f35-94f7c42860b4/config-reloader/0.log --log-level info -n k8s_config-reloader_prometheus-k8s-1_openshift-monitoring_0a15cd78-757d-4322-9f35-94f7c42860b4_0 -P /run/containers/storage/overlay-containers/76140a26677b1512e18864b5c1bfb31a2847d581721a64cd7f7633fb2a62ab66/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/76140a26677b1512e18864b5c1bfb31a2847d581721a64cd7f7633fb2a62ab66/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/76140a26677b1512e18864b5c1bfb31a2847d581721a64cd7f7633fb2a62ab66/userdata -r /usr/bin/runc --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio --syslog -u 76140a26677b1512e18864b5c1bfb31a2847d581721a64cd7f7633fb2a62ab66 -s
root       13037       1  0 08:34 ?        00:00:00 /usr/bin/conmon -b /run/containers/storage/overlay-containers/a36c49e125c9a385e8e32e904e61c404c40135a1c5040d01244692e1a930f730/userdata -c a36c49e125c9a385e8e32e904e61c404c40135a1c5040d01244692e1a930f730 --exit-dir /var/run/crio/exits -l /var/log/pods/openshift-monitoring_prometheus-k8s-1_0a15cd78-757d-4322-9f35-94f7c42860b4/thanos-sidecar/0.log --log-level info -n k8s_thanos-sidecar_prometheus-k8s-1_openshift-monitoring_0a15cd78-757d-4322-9f35-94f7c42860b4_0 -P /run/containers/storage/overlay-containers/a36c49e125c9a385e8e32e904e61c404c40135a1c5040d01244692e1a930f730/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/a36c49e125c9a385e8e32e904e61c404c40135a1c5040d01244692e1a930f730/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/a36c49e125c9a385e8e32e904e61c404c40135a1c5040d01244692e1a930f730/userdata -r /usr/bin/runc --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio --syslog -u a36c49e125c9a385e8e32e904e61c404c40135a1c5040d01244692e1a930f730 -s
root       13087       1  0 08:34 ?        00:00:00 /usr/bin/conmon -b /run/containers/storage/overlay-containers/22d9f341cb2b9f8748852a5b83146844cebb8ec2e4e00ce17016d523f5edb1ec/userdata -c 22d9f341cb2b9f8748852a5b83146844cebb8ec2e4e00ce17016d523f5edb1ec --exit-dir /var/run/crio/exits -l /var/log/pods/openshift-monitoring_prometheus-k8s-1_0a15cd78-757d-4322-9f35-94f7c42860b4/prometheus-proxy/0.log --log-level info -n k8s_prometheus-proxy_prometheus-k8s-1_openshift-monitoring_0a15cd78-757d-4322-9f35-94f7c42860b4_0 -P /run/containers/storage/overlay-containers/22d9f341cb2b9f8748852a5b83146844cebb8ec2e4e00ce17016d523f5edb1ec/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/22d9f341cb2b9f8748852a5b83146844cebb8ec2e4e00ce17016d523f5edb1ec/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/22d9f341cb2b9f8748852a5b83146844cebb8ec2e4e00ce17016d523f5edb1ec/userdata -r /usr/bin/runc --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio --syslog -u 22d9f341cb2b9f8748852a5b83146844cebb8ec2e4e00ce17016d523f5edb1ec -s
root       13142       1  0 08:34 ?        00:00:00 /usr/bin/conmon -b /run/containers/storage/overlay-containers/bacbcddd17d2cfd032c0e890783018bec8d98a06b373918576b0de533332b369/userdata -c bacbcddd17d2cfd032c0e890783018bec8d98a06b373918576b0de533332b369 --exit-dir /var/run/crio/exits -l /var/log/pods/openshift-monitoring_prometheus-k8s-1_0a15cd78-757d-4322-9f35-94f7c42860b4/kube-rbac-proxy/0.log --log-level info -n k8s_kube-rbac-proxy_prometheus-k8s-1_openshift-monitoring_0a15cd78-757d-4322-9f35-94f7c42860b4_0 -P /run/containers/storage/overlay-containers/bacbcddd17d2cfd032c0e890783018bec8d98a06b373918576b0de533332b369/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/bacbcddd17d2cfd032c0e890783018bec8d98a06b373918576b0de533332b369/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/bacbcddd17d2cfd032c0e890783018bec8d98a06b373918576b0de533332b369/userdata -r /usr/bin/runc --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio --syslog -u bacbcddd17d2cfd032c0e890783018bec8d98a06b373918576b0de533332b369 -s
root       13194       1  0 08:34 ?        00:00:00 /usr/bin/conmon -b /run/containers/storage/overlay-containers/1b1d29d044d4fadd61fdc8b3e040c4aaa9a8fb2690cffb562d170d3952dbc9d6/userdata -c 1b1d29d044d4fadd61fdc8b3e040c4aaa9a8fb2690cffb562d170d3952dbc9d6 --exit-dir /var/run/crio/exits -l /var/log/pods/openshift-monitoring_prometheus-k8s-1_0a15cd78-757d-4322-9f35-94f7c42860b4/kube-rbac-proxy-thanos/0.log --log-level info -n k8s_kube-rbac-proxy-thanos_prometheus-k8s-1_openshift-monitoring_0a15cd78-757d-4322-9f35-94f7c42860b4_0 -P /run/containers/storage/overlay-containers/1b1d29d044d4fadd61fdc8b3e040c4aaa9a8fb2690cffb562d170d3952dbc9d6/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/1b1d29d044d4fadd61fdc8b3e040c4aaa9a8fb2690cffb562d170d3952dbc9d6/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/1b1d29d044d4fadd61fdc8b3e040c4aaa9a8fb2690cffb562d170d3952dbc9d6/userdata -r /usr/bin/runc --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio --syslog -u 1b1d29d044d4fadd61fdc8b3e040c4aaa9a8fb2690cffb562d170d3952dbc9d6 -s
root       69497   35507  0 09:20 pts/1    00:00:00 grep prometheus-k8s-1

sh-4.4# ps -ef | grep prometheus-k8s-1  | wc -l
7