鲲鹏机密容器验证

远程证明服务配置

配置验证VirtCCA度量报告的policy。

mkdir -p /opt/confidential-containers/attestation-service/token/simple/policies/opa
vim /opt/confidential-containers/attestation-service/token/simple/policies/opa/default.rego

# default.rego内容如下
package policy
import future.keywords.every
import future.keywords.if
default allow := false
allow if {
    print("Full Input:", input)
    print("Rim:", input["virtcca.realm.rim"])
    print("Ref:", data.reference)
    input["virtcca.realm.rim"] in data.reference["virtcca.realm.rim"]
}

远程证明服务节点预置远程证明根证书、二级CA证书文件。
```
mkdir -p /etc/attestation/attestation-service/verifier/virtcca/ && cd /etc/attestation/attestation-service/verifier/virtcca/
```
访问网站下载证书，并将下面证书放到当前目录，保持下载的证书名称不变。
```
Huawei Equipment Root CA.pem
Huawei IT Product CA.pem
```
- RSA根证书下载链接：https://download.huawei.com/dl/download.do?actionFlag=download&nid=PKI1000000002&partNo=3001&mid=SUP_PKI
- RSA二级CA证书下载链接：https://download.huawei.com/dl/download.do?actionFlag=download&nid=PKI1000000040&partNo=3001&mid=SUP_PKI
- ECC根证书下载链接：https://download.huawei.com/dl/download.do?actionFlag=download&nid=PKI1100000224&partNo=3001&mid=SUP_PKI
- ECC二级CA证书下载链接：https://download.huawei.com/dl/download.do?actionFlag=download&nid=PKI1100000225&partNo=3001&mid=SUP_PKI

通过rvps工具添加RIM基线值。

cd /opt/confidential-containers/attestation-service
cat << EOF > sample
{
    "virtcca.realm.rim": [
        "0f7733fcbaa9059d4d579fab25743868a2b4027290b09dfbc59964fc4b642kkk"
    ]
}
EOF
provenance=$(cat sample | base64 --wrap=0)
cat << EOF > message
{
    "version" : "0.1.0",
    "type": "sample",
    "payload": "$provenance"
}
EOF

# 将基线值注册到rvps服务
kubectl exec -it -n trustee-operator-system trustee-deployment-694bf879d5-7wzwp -c rvps -- bash
./rvps-tool register --path /opt/confidential-containers/attestation-service/message --addr http://127.0.0.1:50003

virtcca.realm.rim 添加的RIM基线值（列表）可通过基线值生成工具生成。

基础工具安装

安装golang。

wget https://golang.google.cn/dl/go1.22.4.linux-arm64.tar.gz
tar -zxvf go1.22.4.linux-arm64.tar.gz -C /usr/local
echo "export GO_HOME=/usr/local/go" >> /etc/profile
echo "export GOPATH=/home/work/go" >> /etc/profile
source /etc/profile
echo "export PATH=${GO_HOME}/bin:${PATH}" >> /etc/profile
source /etc/profile

安装cosign镜像签名工具。

wget https://github.com/sigstore/cosign/releases/download/v2.5.0/cosign-linux-arm64
chmod +x cosign-linux-arm64
mv cosign-linux-arm64 /usr/local/bin/cosign

skopeo编译安装。

git clone -b v1.15.1 https://github.com/containers/skopeo.git $GOPATH/src/github.com/containers/skopeo
yum install -y gpgme-devel device-mapper-devel
cd $GOPATH/src/github.com/containers/skopeo && make bin/skopeo
cp bin/skopeo /usr/local/bin

# 测试拉取镜像到本地，可以看到镜像被成功拉取
mkdir -p /home/work/images && cd /home/work/images
skopeo copy --insecure-policy   
docker://docker.io/library/busybox:latest dir:busybox:latest

容器镜像加解密

官方参考文档： https://github.com/confidential-containers/guest-components/tree/main/attestation-agent/coco_keyprovider https://confidentialcontainers.org/docs/features/encrypted-images/

生成加密秘钥并加密容器镜像

# 生成加密秘钥
KEY_FILE="image_key"
head -c 32 /dev/urandom | openssl enc > "$KEY_FILE"

# 创建加密镜像生成目录
mkdir output

# 创建容器并在容器中使用生成的秘钥加密明文镜像（-s指定被加密的原始镜像）
docker run -v "$PWD/output:/output" \\
-e http_proxy="http://IP:PORT" \\
-e https_proxy="http://IP:PORT" \\
ghcr.io/confidential-containers/staged-images/coco-keyprovider:latest /encrypt.sh \\
-k "$(base64 < image_key)" \\
-i kbs:///default/image_key/busybox_enc \\
-s docker://busybox:latest \\
-d dir:/output

# 推送加密镜像到本地镜像仓
skopeo copy dir:output docker://registry.hw.com:5000/busybox_enc:latest

# 通过下述命令检查镜像是否加密成功
skopeo inspect docker://registry.hw.com:5000/busybox_enc:latest

# 复制加密秘钥到指定路径
cp ./image_key /opt/confidential-containers/kbs/repository/default/image_key/busybox_enc

启动加密镜像容器

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: enc-test
  annotations:
    io.containerd.cri.runtime-handler: "kata-qemu-virtcca"
    io.katacontainers.config.hypervisor.kernel_params: "agent.debug_console agent.log=debug agent.image_policy_file=kbs:///default/security-policy/test agent.enable_signature_verification=true agent.guest_components_rest_api=all agent.aa_kbc_params=cc_kbc::http:[证明服务pod IP]:8080"
spec:
  runtimeClassName: kata-qemu-virtcca
  terminationGracePeriodSeconds: 5
  containers:
  - name: box
    image: registry.hw.com:5000/busybox_enc:latest
    imagePullPolicy: Always
    command:
    - sh
    tty: true
EOF

补充pod.metadata.annotations中的agent.aa_kbc_params参数，需填入远程证明服务的pod IP，示例如下。

# 查看远程证明服务pod，kubectl get pod -n trustee-operator-system -owide
NAMESPACE                  NAME                                    READY   STATUS    IP
trustee-operator-system    trustee-deployment-694bf879d5-7wzwp     3/3     Running   10.244.0.231

# pod annotations里agent.aa_kbc_params=cc_kbc::http:10.244.0.231:8080"

容器镜像签名

CoCo社区参考文档：https://confidentialcontainers.org/docs/features/signed-images/

签名镜像

# 生成密钥对(两次回车可无需设置密码，用户视自身诉求而定)
cosign generate-key-pair

# --tlog-upload=false表示不上传到cosign官方（即离线签名，用户视自身诉求而定）
cosign sign --key cosign.key --tlog-upload=false registry.hw.com:5000/busybox:latest

# 部署签名公钥
mkdir -p /opt/confidential-containers/kbs/repository/default/cosign-key
cp ./cosign.pub /opt/confidential-containers/kbs/repository/default/cosign-key/1

指定镜像仓验证策略

vim /opt/confidential-containers/kbs/repository/default/security-policy/test

{
    "default": [
        {
            "type": "reject"
        }
    ],
    "transports": {
        "docker": {
            "registry.hw.com:5000": [
                {
                    "type": "sigstoreSigned",
                    "keyPath": "kbs:///default/cosign-key/1"
                }
            ]
        }
    }
}

说明： 设置type字段指定具体验证要求，"reject"为拒绝任何镜像，"sigstoreSigned"为要求验证签名，"insecureAcceptAnything"为不进行任何验证。

验签启动容器

签名验签demo：

apiVersion: v1
kind: Pod
metadata:
  name: sign-test
  annotations:
    io.containerd.cri.runtime-handler: "kata-qemu-virtcca"
    io.katacontainers.config.hypervisor.kernel_params: "agent.debug_console agent.log=debug agent.image_policy_file=kbs:///default/security-policy/test agent.enable_signature_verification=true agent.guest_components_rest_api=all agent.aa_kbc_params=cc_kbc::http:[证明服务pod IP]:8080"
spec:
  runtimeClassName: kata-qemu-virtcca
  terminationGracePeriodSeconds: 5
  containers:
  - name: box
    image: registry.hw.com:5000/busybox:latest
    imagePullPolicy: Always
    command:
      - sh
    tty: true

补充pod.metadata.annotations中的agent.aa_kbc_params参数，需填入远程证明服务的pod IP，示例如下。

# 查看远程证明服务pod，kubectl get pod -n trustee-operator-system -owide
NAMESPACE                  NAME                                    READY   STATUS    IP
trustee-operator-system    trustee-deployment-694bf879d5-7wzwp     3/3     Running   10.244.0.231

# pod annotations里agent.aa_kbc_params=cc_kbc::http:10.244.0.231:8080"