带有 TPM 的 Greengrass v2 Pkcs11Provider - 安装后无法连接

Question

【以下的问题经过翻译处理】 概述
========

我正在尝试使用带有“Pkcs11Provider”组件的 TPM2.0 HSM 进行手动 Greengrass v2 配置。

执行安装步骤后，我在 `greengrass.log` 中看到错误并且不确定如何解释。

我试过的
===============

我完成了 [此处](https://docs.aws.amazon.com/greengrass/v2/developerguide/hardware-security.html) 的 5 个步骤，并在 TPM 中存储了密钥和证书：

```
$ sudo ./tpm2_ptool listobjects --label greengrass
- CKA_CLASS: CKO_PRIVATE_KEY
  CKA_ID:
  - '64653534386238323765613433653632'
  CKA_KEY_TYPE: CKK_RSA
  CKA_LABEL: greenkey
  id: 1
- CKA_CLASS: CKO_PUBLIC_KEY
  CKA_ID:
  - '64653534386238323765613433653632'
  CKA_KEY_TYPE: CKK_RSA
  CKA_LABEL: greenkey
  id: 2
- CKA_CLASS: CKO_CERTIFICATE
  CKA_ID:
  - '64653534386238323765613433653632'
  CKA_LABEL: greenkey
  id: 3
$ sudo p11tool --login --list-privkeys 'pkcs11:manufacturer=Infineon;token=greengrass;pin-value=123456'
Object 0:
	URL: pkcs11:model=SLB9670%00%00%00%00%00%00%00%00%00;manufacturer=Infineon;serial=0000000000000000;token=greengrass;id=%64%65%35%34%38%62%38%32%37%65%61%34%33%65%36%32;object=greenkey;type=private
	Type: Private key
	Label: greenkey
	Flags: CKA_PRIVATE; CKA_NEVER_EXTRACTABLE; CKA_SENSITIVE; 
	ID: 64:65:35:34:38:62:38:32:37:65:61:34:33:65:36:32

```

在存储在 TPM 中之前，我用CSR 在 AWS 中生成了证书。

我从 [此处](https://d2s8p88vqu9w66.cloudfront.net/releases/Pkcs11Provider/aws.greengrass.crypto.Pkcs11Provider-latest.jar) 下载了 `Pkcs11Provider` JAR，它是 2022 年 11 月 21 日的 `v2.0.4` .

然后我运行命令 `sudo -E java -Droot="/home/user/greengrass/v2" -Dlog.store=FILE -jar ./GreengrassInstaller/lib/Greengrass.jar --trusted-plugin ./aws.greengrass .crypto.Pkcs11Provider-latest.jar --init-config install.yaml --component-default-user ggc_user:ggc_group --setup-system-service true`。这是配置文件的内容：

```
$ cat ~/install.yaml 
---
system:
  certificateFilePath: "pkcs11:object=greenkey;type=cert"
  privateKeyPath: "pkcs11:object=greenkey;type=private"
  rootCaPath: "/home/user/rootCA.pem"
  rootpath: "/home/user/greengrass/v2"
  thingName: "MyThing"
services:
  aws.greengrass.Nucleus:
    componentType: "NUCLEUS"
    version: "2.7.0"
    configuration:
      awsRegion: "us-west-2"
      iotRoleAlias: "GreengrassV2TokenExchangeRoleAlias"
      iotDataEndpoint: "-ats.iot.us-west-2.amazonaws.com"
      iotCredEndpoint: ".credentials.iot.us-west-2.amazonaws.com"
  aws.greengrass.crypto.Pkcs11Provider:
    configuration:
      name: "tpm2_pkcs11"
      library: "/usr/local/lib/libtpm2_pkcs11.so"
      slot: 1
      userPin: "123456"

```

不知道这是否重要，但我在 AWS IoT Core 面板中将通过 CSR 创建的证书与 Thing 相链接，并为证书提供了常用的访问策略。由于我之前在环境变量中使用令牌创建了自动配置，因此该设备已经存在。我没有删除 Thing，但在链接新证书时删除了自动创建的证书。

错误日志
=========

这是我在 Greengrass 启动时看到的错误：

```
$ sudo tail -n 100 greengrass/v2/logs/greengrass.log
2022-11-22T14:45:17.246Z [INFO] (pool-2-thread-16) com.aws.greengrass.lifecyclemanager.GenericExternalService: generic-service-shutdown. {serviceName=aws.greengrass.Nucleus, currentState=STOPPING}
2022-11-22T14:45:17.274Z [INFO] (aws.greengrass.Nucleus-lifecycle) com.aws.greengrass.lifecyclemanager.GenericExternalService: service-set-state. {serviceName=aws.greengrass.Nucleus, currentState=STOPPING, newState=FINISHED}
2022-11-22T14:45:17.376Z [INFO] (pool-2-thread-8) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: Initializing PKCS11 provider with configuration. {configuration=name=tpm2_pkcs11
library=/usr/local/lib/libtpm2_pkcs11.so
slot=1, serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW}
2022-11-22T14:45:17.915Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW, newState=INSTALLED}
2022-11-22T14:45:17.921Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=INSTALLED, newState=STARTING}
2022-11-22T14:45:17.925Z [INFO] (pool-2-thread-16) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}
2022-11-22T14:45:17.927Z [INFO] (pool-2-thread-16) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}
2022-11-22T14:45:17.929Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=STARTING, newState=RUNNING}
2022-11-22T14:45:17.933Z [INFO] (main-lifecycle) com.aws.greengrass.lifecyclemanager.GenericExternalService: service-set-state. {serviceName=main, currentState=INSTALLED, newState=STARTING}
2022-11-22T14:45:17.939Z [INFO] (pool-2-thread-11) com.aws.greengrass.lifecyclemanager.GenericExternalService: generic-service-finished. Nothing done. {serviceName=main, currentState=STARTING}
2022-11-22T14:45:17.944Z [INFO] (main-lifecycle) com.aws.greengrass.lifecyclemanager.GenericExternalService: service-set-state. {serviceName=main, currentState=STARTING, newState=FINISHED}
2022-11-22T14:45:18.890Z [ERROR] (pool-2-thread-6) com.aws.greengrass.mqttclient.MqttClient: Error subscribing. {topic=$aws/things/MyThing/jobs/$next/namespace-aws-gg-deployment/get/accepted}
java.util.concurrent.CompletionException: software.amazon.awssdk.crt.mqtt.MqttException: Error during getting mqtt connection builder
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:989)
	at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137)
	at com.aws.greengrass.mqttclient.AwsIotMqttClient.connect(AwsIotMqttClient.java:234)
	at com.aws.greengrass.mqttclient.AwsIotMqttClient.subscribe(AwsIotMqttClient.java:153)
	at com.aws.greengrass.mqttclient.MqttClient.subscribe(MqttClient.java:403)
	at com.aws.greengrass.mqttclient.WrapperMqttClientConnection.subscribe(WrapperMqttClientConnection.java:73)
	at com.aws.greengrass.deployment.IotJobsClientWrapper.SubscribeToDescribeJobExecutionAccepted(IotJobsClientWrapper.java:198)
	at software.amazon.awssdk.iot.iotjobs.IotJobsClient.SubscribeToDescribeJobExecutionAccepted(IotJobsClient.java:599)
	at com.aws.greengrass.deployment.IotJobsHelper.subscribeToGetNextJobDescription(IotJobsHelper.java:504)
	at com.aws.greengrass.deployment.IotJobsHelper.subscribeToJobsTopics(IotJobsHelper.java:463)
	at com.aws.greengrass.deployment.IotJobsHelper.lambda$setupCommWithIotJobs$5(IotJobsHelper.java:339)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.crt.mqtt.MqttException: Error during getting mqtt connection builder
	at com.aws.greengrass.mqttclient.MqttClient.lambda$new$0(MqttClient.java:180)
	at com.aws.greengrass.mqttclient.MqttClient.lambda$getNewMqttClient$16(MqttClient.java:766)
	at com.aws.greengrass.mqttclient.AwsIotMqttClient.establishConnection(AwsIotMqttClient.java:256)
	... 14 more

2022-11-22T14:45:18.906Z [WARN] (pool-2-thread-6) com.aws.greengrass.deployment.IotJobsHelper: No connection available during subscribing to Iot Jobs descriptions topic. Will retry in sometime. {ThingName=MyThing}
software.amazon.awssdk.crt.mqtt.MqttException: Error during getting mqtt connection builder
	at com.aws.greengrass.mqttclient.MqttClient.lambda$new$0(MqttClient.java:180)
	at com.aws.greengrass.mqttclient.MqttClient.lambda$getNewMqttClient$16(MqttClient.java:766)
	at com.aws.greengrass.mqttclient.AwsIotMqttClient.establishConnection(AwsIotMqttClient.java:256)
	at com.aws.greengrass.mqttclient.AwsIotMqttClient.connect(AwsIotMqttClient.java:234)
	at com.aws.greengrass.mqttclient.AwsIotMqttClient.subscribe(AwsIotMqttClient.java:153)
	at com.aws.greengrass.mqttclient.MqttClient.subscribe(MqttClient.java:403)
	at com.aws.greengrass.mqttclient.WrapperMqttClientConnection.subscribe(WrapperMqttClientConnection.java:73)
	at com.aws.greengrass.deployment.IotJobsClientWrapper.SubscribeToDescribeJobExecutionAccepted(IotJobsClientWrapper.java:198)
	at software.amazon.awssdk.iot.iotjobs.IotJobsClient.SubscribeToDescribeJobExecutionAccepted(IotJobsClient.java:599)
	at com.aws.greengrass.deployment.IotJobsHelper.subscribeToGetNextJobDescription(IotJobsHelper.java:504)
	at com.aws.greengrass.deployment.IotJobsHelper.subscribeToJobsTopics(IotJobsHelper.java:463)
	at com.aws.greengrass.deployment.IotJobsHelper.lambda$setupCommWithIotJobs$5(IotJobsHelper.java:339)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
```

结论
==========

任何诊断此问题的建议都将受到赞赏！

编辑 1 - SoftHSM
================

正如评论中所指出的，我尝试使用 softhsm 而不是 TPM 函数，以确保 Pkcs11Provider 插件能正常工作。但还是遇到了一些问题。我很确定 Pkcs11Provider 配置的 slot: 是正确的。

```
$ sudo pkcs11-tool --module /usr/local/lib/softhsm/libsofthsm2.so --list-slots
Available slots:
Slot 0 (0x69906b7d): SoftHSM slot ID 0x69906b7d
  token label        : greengrass
  token manufacturer : SoftHSM project
  token model        : SoftHSM v2
  token flags        : rng, login required, PIN initialized, token initialized, other flags=0x20
  hardware version   : 2.6
  firmware version   : 2.6
  serial num         : 2337b990e9906b7d
Slot 1 (0x1): SoftHSM slot ID 0x1
  token state:   uninitialized
  
$ sudo pkcs11-tool --module /usr/local/lib/softhsm/libsofthsm2.so --list-objects --slot 1771072381 --login
Logging in to "greengrass".
Please enter User PIN: 
Certificate Object, type = X.509 cert
  label:      
  ID:         1771072381
Private Key Object; RSA 
  label:      greenkey
  ID:         1771072381
  Usage:      decrypt, sign, unwrap
Public Key Object; RSA 2048 bits
  label:      greenkey
  ID:         1771072381
  Usage:      encrypt, verify, wrap
  
$ sudo cat greengrass/v2/config/effectiveConfig.yaml  | grep "slot\|pkcs"
  certificateFilePath: "pkcs11:object=greenkey;type=cert"
  privateKeyPath: "pkcs11:object=greenkey;type=key"
      name: "softhsm_pkcs11"
      slot: 1771072381

$ sudo tail -f greengrass/v2/logs/greengrass.log
2022-12-02T21:32:38.826Z [INFO] (pool-2-thread-14) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: Initializing PKCS11 provider with configuration. {configuration=name=softhsm_pkcs11
library=/usr/local/lib/softhsm/libsofthsm2.so
slot=1771072381, serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW}
2022-12-02T21:32:39.431Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.sec
urity.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW, newState=INSTALLED}
2022-12-02T21:32:39.438Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=INSTALLED, newState=STARTING}
2022-12-02T21:32:39.442Z [INFO] (pool-2-thread-13) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}
2022-12-02T21:32:39.444Z [INFO] (pool-2-thread-13) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}
2022-12-02T21:32:39.446Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=STARTING, newState=RUNNING}
...
2022-12-02T21:32:41.162Z [ERROR] (pool-2-thread-7) com.aws.greengrass.mqttclient.MqttClient: Error subscribing. {topic=$aws/things/MyThing/jobs/$next/namespace-aws-gg-deployment/get/accepted}
...
2022-12-02T21:32:41.173Z [WARN] (pool-2-thread-7) com.aws.greengrass.deployment.IotJobsHelper: No connection available during subscribing to Iot Jobs descriptions topic. Will retry in sometime. {ThingName=MyThing}
...
2022-12-02T21:32:41.206Z [ERROR] (pool-2-thread-8) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: Private key must be a PKCS11 private type, but was key. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=RUNNING}

```

Answer

【以下的回答经过翻译处理】 我使用了`strace`来确定AWS的`Pkcs11Provider`正在错误的位置查找TPM2 PKCS11 sqlite3数据库：
![pic1](https://repost.aws/media/postImages/original/IMVnuYdF5_QnOlQTg0wUwsfA "pic1")
这就解释了`greengrass.log`中的CKR_OPERATION_NOT_INITIALIZED错误。上述路径中的数据库是空的，因此没有与TPM正在追踪的插槽、标记和对象相匹配的条目......

需要对AWS插件进行指示，以查找TPM2数据库*实际*存在的不同位置，可以通过环境变量（可能还有其他方式）来实现：
![pic2](https://repost.aws/media/postImages/original/IMFYx26T9HRAG6HJvTkqjaLQ "pic2")

带有 TPM 的 Greengrass v2 Pkcs11Provider - 安装后无法连接

【以下的问题经过翻译处理】概述

我试过的

错误日志

结论

编辑 1 - SoftHSM

相关内容

带有 TPM 的 Greengrass v2 Pkcs11Provider - 安装后无法连接

【以下的问题经过翻译处理】 概述

我试过的

错误日志

结论

编辑 1 - SoftHSM

相关内容

【以下的问题经过翻译处理】概述