RFR: 8313367: SunMSCAPI cannot read Local Computer certs w/o Windows elevation [v4]

[rebarbora-mckvak]

On Fri, 22 Mar 2024 22:25:47 GMT, rebarbora-mckvak <duke at openjdk.org> wrote:

>> This fixes the defect described at https://bugs.openjdk.org/browse/JDK-8313367
>> 
>> If the process does not have write permissions, the store is opened as read-only (instead of failing).
>> 
>> Please note that permissions to use a certificate in a local machine store must be granted - in a management console, select a certificate, right-click -> All tasks... -> Manage Private Keys... -> add Full control to user.
>
> rebarbora-mckvak has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8313367: signHash looks for a key in either user or machine store

The new code tested with keys in user and machine store:

> loading store: Windows-MY
> MSCAPI (542): CAPI hCryptProv=2240584244800 hUserKey=2240588561616
> MSCAPI (632): testkey: 1.2.840.113549.1.1.1
> Alias: testkey
> testing private key on SHA256withRSA
> MSCAPI (783): CryptCreateHash error: 80090008 (hCryptProv=2240584244800, hCryptKey=2240588561616), will try PROV_RSA_AES container: {A66FC309-D9F5-40ED-BFD9-6AC8D28A94D3}, keysetType=0
> SUCCESS, siglen: 256

Note the detected keysetType is 0 i.e. user's store.

> loading store: Windows-MY-LOCALMACHINE
> MSCAPI (542): CAPI hCryptProv=2240584244032 hUserKey=2240588562624
> MSCAPI (632): stepan-cert: 1.2.840.113549.1.1.1
> Alias: stepan-cert
> testing private key on SHA256withRSA
> MSCAPI (783): CryptCreateHash error: 80090008 (hCryptProv=2240584244032, hCryptKey=2240588562624), will try PROV_RSA_AES container: {48A88AAD-5CC2-4BBB-A26B-D64BF6A07D21}, keysetType=20
> SUCCESS, siglen: 256

Note the detected keysetType is 0x20 i.e. machine store.

BTW, I believe https://bugs.openjdk.org/browse/JDK-8328184 is caused by not passing `CRYPT_MACHINE_KEYSET` to `CryptAcquireContext` when the key is created in the store.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16687#issuecomment-2016040585