RFR: 8313367: SunMSCAPI cannot read Local Computer certs w/o Windows elevation [v3]

[rebarbora-mckvak]

On Wed, 20 Mar 2024 19:45:32 GMT, Weijun Wang <weijun at openjdk.org> wrote:

>> rebarbora-mckvak has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits:
>> 
>>  - 8313367: signHash finds a key in the local machine store
>>  - 8313367: Local Computer store is opened with max. allowed permissions
>
> src/jdk.crypto.mscapi/windows/native/libsunmscapi/security.cpp line 806:
> 
>> 804:             {
>> 805:                 // If the key is in a local machine store, we need to try again with CRYPT_MACHINE flag.
>> 806:                 // See https://learn.microsoft.com/en-us/troubleshoot/windows-server/windows-security/cryptacquirecontext-troubleshooting
> 
> Since you said "if the key is in a local machine store", do you need to check if this is true before retrying with the `CRYPT_MACHINE_KEYSET` flag?

I am fairly new to Windows crypto API, so I will try to describe what happens on my test system and hopefully it will help to answer your question:

> loading store: Windows-MY-LOCALMACHINE
> MSCAPI (483): --------------------------

my keystore has two private key pairs, the following is loaded using MSCAPI by `..._loadKeysOrCertificateChains`

> MSCAPI (542): CAPI hCryptProv=2599423712528 hUserKey=2599426935088
> MSCAPI (632): stepan-cert: 1.2.840.113549.1.1.1

the second key pair is loaded using CNG:

> MSCAPI (483): --------------------------
> MSCAPI (527): CNG 2599431325072
> MSCAPI (632): Stepan-user: 1.2.840.113549.1.1.1

Now I try to sign data using the first key (CAPI):

> Alias: stepan-cert
> testing private key on SHA256withRSA
> MSCAPI (779): CryptCreateHash error: 80090008 (hCryptProv=2599423712528, hCryptKey=2599426935088), will try PROV_RSA_AES container: {48A88AAD-5CC2-4BBB-A26B-D64BF6A07D21}

`..._signHash` called `CryptCreateHash`, but it failed (NTE_BAD_ALGID), because the provider does not support SHA256. The original code tries to get the name of a container using `CryptGetProvParam(..., PP_CONTAINER, ...)` and tries to acquire a context.

> MSCAPI (788): CryptAcquireContext error: 80090016, may try PROV_RSA_AES (CRYPT_MACHINE)

That fails again with _Keyset does not exist_ error. Here comes my code trying to acquire a context again now with `CRYPT_MACHINE_KEYSET` flag set and that finally succeeds and signs the data.

I did a bit more of digging in Crypto API and it seems we can call `CryptGetProvParam(..., PP_KEYSET_TYPE, ...)` to get the information where the key is stored (user/machine store). I will modify the code in that way.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16687#discussion_r1536248403