"Pluggable" key serialization in JCE/JCA

[Michael StJohns]

On 3/25/2022 4:03 AM, Anders Rundgren wrote:
> Hi Mike & the JDK crypto team,
>
> What I'm saying is that key serialization in Java haven't gotten 
> enough attention.  Examples:
>
> AFAIK, there is still no support for using named curves to construct 
> an EC key.  Names curves are MANDATORY in JOSE/CODE.

Use AlgorithmParameterGenerator with ECGenParameterSpec.  Works like a 
charm.

>
> The gap between EdDSA keys as expressed in JOSE/COSE and the Java API 
> is simply put GIGANTIC:
> https://mail.openjdk.java.net/pipermail/security-dev/2020-August/022379.html 
>
> The only obvious solution is rolling your own:
> https://github.com/cyberphone/openkeystore/blob/master/library/src/org/webpki/crypto/OkpSupport.java 
>
> This code [probably] stinks but it is hardly my fault...

TL;dr - but if you're talking about EDDSA and the equivalent ECDH 
functions their related key  ilk - well, I tried, but lost the battle to 
use the same API as the existing key material.  I'd call that a self 
inflicted wound.

>
> The JDK crypto team is the only party who can clean up this mess.
>
> Since importing X.509 certificates through PKCS12/KeyStore works out 
> of the box without any a prior knowledge of key algorithms, there is 
> already a precedent.  It is almost just copy and paste :)

Yeah, but even then that model  has problems.  Cf the discussion on this 
list about an RSA key that was marked with RSA-OAEP as its algorithm and 
the issued I encountered being able to use it as java key material.   I 
got back a PublicKey, but one that I couldn't coerce into an 
RSAPublicKey without re-encoding the key and changing the algorithm OID 
- it just wasn't supported.  And that model only knows about ASN1 - or 
did someone update the CertificateFactory for JOSE/COSE format 
certificates (if such a thing exists)?

>
> The new crypto systems that are on the way in (BLS, Post Quantum), 
> accentuates this issue.
And the "converter" as you call it will be far behind.
>
> Regarding HSMs, I gave it one more thought and I didn't come up with 
> any connections except maybe in Keytool but HSMs appear to be 
> X.509-oriented which not the case with COSE/JOSE.

Think PKCS11, the PKCS11 Java provider, and the ability to retrieve a 
public key from an HSM for various purposes.  The ability to place a 
private key on the HSM to secure it (there are a number of use cases 
where generating an unextractable private key on the HSM - the usual 
model - can't be used).  HSMs are cryptographically oriented not X509 
oriented and new mechanisms and key types are added all the time.  They 
have tools to do the key loading directly - from well known formats, or 
programatically through things like Java PKCS11 or C PKCS11.

Later, Mike


>
> Regards,
> Anders
>
> On 2022-03-25 3:12, Michael StJohns wrote:
>> On 3/24/2022 12:28 PM, Anders Rundgren wrote:
>>> On 2022-03-24 16:59, Michael StJohns wrote:
>>>> On 3/24/2022 2:46 AM, Anders Rundgren wrote:
>>>>> Hi List,
>>>>>
>>>>> I find it a bit strange that every user of crypto either have to 
>>>>> write
>>>>> or install specific software for converting JOSE/COSE/PEM keys
>>>>> back-and-forth to Java's internal representation. This reduces the
>>>>> value of the abstract types as well.
>>>>>
>>>>> Now there is whole bunch of new algorithms coming to the Java 
>>>>> platform
>>>>> making this task even less attractive.
>>>>>
>>>>> So my question is simply: How about including this part in an 
>>>>> enhanced
>>>>> JCE/JCA?  That is, making the encoder/decoder "pluggable" and hiding
>>>>> this complexity from users and library developers?
>>>>
>>>> Hi Anders -
>>>
>>> Hi Mike,
>>>
>>>>
>>>> Isn't that the exact purpose for KeyFactory and its plugins?
>>>
>>> They presume that you know not only the key algorithm but the
>>> associated parameters as well.  Why should users or library developers
>>> have to deal with that?
>>
>> Um - no.  They presume you know the key algorithm, but (using EC Public
>> keys for example), you can use either of the X509PublicKeySpec (to
>> decode an encoded key), or the ECPublicKeySpec (to deal with building a
>> key from scratch using the parameters.  You can notionally (haven't
>> tried it, but should work) use KeyFactory.getKeySpec() to convert
>> between the two formats or use Key.getEncoded() to get the default
>> encoding for the key.
>>
>> The equivalent on the ECPrivateKey side are the ECPrivateKeySpec and
>> PKCS8EncodedKeySpec clases.
>>
>> PEM is actually not an encoding, but a wrapping of an encoding. The only
>> part of Java that deals with it natively (I believe) is the
>> CertificateFactory for X.509 certificates.  You may have an extra step
>> to add or remove a PEM style envelope (or the equivalent BASE64 bare
>> string), but that's reasonable.
>>
>>>
>>> The requirement to know key algorithm in advance forces you into ASN.1
>>> decoding for dealing with PEMs.  Been there done that.
>>
>> Ah - not sure why you wouldn't know the key algorithm, BUT:  It should
>> theoretically be possible to write a KeyFactory provider that does that
>> decoding for you from the X509PublicKeySpec (or PKCS8PrivateKeySpec) and
>> returns an appropriate PublicKey. Check the actual return type to figure
>> out what type of key.  E.g.:
>>
>> KeyFactory kf = KeyFactory.getInstance("GenericDecoder");
>>
>> As I said below, you'll need to define the equivalent opaque format
>> KeySpec classes to handle COSE and JOSE.  I think that's a class per
>> type of encoding.
>>
>>>
>>>>
>>>> You might need to add KeySpec types for Jose and Cose
>>>> (JOSEEncodedKeySpec and COSEEncodedKeySpec) assuming both of those 
>>>> cover
>>>> all of the secret, public and private key specifications.
>>>>
>>>> Or hmm... GenericEncodedKeySpec (String alg, byte[] encoded key) or
>>>> GenericEncodedKeySpec (String alg, String encodedKey).
>>>>
>>>>
>>>>>
>>>>> Eventually you could end up with something like:
>>>>>
>>>>> PrivateKey privateKey = new KeyConverter().readPrivateKey(byte[] or
>>>>> stream);
>>>>>
>>>>> You would not even have to know in which format the key is 
>>>>> supplied in
>>>>> since this could be accomplished by simple "sniffing".
>>>>
>>>> Nope.  This isn't safe as someone might up with yet another
>>>> representation that looks like one of the "sniffable" ones. You could
>>>> build a private implementation that takes its best shot, but ....
>>>
>>> Finding out if the container is COSE, JOSE, or PEM is (AFAICT)
>>> trivial.  If the guess is incorrect a properly designed decoder should
>>> simply fail.
>>
>> It's trivial NOW because its a closed set of possibilities.  And even
>> then, you're assuming you don't have to detect ASN1 vs raw key encodings
>> vs string encodings.  Simply detecting a) whether a file is character,
>> b) detecting the character encoding, and c) accidentally thinking a
>> character file is actually binary or vice versa is fragile.
>>
>> The addition of the "new" EC variant keys required a substantial
>> reworking of the API to support them and the arguments were fierce.  The
>> additions were not free.
>>
>> I deal with no fewer than 3 encodings for an EC public key.  Two or
>> three for an EC private key. 2 or 3 text versions of a RSA public key
>> along with the X509Encoded  binary version and the TPM binary version.
>> Etc.  I usually know what type of key I'm dealing with when I get it,
>> but I would love a magic DWIM class that did the right thing on all 
>> inputs.
>>
>>
>>>
>>>>
>>>>>
>>>>> To make "pluggability" feasible, I'm trying to convince the JOSE/COSE
>>>>> folks to give each new crypto system a separate namespace as an
>>>>> alternative to overloading OKP (RFC 8037), even if the parameters
>>>>> match technically.  AFAICT, X.509 public keys essentially already
>>>>> adhere to this notion.
>>>>>
>>>>> I would exclude private key import and export in HSMs since these are
>>>>> specific and rare occasions.
>>>>
>>>> Again, no.  Don't do this in an incomplete way - it will come back to
>>>> bite you.  You don't have to implement the plugin that talks to the 
>>>> HSM,
>>>> but you have to define the mechanism/API so that the HSM vendors don't
>>>> curse your first born child later.
>>>
>>>
>>> This is little bit out of my league but targeting the masses is core.
>>
>> This is the problem with open source some times. It may be core, but it
>> is not complete and leaving this as a problem to be worked later is a
>> bit unprofessional.
>>
>> I don't mean any offense here, but there's a problem with basing
>> everything what IOT requires and ignoring everything else.
>>
>> Later, Mike
>>
>>
>>>
>>>
>>> Anders
>>>
>>>>
>>>> Mike
>>>>
>>>>
>>>>>
>>>>> WDYT?
>>>>>
>>>>> Thanx,
>>>>> Anders
>>>>
>>>>
>>>
>>
>