[External] : Re: PEM KeyStore Implementation
Anthony Scarpino
anthony.scarpino at oracle.com
Tue Dec 5 22:35:47 UTC 2023
On 12/1/23 2:46 AM, Karl Scheibelhofer wrote:
> Hi Tony,
>
> here is some thoughts about the PEM API Draft
> https://openjdk.org/jeps/8300911 <https://openjdk.org/jeps/8300911>
> considdering its use for implementing a PEM KeyStore:
>
> * Decoder reads from a Reader. Encoder should also support writing to
> a Writer for consistency.
At one point Writer was part of the API. But after seeing that Writer
already has a write(String) method, there was no additional
functionality the PEM API could offer that the user couldn't code
themselves.
Writer.write(PEMEncoder.encode(data));
vs
PEMEncoder.encodeToWriter(data, Writer);
>
> * Consider supporting byte[] or InputStream/OutputStream to hold the
> PEM encoded content. PEM is ASCII only anyway. There may be no need
> to go through characters and strings in some cases. Some applications
> may implement some kind of trial-and-error in detecting the encoding
> of keys or certificates. In this case, an application typically has
> byte[]/InputStream /OutputStream instead of String.
I had thought about byte[] and opted against it for similar reasons as
above. The PEMDecoder method that took a byte[] would just do a new
String(byte[]). I also felt there was potential confusion with data
formats in a byte[]. I felt byte[] is more likely to be in a binary
format (PKCS8 or X509) and PEM as a String. That the reason
EncodedKeySpec is accepted as a input type instead of byte[]. That
forces the developer to know what data they are passing into the API.
Any of the Input/OutputStream classes, I figured the use can pass their
IO object into InputStreamReader.
>
> * Encoded/Decoder may support Stream<SecurityObject>.
>
> * PEMEncoder and PEMDecoder may have an additional constructor taking
> an OutputStream/Writer or InputStream/Reader respectively, and an
> encode(SecurityObject) and decode() method as complement. Separate
> PEMStreamEncoder and PEMStreamDecoder classes may be even cleaner.
> Because in this case, these classes hold the Reader/Writer/
> InputStream/OutputStream which is some kind of state.
I had looked at Stream, but hadn't found a strong situation to use it.
It sounds interesting in practice, and I can see it being useful for a
Keystore. But I'm not sure much beyond that. I'd have to think about
it and if it belonged in the same class or a new one.
>
> * A decoding operation reading from Reader/InputStream should read
> any explanatory text lines and provide means for an application to
> get them. Metadata in these lines may be necessary. The PEM keystore
> implementation (KarlScheibelhofer/java-crypto-tools: PEM KeyStore
> implementation with a JCA Provider. (github.com)
> <https://github.com/KarlScheibelhofer/java-crypto-tools>) uses such
> explanatory text lines to hold the alias name of the keystore
> entries. There is no other place in PEM to store this information.
> Transporting this metadata, one option could be separate methods,
> e.g. "Stream<String>/List<String>
> readPrefix(Reader/String/InputStream)" providing these lines in
> addition or empty, the decoder stopping at the "-----BEGIN" line.
> Applications can call readPrefix() and decode() alternately, or just
> decode() subsequently, if the lines are of no relevance. If the
> PEMDecoder does not provide means for transporting explanatory text
> lines, the application may need to extract them itself between
> "-----END" and "-----BEGIN" lines. To enable this, the decoder must
> not read a single character outside the "----" boundary lines.
This topic has come up a few times. I do not feel it is the
responsibility of the PEM API to read and write other data. I have
tried to keep the API simple and focused on only encoding and decoding
PEM data.
I view the metadata as the responsibility of the application, for it
knows more about the non-PEM data than the PEM API would. Making the
PEM API responsible for returning the non-PEM data leaves too many
unknowns. Does the API return data line-by-line, per byte, all in one
byte[], or does it return it as a List? In the end, the application
still must parse this metadata, as you know with your KeyStore.
All that being said, I have no problem with a future PEMReader or
PEMWriter class that combines the PEM API with a Reader/Writer and/or
Input/OutputStream. As part of the contribution of the PEM Keystore, we
could work on an IO class that could eventually be part of the public
API. It's better to have a use case to better understand the problems.
As far as viewing to the API code. It's mostly ready, but not ready for
public release just yet. Maybe after the first of the year.
Thanks
Tony
>
> best regards, Karl
>
> Am Mi., 29. Nov. 2023 um 16:23 Uhr schrieb Karl Scheibelhofer
> <karl.scheibelhofer.75 at gmail.com
> <mailto:karl.scheibelhofer.75 at gmail.com>>:
>
> Hi Tony!
>
> I read the darft https://openjdk.org/jeps/8300911
> <https://openjdk.org/jeps/8300911>. It looks quite good to me.
>
> To get a beffer feeling, I would like to use it. Is there a draft
> implememtation of this API?
>
> I could give it a go with the draft version of PEM Keystore I did in
> KarlScheibelhofer/jdk at pem-keystore (github.com)
> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/jdk/tree/pem-keystore__;!!ACWV5N9M2RV99hQ!Lkz2jbw4X6YTEtXCzagCWKxJedAUd_P4geW8efbYnAfIW-B1R5hR72y4bjzFjFGjzFYbZhGENfsanHqsQ9AFHs-bzDjEC8s$>
>
> regards, Karl
> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/jdk/tree/pem-keystore__;!!ACWV5N9M2RV99hQ!Lkz2jbw4X6YTEtXCzagCWKxJedAUd_P4geW8efbYnAfIW-B1R5hR72y4bjzFjFGjzFYbZhGENfsanHqsQ9AFHs-bzDjEC8s$>
>
> GitHub - KarlScheibelhofer/jdk at pem-keystore
> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/jdk/tree/pem-keystore__;!!ACWV5N9M2RV99hQ!Lkz2jbw4X6YTEtXCzagCWKxJedAUd_P4geW8efbYnAfIW-B1R5hR72y4bjzFjFGjzFYbZhGENfsanHqsQ9AFHs-bzDjEC8s$>
>
>
JDK main-line development https://openjdk.org/projects/jdk
> <https://openjdk.org/projects/jdk> - GitHub - KarlScheibelhofer/jdk
> at pem-keystore github.com
> <https://urldefense.com/v3/__http://github.com__;!!ACWV5N9M2RV99hQ!Lkz2jbw4X6YTEtXCzagCWKxJedAUd_P4geW8efbYnAfIW-B1R5hR72y4bjzFjFGjzFYbZhGENfsanHqsQ9AFHs-b-_76hPA$>
>
>
>
> ------------------------------------------------------------------------
>
>
*Von:* Anthony Scarpino <anthony.scarpino at oracle.com
> <mailto:anthony.scarpino at oracle.com>> *Gesendet:* Freitag, 17.
> November 2023 20:52 *An:* Karl Scheibelhofer
> <karl.scheibelhofer at gmx.net <mailto:karl.scheibelhofer at gmx.net>>
> *Cc:* security-dev at openjdk.org <mailto:security-dev at openjdk.org>
> <security-dev at openjdk.org <mailto:security-dev at openjdk.org>>
> *Betreff:* Re: [External] : Re: PEM KeyStore Implementation As you
> may have seen, the PEM API draft is out:
> https://openjdk.org/jeps/8300911 <https://openjdk.org/jeps/8300911>
>
> Tony
>
>
>
> On 10/18/23 3:00 AM, Karl Scheibelhofer wrote:
>> Hi Sean,
>>
>> Yes, I can help with this new PEM API.
>>
>> Let me know, when there is something to review.
>>
>> Best regards,
>>
>> Karl
>>
>>
>> On Tue, Oct 17, 2023, 19:12 Sean Mullan <sean.mullan at oracle.com
>> <mailto:sean.mullan at oracle.com> <mailto:sean.mullan at oracle.com
>> <mailto:sean.mullan at oracle.com>>> wrote:
>>
>> Hi Karl,
>>
>> I discussed your proposal with some other colleagues.
>>
>> We generally feel a PEM KeyStore would be a useful addition to the
>> JDK. This would alleviate usability issues that many users
>> encounter when configuring and deploying applications that store
>> keys or certificates in PEM files.
>>
>> However, we would like to first make sure that your PEM KeyStore
>> implementation will work well with the PEM API that we will be
>> proposing soon. We think this is a perfect opportunity to ensure
>> they work well together and would appreciate your help in
>> reviewing and validating the API - would you be interested in
>> helping out?
>>
>> Once that is done, we can discuss next steps.
>>
>> Thanks, Sean
>>
>>
>>> On Oct 5, 2023, at 9:41 AM, Sean Mullan <sean.mullan at oracle.com
>>> <mailto:sean.mullan at oracle.com> <mailto:sean.mullan at oracle.com
>>> <mailto:sean.mullan at oracle.com>>> wrote:
>>>
>>>
>>>
>>>> On Oct 5, 2023, at 2:48 AM, Karl Scheibelhofer
>>>> <karl.scheibelhofer at gmx.net
>>>> <mailto:karl.scheibelhofer at gmx.net>
> <mailto:karl.scheibelhofer at gmx.net
> <mailto:karl.scheibelhofer at gmx.net>>>
>>>> wrote:
>>>>
>>>> Hi Sean,
>>>>
>>>> Yes, I had a look at the Contributing docs at the OpenJDK site
>>>> before. I also signed the OCA.
>>>
>>> Great, thanks.
>>>
>>>>
>>>> Honestly, I thought there would be some more reaction on the
>>>> suggested PEM KeyStore. It would really be good to discuss the
>>>> topic with others. Is there anything we can do to get others
>>>> in sharing their thoughts on this?
>>>
>>> I think there is a fair amount of interest in it, but reviewing
>>> something significant like this takes a bit of time, as I
>>> mentioned in my prior email. Also, if we do decide to accept the
>>> contribution, we want to make sure it works well with the PEM
>>> API that we are working on - we hope to have a draft of a JEP for
>>> that out in the next few weeks. So I think we probably need a few
>>> weeks to review your contribution.
>>>
>>>>
>>>> There is already a fair amount of documentation und unit
>>>> tests. See
>>>> https://github.com/KarlScheibelhofer/java-crypto-tools/
> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/java-crypto-tools/__;!!ACWV5N9M2RV99hQ!Lkz2jbw4X6YTEtXCzagCWKxJedAUd_P4geW8efbYnAfIW-B1R5hR72y4bjzFjFGjzFYbZhGENfsanHqsQ9AFHs-bsZ3p0RI$>
>
>
>>> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/java-crypto-tools/__;!!ACWV5N9M2RV99hQ!L4Oy66pyQcMu7F5jKCD98FvyWZGBlrWmmpnxhOkj2bAffn_KyL69pJh6Y36l9xIk-U4itzwJyPjubZxjk-SP2qN2C39SWwY$
>>> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/java-crypto-tools/__;!!ACWV5N9M2RV99hQ!L4Oy66pyQcMu7F5jKCD98FvyWZGBlrWmmpnxhOkj2bAffn_KyL69pJh6Y36l9xIk-U4itzwJyPjubZxjk-SP2qN2C39SWwY$>>
>>> .
>>>
>>> Ok.
>>>
>>> —Sean
>>>
>>>>
>>>> Best regards, Karl
>>>>
>>>> On Wed, Oct 4, 2023, 13:58 Sean Mullan <sean.mullan at oracle.com
>>>> <mailto:sean.mullan at oracle.com> <mailto:sean.mullan at oracle.com
>>>> <mailto:sean.mullan at oracle.com>>> wrote:
>>>>
>>>> Hi Karl,
>>>>
>>>> The OpenJDK Developer’s Guide includes a helpful section on
>>>> Contributing to an OpenJDK Project [1]. I suggest you read
>>>> through that if you have not already. In particular, have you
>>>> signed the OCA? I don’t want to review your code/contribution
>>>> until that is done.
>>>>
>>>> For this particular contribution, I don’t think there has been
>>>> enough discussion and evaluation from members of the Security
>>>> project. This would be a fairly major contribution. Keep in
>>>> mind that a contribution doesn’t mean the work ends there.
>>>> There would need to be documentation, tests, and ongoing
>>>> support for the foreseeable future. We need to think about
>>>> these aspects every time we add a new feature, so there needs
>>>> to be a strong motivation for doing it.
>>>>
>>>> Thanks, Sean
>>>>
>>>> [1]
>>>> https://openjdk.org/guide/#contributing-to-an-openjdk-project
> <https://openjdk.org/guide/#contributing-to-an-openjdk-project>
>>>> <https://openjdk.org/guide/#contributing-to-an-openjdk-project
> <https://openjdk.org/guide/#contributing-to-an-openjdk-project>>
>>>>
>>>>> On Oct 4, 2023, at 4:21 AM, Karl Scheibelhofer
>>>> <karl.scheibelhofer at gmx.net
>>>> <mailto:karl.scheibelhofer at gmx.net>
>>>> <mailto:karl.scheibelhofer at gmx.net
> <mailto:karl.scheibelhofer at gmx.net>>> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I would like to contribute my PEM KeyStore implementation
>>>> to the
>>>>> OpenJDK, including integration in the OpenJDK source and
>>>> creating a
>>>>> pull request. What is the recommended way to do this? Who can
>>>>> create a suitable ticket in OpenJDK to document the
>>>>> enhancement and to track the progress?
>>>>>
>>>>> What are the requirements for a pull request to get merged?
>>>>>
>>>>> Best regards
>>>>>
>>>>> Karl
>>>>>
>>>>> Am Mi., 20. Sept. 2023 um 11:26 Uhr schrieb Karl
>>>>> Scheibelhofer <karl.scheibelhofer at gmx.net
>>>>> <mailto:karl.scheibelhofer at gmx.net>
>>>> <mailto:karl.scheibelhofer at gmx.net
>>>> <mailto:karl.scheibelhofer at gmx.net>>>:
>>>>>>
>>>>>> Hi Tony!
>>>>>>
>>>>>> When the PEM API implementation becomes available it would
>>>> make sense
>>>>>> to use it inside the PEM Keystore implementation. It will
>>>> reduce the
>>>>>> code (the internal classes PemReader und PemWriter may
>>>>>> become obsolete), but it does not affect the functionality
>>>>>> of the PEM keystore. Users of the PEM Keystore won't
>>>>>> experience a
>>>> difference.
>>>>>>
>>>>>> Let me know when there is something for the PEM API and I
>>>> will see if
>>>>>> I can assist.
>>>>>>
>>>>>> I would suggest starting with PEM Keystore now and not
>>>> wait for the
>>>>>> PEM API, because the time schedule for it seems vague. I
>>>> would try to
>>>>>> refactor my current PEM Keystore implementation to
>>>> integrate in the
>>>>>> OpenJDK sun.security.provider package. I do not expect any
>>>> API changes
>>>>>> or other compatibility issues with existing code. Then
>>>> consult this
>>>>>> group for feedback before creating a pull request.
>>>>>>
>>>>>> When the PEM API becomes available, rework the PEM
>>>>>> Keystore implementation to use it internally.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Best regards
>>>>>>
>>>>>> Karl Scheibelhofer
>>>>>>
>>>>>> Am Di., 19. Sept. 2023 um 22:31 Uhr schrieb Anthony
>>>>>> Scarpino <anthony.scarpino at oracle.com
>>>>>> <mailto:anthony.scarpino at oracle.com>
>>>> <mailto:anthony.scarpino at oracle.com
> <mailto:anthony.scarpino at oracle.com>>>:
>>>>>>>
>>>>>>> There are no doc links yet.
>>>>>>>
>>>>>>> Tony
>>>>>>>
>>>>>>> On 9/10/23 1:04 AM, Karl Scheibelhofer wrote:
>>>>>>>> Hi Tony,
>>>>>>>>
>>>>>>>> The motivation was mostly about reading PEM keys and
>>>> certificates
>>>>>>>> generated somewhere else. This is common practice in
>>>> enterprise
>>>>>>>> environments I work in. Because corporate key material
>>>> is subject to
>>>>>>>> centralized key management, including generation,
>>>>>>>> backup
>>>> and rollover.
>>>>>>>> PEM is the format most software products can handle.
>>>>>>>> For
>>>> Java
>>>>>>>> applications, having a PEM KeyStore would reduce the
>>>> often required
>>>>>>>> additional step of converting PEM key and certificate
>>>>>>>> in
>>>> a Java
>>>>>>>> Keystore/PKCS#12. Even truststores handling is easier
>>>>>>>> with individual PEM
>>>> certificates
>>>>>>>> instead of a single PKCS#12 Truststore. Adding or
>>>> deleting a single
>>>>>>>> file instead of replacing the complete PKCS#12 store
>>>>>>>> is
>>>> less error
>>>>>>>> prone and cleaner to track in version control. The
>>>> additional benefit
>>>>>>>> of a MAC in PKCS#12 adds little to no security in most
>>>> cases.
>>>>>>>> And being text based, PEM is more version control
>>>> friendly than binary PKCS#12.
>>>>>>>>
>>>>>>>> But to enable sound support of PEM, I also implemented
>>>> writing PEM
>>>>>>>> keys and certificates. This way, one can use the JDK
>>>> keytool to
>>>>>>>> generate key and certificate signing requests in PEM
>>>> format. Getting
>>>>>>>> the certificate from the CA in PEM, one can use PEM
>>>> throughout the
>>>>>>>> process.
>>>>>>>>
>>>>>>>> Do you have any links or documentation on the PEM API
>>>> JEP that you mentioned?
>>>>>>>>
>>>>>>>> Thank you for your feedback and best regards
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>> Am Fr., 8. Sept. 2023 um 21:17 Uhr schrieb Anthony
>>>>>>>> Scarpino <anthony.scarpino at oracle.com
>>>>>>>> <mailto:anthony.scarpino at oracle.com>
>>>> <mailto:anthony.scarpino at oracle.com
> <mailto:anthony.scarpino at oracle.com>>>:
>>>>>>>>>
>>>>>>>>> Hi Karl
>>>>>>>>>
>>>>>>>>> The keystore is interesting and may have some value.
>>>>>>>>>
>>>> Was your use case
>>>>>>>>> mostly reading PEM keys and certificates generated
>>>> elsewhere for use
>>>>>>>>> with a particular application, maybe webservers?
>>>>>>>>> Did
>>>> you see value in
>>>>>>>>> writing to this keystore from Java?
>>>>>>>>>
>>>>>>>>> On the topic of PEM, I hope before the end of the
>>>>>>>>> year
>>>> to have a PEM API
>>>>>>>>> JEP. I would be interested in your API feedback
>>>>>>>>> from
>>>> your keystore
>>>>>>>>> experiences. I think if this keystore contribution
>>>>>>>>> was
>>>> accepted, it
>>>>>>>>> should wait so it can use that API.
>>>>>>>>>
>>>>>>>>> thanks
>>>>>>>>>
>>>>>>>>> Tony
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 9/1/23 12:15 PM, Karl Scheibelhofer wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Working with Java and the JCA KeyStore for decades,
>>>>>>>>>> I
>>>> came across
>>>>>>>>>> many situations where I thought it would be
>>>>>>>>>> convenient
>>>> to be
>>>>>>>>>> able to load private keys and certificates in PEM
>>>> format directly
>>>>>>>>>> using the KeyStore API. Without the need to
>>>>>>>>>> convert
>>>> them to PKCS#12/JKS.
>>>>>>>>>>
>>>>>>>>>> You can find my implementation of a PEM KeyStore
>>>>>>>>>> in
>>>>>>>>>>
>>>> https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/java-crypto-tools__;!!ACWV5N9M2RV99hQ!Oty2x6ce8fseqwbwEZ1eFN9xJCtVxU8aUXn1GXt81SA1JkTeB9GSykdwShzJKOFYUAA1oUtLGaX1kmZV984WRsO-8KQq5dw$
>>>> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/java-crypto-tools__;!!ACWV5N9M2RV99hQ!Oty2x6ce8fseqwbwEZ1eFN9xJCtVxU8aUXn1GXt81SA1JkTeB9GSykdwShzJKOFYUAA1oUtLGaX1kmZV984WRsO-8KQq5dw$>
>>>> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/java-crypto-tools__;!!ACWV5N9M2RV99hQ!Oty2x6ce8fseqwbwEZ1eFN9xJCtVxU8aUXn1GXt81SA1JkTeB9GSykdwShzJKOFYUAA1oUtLGaX1kmZV984WRsO-8KQq5dw$
>>>> <https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/java-crypto-tools__;!!ACWV5N9M2RV99hQ!Oty2x6ce8fseqwbwEZ1eFN9xJCtVxU8aUXn1GXt81SA1JkTeB9GSykdwShzJKOFYUAA1oUtLGaX1kmZV984WRsO-8KQq5dw$>>
>>>> .
>>>>>>>>>>
>>>>>>>>>> I wondered if it would make sense to integrate such
>>>>>>>>>> an
>>>> implementation
>>>>>>>>>> in one of the standard providers of OpenJDK - like
>>>>>>>>>> the
>>>> SUN provider.
>>>>>>>>>> What do you think?
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>>
>>>>>>>>>> Karl
>>>>
>>>
>>
>
More information about the security-dev
mailing list