RFR: JDK-8296995: ostream should handle snprintf(3) errors in release builds [v2]

[Matthias Baesken]

On Mon, 5 Dec 2022 08:07:25 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Small fix for a very unlikely problem. 
>> 
>> All streams in ostream.hpp end up using `os::snprintf()`, which uses `::vsnprintf()`. `vsnprintf(3)`can fail and return -1. 
>> 
>> The chance for this to happen is small. snprintf errors are usually encoding errors though not always (see third example at https://stackoverflow.com/questions/65334245/what-is-an-encoding-error-for-sprintf-that-should-return-1). I found "%ls" in one place in windows coding, so I am not sure we can always exclude the possibility of wide strings being used in our code base, or that of printing with outside-provided format strings.
>> 
>> In case of an error, we assert in debug builds but don't handle it in release. There, this situation gets misdiagnosed later as a buffer overflow because we cast the signedness of the result away (see `outputStream::do_vsnprintf()`). 
>> 
>> ---
>> 
>> The patch is trivial. The most exciting thing is the gtest, I guess.
>> 
>> In release builds, we now treat this condition as an empty string write. I considered printing a clear marker into the stream instead, e.g. "ENCODING ERROR", but ultimately did not do it.
>
> Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains one commit:
> 
>   JDK-8296995-ostream-handle-sprintf-errors

Looks good to me.
I thought about adding some (potentially optional) log or trace in case required_len < 0 but probably it is good as it is.
Please adjust  copyright year in [test_ostream.cpp](https://github.com/openjdk/jdk/pull/11160/files#diff-c0d2076833ffa15ad831383a8ddc594763aef71e53276afeddc8bdaf1b4e887e) .

-------------

Marked as reviewed by mbaesken (Reviewer).

PR: https://git.openjdk.org/jdk/pull/11160