RFR: 8306580: Propagate CDS dumping errors instead of directly exiting the VM [v3]

[Ioi Lam]

On Wed, 5 Jun 2024 21:26:01 GMT, Matias Saavedra Silva <matsaave at openjdk.org> wrote:

>> Currently, when CDS dumping run into an unrecoverable error (e.g., file I/O error, out of memory), it calls MetaspaceShared::unrecoverable_writing_error(), which directly exits the VM.  Some of these errors can be propagated to the caller for a normal exit. 
>> 
>> This change introduces `MetaspaceShared::writing_error()` to report errors without exiting the VM. The function `MetaspaceShared::unrecoverable_writing_error()` now should only be used for errors that require the VM to exit. Verified with tier1-5 tests.
>
> Matias Saavedra Silva has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Thomas comments and cleanup
>  - Added test for file errors

Changes requested by iklam (Reviewer).

src/hotspot/share/cds/metaspaceShared.cpp line 520:

> 518:   if (_builder.reserve_buffer() == nullptr) {
> 519:     this->_failed = true;
> 520:     return;

I think this should be removed, since reserve_buffer() will exit the JVM when OOM happens.

I did some experiments with `docket run --memory=1024m ...` and found that, at least on Linux, when a process runs over the memory limit, it just gets killed immediately by the OOM killer. There's no way to detect an OOM condition and handle it gracefully. So I think we can't (and don't need to) handle OOM errors.

src/hotspot/share/cds/metaspaceShared.cpp line 797:

> 795: 
> 796:   if (op.failed() || !write_static_archive(&builder, op.map_info(), op.heap_info())) {
> 797:     THROW_MSG(vmSymbols::java_io_IOException(), "Encountered error while dumping");

`op.failed()` can be removed, as we have no way of detecting OOM failure.

-------------

PR Review: https://git.openjdk.org/jdk/pull/19370#pullrequestreview-2103537579
PR Review Comment: https://git.openjdk.org/jdk/pull/19370#discussion_r1630572087
PR Review Comment: https://git.openjdk.org/jdk/pull/19370#discussion_r1630572563