System.exit(...) without SecurityManager - implications for Apache Ant project

[Jaikiran Pai]

(resending from the correct subscribed email address)

I see that System.setSecurityManager(...) will start throwing exception 
by default starting Java 18[1]. I have been following the JEP 411, 
mostly in context of the Apache Ant project. I think we (Ant) are in a 
OK shape for Java 17 when it comes to the security manager deprecation 
messages. The initial issue related to flooding of logs in a basic Ant 
build has been addressed in recent Java 17 EA releases.

Going forward, clearly, Ant has to get rid of its usage of the 
SecurityManager. From what I have seen in the Ant code, the most 
critical usage of the security manager (even when a user doesn't 
specifically configure it) is around the prevention of JVM exit through 
user code that can call System.exit(...). Ant, by default sets up a 
custom SecurityManager and then uses the SecurityManager's 
"checkExit(...)" to prevent such calls. The JEP 411 does state that as a 
future task, the System.exit() usecase will be addressed. I thought I 
had seen a JBS issue linked that dealt with this, but now that I search 
for it, I can't find it except for this one 
https://bugs.openjdk.java.net/browse/JDK-8199704 which was raised some 
years back. Is that the one where the work related to any new APIs for 
preventing system exit, without the SecurityManager API usage, will be 
addressed?

Either way, I haven't seen any specific discussion of the use cases or 
proposals for new APIs around this. So I would like to explain how Ant 
currently uses the SecurityManager APIs when it comes to System.exit() 
calls. As many of you might already know, the core of Ant revolves 
around "tasks" which essentially are pluggable classes where the actual 
build related work happens. Ant ships with many builtin/core classes 
which perform basic build activities. These tasks are then clubbed into 
build "target(s)" in a build file for users to invoke. For the context 
of this discussion, we will focus on the builtin "java" task (which is 
one of the tasks for which Ant sets up a SecurityManager at runtime by 
default). The "java" task takes a class name and then launches the 
"main" method of that class. Users have the option to configure a fork 
mode so that this launch happens in a separate JVM than the one that the 
Ant build is currently running in. In the case where fork is disabled, 
Ant uses reflection to call the "main" method of that class within the 
current running JVM of the Ant build. Since this user provided class can 
do anything in its code, Ant sets up a custom SecurityManager at runtime 
and in addition to "granting" certain permissions, it also overrides the 
"checkExit()" API of the SecurityManager like here[2] and here[3]. What 
that effectively does is whenever the user code, that is launched using 
the "java" task, invokes `System.exit(...)` or 
`Runtime.getRuntime().exit(...)` it lets the SecurityManager API 
infrastructure do the "checkExit" checks and if not allowed then throw 
the SecurityException and prevent the JVM from exiting. Additionally, 
Ant's custom SecurityManager implementation wraps any such 
`SecurityException` from "checkExit(...)" into a internal Exception type 
which holds on to the exit code that was used to call the 
`System.exit(...)` or `Runtime.getRuntime().exit(...)`. This custom 
wrapped exception is then caught in Ant and the captured exit code is 
then logged or used to do additional decisions like whether the build 
should be stopped i.e. should further tasks/targets should be invoked or 
not. Of course there are other implications of continuing with the rest 
of the tasks/build when such a SecurityException is thrown for exit(...) 
calls (for example, if any shutdown hooks were registered by the user 
code those won't be run immediately but will get run after the Ant JVM 
exits), but those issues aren't in the scope of what we are discussing here.

So if any of the upcoming Java 18 EA builds introduce the "disallow" by 
default, then if Ant has to test against those EA builds (not just for 
SecurityManager but any other changes), then Ant project will have to 
start setting the "java.security.manager" system property to "allow". 
That's not going to be straightforward, since it would be dealing with 
this in potentially multiple launch scripts or maybe even dynamic launch 
commands issued through the code (I haven't checked where/how many). I 
don't yet know if this property value can be changed at 
runtime/dynamically. If that is allowed, it may make things a bit easier 
since that would allow us to set that property value at the same place 
where we set the custom security manager in the Ant code, but I would 
still like to avoid it if possible.

So given all this, can the new API(s) design/implementation for 
preventing System.exit(...) calls without a SecurityManager be 
prioritized and perhaps be scheduled for release in Java 18 itself, 
which is where the "disallow" will come in as default?

To summarize - Ant would need a way to intercept/prevent JVM exits via 
System.exit(...) or Runtime.getRuntime().exit(...) calls *and* a way to 
get hold of the exit code that was used to trigger such calls (Ant had 
to wrap the SecurityException into a custom exception type just to keep 
track of that exit code value).


[1] https://github.com/openjdk/jdk/pull/5204
[2] 
https://github.com/apache/ant/blob/master/src/main/org/apache/tools/ant/types/Permissions.java#L182
[3] 
https://github.com/apache/ant/blob/master/src/main/org/apache/tools/ant/util/optional/NoExitSecurityManager.java#L34

-Jaikiran