Will we need to use the --enable-native-access option to enable JNI in the future?

[Glavo]

Because my English is very poor, I may not know enough about the
exact meaning of words. If my words are too offensive, I apologize.

What I'm more intersted in is - what make you think that adding extra
> command line flags would be perceived as such a blocker? Do you have
> examples in mind where this might turn out to be problematic in practice?


I think there are a lot of possible blockers:

  * Most obviously, we have lost the ability to simply execute with java
-m.
    Now we can only go back to the classpath and describe it in MANIFEST.MF
(if supported in the future).
    If it is not supported, we even lose the ability to double-click to
open the jar on windows.
    We can only consider wrapping the jar with a third-party solution.
  * Maybe I missed something. I remember there were relevant discussions
before,
    but there still seems to be no way to enable native access for modules
at runtime in Java 17.
    If so, how to enable a module defined at run time using custom
ModuleLayer to use Panama?
    If not, it will be a devastating blow to any dynamic plugin mechanism,
e.g. Minecraft mods.
  * This assumes that I always know all the modules when I start the
program,
    but this is not necessarily the case. For example, the most typical is
the ServiceLoader.
    Some components expect to work only by putting them on the module path,
    such as a implementation of FileSystem or a ScriptEngine.
  * Windows cmd limits the maximum length of a single command to 8192.
    We have some use cases to export a single bat script file that can be
started
    by double clicking for novice users. We are often close to this
boundary, and continuing
    to increase the length of the command line will exacerbate this problem.
  * and more...

Saying that, since JNI is unsafe and enabled-by-default, then Panama should
> just follow suit, seems like an ill-conceived argument. Because something
> else is not great (for historical reasons - modules weren't there when JNI
> was born), doesn't, in my opinion, justify making everything else bad too,
> just because of that precedent.
>

In fact, I am not entirely opposed to restricting native access.
However, the premise is that this option, like its name, really needs
to enable native access. However, at present, even without it,
it does not mean that the module cannot to native access.
This naming is extremely misleading. It can't bring real security to users.

Note also that, in JNI, most of the unsafety comes from within JNI C glue
> code (e.g. the so called JNI functions). This glue code is no longer there
> in Panama - since it can be simply expressed as Java code. So, the Panama
> API faces a risk which was much less real with JNI: having seemingly
> innocent, pure Java code crashing the JVM. All this reasoning is explained,
> in greated details, in JEP 412.
>

I understand this, which is the only reason I can partially agree.
If it just block me at compile time, need me to explicitly enable it
with annotations or compiler parameters, and I'd like to see that.
But why do my end users need to confirm my choice?
Does TA need to be clearly aware that my library, as a third-party library,
is using java code instead of native code to implement binding?

I think adding better support for this in manifest is probably a good thing
> to explore, yes. At the same time your premise (many complains, no one
> likes it), seems at fault; the rationale behind guarding some unsafe
> corners of the Panama API behind a flag were discussed, and considered
> better than the alternatives (e.g. at some point we experimented with an
> unsafe module, which users had to enable manually - which was worse, both
> for usability and API discoverability). Of course in a mailing list such as
> this one (but this is true of many internet media), you will be more likely
> to find negative feedback on any given topic than positive feedback - but
> one shouldn't assume too much from that. To be honest, since we moved to
> the new command line option, this issue has not even been brought up again,
> which makes me think that, in reality, most developers understand where the
> option is coming from.

The premise of my sentence is among the people around me,
among those who have daily close communication with me.
Among these people, game developers, game engine developers
and other types of native binding library authors pay attention to Panama.
They are concerned about the development of JDK, but prefer to
communicate in private groups rather than mailing lists.
At least among these people, I have never received any positive
feedback for these restrictions, not at all. The best attitude is just
indifference,
and most of them complain about it.

Now, I think both arguments can be turned on their heads; let's not think
> about existing code that wants to migrate from JNI to Panama - let's focus
> on _new_ code instead. If you write bindings for a C library post Java 17,
> it is much more likely that you will find that JEP 412 (+ jextract) gives
> you all you want. Will you be put off by extra command line flags? I don't
> think so - if your application requires native libraries, you will likely
> need flags (e.g. java.library.path) anyway. Will you be put off by the fact
> that JNI doesn't share the same restriction? Well, no - since you opted
> into Panama from the beginning, so JNI is most likely in your rearview
> mirror.
>

 I won't talk about how realistic it is for JNI to migrate to Panama.
How to define the _new_? Do you think this is the era of Java 17?
No, this is the era of Java 8, 11 and 17. Before 2031, in these nine years,
there is no reason to support Java 8, but only reason to give up.
In these nine years, any blocker will have a significant impact on
the promotion of an alternative without introducing new capabilities.

Let's also think of users of the Panama API who couldn't care less about
> native interop (there are some in this mailing list) - e.g. developers who
> only use the foreign API to allocate, access and free memory off heap. Very
> likely, these developer would like to take advantage of the extra
> protection provided by the --enable-native-access flag, to make sure they
> don't accidentally depend on unsafe functionalities they do not require.
>

 The same as above, why can't you restrict it at compile time and put it at
run time?

Maurizio Cimadamore <maurizio.cimadamore at oracle.com> 于2021年9月20日周一 下午9:45写道：

>
> On 20/09/2021 14:07, Glavo wrote:
>
> Honestly, as the author of the library, I am extremely disappointed
> with this answer. I will choose to give up my expectation of Panama
> until 2031 (Java 8 and 17 are out of support).
>
> I think that under the current situation, it is extremely stupid to
> restrict Panama by default. Its only significance at present is to
> recommend users to use JNI/JNA/JNR instead of Panama.
>
> I note that the word "extremely stupid" might have come from the automatic
> Google translation - it is a bit on the offsensive, side, but I'll ignore,
> given the circumstances.
>
> What I'm more intersted in is - what make you think that adding extra
> command line flags would be perceived as such a blocker? Do you have
> examples in mind where this might turn out to be problematic in practice?
>
>
> At present, --enable-native-access option has nothing to do with
> safety at all: Because of the existence of JNI, a module can do
> anything without it. Since it can't provide any safety guarantee
> at all, and at the same time it brings some additional usability
> issues, it will only limit the promotion of Panama.
>
> Saying that, since JNI is unsafe and enabled-by-default, then Panama
> should just follow suit, seems like an ill-conceived argument. Because
> something else is not great (for historical reasons - modules weren't there
> when JNI was born), doesn't, in my opinion, justify making everything else
> bad too, just because of that precedent.
>
> It is true that JNI is a permanent safety hole in the JDK, but it would be
> naive to assume we can turn it off overnight; at the same time, since
> Panama is a new API, I think we have an opportunity to make at least the
> new usages of native code safer.
>
> Note also that, in JNI, most of the unsafety comes from within JNI C glue
> code (e.g. the so called JNI functions). This glue code is no longer there
> in Panama - since it can be simply expressed as Java code. So, the Panama
> API faces a risk which was much less real with JNI: having seemingly
> innocent, pure Java code crashing the JVM. All this reasoning is explained,
> in greated details, in JEP 412.
>
>
> The usability issues brought about by this option have been complained
> by other people a long time ago, and I have not seen anyone around
> me who are satisfied with this option. I don't want to repeat all these
> questions here, because I believe you have heard them many times.
> And I think some of these issues  can be alleviated by improving other
> features of the OpenJDK, e.g. adding an alternative to MANIFEST.MF
> for modular programs, allow options such as --add-opens and
> --enable-native-access to be written into it.
>
> I think adding better support for this in manifest is probably a good
> thing to explore, yes. At the same time your premise (many complains, no
> one likes it), seems at fault; the rationale behind guarding some unsafe
> corners of the Panama API behind a flag were discussed, and considered
> better than the alternatives (e.g. at some point we experimented with an
> unsafe module, which users had to enable manually - which was worse, both
> for usability and API discoverability). Of course in a mailing list such as
> this one (but this is true of many internet media), you will be more likely
> to find negative feedback on any given topic than positive feedback - but
> one shouldn't assume too much from that. To be honest, since we moved to
> the new command line option, this issue has not even been brought up again,
> which makes me think that, in reality, most developers understand where the
> option is coming from.
>
>
> Now what I want to emphasize here is that as a more restricted alternative
> to existing features, there is a lot of resistance to migrating to Panama.
> Before restricting JNI, the so-called safety is just a fig leaf,
> it’s just self-deception. This meaningless constraint will not restrict
> the user's behavior at all, but will only hinder the user from choosing
> this road - because there is an unobstructed road nearby.
>
> Again, your argument seems to be "since w/o restricting JNI we can't make
> the platform fully safe, why even try" - but the same argument would apply
> to a lot of other stuff too. We have to start somewhere - note that even if
> JNI would be restricted we'd still have Unsafe around.
> `--enable-native-access` is an attempt to draw a path, where we'd like to
> move the platform in the next N years. Of course I understand that, if you
> pick this particular point in time (e.g. 2021), the picture is not perfect,
> but I think that we have a path to get there. If Panama was just
> unrestricted from the start, it would be much harder to restrict it later
> on (which is why we have to be careful with JNI).
>
>
> I would like to give a proposal here, which I think will minimize the
> obstacles to the promotion of Panama:
>
>   While providing the --enable-native-access option, it also provides
>   a --illegal-native-access=(*silent*|*warn*|*deny*) option.
>
>   At first, the default value is *silent*, and there are no restrictions
> on
>   the use of Panama.
>
>   When you are ready to limit JNI, apply this option to JNI at the same
> time,
>   and adjust the default value to *warn *to warn the use of JNI and
> Panama,
>   and use --enable-native-access to eliminate the warning for a module.
>
>   After several versions, adjust it to *deny *by default to prevent
> unauthorized
>   native operations, and completely remove this option in a later version.
>
> This plan is good on paper, except that if the default is silent for both
> Panama and JNI there is no incentive for people using Panama (which is a
> new API) to get the act together.
>
> Overall, the arguments in this email are a mixture of:
>
> * since JNI is unsafe anyway, what's the point of restricting Panama
> * restricting Panama will hinder adoption in some way
>
> Now, I think both arguments can be turned on their heads; let's not think
> about existing code that wants to migrate from JNI to Panama - let's focus
> on _new_ code instead. If you write bindings for a C library post Java 17,
> it is much more likely that you will find that JEP 412 (+ jextract) gives
> you all you want. Will you be put off by extra command line flags? I don't
> think so - if your application requires native libraries, you will likely
> need flags (e.g. java.library.path) anyway. Will you be put off by the fact
> that JNI doesn't share the same restriction? Well, no - since you opted
> into Panama from the beginning, so JNI is most likely in your rearview
> mirror.
>
> Let's also think of users of the Panama API who couldn't care less about
> native interop (there are some in this mailing list) - e.g. developers who
> only use the foreign API to allocate, access and free memory off heap. Very
> likely, these developer would like to take advantage of the extra
> protection provided by the --enable-native-access flag, to make sure they
> don't accidentally depend on unsafe functionalities they do not require.
>
>
>
> I like the work of Panama very much, but I am very disappointed by
> the current restrictions. For the last time, I want to emphasize that
> restricting Panama alone will only refuse users to use it.
> If you insist on this choice, I will tell you:
>
> Again, "users" is very vague here. There are "users" who have existing
> code bases, who made lots of investments in JNI. I believe for these users
> the decision whether or not to use the new API has to consider a lot more
> factors than just "how many flags will I need with the new API". Then there
> are "users" writing new applications, or frameworks from scratch. These
> users will likely use Panam instead, not because it's newer, or shinier,
> but simply because it reduces the number of pieces they have to maintain
> going forward (e.g. no more JNI intermediate goop). For these users, the
> presence of the new flag is probably not very disruptive - from their
> perspective that's just how things are in Panama.
>
> Thanks for the comments - I think you raise some valid points regarding
> safety and JNI - but I think that ultimately, picking a worse solution just
> because there are other broken areas which won't be fixed immediately is
> not a great decision, and is probably one which will just regret later.
>
> Maurizio
>
>
>   Thank you for your contributions to the Java 8 ecosystem.
>   And I sincerely thank you, because my library users don't have to
>   ask me why they need to upgrade to Java 21.
>
> (This email is written by Google translation. Please forgive me for the
> syntax error)
>
> Maurizio Cimadamore <maurizio.cimadamore at oracle.com> 于2021年9月20日周一
> 下午6:17写道：
>
>> Hi,
>> While we do not any plan to require so now, many of the reasons as to
>> why the flag is added for the foreign Panama API hold true for JNI code.
>> As such, It would be not unreasonable, some day, to use the same
>> mechanism to restricted access to both (since, after all, there is no
>> way for the static compiler, or the JVM, to prove that a program using
>> JNI is sound). If we go down this path, it is very likely that it won't
>> be a flag-day, where, say, JDK 30 does not require the flag, but JDK 31
>> does. It is more likely that we will go through a number of intermediate
>> releases where the lack of a flag will result in a runtime warning (not
>> unlike to what has been done with illegal reflective access violation
>> for modules).
>>
>> Maurizio
>>
>>
>> On 19/09/2021 12:11, Glavo wrote:
>> > At present, it seems that JNI will not be restricted. Will the state be
>> > modified in the future?
>>
>