RFR: JDK-8320005 : Allow loading of shared objects with .a extension on AIX [v7]

[Suchismith Roy]

On Thu, 21 Dec 2023 10:01:04 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>>> > > What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen?
>>> 
>>> > I don't think the problem is with *.a . They would load as the default behaviour of the dlopen. It is only when the dlopen fails for *.so , we give another chance to check for .a file with the same name.
>>> 
>>> No, what I meant, and what must be clarified before going forward with this solution, is the following:
>>> 
>>> * is _every_ `*.a` object on AIX loadable with `dlopen`, and will the result be the same as when loading a `*.so` object
>>> * or, if we present arbitrary `*.a` files to dlopen, is there a chance for dlopen to crash or misbehave.
>>> 
>>> Reason is that I was under the impression that *.a libraries are static libraries and cannot be loaded dynamically. This is what you now try to do.
>>> If we cannot safely answer this question, I would opt for a more narrow solution by hard-wiring known alternative names. So, do the second *.a attempt only for your `ibm_16_am.a` which you know works. That could also be done in a reasonably maintainable manner.
>>> 
>> In AIX, both static and dynamic libraries have *.a extension. And AIX also supports *.so files.Bascially shared objects in AIX have both *.a and *.so extension.  Hence we need to implement this logic.
>> If we try loading a static archive specifically ,how the dlopen would behave , that is something probably @JoKern65  can answer ? 
>> 
>> 
>>> > > Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from?
>>> 
>>> > I am not sure how J9 handles this. I would have to consult .
>>> 
>>> J9 is Open Source, can't you just look? :)
>> 
>> I did try comparing the file structures, and i do not see a similar file structure over there. 
>> I am unable to find the jvmTiAgent code and also os_aix file. So i am not sure which functions over there are doing the same functionality. You have any suggestion on how i can check and correlate ?  
>>> 
>>> > However as per current observation, this issue does not show up on Semuru. This issue is only happening on Adoptium. The team that release these file has always released *.a files which work fine for Semuru.
>>> 
>>> I don't know what Semuru is. What is the context, is that a different VM? Also OpenJDK? J9 derived? 
>> 
>> 
>> Semuru is J9 derived.
>
>> > > > What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen?
>> > 
>> > 
>> > > I don't think the problem is with *.a . They would load as the default behaviour of the dlopen. It is only when the dlopen fails for *.so , we give another chance to check for .a file with the same name.
>> > 
>> > 
>> > No, what I meant, and what must be clarified before going forward with this solution, is the following:
>> > 
>> > * is _every_ `*.a` object on AIX loadable with `dlopen`, and will the result be the same as when loading a `*.so` object
>> > * or, if we present arbitrary `*.a` files to dlopen, is there a chance for dlopen to crash or misbehave.
>> > 
>> > Reason is that I was under the impression that *.a libraries are static libraries and cannot be loaded dynamically. This is what you now try to do.
>> > If we cannot safely answer this question, I would opt for a more narrow solution by hard-wiring known alternative names. So, do the second *.a attempt only for your `ibm_16_am.a` which you know works. That could also be done in a reasonably maintainable manner.
>> 
>> In AIX, both static and dynamic libraries have *.a extension. And AIX also supports *.so files.Bascially shared objects in AIX have both *.a and *.so extension. Hence we need to implement this logic. If we try loading a static archive specifically ,how the dlopen would behave , that is something probably @JoKern65 can answer ?
> 
> Rather, this is a question you have to ask your collegues at IBM that develop the AIX libc.
> 
> Since AIX libc is not open source, we cannot look for ourselves, nor can Joachim (her works at SAP).
> 
>> 
>> > > > Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from?
>> > 
>> > 
>> > > I am not sure how J9 handles this. I would have to consult .
>> > 
>> > 
>> > J9 is Open Source, can't you just look? :)
>> 
>> I did try comparing the file structures, and i do not see a similar file structure over there. I am unable to find the jvmTiAgent code and also os_aix file. So i am not sure which functions over there are doing the same functionality. You have any suggestion on how i can check and correlate ?
> 
> Someone must implement LoadLibrary. Try looking for places where dlopen() is called.
> 
>> 
>> > > However as per current observation, this issue does ...

Hi @tstuefe Clarifications on your questions.



> Hi,
> 
> some requests and questions:
> 

> * Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from?

In J9, there was syshelp native file that did the appending of .a to the filename till java 14. Post that the appending is handled in the JavaClassLoader . However, when I tried to do the same on OpenJDK, it didn't work. So I had to trace into the hotspot code. 
This change will affect some Panama changes too. I think @TheRealMDoerr  has faced issues with respect to another library as well.  Long term, providing a .so variant may not be feasible as it will be continuous process imo.
 



> * What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen?

I enquired on this. The dlopen exits with an error .The program does not crash when trying to load a static library. 

> * What happens if the original path handed to os::dll_load is already a *.a file? Should the logic then be reversed?

Already explained above.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1878327576