An idea: Add a port layer

[Kelly O'Hair]

On May 16, 2011, at 8:20 PM, Jing LV wrote:

> Hi Kelly,
> 
>       Thank a lot for the information, really great help to the topic!
>       (Yes it's a bit odd to see linux source under solaris folder, it took me some time to find it out when I download the source for the first time)
> 
>        So Kelly, is npt still in use? I believe we may start from existing implementation if it already works well there.  Never touched hprof so I'd download them and read.

Yes npt is used by the jvmti agents HPROF and JDWP, maybe elsewhere not sure.

It was designed to be dlopen'd or LoadLibrary'd dynamically because of the horrible shared library
support on Windows (No LD_LIBRARY_PATH or RUNPATH and $ORIGIN).

-kto

>        I strongly agree with you that we'd define the interface in detail, describe the exact behavior to cover the differences between the APIs in different systems. In my experience, one difficulty here is that sometime we need to use some new APIs to improve performance, but new APIs may not exist on all platforms (let's consider asynchronized API for NIO2).
> 
>        Clearly it'll be a big amount of work, but it's worth if the layer would help openjdk in the long run in my opinion.
>        NPT document is very interesting and I'll take a look - thank you Kelly!
> 
> 
> 于 2011-5-17 0:42, Kelly O'Hair 写道:
>> 
>> Just for conversation fodder...
>> 
>> Keep in mind, the directory name "solaris" does not mean just "Solaris", but that source is also used
>> with Linux builds. The directory name needs to change someday, but that's a separate and long discussion.
>> 
>> When I worked on the JVM TI agents and demos I had a unique issue of not being able to leverage
>> Java apis because the agents execute sometimes prior to the VM being initialized.
>> So doing character conversions of command lines was quite messy.
>> At the time I created the npt library (npt.dll or libnpt.so), and tried to model it around JNI and the
>> function pointers but I was never really very happy with it.
>> See jdk/make/java/npt, jdk/src/share/npt, jdk/src/solaris/npt, and jdk/src/windows/npt.
>> Anyway, I'll attach the original document on npt just for conversation, but it never really reached
>> a point where it met this manual. So be kind in your criticism. ;^)
>> 
>> 
>> The hprof agent had it's own issues porting and it might be of interest, it used the function prefix
>> "md_" and the shared include file was hprof_md.h.
>> So 3 files: jdk/src/share/demo/jvmti/hprof/hprof_md.h, jdk/src/solaris/demo/jvmti/hprof/hprof_md.c,
>> and jdk/src/windows/demo/jvmti/hprof/hprof_md.c.
>> 
>> 
>> What I think is critically important is defining this common interface in exact detail and also have
>> a separate set of unit tests to verify the behavior.
>> If you can't keep the #ifdef logic out of the source code using this api, then you have a problem.
>> And do not underestimate Windows, I'll wager that more than 50% of the effort will be on that one.
>> Don't underestimate this kind of cleanup work, and it is a great deal of cleanup work.
>> 
>> -kto
>> 
>> 
>> 
>> 
>> On May 16, 2011, at 9:06 AM, Jing LV wrote:
>> 
>>> Hello Henri,
>>> 
>>> Yes something like that. As far as I understand APR focus on serving Apache http server, the portlib for JDK may work for a better JDK. In fact there are already many portlibs existing, and we can learn from them for a layer for OpenJDK.
>>> I've posted on BSD porting list as well, someone is already interested in it. He sugggest we may use a service provider pattern to make it more flexible for all kinds of platforms.
>>> Any questions/suggestion/comments are welcome!
>>> 
>>> 于 2011-5-16 21:35, Henri Gomez 写道:
>>>> Do you think about something like Apache does with APR (Apache
>>>> Portable Runtime) ?
>>>> 
>>>> 2011/5/15 Jing LV<lvjing at linux.vnet.ibm.com>:
>>>>> Hello MacOS developers,
>>>>> 
>>>>> I see on openjdk we'll have more platforms - except MacOS, there is some
>>>>> discussions about AIX. This is great news to the community, as well as a
>>>>> new challenge to the community to manage different native implementation
>>>>> for new added platform as well as early platform. The challenges we may
>>>>> face are:
>>>>> 1. in current implementation, we have native implementation in
>>>>> windows/linux/solaris directory, and create some same jni methods. But
>>>>> actually they have the same or very similar logic. This is not very
>>>>> manageable. If some logic is change we need to modify implementation on
>>>>> all platforms. and may cause some of problem as no one knows all
>>>>> platforms differences, and the platform developers need to understand
>>>>> the logic before the modification, it may be a extra work for developers
>>>>> like MacOS engineers.
>>>>> 2. different platforms offers different system APIs, and even different
>>>>> versions of system have different APIs; in current implementation I see
>>>>> some code like
>>>>> #ifdef someplatform
>>>>> use some API
>>>>> #endif
>>>>> This increases the complexity of the code, and make code ugly. Also the
>>>>> developer may have much trouble to read and modify if necessary.
>>>>> 3. Openjdk is working on project digjaw/modularization, it may meet some
>>>>> trouble if the native API and logic are separated by platform level, not
>>>>> in functional level.
>>>>> 
>>>>> I am wondering if a port layer, leave all APIs differences in this
>>>>> layer. The jni developers can use this unified API, like "int write(fd,
>>>>> byte[])" should work on all platforms openjdk supported, including
>>>>> linux, windows, MacOS etc. This may help us the developers:
>>>>> 1. the platform developer can focus on covering the API difference and
>>>>> care nothing of the upper logical - say, e.g, focus on write some given
>>>>> bytes into the give fd, do not care what the fd is and how to deal with
>>>>> the buffers etc, so we will write it only once, and only update for new
>>>>> APIs when necessary. Meanwhile, the classlib developers can use an
>>>>> unified system API and focus on the logic, we write the code once for
>>>>> all platforms. It save time and effort on both side.
>>>>> 2. The code is then clear, no #endif is required, this helps the
>>>>> developer to read and understand, and much easier to modify.
>>>>> 3. it may help to modularize the jdk as well.
>>>>> 
>>>>> An new portlib may also have some problems, like modification on the
>>>>> current code, and performance. We need to define the portlayer well, and
>>>>> make excellent build script to avoid performance degradation of the
>>>>> layer. However, in the long run, the portlayer will really help the
>>>>> developers as well as JDK, like some other opensource jdk do.
>>>>> 
>>>>> I believe the portlayer would help MacOS developers a lot in code
>>>>> maintenance, and when updating new features. I'd like to listen to your
>>>>> opinions/comments/suggestions on this topic.
>>>>> 
>>>>> Thanks!
>>>>> 
>>>>> --
>>>>> Best Regards,
>>>>> Jimmy, Jing LV
>>>>> 
>>>>> 
>>>>> 
>>> 
>>> -- 
>>> Best Regards,
>>> Jimmy, Jing LV
>>> 
>>> 
>> 
> 
> 
> -- 
> Best Regards,
> Jimmy, Jing LV