From john.r.rose at oracle.com  Sat Sep  1 03:46:05 2018
From: john.r.rose at oracle.com (John Rose)
Date: Fri, 31 Aug 2018 20:46:05 -0700
Subject: [foreign] RFR 8210264: cleanup semantics of function pointer
 conversion
In-Reply-To: <02dae87a-5560-93d1-f90a-fe9e80980126@oracle.com>
References: <02dae87a-5560-93d1-f90a-fe9e80980126@oracle.com>
Message-ID: <0C0AB958-2510-439D-B777-10FD86AB96F4@oracle.com>

On Aug 31, 2018, at 9:04 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~mcimadamore/panama/8210264/ <http://cr.openjdk.java.net/~mcimadamore/panama/8210264/>
This is a good cleanup.

An object is allowed to implement two functional interfaces, and even
two interfaces which annotated as native callbacks. So it is possible
that allocateCallback will be presented with an ambiguous input.
I suggest that Util.findCallback throw an exception in that case,
instead of returning the first marked FI that that it finds.

This won't harm correct code but will let us exclude some corner
cases.

Another way to slice this is to require a runtime witness as follows:

<T> Callback<T> allocateCallback(Class<T> funcInt, T funcIntfInstance);
default <T> Callback<T> allocateCallback(T funcIntfInstance) {
  return allocateCallback(Utils.findCallback(funcIntfInstance.getClass()), funcIntfInstance);
}

I.e., allow explicit specification of which FI we care about but specify
a sensible default.  The explicit specification has two effects:  First,
it allows cast-free passing of a lambda, and second it allows the
approach to scale to non-native FI types (assuming the scope
has rules for dealing with type conversion).

? John

From samuel.audet at gmail.com  Sat Sep  1 06:28:09 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Sat, 1 Sep 2018 15:28:09 +0900
Subject: [nicl] branch is now closed
In-Reply-To: <793f413e-0567-9290-c44f-d2e8b327fbd6@oracle.com>
References: <4119a11e-df50-75dd-f5fe-5317b1e90d5b@oracle.com>
 <2d30ed4d-802e-dfe9-5e25-bed2cd0ca8a8@gmail.com>
 <60b52c13-a54e-573b-ae21-48a1c6facfa9@oracle.com>
 <8998a90f-24f8-928e-7d8a-e6803736fbe3@gmail.com>
 <4c4a6d23-2d35-9f18-06e9-3abfd5372e09@oracle.com>
 <fd158809-df2a-be15-8c08-97702ed30cf6@gmail.com>
 <793f413e-0567-9290-c44f-d2e8b327fbd6@oracle.com>
Message-ID: <e31a59d5-44ff-06f7-aa6f-cbf4431a7d0a@gmail.com>

Now this is starting to sound like a plan! The point isn't about 
pointers or arrays, I'm not aware of all the discussions that went into 
that decision, and this is the point. All decisions related to Panama 
are made behind closed doors with no input from the community. It's the 
first time I hear about most of what you've just made public below, so 
this is a good start. Please keep making this kind of information 
public. I'm sure it will help make people feel welcome.

 From the new information below, it seems like the plan is basically to 
upgrade a lot of the JVM based on ideas pretty much from LLVM, which 
sounds great, but LLVM happened because people started cooperating with 
each other: Apple, Microsoft, Intel, ARM, Qualcomm, Google, Cray, and 
many other smaller players. To achieve the same level of success with 
the JVM, there is going to be a need to gather the same level of 
cooperation. Your post makes me believe that the OpenJDK community is at 
least interested in the idea...

However, the fact is, there is still no cooperation whatsoever between 
Panama and GraalVM, but the parallel are striking, see this for example: 
  https://cornerwings.github.io/2018/07/graal-native-methods/
	Method		 	Time per iteration
	graal/native-image 	76.11 ns
	jni/unsafe 		109.42 ns
Trying to get these teams on the same page might be a good start here. 
Why isn't this happening? I think it would be a useful exercise to 
understand the internal politics of Oracle, but maybe we don't need to 
know the details, just that the internal disputes are taken care of...

Samuel

On 08/27/2018 07:31 PM, Maurizio Cimadamore wrote:
> 
> 
> On 25/08/18 09:34, Samuel Audet wrote:
>> On 08/21/2018 06:34 PM, Maurizio Cimadamore wrote:
>>>
>>>
>>> On 18/08/18 00:16, Samuel Audet wrote:
>>>> About the Array type, it's still not clear to me why we can't just 
>>>> extend the Pointer with a "length"? It's not like pointers in C 
>>>> don't typically point to memory regions with length > 0: It's just 
>>>> often unknown. I've modeled the Pointer type in JavaCPP after Buffer 
>>>> (with position, limit, and capacity) and I haven't had any issues at 
>>>> all in years. None whatsoever. What is the benefit you perceive in 
>>>> mapping artificially 1 C concept into 2 Java concepts? I was not 
>>>> able to find the rationale from your document... If the idea is to 
>>>> provide safe get()/set() methods, why not simply use a layout? What 
>>>> does the Array type do that a layout could not do? That's basically 
>>>> what I do with the indexer package and it works fine:
>>>> http://bytedeco.org/news/2014/12/23/third-release/
>>>> Again, no issues whatsoever in years, without complicating the API! 
>>> I'm not sure I follow your argument fully. In the C language spec 
>>> there's not one concept, but two - we have array types and pointer 
>>> types. Their behavior is quite different, so, for instance sizeof 
>>> returns quite different things if you compare sizeof(int[]) with 
>>> sizeof(int*). More generally, an array refers to a contiguous region 
>>> of memory filled by N objects of the same kind. Layout-wise, an array 
>>> is much more similar to a struct than it is to a pointer. E.g. int[3] 
>>> and struct { int, int, int } have the same layout.
>>
>> I fully agree there are a few differences that we can't deal with just 
>> pointers, but my point is, why not leave that up to a layout instead 
>> of introducing one more class to the mix? What precisely are the 
>> layouts not capable of accommodating that you had to introduce the 
>> Array class for?
> I think you are mixing up different factors here.
> 
> - There is an API decision: what should be the (Java) carrier type for 
> API points featuring C array types?
> - Then there is the kind of mapping we want to achieve: we used to call 
> this 'raw' mapping - that is, as close as possible to C semantics. More 
> 'civilized' mapping will come to the fore at a later point.
> - Then there is the role of layouts.
> 
> All these thee axis are quite orthogonal to each other.
> 
> In the C world, as I stressed, arrays and pointers are different things, 
> though the C compiler might insert an auto conversion from the former to 
> the latter and blur things a bit. So, I believe the sensible API choices 
> would be:
> 
> 1) use Java array as carriers
> 2) introduce a new carrier Array, and make it Array <: Pointer, to mimic 
> the auto conversion
> 3) introduce a new carrier Array - and have a way to go from Array to 
> Pointer
> 
> As described in quite detail in the design document, using Java arrays 
> as carriers is the wrong move. Java arrays comes with too many 
> assumptions on how things are laid out in memory - and in many cases to 
> go from a native array to a Java array you would need to copy. If this 
> copy happens magically as soon as you call a struct getter returning an 
> array, that is really bad: suppose you want to read an array from struct 
> A, take its base pointer and pass it to function f() - why copy memory?
> 
> So, unless something like Array 2.0 comes to the rescue (that will allow 
> modeling arrays as interfaces), using Java arrays is a prohibitive 
> choice for the 'raw' mapping such as the one we're working on now. Later 
> on, it will be possible to define 'civilized' bindings that automap 
> native arrays to Java arrays 'magically' - but that has to be an opt-in, 
> because of the performance model that comes with it. (very similar 
> argument applies, BTW, to function pointer auto conversion).
>>
>>> Of course there is a relationship between arrays and pointers, and 
>>> that is caused by the fact that an array always decays to a 'base 
>>> pointer' - when the context requires it. But the layout of this base 
>>> pointer is rather different - e.g. if you take the base pointer of an 
>>> int[3], you and up with a pointer whose pointee is 'int'.
>>>
>>> Now, if we had a calculus for layouts which allowed to specify sizes 
>>> as well - and maybe ours can do that with layout annotations:
>>>
>>> u64(length=3):i32
>>>
>>> Then it kind of seems that it would be possible to lump arrays and 
>>> pointers together. But there's one detail that is omitted from this 
>>> layout description: this description does not mandate that the 
>>> elements occur 'right here right now'. That is, if I have a struct 
>>> like this:
>>>
>>> [ u64(length=3):i32 ]
>>>
>>> Does the struct have a field of size 32 * 3 inline, or does the 
>>> struct have a field of size 64 which points to some contiguous region 
>>> of memory storing 3 ints? I believe the layout expression above 
>>> suggests the latter.
>>
>> So, maybe we should fix this in the layouts instead of coming up with 
>> Array just to patch over this?
> And here we arrive at layouts: layouts are for describing region of 
> memory (the region that, in the C lingo would be associated with an 
> 'object'). A region of memory doesn't depend on what you do with it - it 
> just exists, and the layout API gives you a way to describe its shape. 
> So, the layout API has ways to describe sequence-like regions, and 
> things like that.
> 
> When you have a pointer, the layout API can be used to describe the 
> shape of the memory region associated with the pointee.
> 
> But using an 'array' layout and attaching that to a pointer doesn't 
> really give you an array - it gives you a "pointer to an array". So I 
> really can't parse your objections here. If I have struct like this:
> 
> struct foo {
> int x[5];
> }
> 
> I'd like the Java API to return something 'more' than Pointer - given 
> that the C type of 'x' is not int* but int[5]. The model should take 
> this distinction into account. I could understand a discussion between 
> choices (2) and (3) - e.g. should Array just be a pointer + lentgth 
> (e.g. use subtyping) or a separate abstraction; for now, our intuition 
> on API design has told us that splitting arrays from pointers is the 
> right move from an API perspective. So here we're talking API design - 
> layout are totally orthogonal: carrier types in Java APIs and layouts 
> are two different things (**).
> 
> (**) - yes, in the extreme case you could lump _all_ native types 
> together, and just distinguish them using a layout field. But that would 
> be such an awkward API to use that we're not entertaining that position 
> in the design space. When you see a Java API , you'd like to know what 
> to pass to it - and if everything is lumped, you turn a lot of 
> statically checkable errors into dynamically checkable errors, to the 
> point that the API is very hard to use right. So, I don't think layouts 
> should be abused in this particular way.
> 
> 
>>
>>> The Array API point is there to recognize this distinction: an array 
>>> is a lump of memory; if you want you can take a pointer to the first 
>>> array element out of it, and work that way - but the fact that it's 
>>> easy to go in that direction doesn't mean there must be an 'is-a' 
>>> relationship between the two entities. Of course, when you design an 
>>> API there are many such decisions - John calls them with the apt 
>>> definition splitting vs. lumping. At the time we considered both 
>>> lumping (e.g. just use pointers) and splitting (the model we have 
>>> now) and found that the latter offered much more clarity to the 
>>> programmer. While arrays and pointers do overlap, there are certain 
>>> differences (e.g. copy semantics on array struct field access) that, 
>>> at the time we looked at them, pushed us towards the 'split' choice. 
>>> It's not, of course, a choice set in stone, but I think it's one 
>>> we're pretty happy with at the moment (unlike that, e.g. for function 
>>> pointer automatic conversion).
>>
>> Hum, it's sounds like you're effectively saying that layouts are 
>> inappropriate to map, for example, Protocol buffers or FlatBuffers, 
>> which are fast becoming the industry standard. Have you given any 
>> thoughts to Protocol Buffers or FlatBuffers and if so where can we 
>> find preliminary notes about those?
> Nothing of that sort - in fact our layout calculus has been derived both 
> by existing FFI use case _and_ by protocols use cases, see this very 
> detailed email analyzing requirements of layouts in protocols:
> 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-February/000940.html
>>
>>> As for your point on the relationship between Panama and Java arrays 
>>> - Java arrays are essentially unsuitable for the kind of raw mapping 
>>> we wanted to achieve at this stage. This does not preclude us to add 
>>> a 'civilized' binding layer which autoconverts native arrays into 
>>> Java arrays - but since that's an expensive move, it will be an opt in.
>>
>> Good to hear that!
>>
>>> The fact that Java arrays are not interfaces is biting us here, as 
>>> well - Array 2.0 explored the possibility of having an Array<any X> 
>>> interface, specialized in primitive types; with such support 
>>> available we could indeed treat Panama arrays as specific instances 
>>> of such a generalized interface. Until that support is available, we 
>>> have to paper over the differences in other ways, which is what this 
>>> API is attempting to do.
>>
>> Right, but by not supporting normal arrays, you're preventing, for 
>> example, the vector team from benefiting from your work.
>>
>> Vladimir expressed interest in the SVML intrinsics offered by the 
>> Intel C++ Compiler. That's basically a library of inline functions. We 
>> could have a header file with something like:
>>
>> __m128 fast_sin(__m128 x) {
>> ??? return __m128 _mm_sin_ps(x)
>> }
>>
>> __m256 fast_sin(__m256 x) {
>> ??? return __m256 _mm256_sin_ps(x)
>> }
>>
>> And if Panama could inline this in Java, that's it! You've got a fast 
>> parallel version of sin() that is inlined with zero overhead when 
>> called from Java.
> This stuff was already working at some point in the Panama repository. 
> To make it work, we need suitable carrier types for vector types such as 
> the one appearing in your signatures, and, to make things efficient, 
> such carriers must better be value types (see Valhalla).
> 
> At this point we decided to split foreign an vector, for pragmatic 
> reasons (the vector API was undergoing a lot of experimentation one year 
> ago) - but note that the VM backend handling native calls still has 
> support for vectorized types - we just miss the Java carriers to go with 
> them, which will be conveniently provided by the vector API. So, when 
> the two APIs will meet again we will just need to teach the binder about 
> vector carriers, and the rest will just work (and we know this from 
> experience, as the support was there).
>>
>> So, you see, this kind of feature is in strong demand, and by your own 
>> team! I say we should do some dog fooding. Create something that 
>> others will actually want to use!
> I hope I have shown that the cases you are thinking about are covered by 
> our plan - we're not designing in silos and the Panama and Vector teams 
> know about each other (if that is the concern you were trying to express).
> 
> If there is one issue that can be inferred from all of this is: to make 
> a good Panama and Vector API (and here I'm talking about API not VM 
> optimization internals), you need quite a bit of VM firepower. Both 
> Panama and Vector would hugely benefit from (i) value types (e.g. if 
> Panama pointers were values, their creation would be super cheap - ditto 
> for vectorized types), (ii) generic specialization (both Panama classes 
> and Vector classes are naturally generic over Java _primitive_ types) 
> and (iii) Arrays 2.0 - as I've said above, some interface to abstract 
> over the behavior of a Java array would be really useful in Panama.
> 
> So, all this to say - the APIs we are designing right now, do have to 
> cope with some of the expressiveness limitations that come from the lack 
> of such language/VM features. For instance, to counter the lack of 
> generic specialization, Panama and Vector went in different paths: 
> Panama used a single Pointer interface with boxed type parameter (but 
> with MH under the hood to provide specialized access) - while the Vector 
> API went down a path of manual specialization of the code. These are of 
> course both suboptimal choices that we hope to be able to revert when 
> more expressiveness will be available in the underlying layers of the 
> language/VM.
> 
> But this has also nothing to do with pointer vs. arrays - that is just a 
> C language distinction that we deemed important enough to reify in our 
> Panama API. I understand other choices are possible, and I hope to have 
> provided some rationale as to why we went the way we went about it.
> 
> Maurizio
>>
>> If Panama could provide that kind of feature, many other users, at 
>> Skymind and elsewhere, could also use it to implement their own 
>> libraries of "fast math", so we wouldn't have to bother you guys all 
>> the time to implement some fast version of math. We'd just do it all 
>> ourselves as libraries! That would be great, wouldn't?
>>
>> As always, please let us know how we can help. As Johan keeps 
>> reminding me, we could write JEPs and what not, but before we get 
>> there I would need to understand what would be required to make you 
>> interested in such a feature. In other words, what would you be 
>> looking for in the JEP to accept it yourself?
>>
>> Samuel
> 


From henry.jen at oracle.com  Mon Sep  3 02:55:56 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Sun, 2 Sep 2018 19:55:56 -0700
Subject: [nicl] branch is now closed
In-Reply-To: <e31a59d5-44ff-06f7-aa6f-cbf4431a7d0a@gmail.com>
References: <4119a11e-df50-75dd-f5fe-5317b1e90d5b@oracle.com>
 <2d30ed4d-802e-dfe9-5e25-bed2cd0ca8a8@gmail.com>
 <60b52c13-a54e-573b-ae21-48a1c6facfa9@oracle.com>
 <8998a90f-24f8-928e-7d8a-e6803736fbe3@gmail.com>
 <4c4a6d23-2d35-9f18-06e9-3abfd5372e09@oracle.com>
 <fd158809-df2a-be15-8c08-97702ed30cf6@gmail.com>
 <793f413e-0567-9290-c44f-d2e8b327fbd6@oracle.com>
 <e31a59d5-44ff-06f7-aa6f-cbf4431a7d0a@gmail.com>
Message-ID: <067E7B33-BFFC-4784-B52D-E4818B984764@oracle.com>


> On Aug 31, 2018, at 11:28 PM, Samuel Audet <samuel.audet at gmail.com> wrote:
> 
> Now this is starting to sound like a plan! The point isn't about pointers or arrays, I'm not aware of all the discussions that went into that decision, and this is the point. All decisions related to Panama are made behind closed doors with no input from the community. It's the first time I hear about most of what you've just made public below, so this is a good start. Please keep making this kind of information public. I'm sure it will help make people feel welcome.
> 
> From the new information below, it seems like the plan is basically to upgrade a lot of the JVM based on ideas pretty much from LLVM, which sounds great, but LLVM happened because people started cooperating with each other: Apple, Microsoft, Intel, ARM, Qualcomm, Google, Cray, and many other smaller players. To achieve the same level of success with the JVM, there is going to be a need to gather the same level of cooperation. Your post makes me believe that the OpenJDK community is at least interested in the idea...
> 
> However, the fact is, there is still no cooperation whatsoever between Panama and GraalVM, but the parallel are striking, see this for example:  https://cornerwings.github.io/2018/07/graal-native-methods/
> 	Method		 	Time per iteration
> 	graal/native-image 	76.11 ns
> 	jni/unsafe 		109.42 ns
> Trying to get these teams on the same page might be a good start here. Why isn't this happening? I think it would be a useful exercise to understand the internal politics of Oracle, but maybe we don't need to know the details, just that the internal disputes are taken care of?
> 

Correct me if I am wrong, but those are different approaches and not exclusive. We noted that Graal polyglot solution, but it needs source code or LLVM bitcode, while here at Panama is focused on platform native format. 

Cheers,
Henry

> Samuel
> 
> On 08/27/2018 07:31 PM, Maurizio Cimadamore wrote:
>> On 25/08/18 09:34, Samuel Audet wrote:
>>> On 08/21/2018 06:34 PM, Maurizio Cimadamore wrote:
>>>> 
>>>> 
>>>> On 18/08/18 00:16, Samuel Audet wrote:
>>>>> About the Array type, it's still not clear to me why we can't just extend the Pointer with a "length"? It's not like pointers in C don't typically point to memory regions with length > 0: It's just often unknown. I've modeled the Pointer type in JavaCPP after Buffer (with position, limit, and capacity) and I haven't had any issues at all in years. None whatsoever. What is the benefit you perceive in mapping artificially 1 C concept into 2 Java concepts? I was not able to find the rationale from your document... If the idea is to provide safe get()/set() methods, why not simply use a layout? What does the Array type do that a layout could not do? That's basically what I do with the indexer package and it works fine:
>>>>> http://bytedeco.org/news/2014/12/23/third-release/
>>>>> Again, no issues whatsoever in years, without complicating the API! 
>>>> I'm not sure I follow your argument fully. In the C language spec there's not one concept, but two - we have array types and pointer types. Their behavior is quite different, so, for instance sizeof returns quite different things if you compare sizeof(int[]) with sizeof(int*). More generally, an array refers to a contiguous region of memory filled by N objects of the same kind. Layout-wise, an array is much more similar to a struct than it is to a pointer. E.g. int[3] and struct { int, int, int } have the same layout.
>>> 
>>> I fully agree there are a few differences that we can't deal with just pointers, but my point is, why not leave that up to a layout instead of introducing one more class to the mix? What precisely are the layouts not capable of accommodating that you had to introduce the Array class for?
>> I think you are mixing up different factors here.
>> - There is an API decision: what should be the (Java) carrier type for API points featuring C array types?
>> - Then there is the kind of mapping we want to achieve: we used to call this 'raw' mapping - that is, as close as possible to C semantics. More 'civilized' mapping will come to the fore at a later point.
>> - Then there is the role of layouts.
>> All these thee axis are quite orthogonal to each other.
>> In the C world, as I stressed, arrays and pointers are different things, though the C compiler might insert an auto conversion from the former to the latter and blur things a bit. So, I believe the sensible API choices would be:
>> 1) use Java array as carriers
>> 2) introduce a new carrier Array, and make it Array <: Pointer, to mimic the auto conversion
>> 3) introduce a new carrier Array - and have a way to go from Array to Pointer
>> As described in quite detail in the design document, using Java arrays as carriers is the wrong move. Java arrays comes with too many assumptions on how things are laid out in memory - and in many cases to go from a native array to a Java array you would need to copy. If this copy happens magically as soon as you call a struct getter returning an array, that is really bad: suppose you want to read an array from struct A, take its base pointer and pass it to function f() - why copy memory?
>> So, unless something like Array 2.0 comes to the rescue (that will allow modeling arrays as interfaces), using Java arrays is a prohibitive choice for the 'raw' mapping such as the one we're working on now. Later on, it will be possible to define 'civilized' bindings that automap native arrays to Java arrays 'magically' - but that has to be an opt-in, because of the performance model that comes with it. (very similar argument applies, BTW, to function pointer auto conversion).
>>> 
>>>> Of course there is a relationship between arrays and pointers, and that is caused by the fact that an array always decays to a 'base pointer' - when the context requires it. But the layout of this base pointer is rather different - e.g. if you take the base pointer of an int[3], you and up with a pointer whose pointee is 'int'.
>>>> 
>>>> Now, if we had a calculus for layouts which allowed to specify sizes as well - and maybe ours can do that with layout annotations:
>>>> 
>>>> u64(length=3):i32
>>>> 
>>>> Then it kind of seems that it would be possible to lump arrays and pointers together. But there's one detail that is omitted from this layout description: this description does not mandate that the elements occur 'right here right now'. That is, if I have a struct like this:
>>>> 
>>>> [ u64(length=3):i32 ]
>>>> 
>>>> Does the struct have a field of size 32 * 3 inline, or does the struct have a field of size 64 which points to some contiguous region of memory storing 3 ints? I believe the layout expression above suggests the latter.
>>> 
>>> So, maybe we should fix this in the layouts instead of coming up with Array just to patch over this?
>> And here we arrive at layouts: layouts are for describing region of memory (the region that, in the C lingo would be associated with an 'object'). A region of memory doesn't depend on what you do with it - it just exists, and the layout API gives you a way to describe its shape. So, the layout API has ways to describe sequence-like regions, and things like that.
>> When you have a pointer, the layout API can be used to describe the shape of the memory region associated with the pointee.
>> But using an 'array' layout and attaching that to a pointer doesn't really give you an array - it gives you a "pointer to an array". So I really can't parse your objections here. If I have struct like this:
>> struct foo {
>> int x[5];
>> }
>> I'd like the Java API to return something 'more' than Pointer - given that the C type of 'x' is not int* but int[5]. The model should take this distinction into account. I could understand a discussion between choices (2) and (3) - e.g. should Array just be a pointer + lentgth (e.g. use subtyping) or a separate abstraction; for now, our intuition on API design has told us that splitting arrays from pointers is the right move from an API perspective. So here we're talking API design - layout are totally orthogonal: carrier types in Java APIs and layouts are two different things (**).
>> (**) - yes, in the extreme case you could lump _all_ native types together, and just distinguish them using a layout field. But that would be such an awkward API to use that we're not entertaining that position in the design space. When you see a Java API , you'd like to know what to pass to it - and if everything is lumped, you turn a lot of statically checkable errors into dynamically checkable errors, to the point that the API is very hard to use right. So, I don't think layouts should be abused in this particular way.
>>> 
>>>> The Array API point is there to recognize this distinction: an array is a lump of memory; if you want you can take a pointer to the first array element out of it, and work that way - but the fact that it's easy to go in that direction doesn't mean there must be an 'is-a' relationship between the two entities. Of course, when you design an API there are many such decisions - John calls them with the apt definition splitting vs. lumping. At the time we considered both lumping (e.g. just use pointers) and splitting (the model we have now) and found that the latter offered much more clarity to the programmer. While arrays and pointers do overlap, there are certain differences (e.g. copy semantics on array struct field access) that, at the time we looked at them, pushed us towards the 'split' choice. It's not, of course, a choice set in stone, but I think it's one we're pretty happy with at the moment (unlike that, e.g. for function pointer automatic conversion).
>>> 
>>> Hum, it's sounds like you're effectively saying that layouts are inappropriate to map, for example, Protocol buffers or FlatBuffers, which are fast becoming the industry standard. Have you given any thoughts to Protocol Buffers or FlatBuffers and if so where can we find preliminary notes about those?
>> Nothing of that sort - in fact our layout calculus has been derived both by existing FFI use case _and_ by protocols use cases, see this very detailed email analyzing requirements of layouts in protocols:
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-February/000940.html
>>> 
>>>> As for your point on the relationship between Panama and Java arrays - Java arrays are essentially unsuitable for the kind of raw mapping we wanted to achieve at this stage. This does not preclude us to add a 'civilized' binding layer which autoconverts native arrays into Java arrays - but since that's an expensive move, it will be an opt in.
>>> 
>>> Good to hear that!
>>> 
>>>> The fact that Java arrays are not interfaces is biting us here, as well - Array 2.0 explored the possibility of having an Array<any X> interface, specialized in primitive types; with such support available we could indeed treat Panama arrays as specific instances of such a generalized interface. Until that support is available, we have to paper over the differences in other ways, which is what this API is attempting to do.
>>> 
>>> Right, but by not supporting normal arrays, you're preventing, for example, the vector team from benefiting from your work.
>>> 
>>> Vladimir expressed interest in the SVML intrinsics offered by the Intel C++ Compiler. That's basically a library of inline functions. We could have a header file with something like:
>>> 
>>> __m128 fast_sin(__m128 x) {
>>>     return __m128 _mm_sin_ps(x)
>>> }
>>> 
>>> __m256 fast_sin(__m256 x) {
>>>     return __m256 _mm256_sin_ps(x)
>>> }
>>> 
>>> And if Panama could inline this in Java, that's it! You've got a fast parallel version of sin() that is inlined with zero overhead when called from Java.
>> This stuff was already working at some point in the Panama repository. To make it work, we need suitable carrier types for vector types such as the one appearing in your signatures, and, to make things efficient, such carriers must better be value types (see Valhalla).
>> At this point we decided to split foreign an vector, for pragmatic reasons (the vector API was undergoing a lot of experimentation one year ago) - but note that the VM backend handling native calls still has support for vectorized types - we just miss the Java carriers to go with them, which will be conveniently provided by the vector API. So, when the two APIs will meet again we will just need to teach the binder about vector carriers, and the rest will just work (and we know this from experience, as the support was there).
>>> 
>>> So, you see, this kind of feature is in strong demand, and by your own team! I say we should do some dog fooding. Create something that others will actually want to use!
>> I hope I have shown that the cases you are thinking about are covered by our plan - we're not designing in silos and the Panama and Vector teams know about each other (if that is the concern you were trying to express).
>> If there is one issue that can be inferred from all of this is: to make a good Panama and Vector API (and here I'm talking about API not VM optimization internals), you need quite a bit of VM firepower. Both Panama and Vector would hugely benefit from (i) value types (e.g. if Panama pointers were values, their creation would be super cheap - ditto for vectorized types), (ii) generic specialization (both Panama classes and Vector classes are naturally generic over Java _primitive_ types) and (iii) Arrays 2.0 - as I've said above, some interface to abstract over the behavior of a Java array would be really useful in Panama.
>> So, all this to say - the APIs we are designing right now, do have to cope with some of the expressiveness limitations that come from the lack of such language/VM features. For instance, to counter the lack of generic specialization, Panama and Vector went in different paths: Panama used a single Pointer interface with boxed type parameter (but with MH under the hood to provide specialized access) - while the Vector API went down a path of manual specialization of the code. These are of course both suboptimal choices that we hope to be able to revert when more expressiveness will be available in the underlying layers of the language/VM.
>> But this has also nothing to do with pointer vs. arrays - that is just a C language distinction that we deemed important enough to reify in our Panama API. I understand other choices are possible, and I hope to have provided some rationale as to why we went the way we went about it.
>> Maurizio
>>> 
>>> If Panama could provide that kind of feature, many other users, at Skymind and elsewhere, could also use it to implement their own libraries of "fast math", so we wouldn't have to bother you guys all the time to implement some fast version of math. We'd just do it all ourselves as libraries! That would be great, wouldn't?
>>> 
>>> As always, please let us know how we can help. As Johan keeps reminding me, we could write JEPs and what not, but before we get there I would need to understand what would be required to make you interested in such a feature. In other words, what would you be looking for in the JEP to accept it yourself?
>>> 
>>> Samuel
> 


From maurizio.cimadamore at oracle.com  Mon Sep  3 09:53:57 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 3 Sep 2018 10:53:57 +0100
Subject: [nicl] branch is now closed
In-Reply-To: <e31a59d5-44ff-06f7-aa6f-cbf4431a7d0a@gmail.com>
References: <4119a11e-df50-75dd-f5fe-5317b1e90d5b@oracle.com>
 <2d30ed4d-802e-dfe9-5e25-bed2cd0ca8a8@gmail.com>
 <60b52c13-a54e-573b-ae21-48a1c6facfa9@oracle.com>
 <8998a90f-24f8-928e-7d8a-e6803736fbe3@gmail.com>
 <4c4a6d23-2d35-9f18-06e9-3abfd5372e09@oracle.com>
 <fd158809-df2a-be15-8c08-97702ed30cf6@gmail.com>
 <793f413e-0567-9290-c44f-d2e8b327fbd6@oracle.com>
 <e31a59d5-44ff-06f7-aa6f-cbf4431a7d0a@gmail.com>
Message-ID: <e664f7a2-baf0-0591-6af9-ad3bb53666a3@oracle.com>



On 01/09/18 07:28, Samuel Audet wrote:
> Now this is starting to sound like a plan! The point isn't about 
> pointers or arrays, I'm not aware of all the discussions that went 
> into that decision, and this is the point. All decisions related to 
> Panama are made behind closed doors with no input from the community. 
> It's the first time I hear about most of what you've just made public 
> below, so this is a good start. Please keep making this kind of 
> information public. I'm sure it will help make people feel welcome. 
Hi Samuel,
point noted; sometimes emails are just not a suitable medium for 
capturing design decisions. And if there's something we should be doing 
more of, is trying to open up some of the discussions that sometimes we 
have internally (although the ration between open and closed discussion 
is nowhere near what you imagine); there's nothing fundamentally private 
to these communications, think of it as 'laziness' or habit on our part. 
I've been trying to be more upfront about our goals - and have some 
write-ups pushed out _before_ starting to make any significant changes 
to the code base. I guess sometimes that works, sometimes it doesn't and 
we need help from all of you to call us out when it doesn't, so thank you.

All this said, I'd say that most of the information is available out 
there - and when it's not, trust me, it's not because of some evil 
machination on our part; writing things up in a decent way takes time 
too, and we have to balance that too - if we still want to have time 
left to push the code changes out :-)

As? a closing comment, at the OpenJDK Committer Workshop [1] very 
similar concerns were raised - and Brian Goetz proposed that some of 
these projects (Valhalla, Panama, Amber) could have some periodic public 
meetup (e.g. via Zoom) where stakeholders could meet architects and 
developers and make sure that their voice is heard. I think this could 
go a long way in helping us understand how to improve our 
communications. As we move closer to an early access state, I think we 
will host one such experimental call (details TBD), so that we can have 
a chat of what will be in the EA, what will not be in it, and what's our 
plan to move the ball forward.

Cheers
Maurizio

[1] - http://openjdk.java.net/workshop

From maurizio.cimadamore at oracle.com  Mon Sep  3 10:04:34 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 3 Sep 2018 11:04:34 +0100
Subject: [foreign] RFR 8210264: cleanup semantics of function pointer
 conversion
In-Reply-To: <0C0AB958-2510-439D-B777-10FD86AB96F4@oracle.com>
References: <02dae87a-5560-93d1-f90a-fe9e80980126@oracle.com>
 <0C0AB958-2510-439D-B777-10FD86AB96F4@oracle.com>
Message-ID: <34e16eac-10d2-dffa-a35e-a968f6914d96@oracle.com>



On 01/09/18 04:46, John Rose wrote:
> On Aug 31, 2018, at 9:04 AM, Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com 
> <mailto:maurizio.cimadamore at oracle.com>> wrote:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/8210264/ 
>> <http://cr.openjdk.java.net/%7Emcimadamore/panama/8210264/>
>
> This is a good cleanup.
>
> An object is allowed to implement two functional interfaces, and even
> two interfaces which annotated as native callbacks. So it is possible
> that allocateCallback will be presented with an ambiguous input.
> I suggest that Util.findCallback throw an exception in that case,
> instead of returning the first marked FI that that it finds.
>
> This won't harm correct code but will let us exclude some corner
> cases.
>
> Another way to slice this is to require a runtime witness as follows:
>
> <T> Callback<T> allocateCallback(Class<T> funcInt, T funcIntfInstance);
> default <T> Callback<T> allocateCallback(T funcIntfInstance) {
> ? return 
> allocateCallback(Utils.findCallback(funcIntfInstance.getClass()), 
> funcIntfInstance);
> }
>
> I.e., allow explicit specification of which FI we care about but specify
> a sensible default. ?The explicit specification has two effects: ?First,
> it allows cast-free passing of a lambda, and second it allows the
> approach to scale to non-native FI types (assuming the scope
> has rules for dealing with type conversion).
I thought about the latter a lot. Honestly, I'd prefer an API point such 
as the one you describe - e.g. with an explicit Class type witness. 
That's the first iteration of the API I experimented with - and I found 
one big usability hurdle with that: the need to pass in opaque 
functional interface names generated by jextract. E.g. what you will be 
looking at would be something like:

qsort(... scope.allocateCallback(StdLib$Func23.class, (p1, p2) -> ... ) )

While I agree this makes the API a lot tighter and avoids the need of a 
dynamic lookup which I dislike, I can also imagine that users will be 
pretty annoyed at having to dig an obscure functional interface name 
just to make the API happy.

Now, the approach I put forward is not bullet proof in this regard: if 
javac inference fails (although, after changes brought in by Java SE 8 
that should be quite rare), the user will still have to dig a type 
witness by hand:

scope.<StdLib$23>allocateCallback( (p1, p2) -> ... )

But this would be (i) very rare and (ii) we could mitigate a lot of the 
pain by having jextract generate predictable functional interface names. 
E.g. instead of Func$23, using Function_ptr_ptr would be a lot clearer 
(and would allow reuse on the jextract side).

I don't think trick (ii) is enough to make a client swallow the fact of 
putting in an explicit witness on any callback call - but maybe I'm 
worrying too much about compactness of client code? But maybe you are 
suggesting the witness-accepting 'allocateCallback' not as a 
replacement, but as a complement for the version included in my patch?

Maurizio
>
> ? John


From sundararajan.athijegannathan at oracle.com  Mon Sep  3 10:34:13 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Mon, 03 Sep 2018 16:04:13 +0530
Subject: [foreign] RFR jextract should model clang Cursors as Trees
In-Reply-To: <d5121e9e-c94d-04d5-0e51-967ebdf870ee@oracle.com>
References: <5B895B13.7060700@oracle.com>
 <d5121e9e-c94d-04d5-0e51-967ebdf870ee@oracle.com>
Message-ID: <5B8D0E25.2000903@oracle.com>

I've made few further changes:

* Removed main method AsmCodeFactory class

* TreePrinter refactored as a separate class and Parser's (test) main is 
now uses it

* Removed CallbackTree (yes, it was from an earlier attempt) and 
associated visitor method

Updated: http://cr.openjdk.java.net/~sundar/8210263/webrev.01

Thanks.
-Sundar

On 31/08/18, 10:26 PM, Maurizio Cimadamore wrote:
> Great work; I think this goes a long way towards putting some more 
> solid foundation under jextract. It also allows, by putting most of 
> the clang dependencies right into the immediate frontend, to make the 
> rest of the code less sensitive to API changes (e.g. should we go from 
> C to C++ clang API). Some comments below (some of those might be 
> overly general and you might want to deal with them in followup changes):
>
> * Thanks for adding the equals/hashcode method in Source[Location, Range]
>
> * While it's probably early now, I think there are a lot of steps 
> which might be translated into their own visitor - e.g. filtering 
> cursors seems one of them (happens between Context and Parser)
>
> * I also believe that, moving forward, classes such as Context and 
> HeaderFile should be all about keeping state, and not about defining 
> behavior (which something that should be delegated to tree visitors)
>
> * I like that AsmCodeFactory just become a visitor!
>
> * Macro handling became a bit better (since we now compute macros on 
> parsing, and then delegate generation to the code factory) - I like 
> that; but let's also keep in mind that macro computation can become 
> its own tree step
>
> * Nit: why is there a main method in ASMCodeFactory?
>
> * It feels like inside Parser there's a tree printer (see main 
> method). And that this printer should probably be unified with the 
> main Printer class. And, pulling on this string some more, I'd like 
> this printer exposed as its own class, so that we can use it to unit 
> test parsing.
>
> * It feels like we could decouple cursors from the tree even more; for 
> instance, names are not reified in the AST (and types, too). This 
> means that it is not possible, for instance, to define a visitor step 
> which replaces some of the names (e.g. something like that would be 
> needed for typedefs). The code, as now, will keep going back to the 
> cursor.
>
> * what exactly is CallbackTree? It seems unused (and so is one of the 
> two createEnum overloads). I believe that is a leftover of some 
> earlier attempts?
>
> P.S.
> Fetched patch, ran all tests, all clear on Linux x64.
>
> Cheers
> Maurizio
>
>
> On 31/08/18 16:13, Sundararajan Athijegannathan wrote:
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210263
>> Webrev: http://cr.openjdk.java.net/~sundar/8210263/webrev.00/
>>
>> Thanks,
>> -Sundar
>

From maurizio.cimadamore at oracle.com  Mon Sep  3 10:42:15 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 3 Sep 2018 11:42:15 +0100
Subject: [foreign] RFR jextract should model clang Cursors as Trees
In-Reply-To: <5B8D0E25.2000903@oracle.com>
References: <5B895B13.7060700@oracle.com>
 <d5121e9e-c94d-04d5-0e51-967ebdf870ee@oracle.com>
 <5B8D0E25.2000903@oracle.com>
Message-ID: <fce7dd49-1f20-c626-3427-8377f2000671@oracle.com>

Looks good. As a followup I'd also like to see Parser::main removed and 
turned into some jextract debug option, so that we can write tests 
against it.

Maurizio


On 03/09/18 11:34, Sundararajan Athijegannathan wrote:
> I've made few further changes:
>
> * Removed main method AsmCodeFactory class
>
> * TreePrinter refactored as a separate class and Parser's (test) main 
> is now uses it
>
> * Removed CallbackTree (yes, it was from an earlier attempt) and 
> associated visitor method
>
> Updated: http://cr.openjdk.java.net/~sundar/8210263/webrev.01
>
> Thanks.
> -Sundar
>
> On 31/08/18, 10:26 PM, Maurizio Cimadamore wrote:
>> Great work; I think this goes a long way towards putting some more 
>> solid foundation under jextract. It also allows, by putting most of 
>> the clang dependencies right into the immediate frontend, to make the 
>> rest of the code less sensitive to API changes (e.g. should we go 
>> from C to C++ clang API). Some comments below (some of those might be 
>> overly general and you might want to deal with them in followup 
>> changes):
>>
>> * Thanks for adding the equals/hashcode method in Source[Location, 
>> Range]
>>
>> * While it's probably early now, I think there are a lot of steps 
>> which might be translated into their own visitor - e.g. filtering 
>> cursors seems one of them (happens between Context and Parser)
>>
>> * I also believe that, moving forward, classes such as Context and 
>> HeaderFile should be all about keeping state, and not about defining 
>> behavior (which something that should be delegated to tree visitors)
>>
>> * I like that AsmCodeFactory just become a visitor!
>>
>> * Macro handling became a bit better (since we now compute macros on 
>> parsing, and then delegate generation to the code factory) - I like 
>> that; but let's also keep in mind that macro computation can become 
>> its own tree step
>>
>> * Nit: why is there a main method in ASMCodeFactory?
>>
>> * It feels like inside Parser there's a tree printer (see main 
>> method). And that this printer should probably be unified with the 
>> main Printer class. And, pulling on this string some more, I'd like 
>> this printer exposed as its own class, so that we can use it to unit 
>> test parsing.
>>
>> * It feels like we could decouple cursors from the tree even more; 
>> for instance, names are not reified in the AST (and types, too). This 
>> means that it is not possible, for instance, to define a visitor step 
>> which replaces some of the names (e.g. something like that would be 
>> needed for typedefs). The code, as now, will keep going back to the 
>> cursor.
>>
>> * what exactly is CallbackTree? It seems unused (and so is one of the 
>> two createEnum overloads). I believe that is a leftover of some 
>> earlier attempts?
>>
>> P.S.
>> Fetched patch, ran all tests, all clear on Linux x64.
>>
>> Cheers
>> Maurizio
>>
>>
>> On 31/08/18 16:13, Sundararajan Athijegannathan wrote:
>>> Please review.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210263
>>> Webrev: http://cr.openjdk.java.net/~sundar/8210263/webrev.00/
>>>
>>> Thanks,
>>> -Sundar
>>


From sundararajan.athijegannathan at oracle.com  Mon Sep  3 11:16:27 2018
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Mon, 03 Sep 2018 11:16:27 +0000
Subject: hg: panama/dev: 8210263: jextract should model clang Cursors as Trees
Message-ID: <201809031116.w83BGS8t016373@aojmv0008.oracle.com>

Changeset: 18513869450c
Author:    sundar
Date:      2018-09-03 16:53 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/18513869450c

8210263: jextract should model clang Cursors as Trees
Reviewed-by: mcimadamore

! src/jdk.internal.clang/share/classes/jdk/internal/clang/SourceLocation.java
! src/jdk.internal.clang/share/classes/jdk/internal/clang/SourceRange.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/Context.java
- src/jdk.jextract/share/classes/com/sun/tools/jextract/FindSymbol.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/HeaderFile.java
- src/jdk.jextract/share/classes/com/sun/tools/jextract/MacroParser.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/Main.java
- src/jdk.jextract/share/classes/com/sun/tools/jextract/Printer.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/TypeDictionary.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/Utils.java
- src/jdk.jextract/share/classes/com/sun/tools/jextract/Validators.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/FindSymbol.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/MacroParser.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/Parser.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/EnumTree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/FieldTree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/FunctionTree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/HeaderTree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/MacroTree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/Printer.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/SimpleTreeVisitor.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/StructTree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/Tree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreeMaker.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreePrinter.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreeVisitor.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TypedefTree.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/VarTree.java


From maurizio.cimadamore at oracle.com  Mon Sep  3 11:21:34 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Mon, 03 Sep 2018 11:21:34 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809031121.w83BLZOr018204@aojmv0008.oracle.com>

Changeset: 2df4cb9a1cba
Author:    mcimadamore
Date:      2018-09-03 13:24 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/2df4cb9a1cba

Automatic merge with foreign

- src/jdk.jextract/share/classes/com/sun/tools/jextract/FindSymbol.java
- src/jdk.jextract/share/classes/com/sun/tools/jextract/MacroParser.java
- src/jdk.jextract/share/classes/com/sun/tools/jextract/Printer.java
- src/jdk.jextract/share/classes/com/sun/tools/jextract/Validators.java


From razvan.a.lupusoru at intel.com  Tue Sep  4 18:27:08 2018
From: razvan.a.lupusoru at intel.com (Lupusoru, Razvan A)
Date: Tue, 4 Sep 2018 18:27:08 +0000
Subject: VectorAPI SKL Fix for ashiftR
In-Reply-To: <8D6F463991A1574A8A803B8DA605414F3A7E907E@ORSMSX111.amr.corp.intel.com>
References: <8D6F463991A1574A8A803B8DA605414F3A7E907E@ORSMSX111.amr.corp.intel.com>
Message-ID: <48D92A70936F7946BE99F3FF5ECB6461F6683131@ORSMSX105.amr.corp.intel.com>

Looks good to me. Thanks Shravya!

--Razvan

-----Original Message-----
From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Rukmannagari, Shravya
Sent: Friday, August 31, 2018 1:39 PM
To: panama-dev at openjdk.java.net
Subject: VectorAPI SKL Fix for ashiftR

Hi All,
I would like to contribute a fix for arithmetic right shift for SKL.

Could you please review the patch here:

http://cr.openjdk.java.net/~srukmannagar/VectorAPI_SKLFixes/webrev_ashiftR.00/



Thanks,
Shravya.



From shravya.rukmannagari at intel.com  Tue Sep  4 20:45:18 2018
From: shravya.rukmannagari at intel.com (shravya.rukmannagari at intel.com)
Date: Tue, 04 Sep 2018 20:45:18 +0000
Subject: hg: panama/dev: Arithmetic Right Shift Fix for SKL
Message-ID: <201809042045.w84KjIRR015478@aojmv0008.oracle.com>

Changeset: 77c09ee06154
Author:    srukmannagar
Date:      2018-09-04 13:36 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/77c09ee06154

Arithmetic Right Shift Fix for SKL

! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/stubRoutines_x86.hpp
! src/hotspot/cpu/x86/x86.ad


From maurizio.cimadamore at oracle.com  Tue Sep  4 20:51:47 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Tue, 04 Sep 2018 20:51:47 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809042051.w84KpmYC017564@aojmv0008.oracle.com>

Changeset: 26b0d25e0cd5
Author:    mcimadamore
Date:      2018-09-04 22:54 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/26b0d25e0cd5

Automatic merge with vectorIntrinsics



From razvan.a.lupusoru at intel.com  Tue Sep  4 23:21:56 2018
From: razvan.a.lupusoru at intel.com (Lupusoru, Razvan A)
Date: Tue, 4 Sep 2018 23:21:56 +0000
Subject: Vector API Multiple Fixes
Message-ID: <48D92A70936F7946BE99F3FF5ECB6461F6683283@ORSMSX105.amr.corp.intel.com>

Hi everyone,

I would like to contribute a couple of improvements, fixes and cleanups for Vector API:

-          Tests for rearrange

-          Test improvement for "get" to exercise functionality

-          Additional verbosity for SVML testing results

-          Fix for extract encoding

-          Fix for cast byte to double in AVX512

-          Fix byte 512 max all intrinsic

-          Fix for get .ad file predicates + fix for 16f, 4d, and 8d extract implementation

-          Fix byte reduction incorrect encoding

-          Few misspelling fixes and removal of redundant entries

-          Workaround around SVML k1 clobbering

Please check out the patch here:
http://cr.openjdk.java.net/~rlupusoru/panama/webrev_fixesandcleanup1_01/

We are close to getting 100% test pass rate on SKL and SKX. Thanks!

--Razvan

From maurizio.cimadamore at oracle.com  Wed Sep  5 20:02:19 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:02:19 +0000
Subject: hg: panama/dev: 70 new changesets
Message-ID: <201809052002.w85K2Pjs027390@aojmv0008.oracle.com>

Changeset: d7fc38d3fc8d
Author:    mikael
Date:      2018-08-29 13:04 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/d7fc38d3fc8d

8209856: Obsolete error reporter
Reviewed-by: coleenp, stuefe

! make/nb_native/nbproject/configurations.xml
! src/hotspot/share/runtime/arguments.cpp
! src/hotspot/share/runtime/globals.hpp
- src/hotspot/share/utilities/errorReporter.cpp
- src/hotspot/share/utilities/errorReporter.hpp
! src/hotspot/share/utilities/vmError.cpp
! test/hotspot/jtreg/compiler/ciReplay/CiReplayBase.java
! test/hotspot/jtreg/runtime/ErrorHandling/CreateCoredumpOnCrash.java
! test/hotspot/jtreg/runtime/ErrorHandling/ErrorHandler.java
! test/hotspot/jtreg/runtime/ErrorHandling/ProblematicFrameTest.java
! test/hotspot/jtreg/runtime/ErrorHandling/TestOnError.java
! test/hotspot/jtreg/runtime/Safepoint/AssertSafepointCheckConsistency1.java
! test/hotspot/jtreg/runtime/Safepoint/AssertSafepointCheckConsistency2.java
! test/hotspot/jtreg/runtime/Safepoint/AssertSafepointCheckConsistency3.java
! test/hotspot/jtreg/runtime/Safepoint/AssertSafepointCheckConsistency4.java
! test/hotspot/jtreg/runtime/Unsafe/RangeCheck.java
! test/hotspot/jtreg/runtime/memory/ReserveMemory.java
! test/hotspot/jtreg/serviceability/sa/TestJmapCore.java
! test/jdk/jdk/jfr/jvm/TestDumpOnCrash.java

Changeset: 0cd55d573893
Author:    mikael
Date:      2018-08-29 13:50 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/0cd55d573893

8210167: ProblemList vmTestbase/nsk/jvmti/scenarios/allocation/AP10/ap10t001/TestDescription.java
Reviewed-by: coleenp

! test/hotspot/jtreg/ProblemList.txt

Changeset: 8cae49105cbc
Author:    jwilhelm
Date:      2018-08-30 03:49 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/8cae49105cbc

Added tag jdk-12+9 for changeset 31b159f30fb2

! .hgtags

Changeset: 18ca918b4ed9
Author:    ljiang
Date:      2018-08-29 19:14 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/18ca918b4ed9

8210153: localized currency symbol of VES
Reviewed-by: naoto

! src/jdk.localedata/share/classes/sun/util/resources/ext/CurrencyNames_es_VE.properties
! test/jdk/java/text/Format/NumberFormat/CurrencyFormat.java
! test/jdk/java/text/Format/NumberFormat/CurrencySymbols.properties

Changeset: ad2224d4f346
Author:    gadams
Date:      2018-08-28 08:06 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/ad2224d4f346

8170089: nsk/jdi/EventSet/resume/resume008: ERROR: suspendCounts don't match for : Common-Cleaner
Reviewed-by: cjplummer, sspitsyn

! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume001.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume001a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume002.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume002a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume003.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume003a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume004.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume004a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume005.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume005a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume006.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume006a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume007.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume007a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume008.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume008a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume009.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume009a.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume010.java
! test/hotspot/jtreg/vmTestbase/nsk/jdi/EventSet/resume/resume010a.java

Changeset: d395677d99f3
Author:    cjplummer
Date:      2018-08-29 20:13 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/d395677d99f3

8199811: com/sun/jdi/ProcessAttachTest.java fails intermittently: Remote thread failed for unknown reason
Summary: give attach listener thread a chance to finish starting
Reviewed-by: dholmes, gadams

! src/hotspot/os/solaris/attachListener_solaris.cpp
! src/hotspot/os/windows/attachListener_windows.cpp

Changeset: c25f6c562bf5
Author:    ihse
Date:      2018-08-30 08:15 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/c25f6c562bf5

8210150: Allow custom-hook.m4 to include files from CUSTOM_CONFIG_DIR
Reviewed-by: erikj

! make/autoconf/configure

Changeset: eaa569eeb856
Author:    ihse
Date:      2018-08-30 08:19 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/eaa569eeb856

8210160: Remove deprecated configure arguments
Reviewed-by: erikj

! make/autoconf/hotspot.m4
! make/autoconf/jdk-options.m4
! make/autoconf/jdk-version.m4
! make/autoconf/libraries.m4

Changeset: b459c731901b
Author:    eosterlund
Date:      2018-08-30 08:59 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b459c731901b

8210061: ZGC: Remove STW weak processor mode
Reviewed-by: pliden, kbarrett

! src/hotspot/share/gc/z/zRootsIterator.cpp
! src/hotspot/share/gc/z/zRootsIterator.hpp
! src/hotspot/share/gc/z/z_globals.hpp

Changeset: f36e08f052b1
Author:    eosterlund
Date:      2018-08-30 09:12 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f36e08f052b1

8210063: ZGC: Enable load barriers for IN_NATIVE runtime barriers
Reviewed-by: pliden

! src/hotspot/share/gc/z/zBarrierSet.inline.hpp

Changeset: 9aa7ac61e68c
Author:    mbaesken
Date:      2018-08-29 10:11 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/9aa7ac61e68c

8210147: adjust some WSAGetLastError usages in windows network coding
Reviewed-by: clanger, stuefe

! src/java.base/windows/native/libnet/Inet4AddressImpl.c
! src/java.base/windows/native/libnet/Inet6AddressImpl.c
! src/java.base/windows/native/libnet/SocketInputStream.c

Changeset: 3aaf039a3636
Author:    eosterlund
Date:      2018-08-30 09:25 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/3aaf039a3636

8210065: ZGC: Remove mode for treating weaks as strong
Reviewed-by: kbarrett, pliden

! src/hotspot/share/gc/z/zRootsIterator.cpp
! src/hotspot/share/gc/z/zRootsIterator.hpp
! src/hotspot/share/gc/z/z_globals.hpp

Changeset: 9183040e34d8
Author:    lucy
Date:      2018-08-30 09:34 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/9183040e34d8

8209950: SIGBUS in CodeHeapState::print_names()
Reviewed-by: thartmann, kvn

! src/hotspot/share/code/codeHeapState.cpp
! src/hotspot/share/code/compiledMethod.cpp
! src/hotspot/share/code/compiledMethod.hpp
! src/hotspot/share/runtime/sharedRuntime.cpp

Changeset: 1ddd1ec04431
Author:    hseigel
Date:      2018-08-30 09:08 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/1ddd1ec04431

8210168: JCK test .vm.classfmt.ins.code__002.code__00201m1.code__00201m1 hangs with -noverify
Summary: Check for 'bc_length > 0' to handle lengths of -1.
Reviewed-by: coleenp

! src/hotspot/share/interpreter/rewriter.cpp

Changeset: e6250a870739
Author:    iklam
Date:      2018-08-30 08:01 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e6250a870739

8210194: [TESTBUG] jvmti_FollowRefObjects.cpp missing initializer for member _jvmtiHeapCallbacks::heap_reference_callback
Reviewed-by: sspitsyn, iignatyev

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref001/followref001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref002/followref002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref003/followref003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_FollowRefObjects.cpp

Changeset: dc79850e0254
Author:    jcbeyler
Date:      2018-08-30 09:47 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/dc79850e0254

8203356: VM Object Allocation Collector can infinite recurse
Summary: VM Event callback do not provoke a VM alloc event
Reviewed-by: sspitsyn, phh, amenkov, cjplummer

! make/nb_native/nbproject/configurations.xml
! src/hotspot/share/prims/jvmtiExport.cpp
+ test/hotspot/jtreg/serviceability/jvmti/VMEvent/MyPackage/VMEventRecursionTest.java
+ test/hotspot/jtreg/serviceability/jvmti/VMEvent/libVMEventTest.c

Changeset: 02572bed95b6
Author:    akolarkunnu
Date:      2018-08-30 03:01 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/02572bed95b6

8209993: Create a test for SwingSet3 ToolTipDemo
Reviewed-by: serb
Contributed-by: abdul.kolarkunnu at oracle.com

+ test/jdk/sanity/client/SwingSet/src/ToolTipDemoTest.java
+ test/jdk/sanity/client/lib/SwingSet3/src/com/sun/swingset3/demos/tooltip/ToolTipDemo.java
+ test/jdk/sanity/client/lib/SwingSet3/src/com/sun/swingset3/demos/tooltip/resources/ToolTipDemo.properties
+ test/jdk/sanity/client/lib/SwingSet3/src/com/sun/swingset3/demos/tooltip/resources/images/ToolTipDemo.gif
+ test/jdk/sanity/client/lib/SwingSet3/src/com/sun/swingset3/demos/tooltip/resources/images/tooltip_background.png

Changeset: b8eea2a7569a
Author:    amenkov
Date:      2018-08-30 11:53 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b8eea2a7569a

8209604: [TEST] rewrite com/sun/jdi shell tests to java version - step2
Reviewed-by: jcbeyler, sspitsyn, cjplummer

! test/jdk/com/sun/jdi/ArrayLengthDumpTest.java
! test/jdk/com/sun/jdi/BreakpointWithFullGC.java
+ test/jdk/com/sun/jdi/CatchAllTest.java
- test/jdk/com/sun/jdi/CatchAllTest.sh
+ test/jdk/com/sun/jdi/CatchCaughtTest.java
- test/jdk/com/sun/jdi/CatchCaughtTest.sh
+ test/jdk/com/sun/jdi/CommandCommentDelimiter.java
- test/jdk/com/sun/jdi/CommandCommentDelimiter.sh
+ test/jdk/com/sun/jdi/DeoptimizeWalk.java
- test/jdk/com/sun/jdi/DeoptimizeWalk.sh
+ test/jdk/com/sun/jdi/EvalArgs.java
- test/jdk/com/sun/jdi/EvalArgs.sh
+ test/jdk/com/sun/jdi/EvalArraysAsList.java
- test/jdk/com/sun/jdi/EvalArraysAsList.sh
+ test/jdk/com/sun/jdi/EvalInterfaceStatic.java
- test/jdk/com/sun/jdi/EvalInterfaceStatic.sh
+ test/jdk/com/sun/jdi/GetLocalVariables3Test.java
- test/jdk/com/sun/jdi/GetLocalVariables3Test.sh
! test/jdk/com/sun/jdi/JdbExprTest.java
! test/jdk/com/sun/jdi/JdbMethodExitTest.java
! test/jdk/com/sun/jdi/lib/jdb/Jdb.java
! test/jdk/com/sun/jdi/lib/jdb/JdbCommand.java
! test/jdk/com/sun/jdi/lib/jdb/JdbTest.java

Changeset: 4c78f4fd8370
Author:    bpb
Date:      2018-08-30 12:39 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/4c78f4fd8370

8207744: Clean up inconsistent use of opendir/closedir versus opendir64/closedir64
Reviewed-by: bsrbnd, mbaesken, bchristi, simonis

! src/java.base/share/native/libjli/wildcard.c
! src/java.base/unix/native/libjava/ProcessHandleImpl_unix.c
! src/java.base/unix/native/libjava/TimeZone_md.c
! src/java.base/unix/native/libjava/UnixFileSystem_md.c
! src/java.base/unix/native/libjava/childproc.c
! src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c
! src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c

Changeset: c88019b32bc4
Author:    kbarrett
Date:      2018-08-30 16:16 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/c88019b32bc4

8210119: Rename SubTasksDone::is_task_claimed
Summary: Renamed to try_claim_task and inverted result.
Reviewed-by: coleenp, sjohanss

! src/hotspot/share/gc/cms/cmsCardTable.cpp
! src/hotspot/share/gc/cms/cmsHeap.cpp
! src/hotspot/share/gc/cms/concurrentMarkSweepGeneration.cpp
! src/hotspot/share/gc/g1/g1RootProcessor.cpp
! src/hotspot/share/gc/shared/genCollectedHeap.cpp
! src/hotspot/share/gc/shared/preservedMarks.cpp
! src/hotspot/share/gc/shared/weakProcessor.inline.hpp
! src/hotspot/share/gc/shared/workgroup.cpp
! src/hotspot/share/gc/shared/workgroup.hpp
! src/hotspot/share/runtime/safepoint.cpp

Changeset: 3198179d97fa
Author:    avoitylov
Date:      2018-08-30 16:33 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/3198179d97fa

8210164: building Minimal VM fails with error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
Summary: Conditionalize compare with serial_phase_count.
Reviewed-by: kbarrett, shade

! src/hotspot/share/gc/shared/weakProcessorPhases.cpp

Changeset: 56309b1b9d9b
Author:    kbarrett
Date:      2018-08-30 17:03 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/56309b1b9d9b

8209975: Some GCThreadLocalData not initialized
Summary: Perform deferred BarrierSet initializations for NonJavaThreads too.
Reviewed-by: eosterlund, pliden

! src/hotspot/share/gc/shared/barrierSet.cpp
! src/hotspot/share/runtime/thread.cpp
! src/hotspot/share/runtime/thread.hpp

Changeset: e469480420dc
Author:    cjplummer
Date:      2018-08-30 17:59 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e469480420dc

8210118: better jdb test diagnostics when getting "Prompt is not received during ... milliseconds" failures
Summary: print pending reply when prompt is not recieved
Reviewed-by: dholmes, gadams, sspitsyn, jcbeyler

! test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java

Changeset: dbb0e798deeb
Author:    mli
Date:      2018-08-31 10:00 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/dbb0e798deeb

8208280: java/nio/channels/Selector/RegisterDuringSelect.java fails with "key not removed from key set"
Reviewed-by: alanb

! test/jdk/java/nio/channels/Selector/RegisterDuringSelect.java
! test/jdk/java/nio/channels/Selector/SelectAndClose.java
+ test/jdk/java/nio/channels/Selector/SelectorUtils.java

Changeset: d0b71f6163e1
Author:    jjiang
Date:      2018-08-31 10:32 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/d0b71f6163e1

8209362: sun/security/ssl/SSLSocketImpl/ReuseAddr.java failed due to "BindException: Address already in use (Bind failed)"
Summary: Refactor this test with SSLSocketTemplate
Reviewed-by: xuelei

! test/jdk/javax/net/ssl/templates/SSLSocketTemplate.java
! test/jdk/sun/security/ssl/SSLSocketImpl/ReuseAddr.java

Changeset: 72291a181f8d
Author:    mbaesken
Date:      2018-08-30 13:12 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/72291a181f8d

8210205: build fails on AIX in hotspot cpp tests (for example getstacktr001.cpp)
Reviewed-by: mdoerr, stuefe

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/getstacktr001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr003/getstacktr003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr004/getstacktr004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr005/getstacktr005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr006/getstacktr006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr007/getstacktr007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr008/getstacktr008.cpp

Changeset: 2ca553d34949
Author:    ihse
Date:      2018-08-31 09:37 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/2ca553d34949

6657100: Rename sparcWorks to solstudio in HotSpot
Reviewed-by: erikj, dcubed

! make/autoconf/toolchain.m4
! make/nb_native/nbproject/configurations.xml
! src/hotspot/share/interpreter/bytecodeHistogram.hpp
! src/hotspot/share/runtime/vmStructs.hpp
! src/hotspot/share/utilities/count_trailing_zeros.hpp
+ src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
- src/hotspot/share/utilities/globalDefinitions_sparcWorks.hpp
! src/java.base/solaris/native/libjvm_db/libjvm_db.c
! src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/types/basic/BasicTypeDataBase.java

Changeset: 18afb2097ada
Author:    dnsimon
Date:      2018-08-31 11:43 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/18afb2097ada

8210066: [JVMCI] iterateFrames uses wrong GrowableArray API for appending
Reviewed-by: dlong, twisti

! src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
! test/hotspot/jtreg/compiler/jvmci/compilerToVM/MaterializeVirtualObjectTest.java

Changeset: 5eb48e9d607a
Author:    hannesw
Date:      2018-08-31 12:41 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/5eb48e9d607a

8176453: Javadoc search: there are issues with generics in parameters
Reviewed-by: jjg, sundar

! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/search.js

Changeset: 625a5bdde0c5
Author:    coleenp
Date:      2018-08-31 07:03 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/625a5bdde0c5

8210155: Lock ClassLoaderDataGraph
Summary: In preparation for concurrent class unloading.
Reviewed-by: hseigel, eosterlund

! src/hotspot/share/classfile/classLoaderData.cpp
! src/hotspot/share/classfile/classLoaderData.hpp
! src/hotspot/share/classfile/systemDictionary.cpp
! src/hotspot/share/gc/cms/concurrentMarkSweepGeneration.cpp
! src/hotspot/share/jfr/periodic/jfrModuleEvent.cpp
! src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.cpp
! src/hotspot/share/jfr/recorder/repository/jfrEmergencyDump.cpp
! src/hotspot/share/memory/heapInspection.cpp
! src/hotspot/share/memory/heapInspection.hpp
! src/hotspot/share/memory/metaspaceShared.cpp
! src/hotspot/share/memory/universe.cpp
! src/hotspot/share/oops/klassVtable.cpp
! src/hotspot/share/prims/jvmtiEnvBase.cpp
! src/hotspot/share/prims/jvmtiGetLoadedClasses.cpp
! src/hotspot/share/prims/whitebox.cpp
! src/hotspot/share/runtime/java.cpp
! src/hotspot/share/runtime/mutexLocker.cpp
! src/hotspot/share/runtime/mutexLocker.hpp
! src/hotspot/share/services/heapDumper.cpp

Changeset: 48a95b70d4af
Author:    coleenp
Date:      2018-08-31 09:10 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/48a95b70d4af

8207793: [TESTBUG] runtime/Metaspace/FragmentMetaspace.java fails: heap needs to be increased
Summary: Reduce test time and allow OOM.
Reviewed-by: iklam, hseigel

! test/hotspot/jtreg/runtime/Metaspace/FragmentMetaspace.java
! test/hotspot/jtreg/runtime/testlibrary/GeneratedClassLoader.java

Changeset: cdef4df6b0e7
Author:    pchilanomate
Date:      2018-08-31 10:22 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/cdef4df6b0e7

8206424: Use locking for cleaning ProtectionDomainTable
Summary: ServiceThread is now in charge of cleaning ProtectionDomainTable entries
Reviewed-by: coleenp, iklam

! src/hotspot/share/classfile/protectionDomainCache.cpp
! src/hotspot/share/classfile/protectionDomainCache.hpp
! src/hotspot/share/classfile/systemDictionary.cpp
! src/hotspot/share/classfile/systemDictionary.hpp
! src/hotspot/share/prims/whitebox.cpp
! src/hotspot/share/runtime/serviceThread.cpp
+ test/hotspot/jtreg/runtime/Dictionary/CleanProtectionDomain.java
! test/lib/sun/hotspot/WhiteBox.java

Changeset: 2ef81feac8b8
Author:    amenkov
Date:      2018-08-31 09:53 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/2ef81feac8b8

8067354: com/sun/jdi/GetLocalVariables4Test.sh failed
Reviewed-by: jcbeyler, sspitsyn

! test/jdk/ProblemList.txt
+ test/jdk/com/sun/jdi/GetLocalVariables4Test.java
- test/jdk/com/sun/jdi/GetLocalVariables4Test.sh

Changeset: bdac20c6c8dd
Author:    mcimadamore
Date:      2018-08-31 18:01 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/bdac20c6c8dd

8210226: Add support for multiple project folders to idea.sh
Summary: Overhaul templating logic for idea.sh; add support for -o option
Reviewed-by: erikj, ihse

! bin/idea.sh
! make/idea/template/ant.xml
! make/idea/template/compiler.xml
! make/idea/template/jdk.iml
! make/idea/template/misc.xml
! make/idea/template/vcs.xml
! make/idea/template/workspace.xml
! make/langtools/intellij/build.xml
! make/langtools/intellij/template/ant.xml
! make/langtools/intellij/template/misc.xml

Changeset: 3835dc32de5e
Author:    dtitov
Date:      2018-08-31 11:56 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/3835dc32de5e

8209585: [Graal] vmTestbase/nsk/jvmti/scenarios/sampling tests fail with "Too small stack of resumed thread"
Reviewed-by: sspitsyn, amenkov, cjplummer, jcbeyler

! test/hotspot/jtreg/ProblemList-graal.txt
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP02/sp02t001/TestDescription.java
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP02/sp02t001/sp02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP02/sp02t002/TestDescription.java
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP02/sp02t002/sp02t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t001/TestDescription.java
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t001/sp06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t002/TestDescription.java
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t002/sp06t002.cpp

Changeset: 36773a4fe3e7
Author:    kbarrett
Date:      2018-08-31 16:29 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/36773a4fe3e7

8210235: JvmtiTrace::safe_get_current_thread_name is unsafe in debug builds
Summary: Use Thread::current_or_null and handle NULL result.
Reviewed-by: coleenp

! src/hotspot/share/prims/jvmtiTrace.cpp

Changeset: afbb33428df7
Author:    jjg
Date:      2018-08-31 14:54 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/afbb33428df7

8208608: Update --module-source-path to allow explicit source paths for specific modules
Reviewed-by: jlahoda

! src/jdk.compiler/share/classes/com/sun/tools/javac/file/Locations.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/main/Option.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties
! test/langtools/tools/javac/diags/examples.not-yet.txt
! test/langtools/tools/javac/file/SetLocationForModule.java
! test/langtools/tools/javac/modules/ModuleSourcePathTest.java
! test/langtools/tools/javac/modules/PatchModulesTest.java

Changeset: b071f4fff1f1
Author:    goetz
Date:      2018-09-01 18:15 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b071f4fff1f1

8210259: [testbug] IncompatibleOptions.java fails if VM configured without ZGC
Reviewed-by: pliden, kbarrett

! test/hotspot/jtreg/runtime/appcds/sharedStrings/IncompatibleOptions.java
! test/lib/sun/hotspot/gc/GC.java

Changeset: 9720ad0a40b6
Author:    iklam
Date:      2018-09-01 12:02 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/9720ad0a40b6

8210246: NMTUtil::_memory_type_names should be in sync with MemoryType
Reviewed-by: ccheung, jiangli, coleenp

! src/hotspot/share/memory/allocation.hpp
! src/hotspot/share/services/nmtCommon.cpp

Changeset: 54b344d9dd4e
Author:    lucy
Date:      2018-09-03 09:43 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/54b344d9dd4e

8207343: Automate vtable/itable stub size calculation
Reviewed-by: kvn, mdoerr

! src/hotspot/cpu/aarch64/vtableStubs_aarch64.cpp
! src/hotspot/cpu/arm/vtableStubs_arm.cpp
! src/hotspot/cpu/ppc/vtableStubs_ppc_64.cpp
! src/hotspot/cpu/s390/vtableStubs_s390.cpp
! src/hotspot/cpu/sparc/vtableStubs_sparc.cpp
! src/hotspot/cpu/x86/vtableStubs_x86_32.cpp
! src/hotspot/cpu/x86/vtableStubs_x86_64.cpp
! src/hotspot/share/code/vtableStubs.cpp
! src/hotspot/share/code/vtableStubs.hpp
! src/hotspot/share/logging/logTag.hpp

Changeset: dca697c71e5d
Author:    avoitylov
Date:      2018-09-03 13:39 +0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/dca697c71e5d

8207247: AARCH64: Enable Minimal and Client VM builds
Reviewed-by: aph

! src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
! src/hotspot/cpu/aarch64/stubGenerator_aarch64.cpp
! src/hotspot/cpu/aarch64/vm_version_aarch64.cpp

Changeset: 4871c2d2e97e
Author:    bulasevich
Date:      2018-09-03 13:42 +0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/4871c2d2e97e

8209408: Primitive heap access for interpreter BarrierSetAssembler/arm32
Reviewed-by: rkennke

! src/hotspot/cpu/arm/gc/shared/barrierSetAssembler_arm.cpp
! src/hotspot/cpu/arm/interp_masm_arm.cpp
! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/arm/templateTable_arm.cpp
! src/hotspot/cpu/arm/templateTable_arm.hpp

Changeset: b0c81cfd4dc9
Author:    stuefe
Date:      2018-09-03 14:27 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b0c81cfd4dc9

8210314: [aix] NMT does not show "Safepoint" memory type
Reviewed-by: goetz, mbaesken

! src/hotspot/os/aix/safepointMechanism_aix.cpp

Changeset: de1a82a239e2
Author:    stuefe
Date:      2018-09-04 08:06 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/de1a82a239e2

8210307: 8210246 broke NMT jtreg tests
Reviewed-by: goetz, iklam

! src/hotspot/share/services/nmtCommon.cpp

Changeset: 3ee917225506
Author:    weijun
Date:      2018-09-04 14:47 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/3ee917225506

8210338: Better output for GenerationTests.java
Reviewed-by: xuelei

! test/jdk/javax/xml/crypto/dsig/GenerationTests.java

Changeset: 7ed777a14094
Author:    tschatzl
Date:      2018-09-04 12:17 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/7ed777a14094

8210265: Crash in HSpaceCounters::update_used()
Summary: Guard call to update HSpaceCounters with flag
Reviewed-by: shade, sjohanss, kbarrett

! src/hotspot/share/gc/g1/g1MonitoringSupport.cpp
+ test/hotspot/jtreg/gc/TestNoPerfCounter.java

Changeset: c265860d5d45
Author:    tschatzl
Date:      2018-09-04 12:18 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/c265860d5d45

8207200: Committed > max memory usage when getting MemoryUsage
Summary: Make sure that modification of memory usage variables are synchronized with returning them to Java.
Reviewed-by: sangheki, mchung

! src/hotspot/share/gc/g1/g1CollectedHeap.cpp
! src/hotspot/share/gc/g1/g1CollectedHeap.hpp
! src/hotspot/share/gc/g1/g1MemoryPool.cpp
! src/hotspot/share/gc/g1/g1MonitoringSupport.cpp
! src/hotspot/share/gc/g1/g1MonitoringSupport.hpp
! src/hotspot/share/gc/shared/collectedHeap.cpp
! src/hotspot/share/gc/shared/collectedHeap.hpp
! src/hotspot/share/runtime/mutexLocker.cpp
! src/hotspot/share/runtime/mutexLocker.hpp
! src/hotspot/share/services/management.cpp

Changeset: 0514d4c30cf1
Author:    alanb
Date:      2018-09-04 11:35 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/0514d4c30cf1

8210341: (fs) Typos in PosixFileAttributeView javadoc
Reviewed-by: dfuchs

! src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java

Changeset: d5ba88422499
Author:    shade
Date:      2018-09-04 13:19 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d5ba88422499

8210355: Minimal and Zero non-PCH builds fail after JDK-8207343 (Automate vtable/itable stub size calculation)
Reviewed-by: thartmann

! src/hotspot/share/code/vtableStubs.cpp

Changeset: 507c72580842
Author:    shade
Date:      2018-09-04 13:19 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/507c72580842

8210357: Zero builds fail after JDK-8207343 (Automate vtable/itable stub size calculation)
Reviewed-by: thartmann

! src/hotspot/cpu/zero/vtableStubs_zero.cpp

Changeset: 78c7f0c7827d
Author:    gromero
Date:      2018-09-04 11:46 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/78c7f0c7827d

8210320: PPC64: Fix uninitialized variable in C1 LIR assembler code
Reviewed-by: mbaesken, shade, mdoerr

! src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp

Changeset: 25606a4e6693
Author:    bpb
Date:      2018-09-04 09:07 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/25606a4e6693

8210279: (bf) Remove unused package private method java.nio.Buffer.truncate()
Reviewed-by: alanb

! src/java.base/share/classes/java/nio/Buffer.java

Changeset: f9773a631272
Author:    alanb
Date:      2018-09-04 18:03 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/f9773a631272

8210087: Classes in jdk.unsupported not accessible from jconsole plugin
Reviewed-by: erikj

! make/launcher/Launcher-jdk.jconsole.gmk

Changeset: ed04bc1ff453
Author:    dfuchs
Date:      2018-09-04 18:32 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/ed04bc1ff453

8210366: Typo in MethodHandles.Lookup: must be either be
Reviewed-by: rriggs

! src/java.base/share/classes/java/lang/invoke/MethodHandles.java

Changeset: 21154cb84d2a
Author:    kvn
Date:      2018-09-04 12:44 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/21154cb84d2a

8209594: guarantee(this->is8bit(imm8)) failed: Short forward jump exceeds 8-bit offset
Summary: replace short jumps and fix other issues when generated code exceed expected range.
Reviewed-by: dlong, rasbold

! src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp
! src/hotspot/cpu/arm/macroAssembler_arm.hpp
! src/hotspot/cpu/arm/macroAssembler_arm.inline.hpp
! src/hotspot/cpu/ppc/macroAssembler_ppc.hpp
! src/hotspot/cpu/ppc/macroAssembler_ppc.inline.hpp
! src/hotspot/cpu/s390/macroAssembler_s390.cpp
! src/hotspot/cpu/s390/macroAssembler_s390.hpp
! src/hotspot/cpu/sparc/macroAssembler_sparc.hpp
! src/hotspot/cpu/sparc/macroAssembler_sparc.inline.hpp
! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp
! src/hotspot/cpu/x86/templateTable_x86.cpp
! src/hotspot/cpu/x86/x86.ad
! src/hotspot/cpu/zero/assembler_zero.cpp
! src/hotspot/cpu/zero/assembler_zero.hpp
! src/hotspot/share/asm/assembler.cpp
! src/hotspot/share/asm/assembler.hpp
! src/hotspot/share/opto/compile.cpp

Changeset: 3f189f451ff1
Author:    jcbeyler
Date:      2018-08-31 22:55 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/3f189f451ff1

8210182: Remove macros for C compilation from vmTestBase but non jvmti
Summary: Remove the macros and update the code
Reviewed-by: sspitsyn, cjplummer, amenkov

! test/hotspot/jtreg/vmTestbase/gc/g1/unloading/libdefine.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jdi/ThreadReference/forceEarlyReturn/forceEarlyReturn004/libforceEarlyReturn004a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jdi/ThreadReference/forceEarlyReturn/forceEarlyReturn005/libforceEarlyReturn005a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jdwp/ThreadReference/ForceEarlyReturn/forceEarlyReturn002/libforceEarlyReturn002a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jdi/MonitorEnterExecutor.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jni/JNIreferences.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jpda/libNativeMethodsTestThread.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/locks/JNIMonitorLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/locks/LockingThread.cpp
! test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/shared/redefineClasses.cpp

Changeset: 6ad495a34ac8
Author:    jcbeyler
Date:      2018-09-04 14:17 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/6ad495a34ac8

8210192: Hsperf counter ParNew::CMS should be ParNew:CMS
Summary: Rename the counter back to ParNew:CMS and added a test
Reviewed-by: sjohanss, tschatzl
Contributed-by: manc at google.com

! src/hotspot/share/gc/cms/cmsHeap.cpp
+ test/hotspot/jtreg/gc/TestPolicyNamePerfCounter.java
! test/hotspot/jtreg/gc/testlibrary/PerfCounter.java

Changeset: ab722555e66d
Author:    ccheung
Date:      2018-09-04 15:00 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/ab722555e66d

8209736: runtime/RedefineTests/ModifyAnonymous.java fails with NullPointerException when running in CDS mode
Summary: add logging of class names in the allLoadedClasses array; throw RuntimeException upon encountering of a null class.
Reviewed-by: jiangli

! test/hotspot/jtreg/runtime/RedefineTests/ModifyAnonymous.java

Changeset: e9177e7749e7
Author:    iignatyev
Date:      2018-09-04 14:35 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e9177e7749e7

8210039: move OSInfo to top level testlibrary
Reviewed-by: serb, chegar, alanb, dfuchs

! test/jdk/com/oracle/security/ucrypto/TestAES.java
! test/jdk/com/sun/management/OperatingSystemMXBean/TestTotalSwap.java
! test/jdk/java/awt/Choice/ItemStateChangeTest/ItemStateChangeTest.java
! test/jdk/java/awt/Choice/PopupPosTest/PopupPosTest.html
! test/jdk/java/awt/Choice/PopupPosTest/PopupPosTest.java
! test/jdk/java/awt/Choice/ResizeAutoClosesChoice/ResizeAutoClosesChoice.java
! test/jdk/java/awt/Cursor/MultiResolutionCursorTest/MultiResolutionCursorTest.java
! test/jdk/java/awt/Desktop/8064934/bug8064934.java
! test/jdk/java/awt/FileDialog/8003399/bug8003399.java
! test/jdk/java/awt/FileDialog/8017487/bug8017487.java
! test/jdk/java/awt/FileDialog/FileDialogForDirectories/FileDialogForDirectories.html
! test/jdk/java/awt/FileDialog/FileDialogForDirectories/FileDialogForDirectories.java
! test/jdk/java/awt/FileDialog/FileDialogForPackages/FileDialogForPackages.html
! test/jdk/java/awt/FileDialog/FileDialogForPackages/FileDialogForPackages.java
! test/jdk/java/awt/Focus/MouseClickRequestFocusRaceTest/MouseClickRequestFocusRaceTest.java
! test/jdk/java/awt/Frame/MaximizedByPlatform/MaximizedByPlatform.java
! test/jdk/java/awt/KeyboardFocusmanager/ConsumeNextMnemonicKeyTypedTest/ConsumeNextMnemonicKeyTypedTest.html
! test/jdk/java/awt/KeyboardFocusmanager/ConsumeNextMnemonicKeyTypedTest/ConsumeNextMnemonicKeyTypedTest.java
! test/jdk/java/awt/KeyboardFocusmanager/TypeAhead/SubMenuShowTest/SubMenuShowTest.html
! test/jdk/java/awt/KeyboardFocusmanager/TypeAhead/SubMenuShowTest/SubMenuShowTest.java
! test/jdk/java/awt/List/FirstItemRemoveTest/FirstItemRemoveTest.html
! test/jdk/java/awt/List/FirstItemRemoveTest/FirstItemRemoveTest.java
! test/jdk/java/awt/List/KeyEventsTest/KeyEventsTest.html
! test/jdk/java/awt/List/KeyEventsTest/KeyEventsTest.java
! test/jdk/java/awt/Menu/OpensWithNoGrab/OpensWithNoGrab.java
! test/jdk/java/awt/MenuBar/8007006/bug8007006.java
! test/jdk/java/awt/MenuBar/MenuBarSetFont/MenuBarSetFont.java
! test/jdk/java/awt/Mouse/MouseModifiersUnitTest/MouseModifiersUnitTest_Extra.java
! test/jdk/java/awt/Multiscreen/MultiScreenInsetsTest/MultiScreenInsetsTest.java
! test/jdk/java/awt/Robot/RobotWheelTest/RobotWheelTest.java
! test/jdk/java/awt/TextArea/ScrollbarIntersectionTest/ScrollbarIntersectionTest.java
! test/jdk/java/awt/Toolkit/ToolkitPropertyTest/bug7129133.java
! test/jdk/java/awt/TrayIcon/8072769/bug8072769.java
! test/jdk/java/awt/TrayIcon/DblClickActionEventTest/DblClickActionEventTest.html
! test/jdk/java/awt/TrayIcon/DblClickActionEventTest/DblClickActionEventTest.java
! test/jdk/java/awt/TrayIcon/DisposeInActionEventTest/DisposeInActionEventTest.html
! test/jdk/java/awt/TrayIcon/DisposeInActionEventTest/DisposeInActionEventTest.java
! test/jdk/java/awt/datatransfer/HTMLDataFlavors/HTMLDataFlavorTest.java
! test/jdk/java/awt/datatransfer/MissedHtmlAndRtfBug/MissedHtmlAndRtfBug.html
! test/jdk/java/awt/datatransfer/MissedHtmlAndRtfBug/MissedHtmlAndRtfBug.java
! test/jdk/java/awt/dnd/ImageTransferTest/ImageTransferTest.java
! test/jdk/java/awt/event/KeyEvent/8020209/bug8020209.java
! test/jdk/java/awt/event/KeyEvent/DeadKey/DeadKeyMacOSXInputText.java
! test/jdk/java/awt/event/KeyEvent/DeadKey/deadKeyMacOSX.java
! test/jdk/java/awt/event/KeyEvent/SwallowKeyEvents/SwallowKeyEvents.java
! test/jdk/java/awt/image/MultiResolutionImage/NSImageToMultiResolutionImageTest.java
! test/jdk/java/awt/image/multiresolution/MultiDisplayTest/MultiDisplayTest.java
! test/jdk/javax/swing/JButton/4796987/bug4796987.java
! test/jdk/javax/swing/JCheckBox/4449413/bug4449413.java
! test/jdk/javax/swing/JCheckBox/8032667/bug8032667_image_diff.java
! test/jdk/javax/swing/JComboBox/4199622/bug4199622.java
! test/jdk/javax/swing/JFileChooser/4150029/bug4150029.html
! test/jdk/javax/swing/JFileChooser/4150029/bug4150029.java
! test/jdk/javax/swing/JFileChooser/4524490/bug4524490.java
! test/jdk/javax/swing/JFileChooser/6840086/bug6840086.java
! test/jdk/javax/swing/JFileChooser/8041694/bug8041694.java
! test/jdk/javax/swing/JFileChooser/8046391/bug8046391.java
! test/jdk/javax/swing/JFileChooser/8062561/bug8062561.java
! test/jdk/javax/swing/JFrame/8016356/bug8016356.java
! test/jdk/javax/swing/JFrame/NSTexturedJFrame/NSTexturedJFrame.java
! test/jdk/javax/swing/JInternalFrame/4251301/bug4251301.java
! test/jdk/javax/swing/JLabel/6596966/bug6596966.java
! test/jdk/javax/swing/JMenu/6470128/bug6470128.java
! test/jdk/javax/swing/JMenuItem/ActionListenerCalledTwice/ActionListenerCalledTwiceTest.java
! test/jdk/javax/swing/JMenuItem/ShortcutNotDiplayed/ShortcutNotDisplayedTest.java
! test/jdk/javax/swing/JOptionPane/8024926/bug8024926.java
! test/jdk/javax/swing/JPopupMenu/6827786/bug6827786.java
! test/jdk/javax/swing/JPopupMenu/7154841/bug7154841.java
! test/jdk/javax/swing/JScrollBar/bug4202954/bug4202954.java
! test/jdk/javax/swing/JScrollPane/HorizontalMouseWheelOnShiftPressed/HorizontalMouseWheelOnShiftPressed.java
! test/jdk/javax/swing/JSlider/6579827/bug6579827.java
! test/jdk/javax/swing/JTabbedPane/4624207/bug4624207.java
! test/jdk/javax/swing/JTabbedPane/6416920/bug6416920.java
! test/jdk/javax/swing/JTextArea/6940863/bug6940863.java
! test/jdk/javax/swing/UITest/UITest.java
! test/jdk/javax/swing/plaf/aqua/CustomComboBoxFocusTest.java
! test/jdk/javax/swing/plaf/basic/BasicLabelUI/bug7172652.java
! test/jdk/jdk/net/Sockets/Test.java
! test/jdk/jdk/net/Sockets/policy.fail
! test/jdk/jdk/net/Sockets/policy.success
- test/jdk/lib/testlibrary/jdk/testlibrary/OSInfo.java
! test/jdk/sun/awt/dnd/8024061/bug8024061.java
+ test/lib/jdk/test/lib/OSVersion.java
! test/lib/jdk/test/lib/Platform.java
! test/lib/jdk/test/lib/process/ProcessTools.java

Changeset: 2e4cf4ca074c
Author:    sherman
Date:      2018-09-04 17:04 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/2e4cf4ca074c

8197398: (zipfs) Files.walkFileTree walk indefinitelly while processing JAR file with "/" as a directory inside.
Reviewed-by: alanb

! src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java
! test/jdk/jdk/nio/zipfs/ZFSTests.java

Changeset: d7df80487e30
Author:    amlu
Date:      2018-09-05 12:34 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/d7df80487e30

8209832: Refactor jdk/internal/reflect/Reflection/GetCallerClassTest.sh to plain java test
Reviewed-by: alanb, mchung

! test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.java
- test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.sh
+ test/jdk/jdk/internal/reflect/Reflection/SetupGetCallerClass.java

Changeset: 58ea9afe0eb8
Author:    pmuthuswamy
Date:      2018-09-05 11:52 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/58ea9afe0eb8

8209052: Low contrast in docs/api/constant-values.html
Reviewed-by: jjg

! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/stylesheet.css

Changeset: 9ce4a0d718c7
Author:    glaubitz
Date:      2018-09-05 11:15 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/9ce4a0d718c7

8165440: Add Zero support for x86_64-linux-gnux32 target
Reviewed-by: erikj, ihse

! make/autoconf/flags.m4
! make/autoconf/platform.m4
! src/hotspot/os/linux/os_linux.cpp

Changeset: d4099c45f148
Author:    alanb
Date:      2018-09-05 11:52 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/d4099c45f148

8209152: (so) ServerSocketChannel::supportedOptions includes IP_TOS
Reviewed-by: chegar

! src/java.base/share/classes/sun/nio/ch/ServerSocketChannelImpl.java
+ test/jdk/java/nio/channels/etc/PrintSupportedOptions.java

Changeset: 8267d480566f
Author:    eosterlund
Date:      2018-09-05 10:11 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/8267d480566f

8210158: Accessorize JFR getEventWriter() intrinsics
Reviewed-by: kvn, neliasso, roland, rbackman

! src/hotspot/share/c1/c1_LIRGenerator.cpp
! src/hotspot/share/opto/library_call.cpp

Changeset: 0df09dadd445
Author:    ihse
Date:      2018-09-05 14:00 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/0df09dadd445

8182733: aarch64 build documentation misleading
Reviewed-by: shade, dholmes

! doc/building.html
! doc/building.md
! doc/testing.html

Changeset: c5f700de5450
Author:    mbaesken
Date:      2018-09-05 12:48 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/c5f700de5450

8209942: [epsilon] range function for EpsilonTLABElasticity causes compiler warning
Reviewed-by: shade, simonis
Contributed-by: ralf.schmelter at sap.com

! src/hotspot/share/gc/epsilon/epsilon_globals.hpp

Changeset: 364863505019
Author:    sherman
Date:      2018-09-05 09:12 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/364863505019

8210394: (zipfs) jdk/nio/zipfs/ZFSTests.java rootdir.zip: The process cannot access the file because it is being used by another process
Reviewed-by: alanb, jlaskey

! test/jdk/jdk/nio/zipfs/ZFSTests.java

Changeset: 11ad7d302a94
Author:    jcbeyler
Date:      2018-09-05 10:17 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/11ad7d302a94

8208186: SetHeapSamplingInterval handles 1 explicitly
Summary: Explicitly test for the 0 case (sample everything)
Reviewed-by: amenkov, sspitsyn

! src/hotspot/share/runtime/threadHeapSampler.cpp

Changeset: b7b69bc93a2d
Author:    amenkov
Date:      2018-09-05 10:39 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b7b69bc93a2d

8210243: [TEST] rewrite com/sun/jdi shell tests to java version - step3
Reviewed-by: jcbeyler, cjplummer, sspitsyn

! test/jdk/com/sun/jdi/BreakpointWithFullGC.java
! test/jdk/com/sun/jdi/DeoptimizeWalk.java
+ test/jdk/com/sun/jdi/JdbArgTest.java
- test/jdk/com/sun/jdi/JdbArgTest.sh
+ test/jdk/com/sun/jdi/JdbLockTest.java
- test/jdk/com/sun/jdi/JdbLockTest.sh
+ test/jdk/com/sun/jdi/JdbMissStep.java
- test/jdk/com/sun/jdi/JdbMissStep.sh
+ test/jdk/com/sun/jdi/JdbVarargsTest.java
- test/jdk/com/sun/jdi/JdbVarargsTest.sh
+ test/jdk/com/sun/jdi/MixedSuspendTest.java
- test/jdk/com/sun/jdi/MixedSuspendTest.sh
+ test/jdk/com/sun/jdi/NotAField.java
- test/jdk/com/sun/jdi/NotAField.sh
+ test/jdk/com/sun/jdi/NullLocalVariable.java
- test/jdk/com/sun/jdi/NullLocalVariable.sh
! test/jdk/com/sun/jdi/lib/jdb/Jdb.java
! test/jdk/com/sun/jdi/lib/jdb/JdbCommand.java
! test/jdk/com/sun/jdi/lib/jdb/JdbTest.java


From maurizio.cimadamore at oracle.com  Wed Sep  5 20:06:15 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:06:15 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809052006.w85K6FEU028561@aojmv0008.oracle.com>

Changeset: 4b5a4549bfc2
Author:    mcimadamore
Date:      2018-09-05 22:08 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/4b5a4549bfc2

Automatic merge with default

! bin/idea.sh
! make/autoconf/flags.m4
! make/autoconf/libraries.m4
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/share/logging/logTag.hpp
- src/hotspot/share/utilities/errorReporter.cpp
- src/hotspot/share/utilities/errorReporter.hpp
- src/hotspot/share/utilities/globalDefinitions_sparcWorks.hpp
! src/java.base/share/classes/java/nio/Buffer.java
- test/jdk/com/sun/jdi/CatchAllTest.sh
- test/jdk/com/sun/jdi/CatchCaughtTest.sh
- test/jdk/com/sun/jdi/CommandCommentDelimiter.sh
- test/jdk/com/sun/jdi/DeoptimizeWalk.sh
- test/jdk/com/sun/jdi/EvalArgs.sh
- test/jdk/com/sun/jdi/EvalArraysAsList.sh
- test/jdk/com/sun/jdi/EvalInterfaceStatic.sh
- test/jdk/com/sun/jdi/GetLocalVariables3Test.sh
- test/jdk/com/sun/jdi/GetLocalVariables4Test.sh
- test/jdk/com/sun/jdi/JdbArgTest.sh
- test/jdk/com/sun/jdi/JdbLockTest.sh
- test/jdk/com/sun/jdi/JdbMissStep.sh
- test/jdk/com/sun/jdi/JdbVarargsTest.sh
- test/jdk/com/sun/jdi/MixedSuspendTest.sh
- test/jdk/com/sun/jdi/NotAField.sh
- test/jdk/com/sun/jdi/NullLocalVariable.sh
- test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/OSInfo.java


From maurizio.cimadamore at oracle.com  Wed Sep  5 20:06:37 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:06:37 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809052006.w85K6bJ2028897@aojmv0008.oracle.com>

Changeset: d05ba95da2e0
Author:    mcimadamore
Date:      2018-09-05 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d05ba95da2e0

Automatic merge with default

! make/autoconf/toolchain.m4
! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/cpu/x86/x86.ad
! src/hotspot/share/opto/compile.cpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/runtime/globals.hpp
- src/hotspot/share/utilities/errorReporter.cpp
- src/hotspot/share/utilities/errorReporter.hpp
- src/hotspot/share/utilities/globalDefinitions_sparcWorks.hpp
- test/jdk/com/sun/jdi/CatchAllTest.sh
- test/jdk/com/sun/jdi/CatchCaughtTest.sh
- test/jdk/com/sun/jdi/CommandCommentDelimiter.sh
- test/jdk/com/sun/jdi/DeoptimizeWalk.sh
- test/jdk/com/sun/jdi/EvalArgs.sh
- test/jdk/com/sun/jdi/EvalArraysAsList.sh
- test/jdk/com/sun/jdi/EvalInterfaceStatic.sh
- test/jdk/com/sun/jdi/GetLocalVariables3Test.sh
- test/jdk/com/sun/jdi/GetLocalVariables4Test.sh
- test/jdk/com/sun/jdi/JdbArgTest.sh
- test/jdk/com/sun/jdi/JdbLockTest.sh
- test/jdk/com/sun/jdi/JdbMissStep.sh
- test/jdk/com/sun/jdi/JdbVarargsTest.sh
- test/jdk/com/sun/jdi/MixedSuspendTest.sh
- test/jdk/com/sun/jdi/NotAField.sh
- test/jdk/com/sun/jdi/NullLocalVariable.sh
- test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/OSInfo.java


From maurizio.cimadamore at oracle.com  Wed Sep  5 20:06:58 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:06:58 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809052006.w85K6wpl029199@aojmv0008.oracle.com>

Changeset: 61adc4177efb
Author:    mcimadamore
Date:      2018-09-05 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/61adc4177efb

Automatic merge with default

- src/hotspot/share/utilities/errorReporter.cpp
- src/hotspot/share/utilities/errorReporter.hpp
- src/hotspot/share/utilities/globalDefinitions_sparcWorks.hpp
- test/jdk/com/sun/jdi/CatchAllTest.sh
- test/jdk/com/sun/jdi/CatchCaughtTest.sh
- test/jdk/com/sun/jdi/CommandCommentDelimiter.sh
- test/jdk/com/sun/jdi/DeoptimizeWalk.sh
- test/jdk/com/sun/jdi/EvalArgs.sh
- test/jdk/com/sun/jdi/EvalArraysAsList.sh
- test/jdk/com/sun/jdi/EvalInterfaceStatic.sh
- test/jdk/com/sun/jdi/GetLocalVariables3Test.sh
- test/jdk/com/sun/jdi/GetLocalVariables4Test.sh
- test/jdk/com/sun/jdi/JdbArgTest.sh
- test/jdk/com/sun/jdi/JdbLockTest.sh
- test/jdk/com/sun/jdi/JdbMissStep.sh
- test/jdk/com/sun/jdi/JdbVarargsTest.sh
- test/jdk/com/sun/jdi/MixedSuspendTest.sh
- test/jdk/com/sun/jdi/NotAField.sh
- test/jdk/com/sun/jdi/NullLocalVariable.sh
- test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/OSInfo.java


From maurizio.cimadamore at oracle.com  Wed Sep  5 20:07:20 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:07:20 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809052007.w85K7LO5029603@aojmv0008.oracle.com>

Changeset: 305869aa35d0
Author:    mcimadamore
Date:      2018-09-05 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/305869aa35d0

Automatic merge with default

! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/cpu/x86/x86.ad
! src/hotspot/share/classfile/systemDictionary.cpp
! src/hotspot/share/classfile/systemDictionary.hpp
! src/hotspot/share/logging/logTag.hpp
! src/hotspot/share/opto/compile.cpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/runtime/globals.hpp
! src/hotspot/share/runtime/mutexLocker.cpp
! src/hotspot/share/runtime/mutexLocker.hpp
- src/hotspot/share/utilities/errorReporter.cpp
- src/hotspot/share/utilities/errorReporter.hpp
- src/hotspot/share/utilities/globalDefinitions_sparcWorks.hpp
! src/java.base/share/classes/java/lang/invoke/MethodHandles.java
- test/jdk/com/sun/jdi/CatchAllTest.sh
- test/jdk/com/sun/jdi/CatchCaughtTest.sh
- test/jdk/com/sun/jdi/CommandCommentDelimiter.sh
- test/jdk/com/sun/jdi/DeoptimizeWalk.sh
- test/jdk/com/sun/jdi/EvalArgs.sh
- test/jdk/com/sun/jdi/EvalArraysAsList.sh
- test/jdk/com/sun/jdi/EvalInterfaceStatic.sh
- test/jdk/com/sun/jdi/GetLocalVariables3Test.sh
- test/jdk/com/sun/jdi/GetLocalVariables4Test.sh
- test/jdk/com/sun/jdi/JdbArgTest.sh
- test/jdk/com/sun/jdi/JdbLockTest.sh
- test/jdk/com/sun/jdi/JdbMissStep.sh
- test/jdk/com/sun/jdi/JdbVarargsTest.sh
- test/jdk/com/sun/jdi/MixedSuspendTest.sh
- test/jdk/com/sun/jdi/NotAField.sh
- test/jdk/com/sun/jdi/NullLocalVariable.sh
- test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/OSInfo.java


From maurizio.cimadamore at oracle.com  Wed Sep  5 20:07:42 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:07:42 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809052007.w85K7h9w029934@aojmv0008.oracle.com>

Changeset: f0024b1d8ae6
Author:    mcimadamore
Date:      2018-09-05 22:10 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f0024b1d8ae6

Automatic merge with default

! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/share/classfile/systemDictionary.cpp
! src/hotspot/share/classfile/systemDictionary.hpp
! src/hotspot/share/logging/logTag.hpp
- src/hotspot/share/utilities/errorReporter.cpp
- src/hotspot/share/utilities/errorReporter.hpp
- src/hotspot/share/utilities/globalDefinitions_sparcWorks.hpp
! src/java.base/share/classes/java/lang/invoke/MethodHandles.java
- test/jdk/com/sun/jdi/CatchAllTest.sh
- test/jdk/com/sun/jdi/CatchCaughtTest.sh
- test/jdk/com/sun/jdi/CommandCommentDelimiter.sh
- test/jdk/com/sun/jdi/DeoptimizeWalk.sh
- test/jdk/com/sun/jdi/EvalArgs.sh
- test/jdk/com/sun/jdi/EvalArraysAsList.sh
- test/jdk/com/sun/jdi/EvalInterfaceStatic.sh
- test/jdk/com/sun/jdi/GetLocalVariables3Test.sh
- test/jdk/com/sun/jdi/GetLocalVariables4Test.sh
- test/jdk/com/sun/jdi/JdbArgTest.sh
- test/jdk/com/sun/jdi/JdbLockTest.sh
- test/jdk/com/sun/jdi/JdbMissStep.sh
- test/jdk/com/sun/jdi/JdbVarargsTest.sh
- test/jdk/com/sun/jdi/MixedSuspendTest.sh
- test/jdk/com/sun/jdi/NotAField.sh
- test/jdk/com/sun/jdi/NullLocalVariable.sh
- test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/OSInfo.java


From maurizio.cimadamore at oracle.com  Wed Sep  5 20:08:04 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:08:04 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809052008.w85K84Fn000303@aojmv0008.oracle.com>

Changeset: 84b78ea0dd83
Author:    mcimadamore
Date:      2018-09-05 22:10 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/84b78ea0dd83

Automatic merge with foreign

! make/autoconf/toolchain.m4
! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/cpu/x86/x86.ad
! src/hotspot/share/opto/compile.cpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/runtime/globals.hpp
- src/hotspot/share/utilities/errorReporter.cpp
- src/hotspot/share/utilities/errorReporter.hpp
- src/hotspot/share/utilities/globalDefinitions_sparcWorks.hpp
- test/jdk/com/sun/jdi/CatchAllTest.sh
- test/jdk/com/sun/jdi/CatchCaughtTest.sh
- test/jdk/com/sun/jdi/CommandCommentDelimiter.sh
- test/jdk/com/sun/jdi/DeoptimizeWalk.sh
- test/jdk/com/sun/jdi/EvalArgs.sh
- test/jdk/com/sun/jdi/EvalArraysAsList.sh
- test/jdk/com/sun/jdi/EvalInterfaceStatic.sh
- test/jdk/com/sun/jdi/GetLocalVariables3Test.sh
- test/jdk/com/sun/jdi/GetLocalVariables4Test.sh
- test/jdk/com/sun/jdi/JdbArgTest.sh
- test/jdk/com/sun/jdi/JdbLockTest.sh
- test/jdk/com/sun/jdi/JdbMissStep.sh
- test/jdk/com/sun/jdi/JdbVarargsTest.sh
- test/jdk/com/sun/jdi/MixedSuspendTest.sh
- test/jdk/com/sun/jdi/NotAField.sh
- test/jdk/com/sun/jdi/NullLocalVariable.sh
- test/jdk/jdk/internal/reflect/Reflection/GetCallerClassTest.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/OSInfo.java


From maurizio.cimadamore at oracle.com  Wed Sep  5 20:08:24 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 05 Sep 2018 20:08:24 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809052008.w85K8Prv000622@aojmv0008.oracle.com>

Changeset: 2d19af5c2516
Author:    mcimadamore
Date:      2018-09-05 22:10 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/2d19af5c2516

Automatic merge with vectorIntrinsics

! src/hotspot/cpu/x86/macroAssembler_x86.hpp


From jbvernee at xs4all.nl  Thu Sep  6 13:49:18 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 06 Sep 2018 15:49:18 +0200
Subject: Binding a single function symbol with [foreign]
Message-ID: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>

Hello,

I was checking out the [foreign] branch today to see what was currently 
possible with it. I have been following the mailing list for a while 
now, and it's always very interesting to read the emails discussing the 
development of this project (especially the technical details).

It seems that currently the only way to bind a native library is to bind 
the entire library to a Java artifact, like one that is generated by 
jextract, through a call to `Libraries.bind`.

The usage I was looking for was more like this:

     Library lib = Libraries.loadLibrary(lookup(), "msvcrt");

     Symbol printf = lib.lookup("printf"); // possibly need mangled name 
here?
     // imaginary api:
     MethodHandle mh = ((FunctionSymbol) printf).bind(); // Or, manually 
provide layout information to bind?
     Scope scope = Scope.newNativeScope();
     Pointer<Byte> message = scope.toCString("Hello World!");
     int result = (int) mh.invokeExact(message);

i.e. having the ability to bind a single function symbol from some 
library and then being able to call that function, without the need to 
use a tool like jextract and binding the generated artifact.

Do you think also having a more low-level API like this is 
possible/desirable?

Best regards,
Jorn Vernee

From maurizio.cimadamore at oracle.com  Thu Sep  6 17:42:11 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 6 Sep 2018 18:42:11 +0100
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
Message-ID: <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>

Hi Jorn,
thanks for your feedback and your interest in our 'foreign' development.

The issue you bring up is a very good one - that is, the target of a 
binding is, currently, an interface. But one might envision (as you did) 
cases where maybe you just want a single method handle to be the result 
of your bind.

This is technically possible - after all, this is what our binder does 
internally (see NativeInvoker). To get there you need 2 ingredients:

1) the method type you want to use for the call
2) the layout of the function parameters/return type

Now, as I type this I realize that there's a third ingredient that is 
present in sources, but missing in this formulation:

3) the generic signatures of arguments/return types - this is a crucial 
piece to tell the binder whether the result should be a Pointer<Byte> 
vs. Pointer<Integer> and so forth; note that this is more than just 
static typing: a Pointer<Integer> will carry runtime type information of 
the fact that it points to an Integer memory region (with given layout - 
e.g. i32).

That said, we could, in principle, use (2) to infer the same information 
available in (3) - e.g. if the method type says it returns a 
Pointer.class and the layout says u64:i32, we could then infer 
Pointer<Integer>. It seems doable.

After you have (1), (2) and (3) you are finally in the position to 
create a method handle for your native function.

So yes, we could have, in principle a method on Symbol like this:

MethodHandle bindAsFunction(MethodType, Function)

And probably another pair:

MethodHandle bindAsVarGetter(Class<?>, Layout)
MethodHandle bindAsVarSetter(Class<?>, Layout)

(since treatment for functions is different from that of global variables).


Would this be something like this useful to the use case you have in mind?

Cheers
Maurizio



On 06/09/18 14:49, Jorn Vernee wrote:
> Hello,
>
> I was checking out the [foreign] branch today to see what was 
> currently possible with it. I have been following the mailing list for 
> a while now, and it's always very interesting to read the emails 
> discussing the development of this project (especially the technical 
> details).
>
> It seems that currently the only way to bind a native library is to 
> bind the entire library to a Java artifact, like one that is generated 
> by jextract, through a call to `Libraries.bind`.
>
> The usage I was looking for was more like this:
>
> ??? Library lib = Libraries.loadLibrary(lookup(), "msvcrt");
>
> ??? Symbol printf = lib.lookup("printf"); // possibly need mangled 
> name here?
> ??? // imaginary api:
> ??? MethodHandle mh = ((FunctionSymbol) printf).bind(); // Or, 
> manually provide layout information to bind?
> ??? Scope scope = Scope.newNativeScope();
> ??? Pointer<Byte> message = scope.toCString("Hello World!");
> ??? int result = (int) mh.invokeExact(message);
>
> i.e. having the ability to bind a single function symbol from some 
> library and then being able to call that function, without the need to 
> use a tool like jextract and binding the generated artifact.
>
> Do you think also having a more low-level API like this is 
> possible/desirable?
>
> Best regards,
> Jorn Vernee


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:06 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:06 +0000
Subject: hg: panama/dev: Rearrange tests
Message-ID: <201809061750.w86Ho7wq025107@aojmv0008.oracle.com>

Changeset: e12d0b951ecc
Author:    rlupusoru
Date:      2018-08-20 10:30 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e12d0b951ecc

Rearrange tests

! test/jdk/jdk/incubator/vector/AbstractVectorTest.java
! test/jdk/jdk/incubator/vector/Byte128VectorTests.java
! test/jdk/jdk/incubator/vector/Byte256VectorTests.java
! test/jdk/jdk/incubator/vector/Byte512VectorTests.java
! test/jdk/jdk/incubator/vector/Byte64VectorTests.java
! test/jdk/jdk/incubator/vector/Double128VectorTests.java
! test/jdk/jdk/incubator/vector/Double256VectorTests.java
! test/jdk/jdk/incubator/vector/Double512VectorTests.java
! test/jdk/jdk/incubator/vector/Double64VectorTests.java
! test/jdk/jdk/incubator/vector/Float128VectorTests.java
! test/jdk/jdk/incubator/vector/Float256VectorTests.java
! test/jdk/jdk/incubator/vector/Float512VectorTests.java
! test/jdk/jdk/incubator/vector/Float64VectorTests.java
! test/jdk/jdk/incubator/vector/Int128VectorTests.java
! test/jdk/jdk/incubator/vector/Int256VectorTests.java
! test/jdk/jdk/incubator/vector/Int512VectorTests.java
! test/jdk/jdk/incubator/vector/Int64VectorTests.java
! test/jdk/jdk/incubator/vector/Long128VectorTests.java
! test/jdk/jdk/incubator/vector/Long256VectorTests.java
! test/jdk/jdk/incubator/vector/Long512VectorTests.java
! test/jdk/jdk/incubator/vector/Long64VectorTests.java
! test/jdk/jdk/incubator/vector/Short128VectorTests.java
! test/jdk/jdk/incubator/vector/Short256VectorTests.java
! test/jdk/jdk/incubator/vector/Short512VectorTests.java
! test/jdk/jdk/incubator/vector/Short64VectorTests.java
! test/jdk/jdk/incubator/vector/gen-template.sh
+ test/jdk/jdk/incubator/vector/templates/Kernel-Rearrange.template
+ test/jdk/jdk/incubator/vector/templates/Unit-Rearrange.template
! test/jdk/jdk/incubator/vector/templates/Unit-header.template


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:13 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:13 +0000
Subject: hg: panama/dev: Add more verbosity to svml testing output
Message-ID: <201809061750.w86HoD9q025208@aojmv0008.oracle.com>

Changeset: 8a0b182b10ac
Author:    rlupusoru
Date:      2018-08-21 16:54 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/8a0b182b10ac

Add more verbosity to svml testing output

! test/jdk/jdk/incubator/vector/Double128VectorTests.java
! test/jdk/jdk/incubator/vector/Double256VectorTests.java
! test/jdk/jdk/incubator/vector/Double512VectorTests.java
! test/jdk/jdk/incubator/vector/Double64VectorTests.java
! test/jdk/jdk/incubator/vector/Float128VectorTests.java
! test/jdk/jdk/incubator/vector/Float256VectorTests.java
! test/jdk/jdk/incubator/vector/Float512VectorTests.java
! test/jdk/jdk/incubator/vector/Float64VectorTests.java
! test/jdk/jdk/incubator/vector/templates/Unit-header.template


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:20 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:20 +0000
Subject: hg: panama/dev: Remove redundant MaxV and MinV matcher entries
Message-ID: <201809061750.w86HoKrD025271@aojmv0008.oracle.com>

Changeset: e2ad48fba3a8
Author:    rlupusoru
Date:      2018-08-22 10:46 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e2ad48fba3a8

Remove redundant MaxV and MinV matcher entries

! src/hotspot/cpu/x86/x86.ad


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:25 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:25 +0000
Subject: hg: panama/dev: Fix trivial misspelling in cast ad format
Message-ID: <201809061750.w86HoQww025315@aojmv0008.oracle.com>

Changeset: e7182f8fc03f
Author:    rlupusoru
Date:      2018-08-22 13:17 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e7182f8fc03f

Fix trivial misspelling in cast ad format

! src/hotspot/cpu/x86/x86.ad


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:31 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:31 +0000
Subject: hg: panama/dev: Make sure extract64x is marked as being evex
Message-ID: <201809061750.w86HoVQf025357@aojmv0008.oracle.com>

Changeset: b135bbfcfcc2
Author:    rlupusoru
Date:      2018-08-22 16:47 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b135bbfcfcc2

Make sure extract64x is marked as being evex

! src/hotspot/cpu/x86/assembler_x86.cpp


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:36 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:36 +0000
Subject: hg: panama/dev: Fix cast byte to double in AVX512
Message-ID: <201809061750.w86Hobrq025395@aojmv0008.oracle.com>

Changeset: 0414dcc637d0
Author:    rlupusoru
Date:      2018-08-23 17:03 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/0414dcc637d0

Fix cast byte to double in AVX512

! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/x86.ad


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:42 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:42 +0000
Subject: hg: panama/dev: Fix byte512 max all intrinsic
Message-ID: <201809061750.w86Hogfb025438@aojmv0008.oracle.com>

Changeset: 7f84564ce3b7
Author:    rlupusoru
Date:      2018-08-24 10:50 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/7f84564ce3b7

Fix byte512 max all intrinsic

! src/hotspot/cpu/x86/x86.ad


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:48 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:48 +0000
Subject: hg: panama/dev: Improve get intrinsic testing and fix failures
Message-ID: <201809061750.w86HomUc025586@aojmv0008.oracle.com>

Changeset: 6063fc396d79
Author:    rlupusoru
Date:      2018-08-27 17:37 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/6063fc396d79

Improve get intrinsic testing and fix failures

! src/hotspot/cpu/x86/x86.ad
! src/hotspot/share/opto/vectornode.hpp
! test/jdk/jdk/incubator/vector/Byte128VectorTests.java
! test/jdk/jdk/incubator/vector/Byte256VectorTests.java
! test/jdk/jdk/incubator/vector/Byte512VectorTests.java
! test/jdk/jdk/incubator/vector/Byte64VectorTests.java
! test/jdk/jdk/incubator/vector/Double128VectorTests.java
! test/jdk/jdk/incubator/vector/Double256VectorTests.java
! test/jdk/jdk/incubator/vector/Double512VectorTests.java
! test/jdk/jdk/incubator/vector/Double64VectorTests.java
! test/jdk/jdk/incubator/vector/Float128VectorTests.java
! test/jdk/jdk/incubator/vector/Float256VectorTests.java
! test/jdk/jdk/incubator/vector/Float512VectorTests.java
! test/jdk/jdk/incubator/vector/Float64VectorTests.java
! test/jdk/jdk/incubator/vector/Int128VectorTests.java
! test/jdk/jdk/incubator/vector/Int256VectorTests.java
! test/jdk/jdk/incubator/vector/Int512VectorTests.java
! test/jdk/jdk/incubator/vector/Int64VectorTests.java
! test/jdk/jdk/incubator/vector/Long128VectorTests.java
! test/jdk/jdk/incubator/vector/Long256VectorTests.java
! test/jdk/jdk/incubator/vector/Long512VectorTests.java
! test/jdk/jdk/incubator/vector/Long64VectorTests.java
! test/jdk/jdk/incubator/vector/Short128VectorTests.java
! test/jdk/jdk/incubator/vector/Short256VectorTests.java
! test/jdk/jdk/incubator/vector/Short512VectorTests.java
! test/jdk/jdk/incubator/vector/Short64VectorTests.java
! test/jdk/jdk/incubator/vector/templates/Kernel-Get-op.template


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:50:54 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:50:54 +0000
Subject: hg: panama/dev: Fix extract for 16f, 4d, and 16d
Message-ID: <201809061750.w86HoteP025739@aojmv0008.oracle.com>

Changeset: dfc41f47f22f
Author:    rlupusoru
Date:      2018-08-30 11:07 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/dfc41f47f22f

Fix extract for 16f, 4d, and 16d

! src/hotspot/cpu/x86/x86.ad


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:51:02 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:51:02 +0000
Subject: hg: panama/dev: Fix byte reductions incorrect register usage
Message-ID: <201809061751.w86Hp3Ob025893@aojmv0008.oracle.com>

Changeset: ae2a7a71b870
Author:    rlupusoru
Date:      2018-08-30 17:09 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/ae2a7a71b870

Fix byte reductions incorrect register usage

! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/x86.ad


From razvan.a.lupusoru at intel.com  Thu Sep  6 17:51:08 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 06 Sep 2018 17:51:08 +0000
Subject: hg: panama/dev: Workaround svml k1 clobbering
Message-ID: <201809061751.w86Hp9rc026013@aojmv0008.oracle.com>

Changeset: 24a22da308cf
Author:    rlupusoru
Date:      2018-08-31 16:03 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/24a22da308cf

Workaround svml k1 clobbering

! src/hotspot/cpu/x86/x86_64.ad


From maurizio.cimadamore at oracle.com  Thu Sep  6 17:51:46 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 06 Sep 2018 17:51:46 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809061751.w86HplIj026719@aojmv0008.oracle.com>

Changeset: 833600014232
Author:    mcimadamore
Date:      2018-09-06 19:54 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/833600014232

Automatic merge with vectorIntrinsics



From razvan.a.lupusoru at intel.com  Thu Sep  6 17:52:29 2018
From: razvan.a.lupusoru at intel.com (Lupusoru, Razvan A)
Date: Thu, 6 Sep 2018 17:52:29 +0000
Subject: Vector API Multiple Fixes
In-Reply-To: <48D92A70936F7946BE99F3FF5ECB6461F6683283@ORSMSX105.amr.corp.intel.com>
References: <48D92A70936F7946BE99F3FF5ECB6461F6683283@ORSMSX105.amr.corp.intel.com>
Message-ID: <48D92A70936F7946BE99F3FF5ECB6461F66838B5@ORSMSX105.amr.corp.intel.com>

Hi everyone,

I have just pushed the changes below to panama. If you find any issues, please let me know and we can fix after the fact. Thanks!

--Razvan

-----Original Message-----
From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Lupusoru, Razvan A
Sent: Tuesday, September 04, 2018 4:22 PM
To: panama-dev at openjdk.java.net
Subject: Vector API Multiple Fixes

Hi everyone,

I would like to contribute a couple of improvements, fixes and cleanups for Vector API:

-          Tests for rearrange

-          Test improvement for "get" to exercise functionality

-          Additional verbosity for SVML testing results

-          Fix for extract encoding

-          Fix for cast byte to double in AVX512

-          Fix byte 512 max all intrinsic

-          Fix for get .ad file predicates + fix for 16f, 4d, and 8d extract implementation

-          Fix byte reduction incorrect encoding

-          Few misspelling fixes and removal of redundant entries

-          Workaround around SVML k1 clobbering

Please check out the patch here:
http://cr.openjdk.java.net/~rlupusoru/panama/webrev_fixesandcleanup1_01/

We are close to getting 100% test pass rate on SKL and SKX. Thanks!

--Razvan

From maurizio.cimadamore at oracle.com  Thu Sep  6 17:56:46 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 06 Sep 2018 17:56:46 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809061756.w86HulHZ029099@aojmv0008.oracle.com>

Changeset: 57911386eccd
Author:    mcimadamore
Date:      2018-09-06 19:59 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/57911386eccd

Automatic merge with vectorIntrinsics



From jbvernee at xs4all.nl  Thu Sep  6 21:58:39 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 06 Sep 2018 23:58:39 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
Message-ID: <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>

Hi Maurizio,

The API points you give look pretty much like what I had in mind!

Though I'm wondering if the `MethodType`/`Class<?>` parameters are 
needed? From what I understood there is a 1 to 1 mapping from a native 
method signature to a Java method signature, so I'd assume there is also 
a 1 to 1 mapping possible from a `Layout` to a `MethodType` or 
`Class<?>`? So instead of passing the `MethodType`/`Class<?>` as a 
separate argument, could it be derived automatically from the 
`Function`/`Layout` argument?

The use case I was thinking of was the ability to use a dynamic library 
without having access to a header file that describes the interface of 
that library. It occurred to me that a dynamic library in some cases 
holds all the information you need to invoke one of it's functions, or 
the user can provide the missing information manually, so it should 
technically be possible to invoke a function without also needing to 
have have the header file. For instance, some languages (I believe C++ 
for one [1]) use decorated function names to also capture the type 
signature of a function in the resulting library symbol, and it should 
be possible to automatically derive a `Layout` from that. It seems like 
the additional steps of parsing a header file, and spinning a Java 
artifact are not always necessary.

Another use case might be dynamically generated native code (generated 
during the lifetime of our Java process). For instance, to support 
template functions or function-like macros, one not-so-convenient 
solution would be to let the user manually declare a template 
instantiation or a wrapper around a macro to have a corresponding 
function appear in the dynamic library binary. But it would be nicer if 
it were possible to generate an 'instance' of a template or macro on the 
fly. For instance by capturing the template/macro source code, and then 
when a template function/function-like macro is called, handing that 
template/macro source code, parameterized with the dynamic argument 
types of the call site, off to some (external) compiler service which 
then gives you back a pointer to a block of native memory that you can 
interpret as a function, wrap in a method handle and invoke.

In the latter case the needed API might be more like having a method on 
`Pointer` like this:

     MethodHandle wrapAsFunction(Function funcType)

i.e. have the ability to interpret an arbitrary `Pointer` as a function 
pointer that can be invoked. I think something like that might be needed 
any ways as support for native functions that return function pointers?

Since `Library.Symbol` already has a way to convert to a `Pointer` such 
an API could also work for my first use case:

     Library lib = Libraries.loadLibrary(lookup(), "msvcrt");

     Symbol printf = lib.lookup("printf");
     // imaginary api:
     MethodHandle mh = 
printf.getAddress().wrapAsFunction(Function.of(...));

Hopefully that gives a good idea of what I had in mind (nothing too 
concrete at the moment I'm afraid). What do you think?

Jorn

[1]: https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling

Maurizio Cimadamore schreef op 2018-09-06 19:42:
> Hi Jorn,
> thanks for your feedback and your interest in our 'foreign' 
> development.
> 
> The issue you bring up is a very good one - that is, the target of a
> binding is, currently, an interface. But one might envision (as you
> did) cases where maybe you just want a single method handle to be the
> result of your bind.
> 
> This is technically possible - after all, this is what our binder does
> internally (see NativeInvoker). To get there you need 2 ingredients:
> 
> 1) the method type you want to use for the call
> 2) the layout of the function parameters/return type
> 
> Now, as I type this I realize that there's a third ingredient that is
> present in sources, but missing in this formulation:
> 
> 3) the generic signatures of arguments/return types - this is a
> crucial piece to tell the binder whether the result should be a
> Pointer<Byte> vs. Pointer<Integer> and so forth; note that this is
> more than just static typing: a Pointer<Integer> will carry runtime
> type information of the fact that it points to an Integer memory
> region (with given layout - e.g. i32).
> 
> That said, we could, in principle, use (2) to infer the same
> information available in (3) - e.g. if the method type says it returns
> a Pointer.class and the layout says u64:i32, we could then infer
> Pointer<Integer>. It seems doable.
> 
> After you have (1), (2) and (3) you are finally in the position to
> create a method handle for your native function.
> 
> So yes, we could have, in principle a method on Symbol like this:
> 
> MethodHandle bindAsFunction(MethodType, Function)
> 
> And probably another pair:
> 
> MethodHandle bindAsVarGetter(Class<?>, Layout)
> MethodHandle bindAsVarSetter(Class<?>, Layout)
> 
> (since treatment for functions is different from that of global 
> variables).
> 
> 
> Would this be something like this useful to the use case you have in 
> mind?
> 
> Cheers
> Maurizio
> 
> 
> 
> On 06/09/18 14:49, Jorn Vernee wrote:
>> Hello,
>> 
>> I was checking out the [foreign] branch today to see what was 
>> currently possible with it. I have been following the mailing list for 
>> a while now, and it's always very interesting to read the emails 
>> discussing the development of this project (especially the technical 
>> details).
>> 
>> It seems that currently the only way to bind a native library is to 
>> bind the entire library to a Java artifact, like one that is generated 
>> by jextract, through a call to `Libraries.bind`.
>> 
>> The usage I was looking for was more like this:
>> 
>> ??? Library lib = Libraries.loadLibrary(lookup(), "msvcrt");
>> 
>> ??? Symbol printf = lib.lookup("printf"); // possibly need mangled 
>> name here?
>> ??? // imaginary api:
>> ??? MethodHandle mh = ((FunctionSymbol) printf).bind(); // Or, 
>> manually provide layout information to bind?
>> ??? Scope scope = Scope.newNativeScope();
>> ??? Pointer<Byte> message = scope.toCString("Hello World!");
>> ??? int result = (int) mh.invokeExact(message);
>> 
>> i.e. having the ability to bind a single function symbol from some 
>> library and then being able to call that function, without the need to 
>> use a tool like jextract and binding the generated artifact.
>> 
>> Do you think also having a more low-level API like this is 
>> possible/desirable?
>> 
>> Best regards,
>> Jorn Vernee

From maurizio.cimadamore at oracle.com  Thu Sep  6 22:41:55 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 6 Sep 2018 23:41:55 +0100
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
Message-ID: <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>



On 06/09/18 22:58, Jorn Vernee wrote:
> Hi Maurizio,
>
> The API points you give look pretty much like what I had in mind!
>
> Though I'm wondering if the `MethodType`/`Class<?>` parameters are 
> needed? From what I understood there is a 1 to 1 mapping from a native 
> method signature to a Java method signature, so I'd assume there is 
> also a 1 to 1 mapping possible from a `Layout` to a `MethodType` or 
> `Class<?>`? So instead of passing the `MethodType`/`Class<?>` as a 
> separate argument, could it be derived automatically from the 
> `Function`/`Layout` argument?
The way it works now is that a native type, modeled by a LayoutType 
abstraction is the fusion of two elements:

* a layout - which tells you how many bits there are in that memory region
* a Java carrier - the Java type you want to view those bits as

So, assuming you have a u64:i8, e.g. a pointer to a signed 8 bit, it's 
really up to the Java client whether to model that as Pointer<Byte>, 
Pointer<Short>, Pointer<Integer>, Pointer<Long>. The layout info and the 
carrier info attached to a memory region are, up to a degree, orthogonal.

When you declare interface by hands, you use this (maybe w/o realizing) 
by assigning Java types that might be bigger than needed for their 
layout counterpart.

Of course you can infer a 'default' carrier information from a layout, 
but in the general case you can't - think of structs (modeled as 
classes) and function pointers (modeled as functional interfaces) - how 
do you know whether something is a Foo or a Bar, if both Foo and Bar are 
structs with same fields?

So, inferring carriers from layout is a tricky business - and, depending 
on the layout you might or might not be able to do it. I was trying to 
give you a comprehensive picture of the ingredients needed.
>
> The use case I was thinking of was the ability to use a dynamic 
> library without having access to a header file that describes the 
> interface of that library. It occurred to me that a dynamic library in 
> some cases holds all the information you need to invoke one of it's 
> functions, or the user can provide the missing information manually, 
> so it should technically be possible to invoke a function without also 
> needing to have have the header file. For instance, some languages (I 
> believe C++ for one [1]) use decorated function names to also capture 
> the type signature of a function in the resulting library symbol, and 
> it should be possible to automatically derive a `Layout` from that. It 
> seems like the additional steps of parsing a header file, and spinning 
> a Java artifact are not always necessary.
Right - going through an header is not necessary in every use case - 
although that's where our design center is at the moment (and tools like 
jextract will make that super easy).

That said, if you want to generate a method handle out of a native 
function, a method handle has a type, so it seems fair that the user 
provides the type it wants for the call. Of course there could be 
(maybe) an override which infers everything automatically (if it can), 
but the most complete version would always be the carrier + layout one.
>
> Another use case might be dynamically generated native code (generated 
> during the lifetime of our Java process). For instance, to support 
> template functions or function-like macros, one not-so-convenient 
> solution would be to let the user manually declare a template 
> instantiation or a wrapper around a macro to have a corresponding 
> function appear in the dynamic library binary. But it would be nicer 
> if it were possible to generate an 'instance' of a template or macro 
> on the fly. For instance by capturing the template/macro source code, 
> and then when a template function/function-like macro is called, 
> handing that template/macro source code, parameterized with the 
> dynamic argument types of the call site, off to some (external) 
> compiler service which then gives you back a pointer to a block of 
> native memory that you can interpret as a function, wrap in a method 
> handle and invoke.
>
> In the latter case the needed API might be more like having a method 
> on `Pointer` like this:
>
> ??? MethodHandle wrapAsFunction(Function funcType)
>
> i.e. have the ability to interpret an arbitrary `Pointer` as a 
> function pointer that can be invoked. I think something like that 
> might be needed any ways as support for native functions that return 
> function pointers?
For function pointers we have the Callback abstraction (see recent RFR) 
- which is basically a wrapper around a (void) code pointer. For now you 
can construct one by giving it a Java functional interface. That said, 
it is theoretically possible to construct callbacks directly from native 
pointers (which have been filled with code) - in fact we do that to some 
extent already, as when you turn a Java functional interface into a 
Callback, the VM spins up a new stub of code, and the Callback contains 
a pointer to that - so that approach works :-)
>
> Since `Library.Symbol` already has a way to convert to a `Pointer` 
> such an API could also work for my first use case:
>
> ??? Library lib = Libraries.loadLibrary(lookup(), "msvcrt");
>
> ??? Symbol printf = lib.lookup("printf");
> ??? // imaginary api:
> ??? MethodHandle mh = 
> printf.getAddress().wrapAsFunction(Function.of(...));
>
> Hopefully that gives a good idea of what I had in mind (nothing too 
> concrete at the moment I'm afraid). What do you think?
I think these are all good ideas - not sure if we are far along the road 
to start exploring them, as our priority in the short term is in 
stabilizing what we've got ahead of an early access release. But what 
you propose looks sensible, and things that programmers might indeed 
want to do.

Thanks
Maurizio
>
> Jorn
>
> [1]: https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling
>
> Maurizio Cimadamore schreef op 2018-09-06 19:42:
>> Hi Jorn,
>> thanks for your feedback and your interest in our 'foreign' development.
>>
>> The issue you bring up is a very good one - that is, the target of a
>> binding is, currently, an interface. But one might envision (as you
>> did) cases where maybe you just want a single method handle to be the
>> result of your bind.
>>
>> This is technically possible - after all, this is what our binder does
>> internally (see NativeInvoker). To get there you need 2 ingredients:
>>
>> 1) the method type you want to use for the call
>> 2) the layout of the function parameters/return type
>>
>> Now, as I type this I realize that there's a third ingredient that is
>> present in sources, but missing in this formulation:
>>
>> 3) the generic signatures of arguments/return types - this is a
>> crucial piece to tell the binder whether the result should be a
>> Pointer<Byte> vs. Pointer<Integer> and so forth; note that this is
>> more than just static typing: a Pointer<Integer> will carry runtime
>> type information of the fact that it points to an Integer memory
>> region (with given layout - e.g. i32).
>>
>> That said, we could, in principle, use (2) to infer the same
>> information available in (3) - e.g. if the method type says it returns
>> a Pointer.class and the layout says u64:i32, we could then infer
>> Pointer<Integer>. It seems doable.
>>
>> After you have (1), (2) and (3) you are finally in the position to
>> create a method handle for your native function.
>>
>> So yes, we could have, in principle a method on Symbol like this:
>>
>> MethodHandle bindAsFunction(MethodType, Function)
>>
>> And probably another pair:
>>
>> MethodHandle bindAsVarGetter(Class<?>, Layout)
>> MethodHandle bindAsVarSetter(Class<?>, Layout)
>>
>> (since treatment for functions is different from that of global 
>> variables).
>>
>>
>> Would this be something like this useful to the use case you have in 
>> mind?
>>
>> Cheers
>> Maurizio
>>
>>
>>
>> On 06/09/18 14:49, Jorn Vernee wrote:
>>> Hello,
>>>
>>> I was checking out the [foreign] branch today to see what was 
>>> currently possible with it. I have been following the mailing list 
>>> for a while now, and it's always very interesting to read the emails 
>>> discussing the development of this project (especially the technical 
>>> details).
>>>
>>> It seems that currently the only way to bind a native library is to 
>>> bind the entire library to a Java artifact, like one that is 
>>> generated by jextract, through a call to `Libraries.bind`.
>>>
>>> The usage I was looking for was more like this:
>>>
>>> ??? Library lib = Libraries.loadLibrary(lookup(), "msvcrt");
>>>
>>> ??? Symbol printf = lib.lookup("printf"); // possibly need mangled 
>>> name here?
>>> ??? // imaginary api:
>>> ??? MethodHandle mh = ((FunctionSymbol) printf).bind(); // Or, 
>>> manually provide layout information to bind?
>>> ??? Scope scope = Scope.newNativeScope();
>>> ??? Pointer<Byte> message = scope.toCString("Hello World!");
>>> ??? int result = (int) mh.invokeExact(message);
>>>
>>> i.e. having the ability to bind a single function symbol from some 
>>> library and then being able to call that function, without the need 
>>> to use a tool like jextract and binding the generated artifact.
>>>
>>> Do you think also having a more low-level API like this is 
>>> possible/desirable?
>>>
>>> Best regards,
>>> Jorn Vernee


From jbvernee at xs4all.nl  Thu Sep  6 23:41:07 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Fri, 07 Sep 2018 01:41:07 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
Message-ID: <32bd28029f19e4b292d0b26fe1e38433@xs4all.nl>

> The way it works now is that a native type, modeled by a LayoutType
> abstraction is the fusion of two elements:
> 
> * a layout - which tells you how many bits there are in that memory 
> region
> * a Java carrier - the Java type you want to view those bits as
> 
> So, assuming you have a u64:i8, e.g. a pointer to a signed 8 bit, it's
> really up to the Java client whether to model that as Pointer<Byte>,
> Pointer<Short>, Pointer<Integer>, Pointer<Long>. The layout info and
> the carrier info attached to a memory region are, up to a degree,
> orthogonal.

Ah I see! I had been reading through your binder v3 document [1] but how 
Layout, LayoutType and Java Carrier types fitted together hadn't fully 
'clicked' yet, but now I have good picture :)

> That said, if you want to generate a method handle out of a native
> function, a method handle has a type, so it seems fair that the user
> provides the type it wants for the call. Of course there could be
> (maybe) an override which infers everything automatically (if it can),
> but the most complete version would always be the carrier + layout
> one.

Yes, understanding all of the above I can see that passing some kind of 
method type for the Java side is pretty much always necessary to signal 
which carrier types you want to use.

> I think these are all good ideas - not sure if we are far along the
> road to start exploring them, as our priority in the short term is in
> stabilizing what we've got ahead of an early access release. But what
> you propose looks sensible, and things that programmers might indeed
> want to do.
> 
> Thanks
> Maurizio

Thanks for the info!
Jorn

[1]: 
http://cr.openjdk.java.net/~mcimadamore/panama/panama-binder-v3.html

From maurizio.cimadamore at oracle.com  Fri Sep  7 07:52:27 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 7 Sep 2018 08:52:27 +0100
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
Message-ID: <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>



On 06/09/18 23:41, Maurizio Cimadamore wrote:
> Of course you can infer a 'default' carrier information from a layout, 
> but in the general case you can't - think of structs (modeled as 
> classes) and function pointers (modeled as functional interfaces) - 
> how do you know whether something is a Foo or a Bar, if both Foo and 
> Bar are structs with same fields? 
Pulling more on this string as I kept thinking about this: adding 
carrier info will help to distinguish a toplevel Foo from a toplevel Bar 
- e.g. let's make this more concrete:


@NativeStruct("[ i32(get=x) ](foo)")
interface Foo extends Struct<Foo> {
 ?? int x();
}

@NativeStruct("[ i32(get=y) ](bar)")
interface Bar extends Struct<Bar> {
 ?? int y();
}

These two struct declarations are isomorphic - you can use one in place 
of another; since in a .so/.dynlib file there's no info about struct 
names (well in the absence of debugging symbol which I assume to be the 
norm), if you want to dynamically bind to a native function whose 
descriptor is:

( [ i32 ])v

What should the binder do? E.g. what should be the Java type associated 
with the argument of this function? Here's where the MethodType is handy 
- as it could help the inference process understanding whether we want 
our method handle to accept a Foo or a Bar thingie.

But, I realized, this trick doesn't have a lot of mileage: the next 
example up is with pointers - so let's assume the function descriptor is 
now changed to:

( u64: [ i32 ] ) v

Ok, now we have a pointer to a thingie with one 32-bit signed int 
inside. Is that Pointer<Foo> or Pointer<Bar> ? This time, unfortunately, 
the MethodType won't help - class types in method types are erased, so 
the method type for this will be something like

(Pointer)void

Which is not enough for our inference process to resolve the ambiguity.

I believe the next best thing here is for our process to end up with 
some Pointer<?> - that is, a pointer whose layout info is composed of 
the following info:

- a layout (namely, "[ i32 ]")
- no carrier info

Note that, since there's no carrier, attempting to call get() on such a 
Pointer will fail with exception. Now, since the pointer here is in an 
argument position, it's not a big deal - e.g. the binder doesn't need to 
produce one, so we're fine (e.g. the pointer will never be 
dereferenced). But if the pointer was in a return position, as in:

( ) u64: [ i32 ]

This would now be an issue, as the returned Pointer<?> will not support 
any dereference? operation:

Pointer<?> p = mh.invoke();
p.get(); //exception!

So the right way to go about this would be to cast the result to the 
right LayoutType - e.g.

Pointer<?> p = mh.invoke();
p = p.cast(NativeTypes.INT32);
int x = p.get(); //ok!

In other words, I think the invocation game can always be decomposed 
into two stages:

1) the first step, works only on erased types and produces types that 
are correct *up to the generic layer*
2) the second step, is a fixup step that takes info from source 
signatures (or in other, more explicit ways, as above) and turns generic 
carriers 'right' (typically with a cast)

Back to your original question of providing binding to separate method 
handle symbol, I now think the answer is yes, *provided* that what you 
mean is (1) - e.g. that you are willing to make up for losses of generic 
type info resulting from the lack of source information that directs the 
binding process.

Does this help?

Cheers
Maurizio



From Ningsheng.Jian at arm.com  Fri Sep  7 11:04:53 2018
From: Ningsheng.Jian at arm.com (Ningsheng Jian (Arm Technology China))
Date: Fri, 7 Sep 2018 11:04:53 +0000
Subject: [vector] RFC: Add scalable shapes for Arm Scalable Vector Extension
Message-ID: <AM0PR08MB3523229FCD23E33A488724E790000@AM0PR08MB3523.eurprd08.prod.outlook.com>

Hi,

As you may know that Arm SVE architecturally supports scalable vector length, from 128 to 2048bits, multiple of 128bits. Currently Vector API only supports 64, 128, 256 and 512 bit vector shapes. It's inappropriate to pre-generate all SVE supported shapes, hence, we propose a new type of vector for scalable length (or length agnostic) shapes.

Webrev:
http://cr.openjdk.java.net/~njian/sve/vectorapi.webrev00/

Could you please help to take a look at this? The *ScalableVector are the new scalable length vectors and to be compatible with existing architectures, the preferredSpecies() will return those known species first.

We are working on SVE support on OpenJDK, and current Vector API's intrinsics can really help us on our SVE backend work. We have an initial prototype which adds SVE backend and updates a bit for current register allocator to support scalable length vector types. Our WIP SVE backend shows that this scalable vector proposal works fine with current Vector API intrinsics. We would like to collect your comments about this scalable vector patch so that we can move forward for the SVE backend support. I notice that Vladimir is also thinking about alternative loop shapes, that's really helpful for SVE! We will also take a look at that further.

Thanks,
Ningsheng

From jbvernee at xs4all.nl  Fri Sep  7 12:20:10 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Fri, 07 Sep 2018 14:20:10 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
Message-ID: <63968d742770dc79acb065c190f22cbd@xs4all.nl>

Yes, I think that would work. As an API point, in that case you could 
add an overload that also takes the `LayoutType` to cast to, and appends 
a call to `cast` to the bound method handle i.e.:

     MethodHandle bind(MethodType mt, Function f) {...}

     MethodHandle bindWithCast(MethodType mt, Function f, LayoutType<?> 
lt) {
         MethodHandle bound = bind(mt, f);
         bound = collectArguments(insertArguments(MH_PointerCast, 1, lt), 
0, bound);
         return bound;
     }

And if the overload without the `LayoutType` is used while the method 
type returns a `Pointer` throw an IAE?

I guess the `LayoutType` of a returned `Pointer` would be 
`LayoutType.ofVoid` before the cast? Or is there a generic fall-back 
carrier type? Maybe a direct `ByteBuffer` that wraps the returned 
pointer, and uses spill-to-heap for non-pointer returns?

Though the solution I was thinking of yesterday was to use a more 
informative type instead of `MethodType`, that also carries the generic 
type information. Something using `java.lang.reflect.ParameterizedType` 
maybe? In that case the right `LayoutType` for the returned `Pointer` 
could be immediately derived from that, without the need for the extra 
cast?

Jorn

Maurizio Cimadamore schreef op 2018-09-07 09:52:
> On 06/09/18 23:41, Maurizio Cimadamore wrote:
>> Of course you can infer a 'default' carrier information from a layout, 
>> but in the general case you can't - think of structs (modeled as 
>> classes) and function pointers (modeled as functional interfaces) - 
>> how do you know whether something is a Foo or a Bar, if both Foo and 
>> Bar are structs with same fields?
> Pulling more on this string as I kept thinking about this: adding
> carrier info will help to distinguish a toplevel Foo from a toplevel
> Bar - e.g. let's make this more concrete:
> 
> 
> @NativeStruct("[ i32(get=x) ](foo)")
> interface Foo extends Struct<Foo> {
> ?? int x();
> }
> 
> @NativeStruct("[ i32(get=y) ](bar)")
> interface Bar extends Struct<Bar> {
> ?? int y();
> }
> 
> These two struct declarations are isomorphic - you can use one in
> place of another; since in a .so/.dynlib file there's no info about
> struct names (well in the absence of debugging symbol which I assume
> to be the norm), if you want to dynamically bind to a native function
> whose descriptor is:
> 
> ( [ i32 ])v
> 
> What should the binder do? E.g. what should be the Java type
> associated with the argument of this function? Here's where the
> MethodType is handy - as it could help the inference process
> understanding whether we want our method handle to accept a Foo or a
> Bar thingie.
> 
> But, I realized, this trick doesn't have a lot of mileage: the next
> example up is with pointers - so let's assume the function descriptor
> is now changed to:
> 
> ( u64: [ i32 ] ) v
> 
> Ok, now we have a pointer to a thingie with one 32-bit signed int
> inside. Is that Pointer<Foo> or Pointer<Bar> ? This time,
> unfortunately, the MethodType won't help - class types in method types
> are erased, so the method type for this will be something like
> 
> (Pointer)void
> 
> Which is not enough for our inference process to resolve the ambiguity.
> 
> I believe the next best thing here is for our process to end up with
> some Pointer<?> - that is, a pointer whose layout info is composed of
> the following info:
> 
> - a layout (namely, "[ i32 ]")
> - no carrier info
> 
> Note that, since there's no carrier, attempting to call get() on such
> a Pointer will fail with exception. Now, since the pointer here is in
> an argument position, it's not a big deal - e.g. the binder doesn't
> need to produce one, so we're fine (e.g. the pointer will never be
> dereferenced). But if the pointer was in a return position, as in:
> 
> ( ) u64: [ i32 ]
> 
> This would now be an issue, as the returned Pointer<?> will not
> support any dereference? operation:
> 
> Pointer<?> p = mh.invoke();
> p.get(); //exception!
> 
> So the right way to go about this would be to cast the result to the
> right LayoutType - e.g.
> 
> Pointer<?> p = mh.invoke();
> p = p.cast(NativeTypes.INT32);
> int x = p.get(); //ok!
> 
> In other words, I think the invocation game can always be decomposed
> into two stages:
> 
> 1) the first step, works only on erased types and produces types that
> are correct *up to the generic layer*
> 2) the second step, is a fixup step that takes info from source
> signatures (or in other, more explicit ways, as above) and turns
> generic carriers 'right' (typically with a cast)
> 
> Back to your original question of providing binding to separate method
> handle symbol, I now think the answer is yes, *provided* that what you
> mean is (1) - e.g. that you are willing to make up for losses of
> generic type info resulting from the lack of source information that
> directs the binding process.
> 
> Does this help?
> 
> Cheers
> Maurizio

From jbvernee at xs4all.nl  Fri Sep  7 12:49:01 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Fri, 07 Sep 2018 14:49:01 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <63968d742770dc79acb065c190f22cbd@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
Message-ID: <a7c7c58ca8653aab4f0aae8d4169b2e8@xs4all.nl>

> I guess the `LayoutType` of a returned `Pointer` would be
> `LayoutType.ofVoid` before the cast? Or is there a generic fall-back
> carrier type? Maybe a direct `ByteBuffer` that wraps the returned
> pointer, and uses spill-to-heap for non-pointer returns?

Never mind, that would create asymmetrical behaviour; remove an 
indirection if the return type is a Pointer, but not do so otherwise.

Still, just copying whatever the result is into a ByteBuffer might be a 
good default? It's not a _great_ default, because of the copy, but for 
ad-hoc usage it might be good enough? You could always get rid of the 
copy by explicitly supplying a carrier type.

Jorn


From maurizio.cimadamore at oracle.com  Fri Sep  7 12:59:01 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 7 Sep 2018 13:59:01 +0100
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <63968d742770dc79acb065c190f22cbd@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
Message-ID: <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>



On 07/09/18 13:20, Jorn Vernee wrote:
> Yes, I think that would work. As an API point, in that case you could 
> add an overload that also takes the `LayoutType` to cast to, and 
> appends a call to `cast` to the bound method handle i.e.:
>
> ??? MethodHandle bind(MethodType mt, Function f) {...}
>
> ??? MethodHandle bindWithCast(MethodType mt, Function f, LayoutType<?> 
> lt) {
> ??????? MethodHandle bound = bind(mt, f);
> ??????? bound = collectArguments(insertArguments(MH_PointerCast, 1, 
> lt), 0, bound);
> ??????? return bound;
> ??? }
>
> And if the overload without the `LayoutType` is used while the method 
> type returns a `Pointer` throw an IAE?
This is a good idea to allow for more user-friendly behavior.
>
> I guess the `LayoutType` of a returned `Pointer` would be 
> `LayoutType.ofVoid` before the cast? Or is there a generic fall-back 
> carrier type? Maybe a direct `ByteBuffer` that wraps the returned 
> pointer, and uses spill-to-heap for non-pointer returns?
LayoutType.ofVoid is what I was thinking.
>
> Though the solution I was thinking of yesterday was to use a more 
> informative type instead of `MethodType`, that also carries the 
> generic type information. Something using 
> `java.lang.reflect.ParameterizedType` maybe? In that case the right 
> `LayoutType` for the returned `Pointer` could be immediately derived 
> from that, without the need for the extra cast?
This is probably the wrong path - there's no API to construct 
java.lang.reflect.Type(s) in a succint way, so asking something like 
that to users would feel like a step backwards to me.

Maurizio
>
> Jorn
>
> Maurizio Cimadamore schreef op 2018-09-07 09:52:
>> On 06/09/18 23:41, Maurizio Cimadamore wrote:
>>> Of course you can infer a 'default' carrier information from a 
>>> layout, but in the general case you can't - think of structs 
>>> (modeled as classes) and function pointers (modeled as functional 
>>> interfaces) - how do you know whether something is a Foo or a Bar, 
>>> if both Foo and Bar are structs with same fields?
>> Pulling more on this string as I kept thinking about this: adding
>> carrier info will help to distinguish a toplevel Foo from a toplevel
>> Bar - e.g. let's make this more concrete:
>>
>>
>> @NativeStruct("[ i32(get=x) ](foo)")
>> interface Foo extends Struct<Foo> {
>> ?? int x();
>> }
>>
>> @NativeStruct("[ i32(get=y) ](bar)")
>> interface Bar extends Struct<Bar> {
>> ?? int y();
>> }
>>
>> These two struct declarations are isomorphic - you can use one in
>> place of another; since in a .so/.dynlib file there's no info about
>> struct names (well in the absence of debugging symbol which I assume
>> to be the norm), if you want to dynamically bind to a native function
>> whose descriptor is:
>>
>> ( [ i32 ])v
>>
>> What should the binder do? E.g. what should be the Java type
>> associated with the argument of this function? Here's where the
>> MethodType is handy - as it could help the inference process
>> understanding whether we want our method handle to accept a Foo or a
>> Bar thingie.
>>
>> But, I realized, this trick doesn't have a lot of mileage: the next
>> example up is with pointers - so let's assume the function descriptor
>> is now changed to:
>>
>> ( u64: [ i32 ] ) v
>>
>> Ok, now we have a pointer to a thingie with one 32-bit signed int
>> inside. Is that Pointer<Foo> or Pointer<Bar> ? This time,
>> unfortunately, the MethodType won't help - class types in method types
>> are erased, so the method type for this will be something like
>>
>> (Pointer)void
>>
>> Which is not enough for our inference process to resolve the ambiguity.
>>
>> I believe the next best thing here is for our process to end up with
>> some Pointer<?> - that is, a pointer whose layout info is composed of
>> the following info:
>>
>> - a layout (namely, "[ i32 ]")
>> - no carrier info
>>
>> Note that, since there's no carrier, attempting to call get() on such
>> a Pointer will fail with exception. Now, since the pointer here is in
>> an argument position, it's not a big deal - e.g. the binder doesn't
>> need to produce one, so we're fine (e.g. the pointer will never be
>> dereferenced). But if the pointer was in a return position, as in:
>>
>> ( ) u64: [ i32 ]
>>
>> This would now be an issue, as the returned Pointer<?> will not
>> support any dereference? operation:
>>
>> Pointer<?> p = mh.invoke();
>> p.get(); //exception!
>>
>> So the right way to go about this would be to cast the result to the
>> right LayoutType - e.g.
>>
>> Pointer<?> p = mh.invoke();
>> p = p.cast(NativeTypes.INT32);
>> int x = p.get(); //ok!
>>
>> In other words, I think the invocation game can always be decomposed
>> into two stages:
>>
>> 1) the first step, works only on erased types and produces types that
>> are correct *up to the generic layer*
>> 2) the second step, is a fixup step that takes info from source
>> signatures (or in other, more explicit ways, as above) and turns
>> generic carriers 'right' (typically with a cast)
>>
>> Back to your original question of providing binding to separate method
>> handle symbol, I now think the answer is yes, *provided* that what you
>> mean is (1) - e.g. that you are willing to make up for losses of
>> generic type info resulting from the lack of source information that
>> directs the binding process.
>>
>> Does this help?
>>
>> Cheers
>> Maurizio


From vivek.r.deshpande at intel.com  Fri Sep  7 22:54:10 2018
From: vivek.r.deshpande at intel.com (Deshpande, Vivek R)
Date: Fri, 7 Sep 2018 22:54:10 +0000
Subject: [vector] Mask and Shuffle construction
In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE3AC9B@FMSMSX126.amr.corp.intel.com>
References: <E95AF1CC-CDC2-40FD-AE89-0B4DFC28C583@oracle.com>
 <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE3AC3E@FMSMSX126.amr.corp.intel.com>
 <630370FA-2CF0-4461-9DD2-D3577106D01A@oracle.com>
 <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE3AC9B@FMSMSX126.amr.corp.intel.com>
Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A9A09ACC2@ORSMSX106.amr.corp.intel.com>

Hi All

I have updated the intrinsics for the rearrange API according to the changes to the Shuffle Vector.
The webrev is here:
http://cr.openjdk.java.net/~vdeshpande/VectorAPI_swizzle/rearrange/webrev.00/
Could you please review the changes.

Regards,
Vivek

-----Original Message-----
From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Viswanathan, Sandhya
Sent: Thursday, May 24, 2018 5:38 PM
To: Paul Sandoz <paul.sandoz at oracle.com>
Cc: panama-dev at openjdk.java.net
Subject: RE: [vector] Mask and Shuffle construction

Hi Paul,

Thanks for correcting me on this, we should then keep both FromValues and FromArrays versions for now and hope to get to them for efficient backend implementation.
 
On your other question, a ByteVector or byte[] as internal representation for Shuffle would work if we implement shuffle() in terms of swizzle() as you proposed earlier.  Looks like this will also reduce the intrinsic work upfront and we would just need optimized swizzle implementation which is already in place. The swizzle intrinsics would need to be adjusted slightly which we can do easily I think.

We look forward to the improved/cleaner Shuffle implementation per your new proposal. 

Thanks,
Sandhya


-----Original Message-----
From: Paul Sandoz [mailto:paul.sandoz at oracle.com]
Sent: Thursday, May 24, 2018 4:47 PM
To: Viswanathan, Sandhya <sandhya.viswanathan at intel.com>
Cc: panama-dev at openjdk.java.net
Subject: Re: [vector] Mask and Shuffle construction

It?s awkward because we need an obvious way to construct. One approach is to start from a Vector and test, which for Mask we can do now.

In fact? for Mask i think there may be a way out, since in HotSpot a boolean[] is internally represented as byte[] so it might be possible to piggy back off that representation.

For shuffle i am working on a Vector.toShuffle the current implementation is:

@Override
@ForceInline
public Shuffle<Byte, Shapes.S128Bit> toShuffle() {
    byte[] a = toArray();
    int[] sa = new int[a.length];
    for (int i = 0; i < a.length; i++) {
        sa[i] = (int) a[i] & (a.length - 1);
    }
    return SPECIES.shuffleFromValues(sa); }

Not particularly efficient but i wonder if we could also piggy back off byte[] or a ByteVector as the internal representation?

Note that the fromValues methods are just varargs whose implementations pass the array to the fromArray method, so its really the fromArray methods that would need to be optimized.

Paul.


> On May 24, 2018, at 4:18 PM, Viswanathan, Sandhya <sandhya.viswanathan at intel.com> wrote:
> 
> Hi Paul,
> 
> Efficient optimized implementation for maskFromValues and shuffleFromValue is challenging so we should remove those two methods.
> 
> Best Regards,
> Sandhya
> 
> 
> -----Original Message-----
> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On 
> Behalf Of Paul Sandoz
> Sent: Thursday, May 24, 2018 3:45 PM
> To: panama-dev at openjdk.java.net
> Subject: [vector] Mask and Shuffle construction
> 
> Hi,
> 
> What plans do we have to optimize construction of Mask and Shuffle? i cannot recall where we are on this.
> 
> As a summary here are the methods on Species:
> 
>  public abstract Mask<E, S> maskFromValues(boolean... bits);  public 
> abstract Mask<E, S> maskFromArray(boolean[] a, int i);  public 
> abstract Shuffle<E, S> shuffleFromValues(int... indexes);  public 
> abstract Shuffle<E, S> shuffleFromArray(int[] a, int i);
> 
> Thanks,
> Paul.
> 
> 


From samuel.audet at gmail.com  Sun Sep  9 11:59:07 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Sun, 9 Sep 2018 20:59:07 +0900
Subject: [nicl] branch is now closed
In-Reply-To: <067E7B33-BFFC-4784-B52D-E4818B984764@oracle.com>
References: <4119a11e-df50-75dd-f5fe-5317b1e90d5b@oracle.com>
 <2d30ed4d-802e-dfe9-5e25-bed2cd0ca8a8@gmail.com>
 <60b52c13-a54e-573b-ae21-48a1c6facfa9@oracle.com>
 <8998a90f-24f8-928e-7d8a-e6803736fbe3@gmail.com>
 <4c4a6d23-2d35-9f18-06e9-3abfd5372e09@oracle.com>
 <fd158809-df2a-be15-8c08-97702ed30cf6@gmail.com>
 <793f413e-0567-9290-c44f-d2e8b327fbd6@oracle.com>
 <e31a59d5-44ff-06f7-aa6f-cbf4431a7d0a@gmail.com>
 <067E7B33-BFFC-4784-B52D-E4818B984764@oracle.com>
Message-ID: <1f700c1c-03a1-a292-dfa3-13aeafaea714@gmail.com>

On 09/03/2018 11:55 AM, Henry Jen wrote:
>> However, the fact is, there is still no cooperation whatsoever between Panama and GraalVM, but the parallel are striking, see this for example:https://cornerwings.github.io/2018/07/graal-native-methods/
>> 	Method		 	Time per iteration
>> 	graal/native-image 	76.11 ns
>> 	jni/unsafe 		109.42 ns
>> Trying to get these teams on the same page might be a good start here. Why isn't this happening? I think it would be a useful exercise to understand the internal politics of Oracle, but maybe we don't need to know the details, just that the internal disputes are taken care of?
>>
> Correct me if I am wrong, but those are different approaches and not exclusive. We noted that Graal polyglot solution, but it needs source code or LLVM bitcode, while here at Panama is focused on platform native format.

Sulong uses LLVM to support native languages like Fortran and C++, as 
well as their native libraries.

The module to call system functions isn't part of Sulong, it's part of 
the "Substrate VM", which is unrelated to LLVM or Sulong.

That said, unlike Panama, the GraalVM team isn't against relying on 
other libraries such as LLVM to support things like C++... I'll repeat 
as many times as I keep hearing people mentioning C++ alongside Panama: 
You will need to figure out how to cooperate with the community!

Samuel

From samuel.audet at gmail.com  Sun Sep  9 13:07:42 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Sun, 9 Sep 2018 22:07:42 +0900
Subject: [nicl] branch is now closed
In-Reply-To: <e664f7a2-baf0-0591-6af9-ad3bb53666a3@oracle.com>
References: <4119a11e-df50-75dd-f5fe-5317b1e90d5b@oracle.com>
 <2d30ed4d-802e-dfe9-5e25-bed2cd0ca8a8@gmail.com>
 <60b52c13-a54e-573b-ae21-48a1c6facfa9@oracle.com>
 <8998a90f-24f8-928e-7d8a-e6803736fbe3@gmail.com>
 <4c4a6d23-2d35-9f18-06e9-3abfd5372e09@oracle.com>
 <fd158809-df2a-be15-8c08-97702ed30cf6@gmail.com>
 <793f413e-0567-9290-c44f-d2e8b327fbd6@oracle.com>
 <e31a59d5-44ff-06f7-aa6f-cbf4431a7d0a@gmail.com>
 <e664f7a2-baf0-0591-6af9-ad3bb53666a3@oracle.com>
Message-ID: <CAHQD+UyDUojw4fkFCsY43UtNAk7iedJa1Urn4dmTOQsd6mXJGg@mail.gmail.com>

Hi, Maurizio,

Sounds good! Looking forward to the meetings.

And I think we're already seeing more action from ARM, awesome!

Samuel

2018?9?3?(?) 18:54 Maurizio Cimadamore <maurizio.cimadamore at oracle.com>:

>
>
> On 01/09/18 07:28, Samuel Audet wrote:
> > Now this is starting to sound like a plan! The point isn't about
> > pointers or arrays, I'm not aware of all the discussions that went
> > into that decision, and this is the point. All decisions related to
> > Panama are made behind closed doors with no input from the community.
> > It's the first time I hear about most of what you've just made public
> > below, so this is a good start. Please keep making this kind of
> > information public. I'm sure it will help make people feel welcome.
> Hi Samuel,
> point noted; sometimes emails are just not a suitable medium for
> capturing design decisions. And if there's something we should be doing
> more of, is trying to open up some of the discussions that sometimes we
> have internally (although the ration between open and closed discussion
> is nowhere near what you imagine); there's nothing fundamentally private
> to these communications, think of it as 'laziness' or habit on our part.
> I've been trying to be more upfront about our goals - and have some
> write-ups pushed out _before_ starting to make any significant changes
> to the code base. I guess sometimes that works, sometimes it doesn't and
> we need help from all of you to call us out when it doesn't, so thank you.
>
> All this said, I'd say that most of the information is available out
> there - and when it's not, trust me, it's not because of some evil
> machination on our part; writing things up in a decent way takes time
> too, and we have to balance that too - if we still want to have time
> left to push the code changes out :-)
>
> As  a closing comment, at the OpenJDK Committer Workshop [1] very
> similar concerns were raised - and Brian Goetz proposed that some of
> these projects (Valhalla, Panama, Amber) could have some periodic public
> meetup (e.g. via Zoom) where stakeholders could meet architects and
> developers and make sure that their voice is heard. I think this could
> go a long way in helping us understand how to improve our
> communications. As we move closer to an early access state, I think we
> will host one such experimental call (details TBD), so that we can have
> a chat of what will be in the EA, what will not be in it, and what's our
> plan to move the ball forward.
>
> Cheers
> Maurizio
>
> [1] - http://openjdk.java.net/workshop
>

From jbvernee at xs4all.nl  Sun Sep  9 19:45:11 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Sun, 09 Sep 2018 21:45:11 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
 <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
Message-ID: <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>

Hello,

It turns out I was using an older build of the branch (sorry, TIL 'hg 
pull' only works for the active branch). I also applied your recent 
Callback patch, and what I was trying to do was almost already possible 
through another route.

This is the code I'm going for:

     @NativeCallback("(u64:i8)i32")
     public interface PutsCallback {
         int call(Pointer<Byte> message);
     }

     public static void main(String[] args) throws Throwable {
         Library lib = Libraries.loadLibrary(lookup(), "msvcrt");
         Symbol sym = lib.lookup("puts");

         Callback<PutsCallback> clbck = 
sym.getAddress().asCallback(PutsCallback.class);
         PutsCallback puts = clbck.asFunction();

         Scope scope = Scope.newNativeScope();
         Pointer<Byte> message = scope.toCString("Hello!");
         System.out.println(puts.call(message));
     }

Instead of passing an explicit MethodType, the method type is derived 
from the SAM of a functional interface, which also solves the problem of 
missing generic type parameters.

There were 2 changes I made when trying to make this work:

Firstly, I had to implement asCallback, which basically just creates a 
new CallbackImpl with the pointer itself as the entry point.

Secondly, the CallbackImpl::asFunction method uses the Scope of it's 
entry point, but in the case of a library symbol that scope is null, so 
it fails with an NPE. As a workaround I'm creating (and leaking) a new 
native scope in case the scope is null. Maybe a longer term solution 
could be something like library symbol pointers returning a Scope that 
is tied to the lifetime of the library itself?

The code half works; It prints the message, and then it crashes [1] but 
that might just be because I'm on windows which is not supported? At 
least I think this API is pretty nice. I haven't been able to build 
jextract yet, so thorough testing is difficult.

You've already said you're working on other things right now, but at 
least I wanted to send this for future reference.

Jorn

[1] : 
https://gist.github.com/JornVernee/dda1d3e42158dfddddb3ba60060e25ae

From maurizio.cimadamore at oracle.com  Mon Sep 10 11:00:19 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 10 Sep 2018 12:00:19 +0100
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
 <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
 <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>
Message-ID: <87181940-517f-b669-8479-698a9e93d107@oracle.com>



On 09/09/18 20:45, Jorn Vernee wrote:
> Hello,
>
> It turns out I was using an older build of the branch (sorry, TIL 'hg 
> pull' only works for the active branch). I also applied your recent 
> Callback patch, and what I was trying to do was almost already 
> possible through another route.
>
> This is the code I'm going for:
>
> ??? @NativeCallback("(u64:i8)i32")
> ??? public interface PutsCallback {
> ??????? int call(Pointer<Byte> message);
> ??? }
>
> ??? public static void main(String[] args) throws Throwable {
> ??????? Library lib = Libraries.loadLibrary(lookup(), "msvcrt");
> ??????? Symbol sym = lib.lookup("puts");
>
> ??????? Callback<PutsCallback> clbck = 
> sym.getAddress().asCallback(PutsCallback.class);
> ??????? PutsCallback puts = clbck.asFunction();
>
> ??????? Scope scope = Scope.newNativeScope();
> ??????? Pointer<Byte> message = scope.toCString("Hello!");
> ??????? System.out.println(puts.call(message));
> ??? }
>
> Instead of passing an explicit MethodType, the method type is derived 
> from the SAM of a functional interface, which also solves the problem 
> of missing generic type parameters.
>
> There were 2 changes I made when trying to make this work:
>
> Firstly, I had to implement asCallback, which basically just creates a 
> new CallbackImpl with the pointer itself as the entry point.
That's very clever - but what's the difference, may I ask, between this and:

@NativeHeader("puts=(u64:i8)i32")
 ??? public interface PutsLib {
 ??????? int puts(Pointer<Byte> message);
 ??? }

And then bind the interface with Libraries.bind ? Actually, I believe 
this latter version should be more performant, as the spinned 
implementation generated by the binder will just have a direct exact 
method handle call to the native code; whereas in the callback case 
there are some few extra checks (e.g. to check that the scope is still 
alive and also to check whether the entry point is a real native 
function or a synthetic stub generated by the binder, in which case we 
shortcircuit and do a Java 2 Java call directly).
>
> Secondly, the CallbackImpl::asFunction method uses the Scope of it's 
> entry point, but in the case of a library symbol that scope is null, 
> so it fails with an NPE. As a workaround I'm creating (and leaking) a 
> new native scope in case the scope is null. Maybe a longer term 
> solution could be something like library symbol pointers returning a 
> Scope that is tied to the lifetime of the library itself?
The idea (not implemented yet) is that each Library will have a 
meaningful Scope that will stay alive as long as that library is loaded. 
That Scope will also be used internally by the binder to, e.g. allocate 
constants, or global variable values, etc. So your code will be able to 
piggy back on that.
>
> The code half works; It prints the message, and then it crashes [1] 
> but that might just be because I'm on windows which is not supported? 
> At least I think this API is pretty nice. I haven't been able to build 
> jextract yet, so thorough testing is difficult.
Yeah - windows is not supported - I'm actually surprised it works at 
all, given that the registers used by the Windows ABI and system V ABI 
are quite different; for instance, System V (the implemented one) passes 
its first integer argument via the RDI register, whereas the Windows ABI 
uses RCX; so the failure you are seeing is related to the fact that the 
current code doesn't arrange the arguments/return values in a way that 
is compatible with the underlying system ABI on your machine.

But it seems like you are taking the code for some serious spin, and 
that's some invaluable feedback to us - thanks!

Maurizio
>
> You've already said you're working on other things right now, but at 
> least I wanted to send this for future reference.
>
> Jorn
>
> [1] : https://gist.github.com/JornVernee/dda1d3e42158dfddddb3ba60060e25ae


From jbvernee at xs4all.nl  Mon Sep 10 12:24:24 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 10 Sep 2018 14:24:24 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <87181940-517f-b669-8479-698a9e93d107@oracle.com>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
 <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
 <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>
 <87181940-517f-b669-8479-698a9e93d107@oracle.com>
Message-ID: <a8c5c210fc4fb93f4cedb68c02eac4ac@xs4all.nl>

>> Firstly, I had to implement asCallback, which basically just creates a 
>> new CallbackImpl with the pointer itself as the entry point.
> That's very clever - but what's the difference, may I ask, between this 
> and:
> 
> @NativeHeader("puts=(u64:i8)i32")
> ??? public interface PutsLib {
> ??????? int puts(Pointer<Byte> message);
> ??? }
> 
> And then bind the interface with Libraries.bind ? Actually, I believe
> this latter version should be more performant, as the spinned
> implementation generated by the binder will just have a direct exact
> method handle call to the native code; whereas in the callback case
> there are some few extra checks (e.g. to check that the scope is still
> alive and also to check whether the entry point is a real native
> function or a synthetic stub generated by the binder, in which case we
> shortcircuit and do a Java 2 Java call directly).

Yeah, I suppose there is not much difference if I'm using a pointer to a 
library symbol. My views on this have changed, I assumed wrapping a 
single function in a MethodHandle would be easier. But, you still need 
the same information, using an interface is just a different way of 
providing it. Though my idea might still be a good solution for when you 
have an arbitrary void Pointer that you want to make invokable, or when 
you want to 'cast' a Callback to a different type (i.e. 
Callback.entryPoint() -> void Pointer -> 
Pointer.asCallback(SomeOtherCallbackType.class)) ?

I also had another version which used the result of 
`NativeInvoker::getBoundMethodHandle` prefixed with a call to 
`Pointer::addr`, also adding a constructor to NativeInvoker that took a 
MethodType. That was working just as well, but this idea seemed more 
interesting for the average use cases, since it's a more high-level API. 
The only difference in capability I see between the two, is that the 
MethodType + Function combination can be derived dynamically, where as 
with an interface you would need the information to be statically know 
(or spin the interface, which is 'meh' for usability imho).

>> Secondly, the CallbackImpl::asFunction method uses the Scope of it's 
>> entry point, but in the case of a library symbol that scope is null, 
>> so it fails with an NPE. As a workaround I'm creating (and leaking) a 
>> new native scope in case the scope is null. Maybe a longer term 
>> solution could be something like library symbol pointers returning a 
>> Scope that is tied to the lifetime of the library itself?
> The idea (not implemented yet) is that each Library will have a
> meaningful Scope that will stay alive as long as that library is
> loaded. That Scope will also be used internally by the binder to, e.g.
> allocate constants, or global variable values, etc. So your code will
> be able to piggy back on that.

Sounds great!

>> The code half works; It prints the message, and then it crashes [1] 
>> but that might just be because I'm on windows which is not supported? 
>> At least I think this API is pretty nice. I haven't been able to build 
>> jextract yet, so thorough testing is difficult.
> Yeah - windows is not supported - I'm actually surprised it works at
> all, given that the registers used by the Windows ABI and system V ABI
> are quite different; for instance, System V (the implemented one)
> passes its first integer argument via the RDI register, whereas the
> Windows ABI uses RCX; so the failure you are seeing is related to the
> fact that the current code doesn't arrange the arguments/return values
> in a way that is compatible with the underlying system ABI on your
> machine.

Using -Xlog:panama I did see the first argument being put into RCX. 
Maybe that's just a coincidence? tbh I didn't know different ABIs were 
being used. For the standard library I always thought 32bit used cdecl, 
and 64bit used fastcall, which I thought would use the same registers on 
all systems. That's interesting to hear (I don't dabble in ABIs too 
often).

> But it seems like you are taking the code for some serious spin, and
> that's some invaluable feedback to us - thanks!

Glad to be doing it! The OpenJDK code is very interesting to look at 
(though daunting at times). It's also a very cool feeling that I can 
actually change the machinery I'm running my programs on, even if it's 
just to do experiments with :)

Jorn

> Maurizio

From maurizio.cimadamore at oracle.com  Mon Sep 10 13:38:49 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 10 Sep 2018 14:38:49 +0100
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <a8c5c210fc4fb93f4cedb68c02eac4ac@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
 <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
 <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>
 <87181940-517f-b669-8479-698a9e93d107@oracle.com>
 <a8c5c210fc4fb93f4cedb68c02eac4ac@xs4all.nl>
Message-ID: <f340f338-4108-f648-c915-a6134d77b183@oracle.com>



On 10/09/18 13:24, Jorn Vernee wrote:
> Yeah, I suppose there is not much difference if I'm using a pointer to 
> a library symbol. My views on this have changed, I assumed wrapping a 
> single function in a MethodHandle would be easier. But, you still need 
> the same information, using an interface is just a different way of 
> providing it. Though my idea might still be a good solution for when 
> you have an arbitrary void Pointer that you want to make invokable, or 
> when you want to 'cast' a Callback to a different type (i.e. 
> Callback.entryPoint() -> void Pointer -> 
> Pointer.asCallback(SomeOtherCallbackType.class)) ?
Sure - the allocateCallback is, as you note, a way to convert function 
pointers into functional interfaces callable from Java, so that's 
correct. It's just that, in your case, since the symbol is publicly 
available in the library, you are doing by hand, what Libraries::bind 
already does for free ;-)

But yes, if the native code were to allocate some code stub and you 
wanted to call it from Java, in that case you'd have to use the Callback 
mechanism (as that would NOT be a library symbol).

<snip>
> Yeah - windows is not supported - I'm actually surprised it works at
>> all, given that the registers used by the Windows ABI and system V ABI
>> are quite different; for instance, System V (the implemented one)
>> passes its first integer argument via the RDI register, whereas the
>> Windows ABI uses RCX; so the failure you are seeing is related to the
>> fact that the current code doesn't arrange the arguments/return values
>> in a way that is compatible with the underlying system ABI on your
>> machine.
>
> Using -Xlog:panama I did see the first argument being put into RCX. 
> Maybe that's just a coincidence? tbh I didn't know different ABIs were 
> being used. For the standard library I always thought 32bit used 
> cdecl, and 64bit used fastcall, which I thought would use the same 
> registers on all systems. That's interesting to hear (I don't dabble 
> in ABIs too often).
Ah - now that I think about it, the hotspot code abstracts over details 
such as these - as internally it uses register names such as

c_rarg0 .... c_rarg_N,? for integer registers

and

c_farg0 ... c_farm_M, for FP registers

In fact, looking more in details at nativeInvoker_x86.cpp, it looks like 
the right thing happens:

static Register 
integer_argument_registers[INTEGER_ARGUMENT_REGISTERS_NOOF] = {
 ? c_rarg0, c_rarg1, c_rarg2, c_rarg3,
#ifndef _WIN64
 ? c_rarg4, c_rarg5
#endif
};

static Register integer_return_registers[INTEGER_RETURN_REGISTERS_NOOF] = {
 ? rax,
#ifndef _WIN64
 ? rdx
#endif
};

static XMMRegister 
vector_argument_registers[VECTOR_ARGUMENT_REGISTERS_NOOF] = {
 ? c_farg0, c_farg1, c_farg2, c_farg3,
#ifndef _WIN64
 ? c_farg4, c_farg5, c_farg6, c_farg7
#endif
};

static XMMRegister vector_return_registers[VECTOR_RETURN_REGISTERS_NOOF] = {
 ? xmm0,
#ifndef _WIN64
 ? xmm1
#endif
};


So, for simple calls like puts, which take a single integer register 
value and returns via a single register value, things *should* work - 
but it seems like it doesn't. Anyway, the current support is, as I 
wrote, mostly Linux and MacOs centric - and we're not planning to do 
Windows tweak in the short term at least. But your experiment suggests 
that we might not be that far off the mark...

Thanks
Maurizio

From jbvernee at xs4all.nl  Mon Sep 10 14:33:48 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 10 Sep 2018 16:33:48 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <f340f338-4108-f648-c915-a6134d77b183@oracle.com>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
 <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
 <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>
 <87181940-517f-b669-8479-698a9e93d107@oracle.com>
 <a8c5c210fc4fb93f4cedb68c02eac4ac@xs4all.nl>
 <f340f338-4108-f648-c915-a6134d77b183@oracle.com>
Message-ID: <0a11eba191efa7f19f991922a5e7b426@xs4all.nl>

> So, for simple calls like puts, which take a single integer register
> value and returns via a single register value, things *should* work -
> but it seems like it doesn't. Anyway, the current support is, as I
> wrote, mostly Linux and MacOs centric - and we're not planning to do
> Windows tweak in the short term at least. But your experiment suggests
> that we might not be that far off the mark...

I was just reading up on the MS x64 ABI [1], and the main difference 
seems to be that the caller is expected to allocate 32 bytes of 'shadow 
space' on the stack right before the call so that the function can do 
argument register spill into that space (I guess to make it easier to 
forward to cdecl functions?). It doesn't look like the current 
implementation accounts for that (?), so puts is probably corrupting the 
stack values that are there, and it blows up on return.

Jorn

[1] : 
https://en.wikipedia.org/wiki/X86_calling_conventions#Microsoft_x64_calling_convention

From jbvernee at xs4all.nl  Tue Sep 11 08:48:29 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Tue, 11 Sep 2018 10:48:29 +0200
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <0a11eba191efa7f19f991922a5e7b426@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
 <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
 <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>
 <87181940-517f-b669-8479-698a9e93d107@oracle.com>
 <a8c5c210fc4fb93f4cedb68c02eac4ac@xs4all.nl>
 <f340f338-4108-f648-c915-a6134d77b183@oracle.com>
 <0a11eba191efa7f19f991922a5e7b426@xs4all.nl>
Message-ID: <9049f93356e088934813362c15aa2e56@xs4all.nl>

>> So, for simple calls like puts, which take a single integer register
>> value and returns via a single register value, things *should* work -
>> but it seems like it doesn't. Anyway, the current support is, as I
>> wrote, mostly Linux and MacOs centric - and we're not planning to do
>> Windows tweak in the short term at least. But your experiment suggests
>> that we might not be that far off the mark...

Got it working!

In the end I made 3 changes [1] :

* Allocate 32 bytes of shadow space before the call.

* Make sure the number of return registers is set to 1 on windows (it 
was hard-coded as 2).

* Make sure RDI and RSI are preserved by the native stub.

`printf` also works for simple calls. So yeah, not far off the mark at 
all.

Again; just leaving this here for future reference :)

Jorn

[1] : 
https://gist.github.com/JornVernee/240a079abe6791b3fa5c9d14613ef28d

From maurizio.cimadamore at oracle.com  Tue Sep 11 11:26:10 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 11 Sep 2018 12:26:10 +0100
Subject: Binding a single function symbol with [foreign]
In-Reply-To: <9049f93356e088934813362c15aa2e56@xs4all.nl>
References: <732ae233bc0dc4d2df0b148461a08cb0@xs4all.nl>
 <4bc1e2a1-3459-538c-3803-9930111cebf0@oracle.com>
 <c61942c64c41ebcabc37cda5d7c2c1a9@xs4all.nl>
 <627eb3c1-2a2f-f946-05d5-bc92f2449fc4@oracle.com>
 <07dbf628-579e-8f3e-ed9a-d569d3048c79@oracle.com>
 <63968d742770dc79acb065c190f22cbd@xs4all.nl>
 <d060787f-c5de-596a-ef48-8e3c26b8ac0f@oracle.com>
 <bb9502159be92420d8ba8979a7a6d902@xs4all.nl>
 <87181940-517f-b669-8479-698a9e93d107@oracle.com>
 <a8c5c210fc4fb93f4cedb68c02eac4ac@xs4all.nl>
 <f340f338-4108-f648-c915-a6134d77b183@oracle.com>
 <0a11eba191efa7f19f991922a5e7b426@xs4all.nl>
 <9049f93356e088934813362c15aa2e56@xs4all.nl>
Message-ID: <dd7c13f0-a18a-f0cb-970e-b061870ee90c@oracle.com>

Thanks for this Jorn,

that's really wonderful news!

Maurizio


On 11/09/18 09:48, Jorn Vernee wrote:
>>> So, for simple calls like puts, which take a single integer register
>>> value and returns via a single register value, things *should* work -
>>> but it seems like it doesn't. Anyway, the current support is, as I
>>> wrote, mostly Linux and MacOs centric - and we're not planning to do
>>> Windows tweak in the short term at least. But your experiment suggests
>>> that we might not be that far off the mark...
>
> Got it working!
>
> In the end I made 3 changes [1] :
>
> * Allocate 32 bytes of shadow space before the call.
>
> * Make sure the number of return registers is set to 1 on windows (it 
> was hard-coded as 2).
>
> * Make sure RDI and RSI are preserved by the native stub.
>
> `printf` also works for simple calls. So yeah, not far off the mark at 
> all.
>
> Again; just leaving this here for future reference :)
>
> Jorn
>
> [1] : https://gist.github.com/JornVernee/240a079abe6791b3fa5c9d14613ef28d


From sundararajan.athijegannathan at oracle.com  Wed Sep 12 13:48:30 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Wed, 12 Sep 2018 19:18:30 +0530
Subject: [foreign] RFR 8210650: Cursor.isAnonymousStruct is about embedded
 anon struct/union and not just name being empty
Message-ID: <5B99192E.1010709@oracle.com>

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8210650

Webrev: http://cr.openjdk.java.net/~sundar/8210650/webrev.00/

Includes other cleanups regarding equals, toString in Cursor, Type and 
SourceLocation.

Thanks,
-Sundar

From henry.jen at oracle.com  Wed Sep 12 15:09:46 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 12 Sep 2018 08:09:46 -0700
Subject: [foreign] RFR 8210650: Cursor.isAnonymousStruct is about embedded
 anon struct/union and not just name being empty
In-Reply-To: <5B99192E.1010709@oracle.com>
References: <5B99192E.1010709@oracle.com>
Message-ID: <EFD4DBE2-3A60-4DCB-AF40-FCE12927161C@oracle.com>

Looks good. I would like to see changes in jdk.internal.clang code propagate to the FFI test as well, as the FFI based jdk.internal.clang module should at least offer same capabilities as JNI base.

Cheers,
Henry

> On Sep 12, 2018, at 6:48 AM, Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com> wrote:
> 
> Please review.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8210650
> 
> Webrev: http://cr.openjdk.java.net/~sundar/8210650/webrev.00/
> 
> Includes other cleanups regarding equals, toString in Cursor, Type and SourceLocation.
> 
> Thanks,
> -Sundar


From sundararajan.athijegannathan at oracle.com  Wed Sep 12 15:53:55 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Wed, 12 Sep 2018 21:23:55 +0530
Subject: [foreign] RFR 8210650: Cursor.isAnonymousStruct is about embedded
 anon struct/union and not just name being empty
In-Reply-To: <EFD4DBE2-3A60-4DCB-AF40-FCE12927161C@oracle.com>
References: <5B99192E.1010709@oracle.com>
 <EFD4DBE2-3A60-4DCB-AF40-FCE12927161C@oracle.com>
Message-ID: <5B993693.9070404@oracle.com>

Hi,

Thanks for your review.

Updated: http://cr.openjdk.java.net/~sundar/8210650/webrev.01/

Thanks,
-Sundar

On 12/09/18, 8:39 PM, Henry Jen wrote:
> Looks good. I would like to see changes in jdk.internal.clang code propagate to the FFI test as well, as the FFI based jdk.internal.clang module should at least offer same capabilities as JNI base.
>
> Cheers,
> Henry
>
>> On Sep 12, 2018, at 6:48 AM, Sundararajan Athijegannathan<sundararajan.athijegannathan at oracle.com>  wrote:
>>
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210650
>>
>> Webrev: http://cr.openjdk.java.net/~sundar/8210650/webrev.00/
>>
>> Includes other cleanups regarding equals, toString in Cursor, Type and SourceLocation.
>>
>> Thanks,
>> -Sundar

From henry.jen at oracle.com  Wed Sep 12 15:57:08 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 12 Sep 2018 08:57:08 -0700
Subject: [foreign] RFR 8210650: Cursor.isAnonymousStruct is about embedded
 anon struct/union and not just name being empty
In-Reply-To: <5B993693.9070404@oracle.com>
References: <5B99192E.1010709@oracle.com>
 <EFD4DBE2-3A60-4DCB-AF40-FCE12927161C@oracle.com>
 <5B993693.9070404@oracle.com>
Message-ID: <2A1C8E88-CFFF-4095-8B6D-BBE3BCB1C5F3@oracle.com>

+1. 

Cheers,
Henry


> On Sep 12, 2018, at 8:53 AM, Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com> wrote:
> 
> Hi,
> 
> Thanks for your review.
> 
> Updated: http://cr.openjdk.java.net/~sundar/8210650/webrev.01/
> 
> Thanks,
> -Sundar
> 
> On 12/09/18, 8:39 PM, Henry Jen wrote:
>> Looks good. I would like to see changes in jdk.internal.clang code propagate to the FFI test as well, as the FFI based jdk.internal.clang module should at least offer same capabilities as JNI base.
>> 
>> Cheers,
>> Henry
>> 
>>> On Sep 12, 2018, at 6:48 AM, Sundararajan Athijegannathan<sundararajan.athijegannathan at oracle.com>  wrote:
>>> 
>>> Please review.
>>> 
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210650
>>> 
>>> Webrev: http://cr.openjdk.java.net/~sundar/8210650/webrev.00/
>>> 
>>> Includes other cleanups regarding equals, toString in Cursor, Type and SourceLocation.
>>> 
>>> Thanks,
>>> -Sundar


From sundararajan.athijegannathan at oracle.com  Wed Sep 12 16:12:02 2018
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Wed, 12 Sep 2018 16:12:02 +0000
Subject: hg: panama/dev: 8210650: Cursor.isAnonymousStruct is about embedded
 anon struct/union and not just name being empty
Message-ID: <201809121612.w8CGC3Rq011289@aojmv0008.oracle.com>

Changeset: dacf59a79319
Author:    sundar
Date:      2018-09-12 21:49 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/dacf59a79319

8210650: Cursor.isAnonymousStruct is about embedded anon struct/union and not just name being empty
Reviewed-by: henryjen

! src/jdk.internal.clang/share/classes/jdk/internal/clang/Cursor.java
! src/jdk.internal.clang/share/classes/jdk/internal/clang/SourceLocation.java
! src/jdk.internal.clang/share/classes/jdk/internal/clang/Type.java
! src/jdk.internal.clang/share/native/libjclang/jdk_internal_clang.cpp
! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/EnumTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/StructTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/Tree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreePrinter.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Cursor.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/SourceLocation.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Type.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 16:16:43 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 16:16:43 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809121616.w8CGGhBB013283@aojmv0008.oracle.com>

Changeset: b82557c428d6
Author:    mcimadamore
Date:      2018-09-12 18:19 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b82557c428d6

Automatic merge with foreign



From jbvernee at xs4all.nl  Wed Sep 12 17:27:59 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 12 Sep 2018 19:27:59 +0200
Subject: [foreign] NPE when trying to lookup a symbol in the default library
 when no libraries are loaded
Message-ID: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>

Hello,

I ran into an NPE while running some of the tests for foreign. This is 
the (much reduced) code to reproduce:

     Libraries.getDefaultLibrary().lookup("");

Throwing:

     Exception in thread "main" java.lang.NullPointerException
	at 
java.base/java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:2545)
	at java.base/java.lang.ClassLoader$NativeLibrary.findEntry(Native 
Method)
	at 
java.base/java.lang.ClassLoader$NativeLibrary.lookup(ClassLoader.java:2426)
	at com.github.jdnl.Main.main(Main.java:108)

If a NativeHeader annotation does not have a value for the `libraries` 
field, the default library is used for lookup (see 
LibrariesHelper::getSymbolLookupForClass), so that's how I discovered 
this error.

It seems that when you try to lookup a symbol in a library the findEntry 
function will call JNIEnv->FindClass which calls getFromClass 
(jni.cpp:403) to find the class loading the library:

ClassLoader.java:2544

     // Invoked in the VM to determine the context class in JNI_OnLoad
     // and JNI_OnUnload
     static Class<?> getFromClass() {
         return nativeLibraryContext.peek().fromClass;
     }

But if no native library is loaded (i.e. when using the default 
library), this context is empty, peek() returns null and you get an NPE. 
I have verified that the nativeLibraryContext is empty with debugging.

If I load a library (e.g. Libraries.loadLibrary(lookup(), "msvcrt")) 
before the call to lookup("") I get the expected result:

     Exception in thread "main" java.lang.NoSuchMethodException: Cannot 
find symbol  in library <default>
	at 
java.base/java.lang.ClassLoader$NativeLibrary.lookup(ClassLoader.java:2428)
	at com.github.jdnl.Main.main(Main.java:108)

I guess in the tests there is just always some native library that was 
already loaded, so nativeLibraryContext was never empty.

I'm not entirely sure what the default library is supposed to be. Is it 
just a dummy? Or is it backed by a bunch of system libraries? (in the 
former case a fix could be to override `lookup` in the `defaultLibrary` 
anonymous class to short-circuit the lookup process and throw a NSME 
early).

This didn't seem to be an issue caused just by running on windows, so 
I'm reporting.

Best regards,
Jorn Vernee

From sundararajan.athijegannathan at oracle.com  Wed Sep 12 17:41:37 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Wed, 12 Sep 2018 23:11:37 +0530
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
Message-ID: <5B994FD1.2070801@oracle.com>

I tried the following on Mac.

$ jshell

|  Welcome to JShell -- Version 12-internal
|  For an introduction type: /help intro

jshell> import java.foreign.*;

jshell>     Libraries.getDefaultLibrary().lookup("");
|  Exception java.lang.NoSuchMethodException: Cannot find symbol  in 
library <default>
|        at ClassLoader$NativeLibrary.lookup (ClassLoader.java:2428)
|        at (#2:1)


I got the NSME as expected. It seems either it is Windows specific or 
perhaps not the latest panama build [just guessing]

Thanks,
-Sundar

On 12/09/18, 10:57 PM, Jorn Vernee wrote:
> Hello,
>
> I ran into an NPE while running some of the tests for foreign. This is 
> the (much reduced) code to reproduce:
>
>     Libraries.getDefaultLibrary().lookup("");
>
> Throwing:
>
>     Exception in thread "main" java.lang.NullPointerException
>     at 
> java.base/java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:2545)
>     at java.base/java.lang.ClassLoader$NativeLibrary.findEntry(Native 
> Method)
>     at 
> java.base/java.lang.ClassLoader$NativeLibrary.lookup(ClassLoader.java:2426)
>     at com.github.jdnl.Main.main(Main.java:108)
>
> If a NativeHeader annotation does not have a value for the `libraries` 
> field, the default library is used for lookup (see 
> LibrariesHelper::getSymbolLookupForClass), so that's how I discovered 
> this error.
>
> It seems that when you try to lookup a symbol in a library the 
> findEntry function will call JNIEnv->FindClass which calls 
> getFromClass (jni.cpp:403) to find the class loading the library:
>
> ClassLoader.java:2544
>
>     // Invoked in the VM to determine the context class in JNI_OnLoad
>     // and JNI_OnUnload
>     static Class<?> getFromClass() {
>         return nativeLibraryContext.peek().fromClass;
>     }
>
> But if no native library is loaded (i.e. when using the default 
> library), this context is empty, peek() returns null and you get an 
> NPE. I have verified that the nativeLibraryContext is empty with 
> debugging.
>
> If I load a library (e.g. Libraries.loadLibrary(lookup(), "msvcrt")) 
> before the call to lookup("") I get the expected result:
>
>     Exception in thread "main" java.lang.NoSuchMethodException: Cannot 
> find symbol  in library <default>
>     at 
> java.base/java.lang.ClassLoader$NativeLibrary.lookup(ClassLoader.java:2428)
>     at com.github.jdnl.Main.main(Main.java:108)
>
> I guess in the tests there is just always some native library that was 
> already loaded, so nativeLibraryContext was never empty.
>
> I'm not entirely sure what the default library is supposed to be. Is 
> it just a dummy? Or is it backed by a bunch of system libraries? (in 
> the former case a fix could be to override `lookup` in the 
> `defaultLibrary` anonymous class to short-circuit the lookup process 
> and throw a NSME early).
>
> This didn't seem to be an issue caused just by running on windows, so 
> I'm reporting.
>
> Best regards,
> Jorn Vernee

From jbvernee at xs4all.nl  Wed Sep 12 17:38:00 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 12 Sep 2018 19:38:00 +0200
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
Message-ID: <9a2de0d3881276d57594d3278edc4052@xs4all.nl>

Jorn Vernee schreef op 2018-09-12 19:27:
> Hello,
> 
> I ran into an NPE while running some of the tests for foreign. This is
> the (much reduced) code to reproduce:
> 
>     Libraries.getDefaultLibrary().lookup("");

Sorry, when running the tests I get a NSME (I think that one is windows 
related, so I'm looking into that by myself), but when testing manually 
using this code I get an NPE (I don't think that's windows related, so 
I'm reporting).

Jorn

From jbvernee at xs4all.nl  Wed Sep 12 17:45:31 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 12 Sep 2018 19:45:31 +0200
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <5B994FD1.2070801@oracle.com>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com>
Message-ID: <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>

Sundararajan Athijegannathan schreef op 2018-09-12 19:41:
> I tried the following on Mac.
> 
> $ jshell
> 
> |  Welcome to JShell -- Version 12-internal
> |  For an introduction type: /help intro
> 
> jshell> import java.foreign.*;
> 
> jshell>     Libraries.getDefaultLibrary().lookup("");
> |  Exception java.lang.NoSuchMethodException: Cannot find symbol  in
> library <default>
> |        at ClassLoader$NativeLibrary.lookup (ClassLoader.java:2428)
> |        at (#2:1)
> 
> 
> I got the NSME as expected. It seems either it is Windows specific or
> perhaps not the latest panama build [just guessing]
> 
> Thanks,
> -Sundar

In jshell I'm also seeing an NSME, so I guess jshell is loading a native 
library.

I'm just compiling a file with the code to repro in the main, and 
getting and NPE.

Jorn

From sundararajan.athijegannathan at oracle.com  Wed Sep 12 18:08:11 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Wed, 12 Sep 2018 23:38:11 +0530
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com> <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
Message-ID: <5B99560B.9070505@oracle.com>

I don't think it is jshell vs. compiled java program. It must be 
something else - because simplest main code does not reproduce NPE for me.

import java.foreign.*;

public class Main {
   public static void main(String[] args) throws Exception {
      Libraries.getDefaultLibrary().lookup("");
   }
}

java Main
Exception in thread "main" java.lang.NoSuchMethodException: Cannot find 
symbol  in library <default>
     at 
java.base/java.lang.ClassLoader$NativeLibrary.lookup(ClassLoader.java:2428)
     at Main.main(Main.java:5)

fwiw, the "default library" is a pseudo Library that searches all 
symbols in the process.

-Sundar

On 12/09/18, 11:15 PM, Jorn Vernee wrote:
> Sundararajan Athijegannathan schreef op 2018-09-12 19:41:
>> I tried the following on Mac.
>>
>> $ jshell
>>
>> |  Welcome to JShell -- Version 12-internal
>> |  For an introduction type: /help intro
>>
>> jshell> import java.foreign.*;
>>
>> jshell>     Libraries.getDefaultLibrary().lookup("");
>> |  Exception java.lang.NoSuchMethodException: Cannot find symbol  in
>> library <default>
>> |        at ClassLoader$NativeLibrary.lookup (ClassLoader.java:2428)
>> |        at (#2:1)
>>
>>
>> I got the NSME as expected. It seems either it is Windows specific or
>> perhaps not the latest panama build [just guessing]
>>
>> Thanks,
>> -Sundar
>
> In jshell I'm also seeing an NSME, so I guess jshell is loading a 
> native library.
>
> I'm just compiling a file with the code to repro in the main, and 
> getting and NPE.
>
> Jorn

From jbvernee at xs4all.nl  Wed Sep 12 18:25:04 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 12 Sep 2018 20:25:04 +0200
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <5B99560B.9070505@oracle.com>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com> <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
 <5B99560B.9070505@oracle.com>
Message-ID: <2daf985f46ed20f2128845663151f882@xs4all.nl>

Sundararajan Athijegannathan schreef op 2018-09-12 20:08:
> I don't think it is jshell vs. compiled java program. It must be
> something else - because simplest main code does not reproduce NPE for
> me.
> 
> import java.foreign.*;
> 
> public class Main {
>   public static void main(String[] args) throws Exception {
>      Libraries.getDefaultLibrary().lookup("");
>   }
> }
> 
> java Main
> Exception in thread "main" java.lang.NoSuchMethodException: Cannot
> find symbol  in library <default>
>     at
> java.base/java.lang.ClassLoader$NativeLibrary.lookup(ClassLoader.java:2428)
>     at Main.main(Main.java:5)

Ok, then I don't know. Btw, I am using the latest panama build minus the 
last 2 change sets from today, but also with Maurizio's callbacks RFR 
applied [1] (It might that then?).

> fwiw, the "default library" is a pseudo Library that searches all
> symbols in the process.

Ok good to know! I thought as much, but couldn't find where the 
libraries were being searched in the source code. I'm just finding a 
call to dlsym or GetProcAddress for the call to NativeLibrary.findEntry 
. Is the defaultLibrary field being patched over by the VM? Can you 
point me in the right direction?

Thanks,
Jorn

> -Sundar

[1] : 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-August/002565.html

From jbvernee at xs4all.nl  Wed Sep 12 18:31:50 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 12 Sep 2018 20:31:50 +0200
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <2daf985f46ed20f2128845663151f882@xs4all.nl>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com> <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
 <5B99560B.9070505@oracle.com> <2daf985f46ed20f2128845663151f882@xs4all.nl>
Message-ID: <318dcea0a0ca21ff13f1dc3d9f008186@xs4all.nl>

> Ok good to know! I thought as much, but couldn't find where the
> libraries were being searched in the source code. I'm just finding a
> call to dlsym or GetProcAddress for the call to
> NativeLibrary.findEntry . Is the defaultLibrary field being patched
> over by the VM? Can you point me in the right direction?

Never mind I found it :)

void* getDefaultHandle() {
     // FIXME: This is not yet tested on Windows.
     return getProcessHandle();
}

Jorn

From maurizio.cimadamore at oracle.com  Wed Sep 12 20:01:36 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:01:36 +0000
Subject: hg: panama/dev: 68 new changesets
Message-ID: <201809122001.w8CK1f73002306@aojmv0008.oracle.com>

Changeset: 8f594f75e054
Author:    dlong
Date:      2018-09-05 13:10 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/8f594f75e054

8187078: -XX:+VerifyOops finds numerous problems when running JPRT
Reviewed-by: kvn

! src/hotspot/share/c1/c1_LIRGenerator.cpp
! src/hotspot/share/gc/shared/c1/barrierSetC1.cpp
! src/hotspot/share/runtime/java.cpp

Changeset: 35dee171e59c
Author:    iignatyev
Date:      2018-08-10 13:36 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/35dee171e59c

8209386: [error-prone] StreamResourceLeak in jdk.internal.ed module
Reviewed-by: rriggs

! src/jdk.internal.ed/share/classes/jdk/internal/editor/external/ExternalEditor.java

Changeset: 3df9c8591afc
Author:    jcbeyler
Date:      2018-09-05 11:12 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/3df9c8591afc

8208352: Merge HeapMonitorTest and HeapMonitorGCTest code
Summary: Merged the code that enables sampling and allocates
Reviewed-by: cjplummer, sspitsyn, amenkov

! test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitor.java
! test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCTest.java
! test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorTest.java

Changeset: f9d10031512d
Author:    jwilhelm
Date:      2018-09-06 02:50 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f9d10031512d

Added tag jdk-12+10 for changeset 8f594f75e054

! .hgtags

Changeset: b51d348698c2
Author:    jcbeyler
Date:      2018-09-05 19:40 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b51d348698c2

8210198: Clean up JNI_ENV_ARG for vmTestbase/jvmti/Get[A-F] tests
Summary: Remove JNI_ENV macros from the Get[A-F] tests
Reviewed-by: sspitsyn, amenkov, cjplummer

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetAllThreads/allthr001/allthr001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetAllThreads/allthr002/allthr002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetArgumentsSize/argsize001/argsize001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetArgumentsSize/argsize002/argsize002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetAvailableProcessors/getavailproc001/getavailproc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetBytecodes/bytecodes001/bytecodes001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetBytecodes/bytecodes002/bytecodes002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetBytecodes/bytecodes003/bytecodes003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCapabilities/getcaps001/getcaps001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCapabilities/getcaps002/getcaps002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields/getclfld005/getclfld005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields/getclfld006/getclfld006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields/getclfld007/getclfld007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassLoader/getclsldr001/getclsldr001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassLoader/getclsldr002/getclsldr002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassLoader/getclsldr003/getclsldr003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassLoaderClasses/clsldrclss001/clsldrclss001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassLoaderClasses/clsldrclss002/clsldrclss002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd005/getclmthd005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd006/getclmthd006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd007/getclmthd007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassModifiers/getclmdf004/getclmdf004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassModifiers/getclmdf005/getclmdf005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassModifiers/getclmdf006/getclmdf006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassModifiers/getclmdf007/getclmdf007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassSignature/getclsig004/getclsig004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassSignature/getclsig005/getclsig005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassSignature/getclsig006/getclsig006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassStatus/getclstat005/getclstat005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassStatus/getclstat006/getclstat006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassStatus/getclstat007/getclstat007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon001/contmon001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon002/contmon002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentContendedMonitor/contmon003/contmon003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentThreadCpuTime/curthrcputime001/curthrcputime001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetCurrentThreadCpuTimerInfo/curthrtimerinfo001/curthrtimerinfo001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetEnv/GetEnv001/GetEnv001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetEnvironmentLocalStorage/getenvstor001/getenvstor001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetErrorName/geterrname001/geterrname001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetErrorName/geterrname002/geterrname002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetExtensionEvents/extevents001/extevents001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetExtensionFunctions/extfuncs001/extfuncs001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldDeclaringClass/getfldecl001/getfldecl001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldDeclaringClass/getfldecl002/getfldecl002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldDeclaringClass/getfldecl004/getfldecl004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldModifiers/getfldmdf003/getfldmdf003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldModifiers/getfldmdf004/getfldmdf004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldName/getfldnm003/getfldnm003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldName/getfldnm004/getfldnm004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldName/getfldnm005/getfldnm005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFrameCount/framecnt001/framecnt001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFrameCount/framecnt002/framecnt002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFrameCount/framecnt003/framecnt003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFrameLocation/frameloc001/frameloc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFrameLocation/frameloc002/frameloc002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFrameLocation/frameloc003/frameloc003.cpp

Changeset: b6ccd982e33d
Author:    xuelei
Date:      2018-09-05 21:01 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b6ccd982e33d

8210334: TLS 1.3 server fails if ClientHello doesn't have pre_shared_key and psk_key_exchange_modes
Reviewed-by: ascarpino, wetmore

! src/java.base/share/classes/sun/security/ssl/PskKeyExchangeModesExtension.java

Changeset: 0f921a6707d9
Author:    rhalade
Date:      2018-09-05 21:06 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/0f921a6707d9

8210432: Add additional TeliaSonera root certificate
Reviewed-by: mullan

! src/java.base/share/lib/security/cacerts
! test/jdk/lib/security/cacerts/VerifyCACerts.java
+ test/jdk/security/infra/java/security/cert/CertPathValidator/certification/TeliaSoneraCA.java

Changeset: 5b1d1a7d4def
Author:    dholmes
Date:      2018-09-06 02:01 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/5b1d1a7d4def

8199874: [TESTBUG] runtime/Thread/ThreadPriorities.java fails with "expected 0 to equal 10"
Reviewed-by: lfoltan, ccheung

! test/hotspot/jtreg/runtime/Thread/ThreadPriorities.java

Changeset: 0fd48caf8243
Author:    mcimadamore
Date:      2018-09-06 13:13 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/0fd48caf8243

8210318: idea.sh script doesn't work on Mac
Summary: remove usage non-portable sed options
Reviewed-by: erikj, ihse

! bin/idea.sh
! make/langtools/intellij/template/runConfigurations/javac.xml
! make/langtools/intellij/template/runConfigurations/javadoc.xml
! make/langtools/intellij/template/runConfigurations/javap.xml
! make/langtools/intellij/template/runConfigurations/jshell.xml
! make/langtools/intellij/template/runConfigurations/sjavac.xml

Changeset: 805807f15830
Author:    vromero
Date:      2018-09-06 05:44 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/805807f15830

8210435: don't add local variable spots if they are DCE'ed by the compiler
Reviewed-by: mcimadamore

! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java
+ test/langtools/tools/javac/T8210435/NoLocalsMustBeReservedForDCEedVarsTest.java

Changeset: ca3003390cf0
Author:    ccheung
Date:      2018-09-06 09:30 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/ca3003390cf0

8185145: AppCDS custom loader support on Mac OS X
Reviewed-by: dholmes, gziemski

! src/hotspot/share/classfile/classListParser.cpp
! test/lib/jdk/test/lib/Platform.java

Changeset: c3ad012182b1
Author:    naoto
Date:      2018-09-06 10:49 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/c3ad012182b1

8210142: java.util.Calendar.clone() doesn't respect sharedZone flag
Reviewed-by: rriggs

! src/java.base/share/classes/java/util/Calendar.java
! test/jdk/java/util/Calendar/CalendarTest.java

Changeset: a4c50d83af82
Author:    igerasim
Date:      2018-09-06 12:10 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/a4c50d83af82

8210285: CharsetDecoder/Encoder's constructor does not reject NaN
Reviewed-by: sherman, smarks, alanb, martin, darcy

! src/java.base/share/classes/java/nio/charset/Charset-X-Coder.java.template
+ test/jdk/java/nio/charset/CharsetDecoder/NaNinCtor.java
+ test/jdk/java/nio/charset/CharsetEncoder/NaNinCtor.java

Changeset: fe4349d27282
Author:    jjg
Date:      2018-09-06 16:15 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/fe4349d27282

8210009: Source Launcher classloader should support getResource and getResourceAsStream
Reviewed-by: mchung, plevart

! src/jdk.compiler/share/classes/com/sun/tools/javac/launcher/Main.java
+ test/langtools/tools/javac/launcher/GetResourceTest.java
+ test/langtools/tools/javac/launcher/src/CLTest.java

Changeset: a65d8a6fa424
Author:    mikael
Date:      2018-09-06 18:06 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/a65d8a6fa424

8210381: Obsolete EmitSync
Reviewed-by: kvn, dcubed, mdoerr, mbaesken, shade

! src/hotspot/cpu/aarch64/aarch64.ad
! src/hotspot/cpu/ppc/macroAssembler_ppc.cpp
! src/hotspot/cpu/s390/macroAssembler_s390.cpp
! src/hotspot/cpu/sparc/macroAssembler_sparc.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/share/runtime/arguments.cpp
! src/hotspot/share/runtime/globals.hpp

Changeset: a8bdd9c24d37
Author:    xyin
Date:      2018-09-07 09:09 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/a8bdd9c24d37

8042902: Test java/net/Inet6Address/serialize/Inet6AddressSerializationTest.java fails intermittently
Reviewed-by: chegar

! test/jdk/java/net/Inet6Address/serialize/Inet6AddressSerializationTest.java

Changeset: cda49f297cb1
Author:    dlong
Date:      2018-09-06 17:45 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/cda49f297cb1

8209361: [AOT] Unexpected number of references for JVMTI_HEAP_REFERENCE_CONSTANT_POOL [111-->111]: 0 (expected at least 1)
Reviewed-by: coleenp, dholmes

! src/hotspot/share/prims/jvmtiTagMap.cpp

Changeset: 7e6b86eb7914
Author:    roland
Date:      2018-09-06 16:27 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/7e6b86eb7914

8209544: AES encrypt performance regression in jdk11b11
Reviewed-by: kvn, vlivanov

! src/hotspot/share/opto/subnode.cpp

Changeset: 799cddff49fe
Author:    mullan
Date:      2018-09-07 08:02 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/799cddff49fe

6899533: SecureClassLoader and URLClassLoader have unnecessary check for createClassLoader permission
Summary: Remove code that is no longer necessary now that pre-JDK 1.2 SecurityManager methods are not supported.
Reviewed-by: mchung

! src/java.base/share/classes/java/net/URLClassLoader.java
! src/java.base/share/classes/java/security/SecureClassLoader.java

Changeset: 0a8d4f484987
Author:    dholmes
Date:      2018-09-07 08:14 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/0a8d4f484987

8210486: Unused code in check_nest_attributes function
Reviewed-by: mikael, sspitsyn

! src/hotspot/share/prims/jvmtiRedefineClasses.cpp

Changeset: 480486f31b3b
Author:    aleonard
Date:      2018-09-07 11:24 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/480486f31b3b

8209786: JDK12 fails to build on s390x with gcc 7.3
Reviewed-by: ihse, goetz

! make/lib/CoreLibraries.gmk
! src/java.desktop/share/native/libmlib_image/mlib_ImageLookUp_Bit.c

Changeset: 2dddc9394b49
Author:    mcimadamore
Date:      2018-09-07 15:56 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/2dddc9394b49

8210495: compiler crashes because of illegal signature in otherwise legal code
Summary: Disable strict verification of compiler signatures when they do not affect generated bytecode
Reviewed-by: vromero

! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java
+ test/langtools/tools/javac/lambda/8210495/T8210495.java

Changeset: 9f6903174bad
Author:    sherman
Date:      2018-09-07 10:17 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/9f6903174bad

8210345: The Japanese message of FileNotFoundException garbled
Reviewed-by: alanb

! src/java.base/share/native/libjava/io_util.c

Changeset: b613bf6a10b1
Author:    jcbeyler
Date:      2018-09-07 09:50 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b613bf6a10b1

8210429: Clean up JNI_ENV_ARG for vmTestbase/jvmti/Get[G-Z] tests
Summary: Remove the JNI_ENV_ARG for the rest of the Get[G-Z]
Reviewed-by: dholmes, sspitsyn, cjplummer

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetImplementedInterfaces/getintrf005/getintrf005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetImplementedInterfaces/getintrf006/getintrf006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetImplementedInterfaces/getintrf007/getintrf007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetJLocationFormat/getjlocfmt001/getjlocfmt001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetJLocationFormat/getjlocfmt002/getjlocfmt002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetJNIFunctionTable/getjniftab001/getjniftab001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetJNIFunctionTable/getjniftab002/getjniftab002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLineNumberTable/linetab001/linetab001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLineNumberTable/linetab002/linetab002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLineNumberTable/linetab003/linetab003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLoadedClasses/loadedclss001/loadedclss001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLoadedClasses/loadedclss002/loadedclss002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariable/getlocal001/getlocal001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariable/getlocal002/getlocal002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariableTable/localtab001/localtab001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariableTable/localtab002/localtab002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariableTable/localtab003/localtab003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariableTable/localtab004/localtab004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariableTable/localtab005/localtab005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMaxLocals/maxloc001/maxloc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMaxLocals/maxloc002/maxloc002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodDeclaringClass/declcls001/declcls001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodDeclaringClass/declcls002/declcls002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodDeclaringClass/declcls003/declcls003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodLocation/methloc001/methloc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodLocation/methloc002/methloc002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodModifiers/methmod001/methmod001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodModifiers/methmod002/methmod002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodName/methname001/methname001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodName/methname002/methname002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetMethodName/methname003/methname003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectHashCode/objhashcode001/objhashcode001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectMonitorUsage/objmonusage001/objmonusage001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectMonitorUsage/objmonusage002/objmonusage002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectMonitorUsage/objmonusage003/objmonusage003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectMonitorUsage/objmonusage004/objmonusage004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectMonitorUsage/objmonusage005/objmonusage005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectMonitorUsage/objmonusage006/objmonusage006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectSize/objsize001/objsize001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetObjectsWithTags/objwithtags001/objwithtags001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetOwnedMonitorInfo/ownmoninf001/ownmoninf001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetOwnedMonitorInfo/ownmoninf002/ownmoninf002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetOwnedMonitorInfo/ownmoninf003/ownmoninf003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetPhase/getphase001/getphase001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetPhase/getphase002/getphase002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetPotentialCapabilities/getpotcaps001/getpotcaps001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSourceDebugExtension/srcdebugex001/srcdebugex001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSourceDebugExtension/srcdebugex002/srcdebugex002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSourceDebugExtension/srcdebugex003/srcdebugex003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSourceFileName/getsrcfn004/getsrcfn004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSourceFileName/getsrcfn005/getsrcfn005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSourceFileName/getsrcfn006/getsrcfn006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/getstacktr001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr002/getstacktr002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr003/getstacktr003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr004/getstacktr004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr005/getstacktr005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr006/getstacktr006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr007/getstacktr007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr008/getstacktr008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetStackTrace/getstacktr009/getstacktr009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSystemProperties/getsysprops001/getsysprops001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSystemProperties/getsysprops002/getsysprops002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSystemProperty/getsysprop001/getsysprop001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetSystemProperty/getsysprop002/getsysprop002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetTag/gettag001/gettag001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadCpuTime/thrcputime001/thrcputime001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadCpuTime/thrcputime002/thrcputime002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadCpuTimerInfo/thrtimerinfo001/thrtimerinfo001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadGroupChildren/getthrdgrpchld001/getthrdgrpchld001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadGroupInfo/thrgrpinfo001/thrgrpinfo001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadGroupInfo/thrgrpinfo002/thrgrpinfo002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadInfo/thrinfo001/thrinfo001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadInfo/thrinfo002/thrinfo002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadLocalStorage/getthrdstor001/getthrdstor001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadState/thrstat001/thrstat001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadState/thrstat002/thrstat002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadState/thrstat003/thrstat003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadState/thrstat004/thrstat004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetThreadState/thrstat005/thrstat005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetTime/gettime001/gettime001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetTimerInfo/timerinfo001/timerinfo001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetTopThreadGroups/topthrgrp001/topthrgrp001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetTopThreadGroups/topthrgrp002/topthrgrp002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetVersionNumber/getvern001/getvern001.cpp

Changeset: 53e61697a020
Author:    kbarrett
Date:      2018-09-07 14:44 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/53e61697a020

8210131: vmTestbase/nsk/jvmti/scenarios/allocation/AP10/ap10t001/TestDescription.java failed with ObjectFree: GetCurrentThreadCpuTimerInfo returned unexpected error code
Summary: Expanded permitted threads to include all NamedThreads.
Reviewed-by: dcubed, sspitsyn

! src/hotspot/share/prims/jvmtiEnter.xsl
! test/hotspot/jtreg/ProblemList.txt

Changeset: d7dcaacb95dd
Author:    jiangli
Date:      2018-09-07 15:18 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/d7dcaacb95dd

8209971: TestOptionsWithRanges.java crashes in CDS mode with G1UpdateBufferSize=1.
Summary: Fixup archive heap regions before restoring any archived java object at runtime.
Reviewed-by: iklam, ccheung

! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/systemDictionary.cpp
! src/hotspot/share/memory/metaspaceShared.cpp
! src/hotspot/share/memory/metaspaceShared.hpp
! src/hotspot/share/memory/universe.cpp

Changeset: b487c1e914d0
Author:    iignatyev
Date:      2018-09-07 14:01 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b487c1e914d0

8210112: remove jdk.testlibrary.ProcessTools
Reviewed-by: alanb, sspitsyn, jcbeyler

! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/AddModules.java
! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/AddOpens.java
! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/AddReads.java
! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/ExportModule.java
! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java
! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/ModulePathAndCP.java
! test/jdk/com/sun/jdi/BadAgentPath.java
! test/jdk/com/sun/jdi/BadHandshakeTest.java
! test/jdk/com/sun/jdi/DoubleAgentTest.java
! test/jdk/com/sun/jdi/ExclusiveBind.java
! test/jdk/com/sun/jdi/NoLaunchOptionTest.java
! test/jdk/com/sun/jdi/ProcessAttachTest.java
! test/jdk/com/sun/jdi/RunToExit.java
! test/jdk/com/sun/jdi/SuspendNoFlagTest.java
! test/jdk/com/sun/jdi/cds/CDSBreakpointTest.java
! test/jdk/com/sun/jdi/cds/CDSDeleteAllBkptsTest.java
! test/jdk/com/sun/jdi/cds/CDSFieldWatchpoints.java
! test/jdk/com/sun/jdi/cds/CDSJDITest.java
! test/jdk/com/sun/management/HotSpotDiagnosticMXBean/CheckOrigin.java
! test/jdk/com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java
! test/jdk/com/sun/management/OperatingSystemMXBean/TestTotalSwap.java
! test/jdk/com/sun/tools/attach/BasicTests.java
! test/jdk/com/sun/tools/attach/PermissionTest.java
! test/jdk/com/sun/tools/attach/ProviderTest.java
! test/jdk/com/sun/tools/attach/RunnerUtil.java
! test/jdk/com/sun/tools/attach/StartManagementAgent.java
! test/jdk/com/sun/tools/attach/TempDirTest.java
! test/jdk/java/io/BufferedInputStream/LargeCopyWithMark.java
! test/jdk/java/lang/Class/forName/modules/TestDriver.java
! test/jdk/java/lang/Class/getResource/ResourcesTest.java
! test/jdk/java/lang/ClassLoader/EndorsedDirs.java
! test/jdk/java/lang/ClassLoader/ExtDirs.java
! test/jdk/java/lang/ClassLoader/GetDotResource.java
! test/jdk/java/lang/ClassLoader/GetSystemPackage.java
! test/jdk/java/lang/ClassLoader/getResource/GetResource.java
! test/jdk/java/lang/ClassLoader/getResource/automaticmodules/Driver.java
! test/jdk/java/lang/ClassLoader/getResource/modules/ResourcesTest.java
! test/jdk/java/lang/ClassLoader/securityManager/ClassLoaderTest.java
! test/jdk/java/lang/ModuleTests/access/AccessTest.java
! test/jdk/java/lang/Runtime/shutdown/ShutdownInterruptedMain.java
! test/jdk/java/lang/StackWalker/CallerFromMain.java
! test/jdk/java/lang/System/LoggerFinder/modules/Base.java
! test/jdk/java/lang/System/LoggerFinder/modules/JDKLoggerForImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/JDKLoggerForJDKTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/LoggerInImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/NamedLoggerForImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/NamedLoggerForJDKTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/UnnamedLoggerForImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/UnnamedLoggerForJDKTest.java
! test/jdk/java/lang/System/OsVersionTest.java
! test/jdk/java/lang/instrument/DaemonThread/TestDaemonThread.java
! test/jdk/java/lang/instrument/DaemonThread/TestDaemonThreadLauncher.java
! test/jdk/java/lang/instrument/PremainClass/NoPremainAgentTest.java
! test/jdk/java/lang/instrument/PremainClass/PremainClassTest.java
! test/jdk/java/lang/instrument/PremainClass/ZeroArgPremainAgentTest.java
! test/jdk/java/lang/instrument/executableJAR/ExecJarWithAgent.java
! test/jdk/java/lang/instrument/modules/AppendToClassPathModuleTest.java
! test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java
! test/jdk/java/lang/management/MemoryMXBean/RunUtil.java
! test/jdk/java/lang/reflect/Proxy/ProxyClassAccessTest.java
! test/jdk/java/lang/reflect/Proxy/ProxyLayerTest.java
! test/jdk/java/lang/reflect/Proxy/ProxyTest.java
! test/jdk/java/nio/file/spi/SetDefaultProvider.java
! test/jdk/java/rmi/module/ModuleTest.java
! test/jdk/java/security/AccessController/DoPrivAccompliceTest.java
! test/jdk/java/security/KeyStore/PKCS12/KeytoolReaderP12Test.java
! test/jdk/java/security/KeyStore/PKCS12/KeytoolWriteP12Test.java
! test/jdk/java/security/KeyStore/PKCS12/MetadataEmptyTest.java
! test/jdk/java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java
! test/jdk/java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java
! test/jdk/java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java
! test/jdk/java/security/KeyStore/PKCS12/Utils.java
! test/jdk/java/security/Policy/ExtensiblePolicy/ExtensiblePolicyWithJarTest.java
! test/jdk/java/security/Policy/SignedJar/SignedJarTest.java
! test/jdk/java/util/Arrays/TimSortStackSize2.java
! test/jdk/java/util/Locale/bcp47u/SystemPropertyTests.java
! test/jdk/java/util/ResourceBundle/modules/cache/CacheTest.java
! test/jdk/java/util/ResourceBundle/modules/casesensitive/CaseInsensitiveNameClash.java
! test/jdk/java/util/ResourceBundle/modules/security/TestPermission.java
! test/jdk/java/util/logging/TestLoggerWeakRefLeak.java
! test/jdk/java/util/logging/modules/GetResourceBundleTest.java
! test/jdk/java/util/zip/EntryCount64k.java
! test/jdk/javax/management/mxbean/MXBeanWeirdParamTest.java
! test/jdk/javax/management/remote/mandatory/connection/DefaultAgentFilterTest.java
! test/jdk/javax/management/security/AuthorizationTest.java
! test/jdk/javax/management/security/SecurityTest.java
! test/jdk/javax/security/auth/Subject/doAs/NestedActions.java
! test/jdk/jdk/internal/misc/VM/RuntimeArguments.java
! test/jdk/jdk/modules/etc/DefaultModules.java
! test/jdk/jdk/modules/incubator/DefaultImage.java
! test/jdk/jdk/modules/incubator/ImageModules.java
! test/jdk/jdk/modules/scenarios/automaticmodules/RunWithAutomaticModules.java
! test/jdk/jdk/modules/scenarios/container/ContainerTest.java
! test/jdk/jdk/modules/scenarios/overlappingpackages/OverlappingPackagesTest.java
! test/jdk/lib/testlibrary/OutputAnalyzerReportingTest.java
! test/jdk/lib/testlibrary/OutputAnalyzerTest.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputAnalyzer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputBuffer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/ProcessTools.java
- test/jdk/lib/testlibrary/jdk/testlibrary/StreamPumper.java
! test/jdk/native_sanity/simplenativelauncher/ProgramTest.java
! test/jdk/sun/jvmstat/monitor/MonitoredVm/MonitorVmStartTerminate.java
! test/jdk/sun/management/jdp/DynamicLauncher.java
! test/jdk/sun/management/jdp/JdpDefaultsTest.java
! test/jdk/sun/management/jdp/JdpJmxRemoteDynamicPortTest.java
! test/jdk/sun/management/jdp/JdpOffTest.java
! test/jdk/sun/management/jdp/JdpSpecificAddressTest.java
! test/jdk/sun/management/jdp/PortAlreadyInUseTest.java
! test/jdk/sun/management/jmxremote/bootstrap/AbstractFilePermissionTest.java
! test/jdk/sun/management/jmxremote/bootstrap/CustomLauncherTest.java
! test/jdk/sun/management/jmxremote/bootstrap/JMXInterfaceBindingTest.java
! test/jdk/sun/management/jmxremote/bootstrap/LocalManagementTest.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java
! test/jdk/sun/management/jmxremote/startstop/JMXStartStopTest.java
! test/jdk/sun/management/jmxremote/startstop/JMXStatusPerfCountersTest.java
! test/jdk/sun/management/jmxremote/startstop/JMXStatusTest.java
! test/jdk/sun/management/jmxremote/startstop/ManagementAgentJcmd.java
! test/jdk/sun/security/krb5/auto/tools/KinitConfPlusProps.java
! test/jdk/sun/security/ssl/SSLEngineImpl/SSLEngineKeyLimit.java
! test/jdk/sun/security/ssl/SSLSocketImpl/SSLSocketKeyLimit.java
! test/jdk/sun/security/tools/jarsigner/TsacertOptionTest.java
! test/jdk/sun/security/tools/jarsigner/Warning.java
! test/jdk/sun/security/tools/jarsigner/warnings/AliasNotInStoreTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/BadExtendedKeyUsageTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/BadKeyUsageTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/BadNetscapeCertTypeTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/ChainNotValidatedTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/HasExpiredCertTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/HasExpiringCertTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/HasUnsignedEntryTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/MultipleWarningsTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/NoTimestampTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/NotSignedByAliasTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/NotYetValidCertTest.java
! test/jdk/sun/security/tools/jarsigner/warnings/Test.java
! test/jdk/sun/tools/jcmd/JcmdBase.java
! test/jdk/sun/tools/jcmd/TestJcmdDefaults.java
! test/jdk/sun/tools/jcmd/TestJcmdSanity.java
! test/jdk/sun/tools/jhsdb/BasicLauncherTest.java
! test/jdk/sun/tools/jhsdb/HeapDumpTest.java
! test/jdk/sun/tools/jinfo/BasicJInfoTest.java
! test/jdk/sun/tools/jmap/BasicJMapTest.java
! test/jdk/sun/tools/jps/JpsHelper.java
! test/jdk/sun/tools/jps/TestJpsSanity.java
! test/jdk/sun/tools/jstack/BasicJStackTest.java
! test/jdk/sun/tools/jstack/DeadlockDetectionTest.java
! test/jdk/sun/tools/jstat/JStatInterval.java
! test/jdk/sun/tools/jstatd/JstatdTest.java
! test/jdk/sun/tools/jstatd/TestJstatdUsage.java
! test/jdk/tools/jimage/JImageToolTest.java
! test/jdk/tools/jlink/basic/AllModulePath.java
! test/jdk/tools/jlink/basic/BasicTest.java
! test/jdk/tools/jlink/bindservices/BindServices.java
! test/jdk/tools/jlink/plugins/SystemModuleDescriptors/CompiledVersionTest.java
! test/jdk/tools/jlink/plugins/SystemModuleDescriptors/UserModuleTest.java
! test/jdk/tools/launcher/InfoStreams.java
! test/jdk/tools/launcher/modules/addexports/AddExportsTest.java
! test/jdk/tools/launcher/modules/addexports/AddExportsTestWarningError.java
! test/jdk/tools/launcher/modules/addexports/manifest/AddExportsAndOpensInManifest.java
! test/jdk/tools/launcher/modules/addmods/AddModsTest.java
! test/jdk/tools/launcher/modules/addreads/AddReadsTest.java
! test/jdk/tools/launcher/modules/addreads/AddReadsTestWarningError.java
! test/jdk/tools/launcher/modules/basic/BasicTest.java
! test/jdk/tools/launcher/modules/basic/InitErrors.java
! test/jdk/tools/launcher/modules/classpath/JavaClassPathTest.java
! test/jdk/tools/launcher/modules/describe/DescribeModuleTest.java
! test/jdk/tools/launcher/modules/dryrun/DryRunTest.java
! test/jdk/tools/launcher/modules/illegalaccess/IllegalAccessTest.java
! test/jdk/tools/launcher/modules/limitmods/LimitModsTest.java
! test/jdk/tools/launcher/modules/listmods/ListModsTest.java
! test/jdk/tools/launcher/modules/patch/basic/PatchTest.java
! test/jdk/tools/launcher/modules/patch/basic/PatchTestWarningError.java
! test/jdk/tools/launcher/modules/patch/systemmodules/PatchSystemModules.java
! test/jdk/tools/launcher/modules/showmoduleresolution/ShowModuleResolutionTest.java
! test/jdk/tools/launcher/modules/validate/ValidateModulesTest.java
! test/lib/jdk/test/lib/apps/LingeredApp.java
! test/lib/jdk/test/lib/dcmd/CommandExecutor.java
! test/lib/jdk/test/lib/process/OutputAnalyzer.java
! test/lib/jdk/test/lib/process/OutputBuffer.java
! test/lib/jdk/test/lib/process/ProcessTools.java
! test/lib/jdk/test/lib/process/StreamPumper.java

Changeset: 5a1be00ea4f6
Author:    erikj
Date:      2018-09-07 14:54 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/5a1be00ea4f6

8210283: Support git as an SCM alternative in the build
Reviewed-by: ihse, ehelin

+ .gitignore
! make/SourceRevision.gmk
! make/autoconf/basics.m4
! make/autoconf/spec.gmk.in
! make/common/MakeBase.gmk

Changeset: ddc976897c75
Author:    ihse
Date:      2018-09-10 09:58 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/ddc976897c75

8200609: Proper fix for mapfile removal for libjsig
Reviewed-by: erikj, dholmes

! make/lib/Lib-java.base.gmk
- make/mapfiles/libjsig/mapfile-vers-solaris
! src/java.base/unix/native/libjsig/jsig.c

Changeset: 2fdfe34f7262
Author:    ihse
Date:      2018-09-10 09:59 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/2fdfe34f7262

8081858: make dist-clean does not delete all log files
Reviewed-by: erikj

! make/Init.gmk
! make/Main.gmk

Changeset: f7563db3ae1b
Author:    ihse
Date:      2018-09-10 10:02 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f7563db3ae1b

8056217: Remove awt_makecube.cpp
Reviewed-by: erikj, serb

- src/java.desktop/windows/native/common/awt_makecube.cpp

Changeset: d058b410af0a
Author:    eosterlund
Date:      2018-09-10 11:24 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d058b410af0a

8210236: Prepare ciReceiverTypeData::translate_receiver_data_from for concurrent class unloading
Reviewed-by: coleenp, roland

! src/hotspot/share/ci/ciMethodData.cpp

Changeset: 72bdaf11dd6a
Author:    eosterlund
Date:      2018-09-10 13:07 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/72bdaf11dd6a

8210233: Prepare Klass::is_loader_alive() for concurrent class unloading
Reviewed-by: coleenp, pliden

! src/hotspot/share/classfile/classLoaderData.hpp
! src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp
! src/hotspot/share/oops/klass.hpp

Changeset: a30461a359f5
Author:    eosterlund
Date:      2018-09-10 13:07 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/a30461a359f5

8210321: Create NO_KEEPALIVE CLD holder accessor
Reviewed-by: coleenp, kbarrett

! src/hotspot/share/classfile/classLoaderData.cpp
! src/hotspot/share/classfile/classLoaderData.hpp
! src/hotspot/share/classfile/classLoaderData.inline.hpp

Changeset: 9d494115eda4
Author:    amlu
Date:      2018-09-10 20:17 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/9d494115eda4

8209930: Refactor java/util/zip/ZipFile/deletetempjar.sh to plain java test
Reviewed-by: alanb

! test/jdk/java/util/zip/ZipFile/DeleteTempJar.java
+ test/jdk/java/util/zip/ZipFile/DeleteTempJarTest.java
- test/jdk/java/util/zip/ZipFile/deletetempjar.sh

Changeset: 4e99f412148f
Author:    lucy
Date:      2018-09-10 16:40 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/4e99f412148f

8210319: [s390]: Use of shift operators not covered by cpp standard
Reviewed-by: mdoerr, goetz

! src/hotspot/cpu/s390/macroAssembler_s390.cpp

Changeset: 1f70116be2df
Author:    mchung
Date:      2018-09-10 12:48 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/1f70116be2df

8210502: jdeps does not handle properly on analyzing a mixture of MR JARs and non-MR JARs
Reviewed-by: alanb

! src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ClassFileReader.java
! test/langtools/tools/jdeps/MultiReleaseJar.java
+ test/langtools/tools/jdeps/foo/module-info.java

Changeset: b8b0da4a5f49
Author:    coleenp
Date:      2018-09-10 16:33 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/b8b0da4a5f49

8208697: vmTestbase/metaspace/stressHierarchy/stressHierarchy012/TestDescription.java fails with OutOfMemoryError: Metaspace
Summary: remove timeoutHandler class and let Stresser handle timeout, remove 30 threads filling metaspace, and remove related unused files.
Reviewed-by: lfoltan, mseledtsov

! test/hotspot/jtreg/ProblemList.txt
! test/hotspot/jtreg/vmTestbase/metaspace/stressHierarchy/common/PerformChecksHelper.java
! test/hotspot/jtreg/vmTestbase/metaspace/stressHierarchy/common/StressHierarchyBaseClass.java
! test/hotspot/jtreg/vmTestbase/metaspace/stressHierarchy/stressHierarchy012/TestDescription.java
! test/hotspot/jtreg/vmTestbase/nsk/share/classload/GeneratingClassLoader.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutHandler.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutWatchdog.java
- test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingHeap.java
! test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingMetaspace.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/CompileAndDeoptimize.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/MetaspaceStresser.java

Changeset: 1e39953aaed8
Author:    iignatyev
Date:      2018-09-10 14:23 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/1e39953aaed8

8182404: remove jdk.testlibrary.JDKToolFinder and JDKToolLauncher
Reviewed-by: amenkov, jcbeyler, alanb

! test/hotspot/jtreg/serviceability/sa/DeadlockDetectionTest.java
! test/jdk/java/lang/ClassLoader/getResource/GetResource.java
! test/jdk/java/lang/System/LoggerFinder/modules/Base.java
! test/jdk/java/lang/System/LoggerFinder/modules/JDKLoggerForImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/JDKLoggerForJDKTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/LoggerInImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/NamedLoggerForImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/NamedLoggerForJDKTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/UnnamedLoggerForImageTest.java
! test/jdk/java/lang/System/LoggerFinder/modules/UnnamedLoggerForJDKTest.java
! test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java
! test/jdk/java/lang/management/MemoryMXBean/RunUtil.java
! test/jdk/java/net/MulticastSocket/MultiDead.java
! test/jdk/java/net/spi/URLStreamHandlerProvider/Basic.java
! test/jdk/java/security/KeyStore/PKCS12/EntryProtectionTest.java
! test/jdk/java/security/KeyStore/PKCS12/KeytoolReaderP12Test.java
! test/jdk/java/security/KeyStore/PKCS12/KeytoolWriteP12Test.java
! test/jdk/java/security/KeyStore/PKCS12/MetadataEmptyTest.java
! test/jdk/java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java
! test/jdk/java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java
! test/jdk/java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java
! test/jdk/java/security/KeyStore/PKCS12/Utils.java
! test/jdk/javax/management/mxbean/MXBeanWeirdParamTest.java
! test/jdk/javax/management/security/AuthorizationTest.java
! test/jdk/javax/management/security/SecurityTest.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolFinder.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolLauncher.java
! test/jdk/sun/management/jmxremote/startstop/ManagementAgentJcmd.java
! test/jdk/sun/security/tools/jarsigner/Warning.java
! test/jdk/sun/tools/jcmd/JcmdBase.java
! test/jdk/sun/tools/jhsdb/BasicLauncherTest.java
! test/jdk/sun/tools/jhsdb/HeapDumpTest.java
! test/jdk/sun/tools/jhsdb/heapconfig/TmtoolTestScenario.java
! test/jdk/sun/tools/jinfo/BasicJInfoTest.java
! test/jdk/sun/tools/jmap/BasicJMapTest.java
! test/jdk/sun/tools/jps/JpsHelper.java
! test/jdk/sun/tools/jstack/BasicJStackTest.java
! test/jdk/sun/tools/jstack/DeadlockDetectionTest.java
! test/jdk/sun/tools/jstat/JStatInterval.java
! test/jdk/sun/tools/jstat/jstatClassloadOutput1.sh
! test/jdk/sun/tools/jstatd/JstatdTest.java
! test/jdk/sun/tools/jstatd/TestJstatdUsage.java
! test/jdk/tools/jar/compat/CLICompatibility.java
! test/jdk/tools/jar/modularJar/Basic.java
! test/jdk/tools/launcher/modules/patch/systemmodules/PatchSystemModules.java

Changeset: fd3b632801aa
Author:    jiangli
Date:      2018-09-10 18:30 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/fd3b632801aa

8210515: [TESTBUG]CheckArchivedModuleApp.java needs to check if EnableJVMCI is set.
Summary: System module objects are not archived when EnableJVMCI is set to true.
Reviewed-by: iklam, ccheung

! test/hotspot/jtreg/ProblemList-graal.txt
! test/hotspot/jtreg/runtime/appcds/cacheObject/CheckArchivedModuleApp.java

Changeset: 9d89c0835ac1
Author:    dholmes
Date:      2018-09-10 18:57 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/9d89c0835ac1

8210512: [Testbug] vmTestbase/nsk/jdi/ObjectReference/referringObjects/referringObjects002/referringObjects002.java fails with unexpected size of ClassLoaderReference.referringObjects
Summary: Account for the self-reference that every class has in the constant pool
Reviewed-by: sspitsyn, jcbeyler

! test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/referringObjects/referringObjects002/referringObjects002.java

Changeset: b83571bbc147
Author:    kbarrett
Date:      2018-09-10 19:18 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/b83571bbc147

8210511: TestSingleWriterSynchronizer can deadlock
Summary: Check for safepoints in test loops.
Reviewed-by: coleenp, eosterlund

! test/hotspot/gtest/utilities/test_singleWriterSynchronizer.cpp

Changeset: f036f767708e
Author:    dlong
Date:      2018-09-10 16:33 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/f036f767708e

8210434: [Graal] 8209301 prevents GitHub Graal from compiling with latest JDK
Reviewed-by: dnsimon, kvn

! src/jdk.aot/share/classes/jdk.tools.jaotc/src/jdk/tools/jaotc/AOTCompiledClass.java
! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectType.java

Changeset: fca6c9aca3e6
Author:    weijun
Date:      2018-09-11 08:48 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/fca6c9aca3e6

8205507: jdk/javax/xml/crypto/dsig/GenerationTests.java timed out
Reviewed-by: mullan

! test/jdk/javax/xml/crypto/dsig/GenerationTests.java

Changeset: 59de57e466ba
Author:    xyin
Date:      2018-09-11 09:27 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/59de57e466ba

8209773: Refactor shell test javax/naming/module/basic.sh to java
Reviewed-by: vtewari, alanb

+ test/jdk/javax/naming/module/RunBasic.java
- test/jdk/javax/naming/module/basic.sh

Changeset: 0dca9b29bf30
Author:    tschatzl
Date:      2018-09-11 09:13 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/0dca9b29bf30

8210463: Recalculate_used() always sets time taken in G1GCPhaseTimes
Reviewed-by: phh, sjohanss

! src/hotspot/share/gc/g1/g1CollectedHeap.cpp

Changeset: bbc7157ad9c5
Author:    tschatzl
Date:      2018-09-11 09:14 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/bbc7157ad9c5

8210467: Remove unused G1CollectedHeap::_max_heap_capacity
Reviewed-by: sjohanss, phh

! src/hotspot/share/gc/g1/g1CollectedHeap.hpp

Changeset: f912267934e0
Author:    vtewari
Date:      2018-09-11 17:48 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/f912267934e0

8205330: InitialDirContext ctor sometimes throws NPE if the server has sent a disconnection
Reviewed-by: chegar, dfuchs

! src/java.naming/share/classes/com/sun/jndi/ldap/LdapClient.java

Changeset: 49e1b21d9878
Author:    hseigel
Date:      2018-09-11 09:53 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/49e1b21d9878

8210470: Remove unused Verifier::verify() Verifier::Mode argument
Summary: Remove the unused argument.
Reviewed-by: coleenp, jiangli

! src/hotspot/share/classfile/verifier.cpp
! src/hotspot/share/classfile/verifier.hpp
! src/hotspot/share/oops/instanceKlass.cpp
! src/hotspot/share/oops/instanceKlass.hpp
! src/hotspot/share/prims/jvmtiRedefineClasses.cpp

Changeset: 74dde8b66b7f
Author:    coleenp
Date:      2018-09-11 09:42 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/74dde8b66b7f

8210422: runtime/modules/ModuleStress/ExportModuleStressTest.java - assertion failed: address not aligned: 0x00000008baadbabe
Summary: CLDG_lock caused safepoint in inconsistent state
Reviewed-by: lfoltan, eosterlund, kbarrett

! src/hotspot/share/classfile/classLoaderData.cpp
! src/hotspot/share/classfile/classLoaderData.inline.hpp
! src/hotspot/share/classfile/classLoaderStats.cpp
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/gc/parallel/psCompactionManager.cpp
! src/hotspot/share/jfr/periodic/jfrPeriodic.cpp
! src/hotspot/share/prims/jvmtiGetLoadedClasses.cpp
! src/hotspot/share/prims/whitebox.cpp

Changeset: 543a3fb81c4c
Author:    jcbeyler
Date:      2018-09-11 10:12 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/543a3fb81c4c

8210385: Clean up JNI_ENV_ARG and factorize the macros for vmTestbase/jvmti[A-N] tests
Summary: Remove JNI_ENV and JVMTI_ENV macros for part of the jvmti tests
Reviewed-by: amenkov, dholmes

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Allocate/alloc001/alloc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassPrepare/classprep001/classprep001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClearBreakpoint/clrbrk001/clrbrk001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClearBreakpoint/clrbrk002/clrbrk002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClearBreakpoint/clrbrk005/clrbrk005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClearFieldAccessWatch/clrfldw001/clrfldw001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClearFieldAccessWatch/clrfldw002/clrfldw002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClearFieldModificationWatch/clrfmodw001/clrfmodw001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClearFieldModificationWatch/clrfmodw002/clrfmodw002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/CreateRawMonitor/crrawmon001/crrawmon001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/CreateRawMonitor/crrawmon002/crrawmon002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Deallocate/dealloc001/dealloc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/DestroyRawMonitor/drrawmon001/drrawmon001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/DestroyRawMonitor/drrawmon003/drrawmon003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/DestroyRawMonitor/drrawmon004/drrawmon004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Exception/exception001/exception001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ExceptionCatch/excatch001/excatch001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FieldAccess/fieldacc001/fieldacc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FieldAccess/fieldacc002/fieldacc002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FieldAccess/fieldacc003/fieldacc003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FieldAccess/fieldacc004/fieldacc004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FieldModification/fieldmod001/fieldmod001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FieldModification/fieldmod002/fieldmod002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FramePop/framepop001/framepop001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/FramePop/framepop002/framepop002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetJNIFunctionTable/getjniftab001/getjniftab001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetJNIFunctionTable/getjniftab002/getjniftab002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/InterruptThread/intrpthrd002/intrpthrd002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/InterruptThread/intrpthrd003/intrpthrd003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsArrayClass/isarray004/isarray004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsArrayClass/isarray005/isarray005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsFieldSynthetic/isfldsin002/isfldsin002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsFieldSynthetic/isfldsin003/isfldsin003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsInterface/isintrf004/isintrf004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsInterface/isintrf005/isintrf005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsMethodNative/isnative001/isnative001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsMethodNative/isnative002/isnative002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsMethodSynthetic/issynth001/issynth001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsMethodSynthetic/issynth002/issynth002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodEntry/mentry001/mentry001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodEntry/mentry002/mentry002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodExit/mexit001/mexit001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodExit/mexit002/mexit002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NotifyFramePop/nframepop001/nframepop001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NotifyFramePop/nframepop002/nframepop002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NotifyFramePop/nframepop003/nframepop003.cpp

Changeset: 0fa33d4e721e
Author:    pchilanomate
Date:      2018-09-11 13:34 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/0fa33d4e721e

8210300: runtime/MemberName/MemberNameLeak.java fails with RuntimeException
Summary: Added flag -XX:+UnlockDiagnosticVMOptions to tests failing in product builds
Reviewed-by: dcubed, dholmes

! test/hotspot/jtreg/runtime/Dictionary/CleanProtectionDomain.java
! test/hotspot/jtreg/runtime/MemberName/MemberNameLeak.java

Changeset: 9012aeaf993b
Author:    iklam
Date:      2018-09-05 18:14 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/9012aeaf993b

8210289: ArchivedKlassSubGraphInfoRecord is incomplete
Reviewed-by: jiangli, ccheung

! src/hotspot/share/memory/heapShared.cpp
! src/hotspot/share/memory/heapShared.hpp
! src/hotspot/share/memory/metaspaceShared.cpp

Changeset: ebd5b1ad971a
Author:    mikael
Date:      2018-09-11 13:54 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/ebd5b1ad971a

8210514: Obsolete SyncVerbose
Reviewed-by: coleenp, dcubed

! src/hotspot/share/prims/jvmtiRawMonitor.cpp
! src/hotspot/share/runtime/arguments.cpp
! src/hotspot/share/runtime/globals.hpp
! src/hotspot/share/runtime/objectMonitor.cpp
! src/hotspot/share/runtime/objectMonitor.hpp
! src/hotspot/share/runtime/synchronizer.cpp
! src/hotspot/share/runtime/thread.cpp

Changeset: 4ffb0a33f265
Author:    igerasim
Date:      2018-09-11 14:51 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/4ffb0a33f265

8210347: Combine subsequent calls to Set.contains() and Set.add()
Reviewed-by: smarks, bpb

! src/java.base/share/classes/java/lang/ModuleLayer.java
! src/java.base/share/classes/java/lang/module/Configuration.java
! src/java.base/share/classes/java/util/ServiceLoader.java
! src/java.base/share/classes/java/util/stream/DistinctOps.java

Changeset: 2368e8e9cec6
Author:    mikael
Date:      2018-09-11 20:37 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/2368e8e9cec6

8210513: Obsolete SyncFlags
Reviewed-by: coleenp, dcubed, dholmes

! src/hotspot/share/runtime/arguments.cpp
! src/hotspot/share/runtime/globals.hpp
! src/hotspot/share/runtime/objectMonitor.cpp

Changeset: 8123901bc3d1
Author:    rkennke
Date:      2018-08-31 16:28 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/8123901bc3d1

8210187: Explicit barriers for C2
Reviewed-by: eosterlund, shade, roland, pliden

! src/hotspot/share/gc/shared/c2/barrierSetC2.hpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/opto/stringopts.cpp

Changeset: be8fe2a352be
Author:    thartmann
Date:      2018-09-12 09:23 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/be8fe2a352be

8210387: C2 compilation fails with "assert(node->_last_del == _last) failed: must have deleted the edge just produced"
Summary: Refresh iterator and start from the beginning while there is progress when removing dead regions.
Reviewed-by: kvn

! src/hotspot/share/opto/phaseX.cpp
+ test/hotspot/jtreg/compiler/c2/TestUnreachableRegionDuringCCP.java

Changeset: 8c7198cac800
Author:    adinn
Date:      2018-09-12 09:12 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/8c7198cac800

8210578: AArch64: Invalid encoding for fmlsvs instruction
Summary: sub_op code for fmslvs should be 1 not 0
Reviewed-by: roland

! src/hotspot/cpu/aarch64/assembler_aarch64.hpp

Changeset: 469ab7c92a32
Author:    tschatzl
Date:      2018-09-12 11:08 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/469ab7c92a32

8209843: Optimize oop scan closure closures wrt to reference processing in G1
Summary: Set more appropriate reference iteration mode for G1 closures.
Reviewed-by: kbarrett, pliden

! src/hotspot/share/gc/g1/g1OopClosures.hpp
! src/hotspot/share/oops/instanceRefKlass.inline.hpp

Changeset: a15a61e954c0
Author:    ihse
Date:      2018-09-12 12:23 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/a15a61e954c0

8059019: sdp.conf.template should be copied on linux too
Reviewed-by: alanb, erikj

! make/copy/Copy-java.base.gmk

Changeset: 355bd23b46e5
Author:    mdoerr
Date:      2018-09-12 12:54 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/355bd23b46e5

8210497: [PPC64] Vector registers not saved across safepoint
Reviewed-by: goetz, lucy

! src/hotspot/cpu/ppc/ppc.ad
! src/hotspot/cpu/ppc/register_ppc.cpp
! src/hotspot/cpu/ppc/register_ppc.hpp
! src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp
! src/hotspot/cpu/ppc/vmreg_ppc.inline.hpp

Changeset: 3aafd7015d87
Author:    mhorie
Date:      2018-09-12 14:24 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/3aafd7015d87

8208171: PPC64: Enrich SLP support
Reviewed-by: mdoerr, gromero

! src/hotspot/cpu/ppc/assembler_ppc.hpp
! src/hotspot/cpu/ppc/assembler_ppc.inline.hpp
! src/hotspot/cpu/ppc/ppc.ad
! src/hotspot/cpu/ppc/register_definitions_ppc.cpp
! src/hotspot/cpu/ppc/register_ppc.cpp
! src/hotspot/cpu/ppc/register_ppc.hpp

Changeset: f0f5d23449d3
Author:    erikj
Date:      2018-09-12 08:46 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/f0f5d23449d3

8210519: build/releaseFile/CheckSource.java failed additional sources found
Reviewed-by: mikael, dholmes, ihse

! test/jdk/build/releaseFile/CheckSource.java

Changeset: d424675a9743
Author:    jlaskey
Date:      2018-09-12 14:19 -0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/d424675a9743

8206981: Compiler support for Raw String Literals
Reviewed-by: mcimadamore, briangoetz, abuckley, jjg, vromero, jlahoda

! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Preview.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Source.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavaTokenizer.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/UnicodeReader.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties
+ test/langtools/tools/javac/RawStringLiteralLang.java
+ test/langtools/tools/javac/RawStringLiteralLangAPI.java
! test/langtools/tools/javac/diags/examples/IllegalChar.java

Changeset: 975d3636a2f9
Author:    jlaskey
Date:      2018-09-12 14:19 -0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/975d3636a2f9

8200434: String::align, String::indent
Reviewed-by: abuckley, smarks, sherman, rriggs, jrose, sundar, igerasim, briangoetz, darcy, jjg

! src/java.base/share/classes/java/lang/String.java
! src/java.base/share/classes/java/lang/StringLatin1.java
! src/java.base/share/classes/java/lang/StringUTF16.java
+ test/jdk/java/lang/String/AlignIndent.java

Changeset: 13a63d4a3f8d
Author:    jcbeyler
Date:      2018-09-12 10:27 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/13a63d4a3f8d

8210593: Clean up JNI_ENV_ARG and factorize the macros for vmTestbase/jvmti[N-R] tests
Summary: Remove JNI_ENV/JVMTI_ENV macros from N to R jvmti tests
Reviewed-by: amenkov, dholmes

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe001/popframe001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe002/popframe002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe003/popframe003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe004/popframe004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe006/popframe006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe007/popframe007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe008/popframe008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe009/popframe009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe010/popframe010.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe011/popframe011.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorEnter/rawmonenter001/rawmonenter001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorEnter/rawmonenter002/rawmonenter002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorEnter/rawmonenter003/rawmonenter003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorEnter/rawmonenter004/rawmonenter004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorExit/rawmonexit001/rawmonexit001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorExit/rawmonexit002/rawmonexit002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorExit/rawmonexit003/rawmonexit003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorExit/rawmonexit005/rawmonexit005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotify/rawmnntfy001/rawmnntfy001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotify/rawmnntfy002/rawmnntfy002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotify/rawmnntfy003/rawmnntfy003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotify/rawmnntfy004/rawmnntfy004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotifyAll/rawmnntfyall001/rawmnntfyall001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotifyAll/rawmnntfyall002/rawmnntfyall002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotifyAll/rawmnntfyall003/rawmnntfyall003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorNotifyAll/rawmnntfyall004/rawmnntfyall004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorWait/rawmnwait001/rawmnwait001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorWait/rawmnwait002/rawmnwait002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorWait/rawmnwait003/rawmnwait003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorWait/rawmnwait004/rawmnwait004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RawMonitorWait/rawmnwait005/rawmnwait005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/StressRedefine/stressRedefine.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass001/redefclass001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass002/redefclass002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass003/redefclass003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass004/redefclass004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass005/redefclass005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass006/redefclass006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass008/redefclass008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass009/redefclass009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass010/redefclass010.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass011/redefclass011.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass012/redefclass012.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass013/redefclass013.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass014/redefclass014.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass015/redefclass015.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass016/redefclass016.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass017/redefclass017.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass018/redefclass018.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass019/redefclass019.cpp

Changeset: e7459270ca63
Author:    amenkov
Date:      2018-09-12 12:29 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e7459270ca63

8210560: [TEST] convert com/sun/jdi redefineClass-related tests
Reviewed-by: jcbeyler, sspitsyn

- test/jdk/com/sun/jdi/Redefine-g.sh
+ test/jdk/com/sun/jdi/RedefineAbstractClass.java
- test/jdk/com/sun/jdi/RedefineAbstractClass.sh
+ test/jdk/com/sun/jdi/RedefineAddPrivateMethod.java
- test/jdk/com/sun/jdi/RedefineAddPrivateMethod.sh
+ test/jdk/com/sun/jdi/RedefineAnnotation.java
- test/jdk/com/sun/jdi/RedefineAnnotation.sh
+ test/jdk/com/sun/jdi/RedefineChangeClassOrder.java
- test/jdk/com/sun/jdi/RedefineChangeClassOrder.sh
+ test/jdk/com/sun/jdi/RedefineClasses.java
- test/jdk/com/sun/jdi/RedefineClasses.sh
+ test/jdk/com/sun/jdi/RedefineClearBreakpoint.java
- test/jdk/com/sun/jdi/RedefineClearBreakpoint.sh
+ test/jdk/com/sun/jdi/RedefineG.java
+ test/jdk/com/sun/jdi/RedefineImplementor.java
- test/jdk/com/sun/jdi/RedefineImplementor.sh
+ test/jdk/com/sun/jdi/lib/jdb/ClassTransformer.java
! test/jdk/com/sun/jdi/lib/jdb/JdbCommand.java
! test/jdk/com/sun/jdi/lib/jdb/JdbTest.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 20:06:19 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:06:19 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809122006.w8CK6KWA004015@aojmv0008.oracle.com>

Changeset: b16a0eb25868
Author:    mcimadamore
Date:      2018-09-12 22:08 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b16a0eb25868

Automatic merge with default

- make/mapfiles/libjsig/mapfile-vers-solaris
! src/hotspot/cpu/aarch64/aarch64.ad
! src/hotspot/cpu/ppc/ppc.ad
! src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/runtime/globals.hpp
- src/java.desktop/windows/native/common/awt_makecube.cpp
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutHandler.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutWatchdog.java
- test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingHeap.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/CompileAndDeoptimize.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/MetaspaceStresser.java
- test/jdk/com/sun/jdi/Redefine-g.sh
- test/jdk/com/sun/jdi/RedefineAbstractClass.sh
- test/jdk/com/sun/jdi/RedefineAddPrivateMethod.sh
- test/jdk/com/sun/jdi/RedefineAnnotation.sh
- test/jdk/com/sun/jdi/RedefineChangeClassOrder.sh
- test/jdk/com/sun/jdi/RedefineClasses.sh
- test/jdk/com/sun/jdi/RedefineClearBreakpoint.sh
- test/jdk/com/sun/jdi/RedefineImplementor.sh
- test/jdk/java/util/zip/ZipFile/deletetempjar.sh
- test/jdk/javax/naming/module/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolFinder.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolLauncher.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputAnalyzer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputBuffer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/ProcessTools.java
- test/jdk/lib/testlibrary/jdk/testlibrary/StreamPumper.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 20:06:42 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:06:42 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809122006.w8CK6gKu004426@aojmv0008.oracle.com>

Changeset: 118c645be55d
Author:    mcimadamore
Date:      2018-09-12 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/118c645be55d

Automatic merge with default

! bin/idea.sh
! make/autoconf/basics.m4
! make/autoconf/spec.gmk.in
- make/mapfiles/libjsig/mapfile-vers-solaris
- src/java.desktop/windows/native/common/awt_makecube.cpp
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutHandler.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutWatchdog.java
- test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingHeap.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/CompileAndDeoptimize.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/MetaspaceStresser.java
- test/jdk/com/sun/jdi/Redefine-g.sh
- test/jdk/com/sun/jdi/RedefineAbstractClass.sh
- test/jdk/com/sun/jdi/RedefineAddPrivateMethod.sh
- test/jdk/com/sun/jdi/RedefineAnnotation.sh
- test/jdk/com/sun/jdi/RedefineChangeClassOrder.sh
- test/jdk/com/sun/jdi/RedefineClasses.sh
- test/jdk/com/sun/jdi/RedefineClearBreakpoint.sh
- test/jdk/com/sun/jdi/RedefineImplementor.sh
- test/jdk/java/util/zip/ZipFile/deletetempjar.sh
- test/jdk/javax/naming/module/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolFinder.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolLauncher.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputAnalyzer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputBuffer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/ProcessTools.java
- test/jdk/lib/testlibrary/jdk/testlibrary/StreamPumper.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 20:07:03 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:07:03 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809122007.w8CK74sr004848@aojmv0008.oracle.com>

Changeset: 9918f5cf4af7
Author:    mcimadamore
Date:      2018-09-12 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/9918f5cf4af7

Automatic merge with default

- make/mapfiles/libjsig/mapfile-vers-solaris
- src/java.desktop/windows/native/common/awt_makecube.cpp
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutHandler.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutWatchdog.java
- test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingHeap.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/CompileAndDeoptimize.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/MetaspaceStresser.java
- test/jdk/com/sun/jdi/Redefine-g.sh
- test/jdk/com/sun/jdi/RedefineAbstractClass.sh
- test/jdk/com/sun/jdi/RedefineAddPrivateMethod.sh
- test/jdk/com/sun/jdi/RedefineAnnotation.sh
- test/jdk/com/sun/jdi/RedefineChangeClassOrder.sh
- test/jdk/com/sun/jdi/RedefineClasses.sh
- test/jdk/com/sun/jdi/RedefineClearBreakpoint.sh
- test/jdk/com/sun/jdi/RedefineImplementor.sh
- test/jdk/java/util/zip/ZipFile/deletetempjar.sh
- test/jdk/javax/naming/module/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolFinder.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolLauncher.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputAnalyzer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputBuffer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/ProcessTools.java
- test/jdk/lib/testlibrary/jdk/testlibrary/StreamPumper.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 20:07:25 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:07:25 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809122007.w8CK7Pif005296@aojmv0008.oracle.com>

Changeset: 78a4d44d4723
Author:    mcimadamore
Date:      2018-09-12 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/78a4d44d4723

Automatic merge with default

- make/mapfiles/libjsig/mapfile-vers-solaris
! src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/classfile/systemDictionary.cpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/runtime/globals.hpp
- src/java.desktop/windows/native/common/awt_makecube.cpp
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutHandler.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutWatchdog.java
- test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingHeap.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/CompileAndDeoptimize.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/MetaspaceStresser.java
- test/jdk/com/sun/jdi/Redefine-g.sh
- test/jdk/com/sun/jdi/RedefineAbstractClass.sh
- test/jdk/com/sun/jdi/RedefineAddPrivateMethod.sh
- test/jdk/com/sun/jdi/RedefineAnnotation.sh
- test/jdk/com/sun/jdi/RedefineChangeClassOrder.sh
- test/jdk/com/sun/jdi/RedefineClasses.sh
- test/jdk/com/sun/jdi/RedefineClearBreakpoint.sh
- test/jdk/com/sun/jdi/RedefineImplementor.sh
- test/jdk/java/util/zip/ZipFile/deletetempjar.sh
- test/jdk/javax/naming/module/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolFinder.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolLauncher.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputAnalyzer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputBuffer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/ProcessTools.java
- test/jdk/lib/testlibrary/jdk/testlibrary/StreamPumper.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 20:07:46 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:07:46 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809122007.w8CK7l5W005677@aojmv0008.oracle.com>

Changeset: b9ebb1bb8354
Author:    mcimadamore
Date:      2018-09-12 22:10 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b9ebb1bb8354

Automatic merge with default

- make/mapfiles/libjsig/mapfile-vers-solaris
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/classfile/systemDictionary.cpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
- src/java.desktop/windows/native/common/awt_makecube.cpp
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutHandler.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutWatchdog.java
- test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingHeap.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/CompileAndDeoptimize.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/MetaspaceStresser.java
- test/jdk/com/sun/jdi/Redefine-g.sh
- test/jdk/com/sun/jdi/RedefineAbstractClass.sh
- test/jdk/com/sun/jdi/RedefineAddPrivateMethod.sh
- test/jdk/com/sun/jdi/RedefineAnnotation.sh
- test/jdk/com/sun/jdi/RedefineChangeClassOrder.sh
- test/jdk/com/sun/jdi/RedefineClasses.sh
- test/jdk/com/sun/jdi/RedefineClearBreakpoint.sh
- test/jdk/com/sun/jdi/RedefineImplementor.sh
- test/jdk/java/util/zip/ZipFile/deletetempjar.sh
- test/jdk/javax/naming/module/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolFinder.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolLauncher.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputAnalyzer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputBuffer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/ProcessTools.java
- test/jdk/lib/testlibrary/jdk/testlibrary/StreamPumper.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 20:08:08 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:08:08 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809122008.w8CK88I1006073@aojmv0008.oracle.com>

Changeset: 836a2a98662c
Author:    mcimadamore
Date:      2018-09-12 22:10 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/836a2a98662c

Automatic merge with foreign

- make/mapfiles/libjsig/mapfile-vers-solaris
! src/hotspot/cpu/aarch64/aarch64.ad
! src/hotspot/cpu/ppc/ppc.ad
! src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/runtime/globals.hpp
- src/java.desktop/windows/native/common/awt_makecube.cpp
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutHandler.java
- test/hotspot/jtreg/vmTestbase/nsk/share/test/timeoutwatchdog/TimeoutWatchdog.java
- test/hotspot/jtreg/vmTestbase/vm/share/gc/TriggerUnloadingByFillingHeap.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/CompileAndDeoptimize.java
- test/hotspot/jtreg/vmTestbase/vm/share/vmstresser/MetaspaceStresser.java
- test/jdk/com/sun/jdi/Redefine-g.sh
- test/jdk/com/sun/jdi/RedefineAbstractClass.sh
- test/jdk/com/sun/jdi/RedefineAddPrivateMethod.sh
- test/jdk/com/sun/jdi/RedefineAnnotation.sh
- test/jdk/com/sun/jdi/RedefineChangeClassOrder.sh
- test/jdk/com/sun/jdi/RedefineClasses.sh
- test/jdk/com/sun/jdi/RedefineClearBreakpoint.sh
- test/jdk/com/sun/jdi/RedefineImplementor.sh
- test/jdk/java/util/zip/ZipFile/deletetempjar.sh
- test/jdk/javax/naming/module/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolFinder.java
- test/jdk/lib/testlibrary/jdk/testlibrary/JDKToolLauncher.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputAnalyzer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/OutputBuffer.java
- test/jdk/lib/testlibrary/jdk/testlibrary/ProcessTools.java
- test/jdk/lib/testlibrary/jdk/testlibrary/StreamPumper.java


From maurizio.cimadamore at oracle.com  Wed Sep 12 20:08:27 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 20:08:27 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809122008.w8CK8RRr006475@aojmv0008.oracle.com>

Changeset: f7ac3e34f1ab
Author:    mcimadamore
Date:      2018-09-12 22:10 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f7ac3e34f1ab

Automatic merge with vectorIntrinsics



From jbvernee at xs4all.nl  Wed Sep 12 20:35:11 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 12 Sep 2018 22:35:11 +0200
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <318dcea0a0ca21ff13f1dc3d9f008186@xs4all.nl>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com> <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
 <5B99560B.9070505@oracle.com> <2daf985f46ed20f2128845663151f882@xs4all.nl>
 <318dcea0a0ca21ff13f1dc3d9f008186@xs4all.nl>
Message-ID: <c50c81a047cb8dce2a4f4d769d418598@xs4all.nl>

> Never mind I found it :)
> 
> void* getDefaultHandle() {
>     // FIXME: This is not yet tested on Windows.
>     return getProcessHandle();
> }

And here's a possible fix [1]

This fixes 2 bugs:

1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass 
threw an NPE (at least on windows). That is fixed by returning 
defaultLibrary.fromClass when the nativeLibraryContext is empty.

2.) The default library search was not working on windows. It was using 
a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not on 
windows (see https://stackoverflow.com/q/23437007). I have changed the 
implementation from using a default handle to using a new native 
function findEntryInProcess, which does the right thing for Windows 
(iterate over all loaded modules), and does the same thing it used to 
for unix. findEntry is now a Java method, and the original findEntry is 
renamed to findEntry0. The NativeLibrary implementation of findEntry 
forwards to findEntry0, and the anonymous class of the default library 
overrides to forward to findEntryInProcess.

For future reference,
Jorn

[1] : 
https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8

From maurizio.cimadamore at oracle.com  Wed Sep 12 20:49:19 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 12 Sep 2018 21:49:19 +0100
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <318dcea0a0ca21ff13f1dc3d9f008186@xs4all.nl>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com> <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
 <5B99560B.9070505@oracle.com> <2daf985f46ed20f2128845663151f882@xs4all.nl>
 <318dcea0a0ca21ff13f1dc3d9f008186@xs4all.nl>
Message-ID: <4f90f9ba-e2f4-0af7-a5c7-5ed429e110b7@oracle.com>

Yeah - was just about to write that.

I think currently we rely on RTLD_DEFAULT which is available in mac and 
linux. But I don't think it's very portable and I've read too that it 
doesn't work too well with Windows.

I think a more robust alternative would be to always explicitly denote 
the library name - as JNR does. For instance, if you want the standard C 
lib, just loadLibray("c"). E.g. remove the getDefaultLibrary() method 
from the API.

Maurizio




On 12/09/18 19:31, Jorn Vernee wrote:
>> Ok good to know! I thought as much, but couldn't find where the
>> libraries were being searched in the source code. I'm just finding a
>> call to dlsym or GetProcAddress for the call to
>> NativeLibrary.findEntry . Is the defaultLibrary field being patched
>> over by the VM? Can you point me in the right direction?
>
> Never mind I found it :)
>
> void* getDefaultHandle() {
> ??? // FIXME: This is not yet tested on Windows.
> ??? return getProcessHandle();
> }
>
> Jorn


From jbvernee at xs4all.nl  Wed Sep 12 20:54:50 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 12 Sep 2018 22:54:50 +0200
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <4f90f9ba-e2f4-0af7-a5c7-5ed429e110b7@oracle.com>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com> <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
 <5B99560B.9070505@oracle.com> <2daf985f46ed20f2128845663151f882@xs4all.nl>
 <318dcea0a0ca21ff13f1dc3d9f008186@xs4all.nl>
 <4f90f9ba-e2f4-0af7-a5c7-5ed429e110b7@oracle.com>
Message-ID: <55e51c011afe356ab4c11ac2b6ef5e7b@xs4all.nl>

> Yeah - was just about to write that.
> 
> I think currently we rely on RTLD_DEFAULT which is available in mac
> and linux. But I don't think it's very portable and I've read too that
> it doesn't work too well with Windows.
> 
> I think a more robust alternative would be to always explicitly denote
> the library name - as JNR does. For instance, if you want the standard
> C lib, just loadLibray("c"). E.g. remove the getDefaultLibrary()
> method from the API.
> 
> Maurizio

Yeah, I agree, that also prevents ambiguities when the same symbol is 
loaded from different libraries.

Thanks for thinking along :) (sorry for being a distraction).

Jorn

From vivek.r.deshpande at intel.com  Wed Sep 12 21:47:52 2018
From: vivek.r.deshpande at intel.com (vivek.r.deshpande at intel.com)
Date: Wed, 12 Sep 2018 21:47:52 +0000
Subject: hg: panama/dev: intrinsics for rearrange
Message-ID: <201809122147.w8CLlrkC008215@aojmv0008.oracle.com>

Changeset: 766a1b4e938c
Author:    vdeshpande
Date:      2018-09-12 14:45 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/766a1b4e938c

intrinsics for rearrange

! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
! src/hotspot/cpu/x86/stubRoutines_x86.hpp
! src/hotspot/cpu/x86/stubRoutines_x86_64.cpp
! src/hotspot/cpu/x86/vmStructs_x86.hpp
! src/hotspot/cpu/x86/vm_version_x86.cpp
! src/hotspot/cpu/x86/vm_version_x86.hpp
! src/hotspot/cpu/x86/x86.ad
! src/hotspot/share/adlc/formssel.cpp
! src/hotspot/share/ci/ciType.cpp
! src/hotspot/share/ci/ciType.hpp
! src/hotspot/share/classfile/vmSymbols.hpp
! src/hotspot/share/jvmci/vmStructs_jvmci.cpp
! src/hotspot/share/opto/classes.hpp
! src/hotspot/share/opto/compile.cpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/opto/vectornode.hpp
! src/hotspot/share/runtime/vmStructs.cpp
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte128Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte256Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte512Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte64Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double128Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double256Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double512Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double64Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float128Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float256Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float512Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float64Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int256Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int512Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int64Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long128Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long256Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long512Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long64Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short128Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short256Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short512Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short64Vector.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorIntrinsics.java
! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template


From maurizio.cimadamore at oracle.com  Wed Sep 12 21:51:48 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 12 Sep 2018 21:51:48 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809122151.w8CLpmPQ009949@aojmv0008.oracle.com>

Changeset: 3e8bb5690f56
Author:    mcimadamore
Date:      2018-09-12 23:54 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/3e8bb5690f56

Automatic merge with vectorIntrinsics



From vivek.r.deshpande at intel.com  Thu Sep 13 00:05:12 2018
From: vivek.r.deshpande at intel.com (vivek.r.deshpande at intel.com)
Date: Thu, 13 Sep 2018 00:05:12 +0000
Subject: hg: panama/dev: fix rearrange32s
Message-ID: <201809130005.w8D05DTr016952@aojmv0008.oracle.com>

Changeset: 100c19a5128e
Author:    vdeshpande
Date:      2018-09-12 17:04 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/100c19a5128e

fix rearrange32s

! src/hotspot/cpu/x86/x86.ad


From maurizio.cimadamore at oracle.com  Thu Sep 13 00:06:44 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 13 Sep 2018 00:06:44 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809130006.w8D06jgo018060@aojmv0008.oracle.com>

Changeset: f78137d6bb01
Author:    mcimadamore
Date:      2018-09-13 02:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f78137d6bb01

Automatic merge with vectorIntrinsics



From sundararajan.athijegannathan at oracle.com  Thu Sep 13 03:58:59 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Thu, 13 Sep 2018 09:28:59 +0530
Subject: [foreign] NPE when trying to lookup a symbol in the default
 library when no libraries are loaded
In-Reply-To: <c50c81a047cb8dce2a4f4d769d418598@xs4all.nl>
References: <e4513ff7e881d9988fead1f356a8bb6e@xs4all.nl>
 <5B994FD1.2070801@oracle.com> <8b242e4593bc38f9c1f6c339f7140ae7@xs4all.nl>
 <5B99560B.9070505@oracle.com> <2daf985f46ed20f2128845663151f882@xs4all.nl>
 <318dcea0a0ca21ff13f1dc3d9f008186@xs4all.nl>
 <c50c81a047cb8dce2a4f4d769d418598@xs4all.nl>
Message-ID: <5B99E083.9060104@oracle.com>

Magic of timezone difference :) Nice to see that you've located the issue...

-Sundar

On 13/09/18, 2:05 AM, Jorn Vernee wrote:
>> Never mind I found it :)
>>
>> void* getDefaultHandle() {
>>     // FIXME: This is not yet tested on Windows.
>>     return getProcessHandle();
>> }
>
> And here's a possible fix [1]
>
> This fixes 2 bugs:
>
> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass 
> threw an NPE (at least on windows). That is fixed by returning 
> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>
> 2.) The default library search was not working on windows. It was 
> using a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but 
> not on windows (see https://stackoverflow.com/q/23437007). I have 
> changed the implementation from using a default handle to using a new 
> native function findEntryInProcess, which does the right thing for 
> Windows (iterate over all loaded modules), and does the same thing it 
> used to for unix. findEntry is now a Java method, and the original 
> findEntry is renamed to findEntry0. The NativeLibrary implementation 
> of findEntry forwards to findEntry0, and the anonymous class of the 
> default library overrides to forward to findEntryInProcess.
>
> For future reference,
> Jorn
>
> [1] : https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8

From maurizio.cimadamore at oracle.com  Fri Sep 14 13:19:26 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 14 Sep 2018 14:19:26 +0100
Subject: [foreign] some JMH benchmarks
Message-ID: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>

Hi,
over the last few weeks I've been busy playing with Panama and assessing 
performances with JMH. For those just interested in raw numbers, the 
results of my explorations can be found here [1]. But as all benchmarks, 
I think it's better to spend few words to understand what these numbers 
actually _mean_.

To evaluate the performances of Panama I have first created a baseline 
using JNI - more specifically I wanted to assess performances of three 
calls (all part of the C std library), namely `getpid`, `exp` and `qsort`.

The first example is the de facto benchmark for FFIs - since it does 
relatively little computation, it is a good test to measure the 
'latency' of the FFI approach (e.g. how long does it take to go to 
native). The second example is also relatively simple, but the this time 
the function takes a double argument. The third test is akin to an FFI 
torture test, since not only it passes substantially more arguments (4) 
but one of these arguments is also a callback - a pointer to a function 
that is used to sort the contents of the input array.

As expected, the first batch of JNI results confirms our expectations: 
`getpid` is the fastest, followed by `exp`, and then followed by 
`qsort`. Note that qsort is not even close in terms of raw numbers to 
the other two tests - that's because, to sort the array we need to do (N 
* log N) upcalls into Java. In the benchmark, N = 8 and we do the 
upcalls using the JNI function JNIEnv::CallIntMethod.

Now let's examine the second batch of results; these call `getpid`, 
`exp` and `qsort` using Panama. The numbers here are considerably lower 
than the JNI ones for all the three benchmark - although the first two 
seem to be the most problematic. To explain these results we need to 
peek under the hood. Panama implements foreign calls through a so called 
'universal adapter' which, given a calling scheme and a bunch of 
arguments (machine words) shuffles these arguments in the right 
registers/stack slot and then jumps to the target native function - 
after which another round of adaptation must be performed (e.g. to 
recover the return value from the right register/memory location).

Needless to say, all this generality comes at a cost - some of the cost 
is in Java - e.g. all arguments have to be packaged up into a long array 
(although this component doesn't seem to show up much in the generated 
JVM compiled code). A lot of the cost is in the adapter logic itself - 
which has to look at the 'call recipe' and move arguments around 
accordingly - more specifically, in order to call the native function, 
the adapter creates a bunch of helper C++ objects and structs which 
model the CPU state (e.g. in the ShuffleDowncallContext struct, we find 
a field for each register to be modeled in the target architecture). The 
adapter has to first move the values coming from the Java world (stored 
in the aforementioned long array) into the right context fields (and it 
needs to do so by looking at the recipe, which involves iteration over 
the recipe elements). After that's done, we can jump into the assembly 
stub that does the native call - this stub will take as input one of 
those ShuffleDowncallContext structure and will load the corresponding 
registers/create necessary stack slots ahead of the call.

As you can see, there's quite a lot of action going on here, and this 
explains the benchmark numbers; of course, if you are calling a native 
function that does a lot of computation, this adaptation cost will wash 
out - but for relatively quick calls such as 'getpid' and 'exp' the 
latency dominates the picture.

Digression: the callback logic suffers pretty much from the same issues, 
albeit in a reversed order - this time it's the Java code which receives 
a 'snapshot' of the register values from a generated assembly adapter; 
the Java code can then read such values (using the Pointer API), turn 
them into Java objects, call the target Java method and store the 
results (after another conversion) in the right location of the 
snapshot. The assembly adapter will then pick up the values set onto the 
snapshot by the Java code, store them into the corresponding registers 
and return control to the native callee. In the remainder of this email 
we will not discuss callbacks in details - we will just posit that for 
any optimization technique that can be defined, there exists a _dual_ 
strategy that works with callbacks.

How can we make sensible native calls go faster? Well, one obvious way 
would be to optimize the universal adapter so that we get a specialized 
assembly stub for each code shape. If we do that, we can move pretty 
much all of the computation described above from execution time to the 
stub generation time, so that, by the time we have to call the native 
function, we just have to populate the right registers (the specialized 
stub knows where to find them) and jump. While this sounds a good 
approach, it feels like there's also a move for the JIT somewhere in 
there - after all, the JVM knows which calls are hot and in need for 
optimization, so perhaps this specialization process (some or all of it) 
could happen dynamically. And this is indeed an approach we'd like to 
aim for in the long run.

Now, few years ago, Vlad put together a patch which now lives in the 
'linkToNative' branch [6, 7] - the goal of this patch is to implement 
the approach described above: generate a specialized assembly adapter 
for a given native signature, and then leverage the JIT to optimize it 
away, turning the adapter into a bare, direct, native method call. As 
you can see from the third batch of benchmarks, if we tweak Panama to 
use the linkToNative machinery, the speed up is really impressive, and 
we end up being much faster than JNI (up to 4x for getPid).

Unfortunately, the technology in the linkToNative branch is not ready 
from prime time (yet) - first, it doesn't cover some useful cases (e.g. 
varargs, multiple returns via registers, arguments passed in memory). 
That is, the logic assumes there's a 1-1 mapping between a Java 
signature and the native function to be called - and that the arguments 
passed from Java will either be longs or doubles. While we can 
workaround this limitation and define the necessary marshalling logic in 
Java (as I have done to run this benchmark), some of the limitations 
(multiple returns, structs passed by value which are too big) cannot 
simply be worked around. But that's fine, we can still have a fast path 
for those calls which have certain characteristics and a slow path 
(through the universal adapter) for all the other calls.

But there's a second and more serious issue lurking: as you can see in 
the benchmark, I was not able to get the qsort benchmark running when 
using the linkToNative backend. The reason is that the linkToNative code 
is still pretty raw, and it doesn't fully adhere to the JVM internal 
conventions - e.g. there are missing thread state transitions which, in 
the case of upcalls into Java, create issues when it comes to garbage 
collection, as the GC cannot parse the native stack in the correct way.

This means that, while there's a clear shining path ahead of us, it is 
simply too early to just use the linkToNative backend from Panama. For 
this reason, I've been looking into some kind of stopgap solution - 
another way of optimizing native calls (and upcalls into Java) that 
doesn't require too much VM magic. Now, a crucial observation is that, 
in many native calls, there is indeed a 1-1 mapping between Java 
arguments and native arguments (and back, for return values). That is, 
we can think of calling a native function as a process that takes a 
bunch of Java arguments, turn them into native arguments (either double 
or longs), calls the native methods and then turns back the result into 
Java.

The mapping between Java arguments and C values is quite simple:

* primitives: either long or double, depending on whether they describe 
an integral value or a floating point one.
* pointers: they convert to a long
* callbacks: they also convert to a long
* structs: they are recursively decomposed into fields and each field is 
marshalled separately (assuming the struct is not too big, in which case 
is passed in memory)

So, in principle, we could define a bunch of native entry points in the 
VM, one per shape, which take a bunch of long and doubles and call an 
underlying function with those arguments. For instance, let's consider 
the case of a native function which is modelled in Java as:

int m(Pointer<Foo>, double)

To call this native function we have to first turn the Java arguments 
into a (long, double) pair. Then we need to call a native adapter that 
looks like the following:

jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, 
jlong arg0, jdouble arg1) {
 ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
}

And this will take care of calling the native function and returning the 
value back. This is, admittedly, a very simple solution; of course there 
are limitations: we have to define a bunch of specialized native entry 
point (and Java entry points, for callbacks). But here we can play a 
trick: most of moderns ABI pass arguments in registers; for instance 
System V ABI [5] uses up to 6 (!!) integer registers and 7 (!!) MMXr 
registers for FP values - this gives us a total of 13 registers 
available for argument passing. Which covers quite a lot of cases. Now, 
if we have a call where _all_ arguments are passed in registers, then 
the order in which these arguments are declared in the adapter doesn't 
matter! That is, since FP-values will always be passed in different 
register from integral values, we can just define entry points which 
look like these:

invokeNative_V_DDDDD
invokeNative_V_JDDDD
invokeNative_V_JJDDD
invokeNative_V_JJJDD
invokeNative_V_JJJJD
invokeNative_V_JJJJJ

That is, for a given arity (5 in this case), we can just put all long 
arguments in front, and the double arguments after that. That is, we 
don't need to generate all possible permutations of J/D in all positions 
- as the adapter will always do the same thing (read: load from same 
registers) for all equivalent combinations. This keeps the number of 
entry points in check -? and it also poses some challenges to the Java 
logic in charge of marshalling/unmarshalling, as there's an extra 
permutation step involved (although that is not something super-hard to 
address).

You can see the performance numbers associated with this invocation 
scheme (which I've dubbed 'direct') in the 4th batch of the benchmark 
results. These numbers are on par (and slightly better) with JNI in all 
the three cases considered which is, I think, a very positive result, 
given that to write these benchmarks I did not have to write a single 
line of JNI code. In other words, this optimization gives you the same 
speed as JNI, with improved ease of use (**).

Now, since the 'direct' optimization builds on top of the VM native call 
adapters, this approach is significantly more robust than linkToNative 
and I have not run into any weird VM crashes when playing with it. The 
downside of that, is that, for obvious reasons, this approach cannot get 
much faster than JNI - that is, it cannot get close to the numbers 
obtained with the linkToNative backend, which features much deeper 
optimizations. But I think that, despite its limitations, it's still a 
good opportunistic improvement that is worth pursuing in the short term 
(while we sort out the linkToNative story). For this reason, I will soon 
be submitting a review which incorporates the changes for the 'direct' 
invocation schemes.

Cheers
Maurizio

[1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
[2] - https://github.com/jnr/jnr-ffi
[3] - https://github.com/jnr/jffi
[4] - https://sourceware.org/libffi/
[5] - 
https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
[6] - http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
[7] - http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354

(**) the benchmark also contains a 5th row in which I repeated same 
tests, this time using JNR [2]. JNR is built on top of libjffi [3], a 
JNI library in turn built on top of the popular libffi [4]. I wanted to 
have some numbers about JNR because that's another solution that allows 
for better ease to use, taking care of marshalling Java values into C 
and back; since the goals of JNR are similar in spirit with some of the 
goals of the Panama/foreign work, I thought it would be worth having a 
comparison of these approaches. For the records, I think the JNR numbers 
are very respectable given that JNR had to do all the hard work outside 
of the JDK!



From maurizio.cimadamore at oracle.com  Fri Sep 14 16:43:11 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 14 Sep 2018 16:43:11 +0000
Subject: hg: panama/dev: 8210264: cleanup semantics of function pointer
 conversion
Message-ID: <201809141643.w8EGhCaT010489@aojmv0008.oracle.com>

Changeset: 0b888a6e7244
Author:    mcimadamore
Date:      2018-09-14 17:41 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/0b888a6e7244

8210264: cleanup semantics of function pointer conversion
Reviewed-by: jrose

! src/hotspot/cpu/x86/nativeInvoker_x86.cpp
! src/java.base/share/classes/java/foreign/Scope.java
! src/java.base/share/classes/java/foreign/memory/Callback.java
! src/java.base/share/classes/java/foreign/memory/LayoutType.java
! src/java.base/share/classes/jdk/internal/foreign/CallbackImplGenerator.java
! src/java.base/share/classes/jdk/internal/foreign/HeaderImplGenerator.java
! src/java.base/share/classes/jdk/internal/foreign/NativeInvoker.java
! src/java.base/share/classes/jdk/internal/foreign/ScopeImpl.java
! src/java.base/share/classes/jdk/internal/foreign/UpcallHandler.java
- src/java.base/share/classes/jdk/internal/foreign/UpcallStub.java
! src/java.base/share/classes/jdk/internal/foreign/Util.java
! src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java
+ src/java.base/share/classes/jdk/internal/foreign/memory/CallbackImpl.java
! src/java.base/share/classes/jdk/internal/foreign/memory/LayoutTypeImpl.java
! src/java.base/share/classes/jdk/internal/foreign/memory/References.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/CallbackType.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/HeaderFile.java
! test/jdk/com/sun/tools/jextract/globalFuncPointer.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Cursor.java
! test/jdk/com/sun/tools/jextract/testArrayFuncParam/FuncArrayParamTest.java
! test/jdk/com/sun/tools/jextract/testStruct/StructTest.java
! test/jdk/java/foreign/StdLibTest.java
! test/jdk/java/foreign/Upcall/CallbackSort.java
! test/jdk/java/foreign/Upcall/DoubleUpcall.java
! test/jdk/java/foreign/Upcall/StructUpcall.java
! test/jdk/java/foreign/Upcall/Upcall.java
! test/jdk/java/foreign/Upcall/libUpcall.c
! test/jdk/java/foreign/qsort/Qsort.java
! test/jdk/java/foreign/qsort/stdlib.java
! test/jdk/java/foreign/types/FunctionAccessTest.java


From maurizio.cimadamore at oracle.com  Fri Sep 14 16:45:30 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 14 Sep 2018 17:45:30 +0100
Subject: [foreign] RFR 8210264: cleanup semantics of function pointer
 conversion
In-Reply-To: <0C0AB958-2510-439D-B777-10FD86AB96F4@oracle.com>
References: <02dae87a-5560-93d1-f90a-fe9e80980126@oracle.com>
 <0C0AB958-2510-439D-B777-10FD86AB96F4@oracle.com>
Message-ID: <b7ff911d-a7f0-e6d3-c755-40dc507d5645@oracle.com>

FTR, I've just pushed this. I've taken into account the review comments, 
and did two things:

* Util.findCallback will now collect all candidates and throw if 
|candidates| != 1

* I've added another variant of Scope::allocateCallback which takes an 
explicit carrier witness, and amended the javadoc so that it's clear 
that the no-carrier version is built on top of the carrier-ful one, with 
some inference logic spruced up in the middle.

Thanks
Maurizio


On 01/09/18 04:46, John Rose wrote:
> On Aug 31, 2018, at 9:04 AM, Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com 
> <mailto:maurizio.cimadamore at oracle.com>> wrote:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/8210264/ 
>> <http://cr.openjdk.java.net/%7Emcimadamore/panama/8210264/>
>
> This is a good cleanup.
>
> An object is allowed to implement two functional interfaces, and even
> two interfaces which annotated as native callbacks. So it is possible
> that allocateCallback will be presented with an ambiguous input.
> I suggest that Util.findCallback throw an exception in that case,
> instead of returning the first marked FI that that it finds.
>
> This won't harm correct code but will let us exclude some corner
> cases.
>
> Another way to slice this is to require a runtime witness as follows:
>
> <T> Callback<T> allocateCallback(Class<T> funcInt, T funcIntfInstance);
> default <T> Callback<T> allocateCallback(T funcIntfInstance) {
> ? return 
> allocateCallback(Utils.findCallback(funcIntfInstance.getClass()), 
> funcIntfInstance);
> }
>
> I.e., allow explicit specification of which FI we care about but specify
> a sensible default. ?The explicit specification has two effects: ?First,
> it allows cast-free passing of a lambda, and second it allows the
> approach to scale to non-native FI types (assuming the scope
> has rules for dealing with type conversion).
>
> ? John


From maurizio.cimadamore at oracle.com  Fri Sep 14 16:47:45 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 14 Sep 2018 16:47:45 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809141647.w8EGlkCO012515@aojmv0008.oracle.com>

Changeset: ba86d01c81c1
Author:    mcimadamore
Date:      2018-09-14 18:50 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/ba86d01c81c1

Automatic merge with foreign

- src/java.base/share/classes/jdk/internal/foreign/UpcallStub.java


From jbvernee at xs4all.nl  Fri Sep 14 17:39:02 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Fri, 14 Sep 2018 19:39:02 +0200
Subject: [foreign] some JMH benchmarks
In-Reply-To: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
Message-ID: <0b5807b991f0ac860114179d858e033c@xs4all.nl>

> So, in principle, we could define a bunch of native entry points in
> the VM, one per shape, which take a bunch of long and doubles and call
> an underlying function with those arguments. For instance, let's
> consider the case of a native function which is modelled in Java as:
> 
> int m(Pointer<Foo>, double)
> 
> To call this native function we have to first turn the Java arguments
> into a (long, double) pair. Then we need to call a native adapter that
> looks like the following:
> 
> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr,
> jlong arg0, jdouble arg1) {
> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
> }
> 
> And this will take care of calling the native function and returning
> the value back. This is, admittedly, a very simple solution; of course
> there are limitations: we have to define a bunch of specialized native
> entry point (and Java entry points, for callbacks). But here we can
> play a trick: most of moderns ABI pass arguments in registers; for
> instance System V ABI [5] uses up to 6 (!!) integer registers and 7
> (!!) MMXr registers for FP values - this gives us a total of 13
> registers available for argument passing. Which covers quite a lot of
> cases. Now, if we have a call where _all_ arguments are passed in
> registers, then the order in which these arguments are declared in the
> adapter doesn't matter! That is, since FP-values will always be passed
> in different register from integral values, we can just define entry
> points which look like these:
> 
> invokeNative_V_DDDDD
> invokeNative_V_JDDDD
> invokeNative_V_JJDDD
> invokeNative_V_JJJDD
> invokeNative_V_JJJJD
> invokeNative_V_JJJJJ
> 
> That is, for a given arity (5 in this case), we can just put all long
> arguments in front, and the double arguments after that. That is, we
> don't need to generate all possible permutations of J/D in all
> positions - as the adapter will always do the same thing (read: load
> from same registers) for all equivalent combinations. This keeps the
> number of entry points in check -? and it also poses some challenges
> to the Java logic in charge of marshalling/unmarshalling, as there's
> an extra permutation step involved (although that is not something
> super-hard to address).

I'm wondering if the 5 native end points for an arity of 5 are enough. 
Don't you also need 5 for when the function returns a long and 5 more 
for when the function returns a double?

I have a suggestion to bypass having to write out all the permutations 
though. What if, on the Java side, whenever there is a method that has a 
shape that can be optimized in this way (which is ABI dependent), spin 
and load a class which defines a single static native method with the 
needed signature, and annotate it. Then, in NativeLookup::lookup, detect 
this annotation, and instead of trying to look up the symbol in a loaded 
library generate a forwarding stub and link the native method to that 
instead. Then you can take a MethodHandle to the native method in the 
anonymous class and use that in the backing implementation.

I'm not sure if it's all that easy though, what do you think?

Jorn



From maurizio.cimadamore at oracle.com  Fri Sep 14 18:04:09 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 14 Sep 2018 19:04:09 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <0b5807b991f0ac860114179d858e033c@xs4all.nl>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <0b5807b991f0ac860114179d858e033c@xs4all.nl>
Message-ID: <4a9515ba-b436-ff8a-d6c5-550ee549cc64@oracle.com>


> I'm wondering if the 5 native end points for an arity of 5 are enough. 
> Don't you also need 5 for when the function returns a long and 5 more 
> for when the function returns a double?
This is an example with arity 5 and return void. Don't take this too 
literally :-)

Things will become clearer once you see the webrev, which I'm about to 
post.
>
> I have a suggestion to bypass having to write out all the permutations 
> though. What if, on the Java side, whenever there is a method that has 
> a shape that can be optimized in this way (which is ABI dependent), 
> spin and load a class which defines a single static native method with 
> the needed signature, and annotate it. Then, in NativeLookup::lookup, 
> detect this annotation, and instead of trying to look up the symbol in 
> a loaded library generate a forwarding stub and link the native method 
> to that instead. Then you can take a MethodHandle to the native method 
> in the anonymous class and use that in the backing implementation.
This is a good suggestion, although for now I'd like to keep things 
simple. In principle I agree, with some spinning (and some VM fixup) we 
could be able to get there. You also need to spin upcalls entry points 
too :-)

Maurizio
>
> I'm not sure if it's all that easy though, what do you think?
>
> Jorn
>
>


From maurizio.cimadamore at oracle.com  Fri Sep 14 18:04:48 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 14 Sep 2018 19:04:48 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
Message-ID: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>

Hi,
as mentioned in [1], this patch adds binder support for the so called 
'direct' invocation scheme, which allows for greater native invocation 
downcall/upcall performances by means of specialized adapters. The core 
idea, also described in [1], is to define adapters of the kind:

invokeNative_V_DDDDD
invokeNative_V_JDDDD
invokeNative_V_JJDDD
invokeNative_V_JJJDD
invokeNative_V_JJJJD
invokeNative_V_JJJJJ

Where long arguments come before double arguments (and do this for each 
arity e.g. <=5).

If all arguments are passed in register, then this reordering doesn't 
affect behavior, and greatly limits the number of permutations to be 
supported/generated.

The downcall part (java to native) is relative straightforward: the 
directNativeInvoker.cpp file defines a bunch of native entry points, one 
per shape, which cast the input address to a function pointer of the 
desired shape, and then call it:

jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, 
jlong arg0, jdouble arg1) {
 ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
}

The upcall business is a little trickier: first, if we are only to 
optimize upcalls where argument passing happens in registers, then it's 
crucial to note that by the time we get into the assembly stub, all the 
registers will have been populated by the native code to contain the 
right arguments in the right places. So we can avoid all the shuffling 
in the assembly adapter and simply jump onto a C function that looks 
like this:

long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
 ????????????????????????????????????? double d0, double d1, double d2, 
double d3,
 ?????????????????????????????????????? unsigned int mask, jobject rec) 
{ ... }

Note here that the first 8 arguments are just longs and doubles, and 
those will be expected to be in registers, according to the System V 
ABI. (In windows, the situation will be a bit different as less integer 
registers are available, so this will need some work there).

So, to recap, the assembly upcall stub simply 'append' the receiver 
object and a 'signature mask' in the last two available C registers and 
then jump onto the helper function. The helper function will find all 
the desired arguments in the right places - there will be, in the 
general case, some unused arguments, but that's fine, after all it 
didn't cost anything to us to load them in the first place!

Note that we have three helper variants, one for each return type { 
long, double, void }. This is required as we need the C helper to return 
a value of the right type which will generate the right assembly 
sequence to store the result in the right register (either integer or MMX).

So, with three helpers we can support all the shapes with up to 8 
arguments. On the Java side we have, of course, to define a specialized 
entry point for each shape.

All the magic for adapting method handle to and from the specialized 
adapters happen in the DirectSignatureShuffler class; this class is 
responsible for adapting each argument e.g. from Java to native value, 
and then reordering the adapted method handle to match the order in 
which arguments are expected by the adapter (e.g. move all longs in 
front). The challenge was in having DirectSignatureShuffle to be fully 
symmetric - e.g. I did not want to have different code paths for upcalls 
and downcalls, so the code tries quite hard to be parametric in the 
shuffling direction (java->native or native->java) - which means that 
adapters will be applied in one way or in the inverse way depending on 
the shuffling direction (and as to whether we are adapting an argument 
or a return). Since method handle filters are composable, it all works 
out quite beautifully.

Note that the resulting, adapted MH is stored in a @Stable field to tell 
the JIT to optimize the heck out of it (as if it were a static constant).

This patch contains several other changes - which I discuss briefly below:

* we need to setup a framework in which new invocation strategies can be 
plugged in - note that we now have essentially 4 cases:

{ NativeInvoker, UpcallHandler } x { Universal, Direct }

When the code wants e.g. a NativeInvoker, it asks for one to the 
NativeInvoker::of factory (UpcallHandler work in a similar way); this 
factory will attempt to go down the fast path - if an error occurs when 
computing the fast path, the call will fallback to the universal (slow) 
path.

Most of the changes you see in the Java code are associated to this 
refactoring - e.g. all clients of NativeInvoker/UpcallHandler should now 
go through the factory

* CallbackImplGenerator had a major issue since the new factory for 
NativeInvoker wants to bind an address eagerly (this is required e.g. to 
be forward compatible with linkToNative backend); which means that at 
construction time we have to get the address of the callback, call the 
NativeInvoker factory and then stash the target method handle into a 
field of the anon callback class. Vlad tells me that fields of anon 
classes are always 'trusted' by the JIT, which means they should be 
treated as '@Stable' (note that I can't put a @Stable annotation there, 
since this code will be spinned in user-land).

* There are a bunch of properties that can be set to either force slow 
path or force 'direct' path; in the latter case, if an error occurs when 
instantiating the direct wrapper, an exception is thrown. This mode is 
very useful for testing, and I indeed have tried to run all our tests 
with this flag enabled, to see how many places could not be optimized.

* I've also reorganized all the native code in hotspot/prims so that we 
have a separate file for each scheme (and so that native Java methods 
could be added where they really belong). This should also help in the 
long run as it should make adding/removing a given scheme easier.

* I've also added a small test which tries to pass structs of different 
sizes, but I will also work on a more complex test which will 
stress-test all invocation modes in a more complete fashion. With 
respect to testing, I've also done a fastdebug build and ran all tests 
with that (as fastdebug catches way many more hotspot assertion than the 
product version); everything passed.

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8210757/

I'd like to thank Vladimir Ivanov for the prompt support whenever I got 
stuck down the macro assembler rabbit hole :-)

Cheers
Maurizio

[1] - 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html




From maurizio.cimadamore at oracle.com  Fri Sep 14 20:19:53 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 14 Sep 2018 21:19:53 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <4a9515ba-b436-ff8a-d6c5-550ee549cc64@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <0b5807b991f0ac860114179d858e033c@xs4all.nl>
 <4a9515ba-b436-ff8a-d6c5-550ee549cc64@oracle.com>
Message-ID: <6dcbc5df-69c6-2a6a-36cd-8c2cca7315ca@oracle.com>



On 14/09/18 19:04, Maurizio Cimadamore wrote:
>> I have a suggestion to bypass having to write out all the 
>> permutations though. What if, on the Java side, whenever there is a 
>> method that has a shape that can be optimized in this way (which is 
>> ABI dependent), spin and load a class which defines a single static 
>> native method with the needed signature, and annotate it. Then, in 
>> NativeLookup::lookup, detect this annotation, and instead of trying 
>> to look up the symbol in a loaded library generate a forwarding stub 
>> and link the native method to that instead. Then you can take a 
>> MethodHandle to the native method in the anonymous class and use that 
>> in the backing implementation.
> This is a good suggestion, although for now I'd like to keep things 
> simple. In principle I agree, with some spinning (and some VM fixup) 
> we could be able to get there. You also need to spin upcalls entry 
> points too :-)
thinking more about this, I'm not sure you can go all the way there. 
Yes, we could save the Java entry points - but somewhere, you still will 
need to define the native entry point (one for each shape). So, overall, 
I think what I have is a pragmatic (and honest) approach - where there's 
equal replication on the Java and native side. Spinning classes will 
help with Java replication but not much with the native one.

The good thing is that this problem has already been solved: the 
linkToNative machinery generates a 'native method handle' out of thin 
air with a given shape and target address. With machinery like that, 
then you can really get rid of hand-written entry points on either side 
of the FFI fence. Everything else is, in a way, a workaround.

Maurizio


From shravya.rukmannagari at intel.com  Fri Sep 14 22:55:57 2018
From: shravya.rukmannagari at intel.com (Rukmannagari, Shravya)
Date: Fri, 14 Sep 2018 22:55:57 +0000
Subject: VectorAPI SKL Fix for get
Message-ID: <8D6F463991A1574A8A803B8DA605414F3A7EB1FD@ORSMSX111.amr.corp.intel.com>

Hi All,
I would like to contribute a fix for get(extract) for SKL.

Could you please review the patch here:

http://cr.openjdk.java.net/~srukmannagar/VectorAPI_SKLFixes/webrev_get.00/



Thanks,
Shravya.



From jbvernee at xs4all.nl  Sat Sep 15 00:04:58 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Sat, 15 Sep 2018 02:04:58 +0200
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
Message-ID: <7720eb81fb733d881042b6b0babd94e4@xs4all.nl>

Maurizio Cimadamore schreef op 2018-09-14 20:04:
> Hi,
> as mentioned in [1], this patch adds binder support for the so called
> 'direct' invocation scheme, which allows for greater native invocation
> downcall/upcall performances by means of specialized adapters. The
> core idea, also described in [1], is to define adapters of the kind:
> 
> invokeNative_V_DDDDD
> invokeNative_V_JDDDD
> invokeNative_V_JJDDD
> invokeNative_V_JJJDD
> invokeNative_V_JJJJD
> invokeNative_V_JJJJJ
> 
> Where long arguments come before double arguments (and do this for
> each arity e.g. <=5).
> 
> If all arguments are passed in register, then this reordering doesn't
> affect behavior, and greatly limits the number of permutations to be
> supported/generated.

On windows the story seems to be more difficult then I initially 
thought. On SysV, if you have a C function like this:

void f(long l, double d);

`l` will be passed in the first integer register, and `d` will be passed 
in the first float/vector register. But on windows, `d` will be passed 
in the **second** float/vector register, and if there was a another 
integer argument it would be passed in the third integer register [1]. 
This becomes worse with varargs, which requires floats to be passed in 
both the integer and float/vector registers.

So I don't think reordering parameters will work on windows, since the 
parameter index dictates which register it uses. However, you should be 
able to still use the downcall-with-shuffling strategy (though I don't 
have that working yet for mixed argument classes).

> The downcall part (java to native) is relative straightforward: the
> directNativeInvoker.cpp file defines a bunch of native entry points,
> one per shape, which cast the input address to a function pointer of
> the desired shape, and then call it:
> 
> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr,
> jlong arg0, jdouble arg1) {
>     return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
> }

As an optimization here, I think you should make the function address 
the last argument, since that prevents having to shuffle the other 
arguments between registers before calling the function [2]

> * we need to setup a framework in which new invocation strategies can
> be plugged in - note that we now have essentially 4 cases:
> 
> { NativeInvoker, UpcallHandler } x { Universal, Direct }
> 
> When the code wants e.g. a NativeInvoker, it asks for one to the
> NativeInvoker::of factory (UpcallHandler work in a similar way); this
> factory will attempt to go down the fast path - if an error occurs
> when computing the fast path, the call will fallback to the universal
> (slow) path.

This sounds like a great idea!

Jorn

[1] : 
https://docs.microsoft.com/en-us/cpp/build/parameter-passing?view=vs-2017#example-of-argument-passing-3---mixed-ints-and-floats
[2] : https://godbolt.org/z/JjPJca

From jbvernee at xs4all.nl  Sat Sep 15 20:29:29 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Sat, 15 Sep 2018 22:29:29 +0200
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <7720eb81fb733d881042b6b0babd94e4@xs4all.nl>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <7720eb81fb733d881042b6b0babd94e4@xs4all.nl>
Message-ID: <77440a77b29260cf68679ba91e7bfdc4@xs4all.nl>

Jorn Vernee schreef op 2018-09-15 02:04:
>> The downcall part (java to native) is relative straightforward: the
>> directNativeInvoker.cpp file defines a bunch of native entry points,
>> one per shape, which cast the input address to a function pointer of
>> the desired shape, and then call it:
>> 
>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr,
>> jlong arg0, jdouble arg1) {
>>     return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>> }
> 
> As an optimization here, I think you should make the function address
> the last argument, since that prevents having to shuffle the other
> arguments between registers before calling the function [2]

Never mind. I totally forgot about the leading JNIEnv and jobject 
arguments in my experiment, so there's shuffling either way :) [1]

Jorn

[1] : https://godbolt.org/z/J2td8X

From maurizio.cimadamore at oracle.com  Sat Sep 15 20:31:31 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Sat, 15 Sep 2018 21:31:31 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <7720eb81fb733d881042b6b0babd94e4@xs4all.nl>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <7720eb81fb733d881042b6b0babd94e4@xs4all.nl>
Message-ID: <d581f99b-61c7-1dd7-f6a3-da0a02982701@oracle.com>



On 15/09/18 01:04, Jorn Vernee wrote:
> Maurizio Cimadamore schreef op 2018-09-14 20:04:
>> Hi,
>> as mentioned in [1], this patch adds binder support for the so called
>> 'direct' invocation scheme, which allows for greater native invocation
>> downcall/upcall performances by means of specialized adapters. The
>> core idea, also described in [1], is to define adapters of the kind:
>>
>> invokeNative_V_DDDDD
>> invokeNative_V_JDDDD
>> invokeNative_V_JJDDD
>> invokeNative_V_JJJDD
>> invokeNative_V_JJJJD
>> invokeNative_V_JJJJJ
>>
>> Where long arguments come before double arguments (and do this for
>> each arity e.g. <=5).
>>
>> If all arguments are passed in register, then this reordering doesn't
>> affect behavior, and greatly limits the number of permutations to be
>> supported/generated.
>
> On windows the story seems to be more difficult then I initially 
> thought. On SysV, if you have a C function like this:
>
> void f(long l, double d);
>
> `l` will be passed in the first integer register, and `d` will be 
> passed in the first float/vector register. But on windows, `d` will be 
> passed in the **second** float/vector register, and if there was a 
> another integer argument it would be passed in the third integer 
> register [1]. This becomes worse with varargs, which requires floats 
> to be passed in both the integer and float/vector registers.
>
> So I don't think reordering parameters will work on windows, since the 
> parameter index dictates which register it uses. However, you should 
> be able to still use the downcall-with-shuffling strategy (though I 
> don't have that working yet for mixed argument classes).
Ouch - yes, it does seem that Windows allocates registers in a 
positional way ; that said, it doesn't change the picture much - in a 
way I was fully aware (and probably should have made more explicit) that 
this fast path is an opportunistic optimization that we can take 
depending on the ABI.

The real solution, as mentioned, going forward, is to generate 
specialized JNI stubs on the fly - so that's gonna be the next stop (and 
the one after that is to teach C2/Graal about such stubs so that they 
could be optimized). But the main point here is that, w/o sprucing some 
kind of specialization in the entry points generated by the VM it will 
be impossible to achieve sensible performances.

Back to Windows, it seems like its fastcall strategy allows it to use up 
to 4 registers, period. So, that's not too terrible in terms of number 
of manually generated entry points, if we wanted to go there (not for 
today :-)).
>
>> The downcall part (java to native) is relative straightforward: the
>> directNativeInvoker.cpp file defines a bunch of native entry points,
>> one per shape, which cast the input address to a function pointer of
>> the desired shape, and then call it:
>>
>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr,
>> jlong arg0, jdouble arg1) {
>> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>> }
>
> As an optimization here, I think you should make the function address 
> the last argument, since that prevents having to shuffle the other 
> arguments between registers before calling the function [2]
I'm not sure I 100% get this (and I can't seem to be able to open your 
link).

Once you are inside this adapter, you have already the two first integer 
registers taken by the JNI arguments (*env and _unused, in this 
example). Which seems to suggest that, regardless of where we put 
'addr', the compiled code will still need to move { arg0, arg1 } in the 
first two integer registers?

Maurizio
>
>> * we need to setup a framework in which new invocation strategies can
>> be plugged in - note that we now have essentially 4 cases:
>>
>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>
>> When the code wants e.g. a NativeInvoker, it asks for one to the
>> NativeInvoker::of factory (UpcallHandler work in a similar way); this
>> factory will attempt to go down the fast path - if an error occurs
>> when computing the fast path, the call will fallback to the universal
>> (slow) path.
>
> This sounds like a great idea!
>
> Jorn
>
> [1] : 
> https://docs.microsoft.com/en-us/cpp/build/parameter-passing?view=vs-2017#example-of-argument-passing-3---mixed-ints-and-floats
> [2] : https://godbolt.org/z/JjPJca


From maurizio.cimadamore at oracle.com  Sat Sep 15 20:32:55 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Sat, 15 Sep 2018 21:32:55 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <77440a77b29260cf68679ba91e7bfdc4@xs4all.nl>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <7720eb81fb733d881042b6b0babd94e4@xs4all.nl>
 <77440a77b29260cf68679ba91e7bfdc4@xs4all.nl>
Message-ID: <63bfdcd3-c60d-089d-897a-4acb6240e9a4@oracle.com>



On 15/09/18 21:29, Jorn Vernee wrote:
> Never mind. I totally forgot about the leading JNIEnv and jobject 
> arguments in my experiment, so there's shuffling either way 
Heh - just replied to your email ;-)

Good to know we're on the same page.

Maurizio

From samuel.audet at gmail.com  Mon Sep 17 10:00:21 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Mon, 17 Sep 2018 19:00:21 +0900
Subject: [foreign] some JMH benchmarks
In-Reply-To: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
Message-ID: <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>

Thanks for the figures Maurizio! It's finally good to be speaking in 
numbers. :)

However, you're not providing a lot of details about how you actually 
ran the experiments. So I've decided to run a JMH benchmark on what we 
get by default with JavaCPP and this declaration:

     @Platform(include = "math.h")
     public class MyBenchmark {
         static { Loader.load(); }

         @NoException
         public static native double exp(double x);

         @State(Scope.Thread)
         public static class MyState {
             double x;

             @Setup(Level.Iteration)
             public void setupMethod() {
                 x = Math.random();
             }
         }

         @Benchmark
         public void testMethod(MyState s, Blackhole bh) {
             bh.consume(exp(s.x));
         }
     }

The relevant portion of generated JNI looks like this:

     JNIEXPORT jdouble JNICALL Java_org_sample_MyBenchmark_exp(JNIEnv* 
env, jclass cls, jdouble arg0) {
         jdouble rarg = 0;
         double rval = exp(arg0);
         rarg = (jdouble)rval;
         return rarg;
     }

And with access to just 2 virtual cores of an Intel(R) Xeon(R) CPU 
E5-2673 v4 @ 2.30GHz and 8 GB of RAM on the cloud (so probably slower 
than your E5-2665 @ 2.40GHz) running Ubuntu 14.04 with GCC 4.9 and 
OpenJDK 8, I get these numbers:
Benchmark                Mode  Cnt         Score        Error  Units
MyBenchmark.testMethod  thrpt   25  37183556.094 ? 460795.746  ops/s

I'm not sure how that compares with your numbers exactly, but it does 
seem to me that what you get for JNI is a bit low. If you could provide 
more details about how to reproduce your results, that would be great.

Samuel


On 09/14/2018 10:19 PM, Maurizio Cimadamore wrote:
> Hi,
> over the last few weeks I've been busy playing with Panama and assessing 
> performances with JMH. For those just interested in raw numbers, the 
> results of my explorations can be found here [1]. But as all benchmarks, 
> I think it's better to spend few words to understand what these numbers 
> actually _mean_.
> 
> To evaluate the performances of Panama I have first created a baseline 
> using JNI - more specifically I wanted to assess performances of three 
> calls (all part of the C std library), namely `getpid`, `exp` and `qsort`.
> 
> The first example is the de facto benchmark for FFIs - since it does 
> relatively little computation, it is a good test to measure the 
> 'latency' of the FFI approach (e.g. how long does it take to go to 
> native). The second example is also relatively simple, but the this time 
> the function takes a double argument. The third test is akin to an FFI 
> torture test, since not only it passes substantially more arguments (4) 
> but one of these arguments is also a callback - a pointer to a function 
> that is used to sort the contents of the input array.
> 
> As expected, the first batch of JNI results confirms our expectations: 
> `getpid` is the fastest, followed by `exp`, and then followed by 
> `qsort`. Note that qsort is not even close in terms of raw numbers to 
> the other two tests - that's because, to sort the array we need to do (N 
> * log N) upcalls into Java. In the benchmark, N = 8 and we do the 
> upcalls using the JNI function JNIEnv::CallIntMethod.
> 
> Now let's examine the second batch of results; these call `getpid`, 
> `exp` and `qsort` using Panama. The numbers here are considerably lower 
> than the JNI ones for all the three benchmark - although the first two 
> seem to be the most problematic. To explain these results we need to 
> peek under the hood. Panama implements foreign calls through a so called 
> 'universal adapter' which, given a calling scheme and a bunch of 
> arguments (machine words) shuffles these arguments in the right 
> registers/stack slot and then jumps to the target native function - 
> after which another round of adaptation must be performed (e.g. to 
> recover the return value from the right register/memory location).
> 
> Needless to say, all this generality comes at a cost - some of the cost 
> is in Java - e.g. all arguments have to be packaged up into a long array 
> (although this component doesn't seem to show up much in the generated 
> JVM compiled code). A lot of the cost is in the adapter logic itself - 
> which has to look at the 'call recipe' and move arguments around 
> accordingly - more specifically, in order to call the native function, 
> the adapter creates a bunch of helper C++ objects and structs which 
> model the CPU state (e.g. in the ShuffleDowncallContext struct, we find 
> a field for each register to be modeled in the target architecture). The 
> adapter has to first move the values coming from the Java world (stored 
> in the aforementioned long array) into the right context fields (and it 
> needs to do so by looking at the recipe, which involves iteration over 
> the recipe elements). After that's done, we can jump into the assembly 
> stub that does the native call - this stub will take as input one of 
> those ShuffleDowncallContext structure and will load the corresponding 
> registers/create necessary stack slots ahead of the call.
> 
> As you can see, there's quite a lot of action going on here, and this 
> explains the benchmark numbers; of course, if you are calling a native 
> function that does a lot of computation, this adaptation cost will wash 
> out - but for relatively quick calls such as 'getpid' and 'exp' the 
> latency dominates the picture.
> 
> Digression: the callback logic suffers pretty much from the same issues, 
> albeit in a reversed order - this time it's the Java code which receives 
> a 'snapshot' of the register values from a generated assembly adapter; 
> the Java code can then read such values (using the Pointer API), turn 
> them into Java objects, call the target Java method and store the 
> results (after another conversion) in the right location of the 
> snapshot. The assembly adapter will then pick up the values set onto the 
> snapshot by the Java code, store them into the corresponding registers 
> and return control to the native callee. In the remainder of this email 
> we will not discuss callbacks in details - we will just posit that for 
> any optimization technique that can be defined, there exists a _dual_ 
> strategy that works with callbacks.
> 
> How can we make sensible native calls go faster? Well, one obvious way 
> would be to optimize the universal adapter so that we get a specialized 
> assembly stub for each code shape. If we do that, we can move pretty 
> much all of the computation described above from execution time to the 
> stub generation time, so that, by the time we have to call the native 
> function, we just have to populate the right registers (the specialized 
> stub knows where to find them) and jump. While this sounds a good 
> approach, it feels like there's also a move for the JIT somewhere in 
> there - after all, the JVM knows which calls are hot and in need for 
> optimization, so perhaps this specialization process (some or all of it) 
> could happen dynamically. And this is indeed an approach we'd like to 
> aim for in the long run.
> 
> Now, few years ago, Vlad put together a patch which now lives in the 
> 'linkToNative' branch [6, 7] - the goal of this patch is to implement 
> the approach described above: generate a specialized assembly adapter 
> for a given native signature, and then leverage the JIT to optimize it 
> away, turning the adapter into a bare, direct, native method call. As 
> you can see from the third batch of benchmarks, if we tweak Panama to 
> use the linkToNative machinery, the speed up is really impressive, and 
> we end up being much faster than JNI (up to 4x for getPid).
> 
> Unfortunately, the technology in the linkToNative branch is not ready 
> from prime time (yet) - first, it doesn't cover some useful cases (e.g. 
> varargs, multiple returns via registers, arguments passed in memory). 
> That is, the logic assumes there's a 1-1 mapping between a Java 
> signature and the native function to be called - and that the arguments 
> passed from Java will either be longs or doubles. While we can 
> workaround this limitation and define the necessary marshalling logic in 
> Java (as I have done to run this benchmark), some of the limitations 
> (multiple returns, structs passed by value which are too big) cannot 
> simply be worked around. But that's fine, we can still have a fast path 
> for those calls which have certain characteristics and a slow path 
> (through the universal adapter) for all the other calls.
> 
> But there's a second and more serious issue lurking: as you can see in 
> the benchmark, I was not able to get the qsort benchmark running when 
> using the linkToNative backend. The reason is that the linkToNative code 
> is still pretty raw, and it doesn't fully adhere to the JVM internal 
> conventions - e.g. there are missing thread state transitions which, in 
> the case of upcalls into Java, create issues when it comes to garbage 
> collection, as the GC cannot parse the native stack in the correct way.
> 
> This means that, while there's a clear shining path ahead of us, it is 
> simply too early to just use the linkToNative backend from Panama. For 
> this reason, I've been looking into some kind of stopgap solution - 
> another way of optimizing native calls (and upcalls into Java) that 
> doesn't require too much VM magic. Now, a crucial observation is that, 
> in many native calls, there is indeed a 1-1 mapping between Java 
> arguments and native arguments (and back, for return values). That is, 
> we can think of calling a native function as a process that takes a 
> bunch of Java arguments, turn them into native arguments (either double 
> or longs), calls the native methods and then turns back the result into 
> Java.
> 
> The mapping between Java arguments and C values is quite simple:
> 
> * primitives: either long or double, depending on whether they describe 
> an integral value or a floating point one.
> * pointers: they convert to a long
> * callbacks: they also convert to a long
> * structs: they are recursively decomposed into fields and each field is 
> marshalled separately (assuming the struct is not too big, in which case 
> is passed in memory)
> 
> So, in principle, we could define a bunch of native entry points in the 
> VM, one per shape, which take a bunch of long and doubles and call an 
> underlying function with those arguments. For instance, let's consider 
> the case of a native function which is modelled in Java as:
> 
> int m(Pointer<Foo>, double)
> 
> To call this native function we have to first turn the Java arguments 
> into a (long, double) pair. Then we need to call a native adapter that 
> looks like the following:
> 
> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, 
> jlong arg0, jdouble arg1) {
>  ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
> }
> 
> And this will take care of calling the native function and returning the 
> value back. This is, admittedly, a very simple solution; of course there 
> are limitations: we have to define a bunch of specialized native entry 
> point (and Java entry points, for callbacks). But here we can play a 
> trick: most of moderns ABI pass arguments in registers; for instance 
> System V ABI [5] uses up to 6 (!!) integer registers and 7 (!!) MMXr 
> registers for FP values - this gives us a total of 13 registers 
> available for argument passing. Which covers quite a lot of cases. Now, 
> if we have a call where _all_ arguments are passed in registers, then 
> the order in which these arguments are declared in the adapter doesn't 
> matter! That is, since FP-values will always be passed in different 
> register from integral values, we can just define entry points which 
> look like these:
> 
> invokeNative_V_DDDDD
> invokeNative_V_JDDDD
> invokeNative_V_JJDDD
> invokeNative_V_JJJDD
> invokeNative_V_JJJJD
> invokeNative_V_JJJJJ
> 
> That is, for a given arity (5 in this case), we can just put all long 
> arguments in front, and the double arguments after that. That is, we 
> don't need to generate all possible permutations of J/D in all positions 
> - as the adapter will always do the same thing (read: load from same 
> registers) for all equivalent combinations. This keeps the number of 
> entry points in check -? and it also poses some challenges to the Java 
> logic in charge of marshalling/unmarshalling, as there's an extra 
> permutation step involved (although that is not something super-hard to 
> address).
> 
> You can see the performance numbers associated with this invocation 
> scheme (which I've dubbed 'direct') in the 4th batch of the benchmark 
> results. These numbers are on par (and slightly better) with JNI in all 
> the three cases considered which is, I think, a very positive result, 
> given that to write these benchmarks I did not have to write a single 
> line of JNI code. In other words, this optimization gives you the same 
> speed as JNI, with improved ease of use (**).
> 
> Now, since the 'direct' optimization builds on top of the VM native call 
> adapters, this approach is significantly more robust than linkToNative 
> and I have not run into any weird VM crashes when playing with it. The 
> downside of that, is that, for obvious reasons, this approach cannot get 
> much faster than JNI - that is, it cannot get close to the numbers 
> obtained with the linkToNative backend, which features much deeper 
> optimizations. But I think that, despite its limitations, it's still a 
> good opportunistic improvement that is worth pursuing in the short term 
> (while we sort out the linkToNative story). For this reason, I will soon 
> be submitting a review which incorporates the changes for the 'direct' 
> invocation schemes.
> 
> Cheers
> Maurizio
> 
> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
> [2] - https://github.com/jnr/jnr-ffi
> [3] - https://github.com/jnr/jffi
> [4] - https://sourceware.org/libffi/
> [5] - 
> https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf 
> 
> [6] - http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
> [7] - http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354
> 
> (**) the benchmark also contains a 5th row in which I repeated same 
> tests, this time using JNR [2]. JNR is built on top of libjffi [3], a 
> JNI library in turn built on top of the popular libffi [4]. I wanted to 
> have some numbers about JNR because that's another solution that allows 
> for better ease to use, taking care of marshalling Java values into C 
> and back; since the goals of JNR are similar in spirit with some of the 
> goals of the Panama/foreign work, I thought it would be worth having a 
> comparison of these approaches. For the records, I think the JNR numbers 
> are very respectable given that JNR had to do all the hard work outside 
> of the JDK!
> 
> 


From maurizio.cimadamore at oracle.com  Mon Sep 17 12:37:02 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 17 Sep 2018 13:37:02 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
Message-ID: <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>

Hi Samuel,
I was planning to upload the benchmark IDE project in the near future (I 
need to clean that up a bit, so that it can be opened at ease):

My getpid example is like this; this is the Java decl:

public class GetPid {

 ??? static {
 ??????? System.loadLibrary("getpid");
 ??? }

 ??? native static long getpid();

 ??? native double exp(double base);
}

This is the JNI code:

JNIEXPORT jlong JNICALL Java_org_panama_GetPid_getpid
 ? (JNIEnv *env, jobject recv) {
 ?? return getpid();
}

JNIEXPORT jdouble JNICALL Java_org_panama_GetPid_exp
 ? (JNIEnv *env, jobject recv, jdouble arg) {
 ?? return exp(arg);
}

And this is the benchmark:

class PanamaBenchmark {
 ??? static GetPid pidlib = new GetPid();

 ??? @Benchmark
 ??? public long testJNIPid() {
 ??????? return pidlib.getpid();
 ??? }

 ??? @Benchmark
 ??? public double testJNIExp() {
 ??????? return pidlib.exp(10d);
 ??? }
}


I think this should be rather standard?

I'm on Ubuntu 16.04.1, and using GCC 5.4.0. The command I use to compile 
the C lib is this:

gcc -I<path to jni,h> -l<path to jni lib> -shared -o libgetpid.so -fPIC 
GetPid.c

One difference I see between our two examples is the use of BlackHole. 
In my bench, I'm just returning the call to 'exp' - which should be 
equivalent, and, actually, preferred, as described here:

http://hg.openjdk.java.net/code-tools/jmh/file/3769055ad883/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_09_Blackholes.java#l51

Another minor difference I see is that I pass a constant argument, while 
you generate a random number on each iteration.

I tried to cut and paste your benchmark and I got this:

Benchmark??????????????????? Mode? Cnt???????? Score Error? Units
PanamaBenchmark.testMethod? thrpt??? 5? 26362701.827 ? 1357012.981? ops/s


Which looks exactly the same as what I've got. So, for whatever reason, 
my machine seems to be slower than the one you are using. For what is 
worth, this website [1] seems to confirm the difference. While clock 
speeds are similar, your machine has more Ghz in Turbo boost mode and 
it's 3-4 years newer than mine, so I'd expect that to make a difference 
in terms of internal optimizations etc. Note that I'm able to beat the 
numbers of my workstation using my laptop which sports a slightly higher 
frequency and only has 2 cores and 8G of RAM.

[1] - 
https://www.cpubenchmark.net/compare/Intel-Xeon-E5-2673-v4-vs-Intel-Xeon-E5-2665/2888vs1439

Maurizio



On 17/09/18 11:00, Samuel Audet wrote:
> Thanks for the figures Maurizio! It's finally good to be speaking in 
> numbers. :)
>
> However, you're not providing a lot of details about how you actually 
> ran the experiments. So I've decided to run a JMH benchmark on what we 
> get by default with JavaCPP and this declaration:
>
> ??? @Platform(include = "math.h")
> ??? public class MyBenchmark {
> ??????? static { Loader.load(); }
>
> ??????? @NoException
> ??????? public static native double exp(double x);
>
> ??????? @State(Scope.Thread)
> ??????? public static class MyState {
> ??????????? double x;
>
> ??????????? @Setup(Level.Iteration)
> ??????????? public void setupMethod() {
> ??????????????? x = Math.random();
> ??????????? }
> ??????? }
>
> ??????? @Benchmark
> ??????? public void testMethod(MyState s, Blackhole bh) {
> ??????????? bh.consume(exp(s.x));
> ??????? }
> ??? }
>
> The relevant portion of generated JNI looks like this:
>
> ??? JNIEXPORT jdouble JNICALL Java_org_sample_MyBenchmark_exp(JNIEnv* 
> env, jclass cls, jdouble arg0) {
> ??????? jdouble rarg = 0;
> ??????? double rval = exp(arg0);
> ??????? rarg = (jdouble)rval;
> ??????? return rarg;
> ??? }
>
> And with access to just 2 virtual cores of an Intel(R) Xeon(R) CPU 
> E5-2673 v4 @ 2.30GHz and 8 GB of RAM on the cloud (so probably slower 
> than your E5-2665 @ 2.40GHz) running Ubuntu 14.04 with GCC 4.9 and 
> OpenJDK 8, I get these numbers:
> Benchmark??????????????? Mode? Cnt???????? Score??????? Error Units
> MyBenchmark.testMethod? thrpt?? 25? 37183556.094 ? 460795.746 ops/s
>
> I'm not sure how that compares with your numbers exactly, but it does 
> seem to me that what you get for JNI is a bit low. If you could 
> provide more details about how to reproduce your results, that would 
> be great.
>
> Samuel
>
>
> On 09/14/2018 10:19 PM, Maurizio Cimadamore wrote:
>> Hi,
>> over the last few weeks I've been busy playing with Panama and 
>> assessing performances with JMH. For those just interested in raw 
>> numbers, the results of my explorations can be found here [1]. But as 
>> all benchmarks, I think it's better to spend few words to understand 
>> what these numbers actually _mean_.
>>
>> To evaluate the performances of Panama I have first created a 
>> baseline using JNI - more specifically I wanted to assess 
>> performances of three calls (all part of the C std library), namely 
>> `getpid`, `exp` and `qsort`.
>>
>> The first example is the de facto benchmark for FFIs - since it does 
>> relatively little computation, it is a good test to measure the 
>> 'latency' of the FFI approach (e.g. how long does it take to go to 
>> native). The second example is also relatively simple, but the this 
>> time the function takes a double argument. The third test is akin to 
>> an FFI torture test, since not only it passes substantially more 
>> arguments (4) but one of these arguments is also a callback - a 
>> pointer to a function that is used to sort the contents of the input 
>> array.
>>
>> As expected, the first batch of JNI results confirms our 
>> expectations: `getpid` is the fastest, followed by `exp`, and then 
>> followed by `qsort`. Note that qsort is not even close in terms of 
>> raw numbers to the other two tests - that's because, to sort the 
>> array we need to do (N * log N) upcalls into Java. In the benchmark, 
>> N = 8 and we do the upcalls using the JNI function 
>> JNIEnv::CallIntMethod.
>>
>> Now let's examine the second batch of results; these call `getpid`, 
>> `exp` and `qsort` using Panama. The numbers here are considerably 
>> lower than the JNI ones for all the three benchmark - although the 
>> first two seem to be the most problematic. To explain these results 
>> we need to peek under the hood. Panama implements foreign calls 
>> through a so called 'universal adapter' which, given a calling scheme 
>> and a bunch of arguments (machine words) shuffles these arguments in 
>> the right registers/stack slot and then jumps to the target native 
>> function - after which another round of adaptation must be performed 
>> (e.g. to recover the return value from the right register/memory 
>> location).
>>
>> Needless to say, all this generality comes at a cost - some of the 
>> cost is in Java - e.g. all arguments have to be packaged up into a 
>> long array (although this component doesn't seem to show up much in 
>> the generated JVM compiled code). A lot of the cost is in the adapter 
>> logic itself - which has to look at the 'call recipe' and move 
>> arguments around accordingly - more specifically, in order to call 
>> the native function, the adapter creates a bunch of helper C++ 
>> objects and structs which model the CPU state (e.g. in the 
>> ShuffleDowncallContext struct, we find a field for each register to 
>> be modeled in the target architecture). The adapter has to first move 
>> the values coming from the Java world (stored in the aforementioned 
>> long array) into the right context fields (and it needs to do so by 
>> looking at the recipe, which involves iteration over the recipe 
>> elements). After that's done, we can jump into the assembly stub that 
>> does the native call - this stub will take as input one of those 
>> ShuffleDowncallContext structure and will load the corresponding 
>> registers/create necessary stack slots ahead of the call.
>>
>> As you can see, there's quite a lot of action going on here, and this 
>> explains the benchmark numbers; of course, if you are calling a 
>> native function that does a lot of computation, this adaptation cost 
>> will wash out - but for relatively quick calls such as 'getpid' and 
>> 'exp' the latency dominates the picture.
>>
>> Digression: the callback logic suffers pretty much from the same 
>> issues, albeit in a reversed order - this time it's the Java code 
>> which receives a 'snapshot' of the register values from a generated 
>> assembly adapter; the Java code can then read such values (using the 
>> Pointer API), turn them into Java objects, call the target Java 
>> method and store the results (after another conversion) in the right 
>> location of the snapshot. The assembly adapter will then pick up the 
>> values set onto the snapshot by the Java code, store them into the 
>> corresponding registers and return control to the native callee. In 
>> the remainder of this email we will not discuss callbacks in details 
>> - we will just posit that for any optimization technique that can be 
>> defined, there exists a _dual_ strategy that works with callbacks.
>>
>> How can we make sensible native calls go faster? Well, one obvious 
>> way would be to optimize the universal adapter so that we get a 
>> specialized assembly stub for each code shape. If we do that, we can 
>> move pretty much all of the computation described above from 
>> execution time to the stub generation time, so that, by the time we 
>> have to call the native function, we just have to populate the right 
>> registers (the specialized stub knows where to find them) and jump. 
>> While this sounds a good approach, it feels like there's also a move 
>> for the JIT somewhere in there - after all, the JVM knows which calls 
>> are hot and in need for optimization, so perhaps this specialization 
>> process (some or all of it) could happen dynamically. And this is 
>> indeed an approach we'd like to aim for in the long run.
>>
>> Now, few years ago, Vlad put together a patch which now lives in the 
>> 'linkToNative' branch [6, 7] - the goal of this patch is to implement 
>> the approach described above: generate a specialized assembly adapter 
>> for a given native signature, and then leverage the JIT to optimize 
>> it away, turning the adapter into a bare, direct, native method call. 
>> As you can see from the third batch of benchmarks, if we tweak Panama 
>> to use the linkToNative machinery, the speed up is really impressive, 
>> and we end up being much faster than JNI (up to 4x for getPid).
>>
>> Unfortunately, the technology in the linkToNative branch is not ready 
>> from prime time (yet) - first, it doesn't cover some useful cases 
>> (e.g. varargs, multiple returns via registers, arguments passed in 
>> memory). That is, the logic assumes there's a 1-1 mapping between a 
>> Java signature and the native function to be called - and that the 
>> arguments passed from Java will either be longs or doubles. While we 
>> can workaround this limitation and define the necessary marshalling 
>> logic in Java (as I have done to run this benchmark), some of the 
>> limitations (multiple returns, structs passed by value which are too 
>> big) cannot simply be worked around. But that's fine, we can still 
>> have a fast path for those calls which have certain characteristics 
>> and a slow path (through the universal adapter) for all the other calls.
>>
>> But there's a second and more serious issue lurking: as you can see 
>> in the benchmark, I was not able to get the qsort benchmark running 
>> when using the linkToNative backend. The reason is that the 
>> linkToNative code is still pretty raw, and it doesn't fully adhere to 
>> the JVM internal conventions - e.g. there are missing thread state 
>> transitions which, in the case of upcalls into Java, create issues 
>> when it comes to garbage collection, as the GC cannot parse the 
>> native stack in the correct way.
>>
>> This means that, while there's a clear shining path ahead of us, it 
>> is simply too early to just use the linkToNative backend from Panama. 
>> For this reason, I've been looking into some kind of stopgap solution 
>> - another way of optimizing native calls (and upcalls into Java) that 
>> doesn't require too much VM magic. Now, a crucial observation is 
>> that, in many native calls, there is indeed a 1-1 mapping between 
>> Java arguments and native arguments (and back, for return values). 
>> That is, we can think of calling a native function as a process that 
>> takes a bunch of Java arguments, turn them into native arguments 
>> (either double or longs), calls the native methods and then turns 
>> back the result into Java.
>>
>> The mapping between Java arguments and C values is quite simple:
>>
>> * primitives: either long or double, depending on whether they 
>> describe an integral value or a floating point one.
>> * pointers: they convert to a long
>> * callbacks: they also convert to a long
>> * structs: they are recursively decomposed into fields and each field 
>> is marshalled separately (assuming the struct is not too big, in 
>> which case is passed in memory)
>>
>> So, in principle, we could define a bunch of native entry points in 
>> the VM, one per shape, which take a bunch of long and doubles and 
>> call an underlying function with those arguments. For instance, let's 
>> consider the case of a native function which is modelled in Java as:
>>
>> int m(Pointer<Foo>, double)
>>
>> To call this native function we have to first turn the Java arguments 
>> into a (long, double) pair. Then we need to call a native adapter 
>> that looks like the following:
>>
>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, 
>> jlong arg0, jdouble arg1) {
>> ???? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>> }
>>
>> And this will take care of calling the native function and returning 
>> the value back. This is, admittedly, a very simple solution; of 
>> course there are limitations: we have to define a bunch of 
>> specialized native entry point (and Java entry points, for 
>> callbacks). But here we can play a trick: most of moderns ABI pass 
>> arguments in registers; for instance System V ABI [5] uses up to 6 
>> (!!) integer registers and 7 (!!) MMXr registers for FP values - this 
>> gives us a total of 13 registers available for argument passing. 
>> Which covers quite a lot of cases. Now, if we have a call where _all_ 
>> arguments are passed in registers, then the order in which these 
>> arguments are declared in the adapter doesn't matter! That is, since 
>> FP-values will always be passed in different register from integral 
>> values, we can just define entry points which look like these:
>>
>> invokeNative_V_DDDDD
>> invokeNative_V_JDDDD
>> invokeNative_V_JJDDD
>> invokeNative_V_JJJDD
>> invokeNative_V_JJJJD
>> invokeNative_V_JJJJJ
>>
>> That is, for a given arity (5 in this case), we can just put all long 
>> arguments in front, and the double arguments after that. That is, we 
>> don't need to generate all possible permutations of J/D in all 
>> positions - as the adapter will always do the same thing (read: load 
>> from same registers) for all equivalent combinations. This keeps the 
>> number of entry points in check - and it also poses some challenges 
>> to the Java logic in charge of marshalling/unmarshalling, as there's 
>> an extra permutation step involved (although that is not something 
>> super-hard to address).
>>
>> You can see the performance numbers associated with this invocation 
>> scheme (which I've dubbed 'direct') in the 4th batch of the benchmark 
>> results. These numbers are on par (and slightly better) with JNI in 
>> all the three cases considered which is, I think, a very positive 
>> result, given that to write these benchmarks I did not have to write 
>> a single line of JNI code. In other words, this optimization gives 
>> you the same speed as JNI, with improved ease of use (**).
>>
>> Now, since the 'direct' optimization builds on top of the VM native 
>> call adapters, this approach is significantly more robust than 
>> linkToNative and I have not run into any weird VM crashes when 
>> playing with it. The downside of that, is that, for obvious reasons, 
>> this approach cannot get much faster than JNI - that is, it cannot 
>> get close to the numbers obtained with the linkToNative backend, 
>> which features much deeper optimizations. But I think that, despite 
>> its limitations, it's still a good opportunistic improvement that is 
>> worth pursuing in the short term (while we sort out the linkToNative 
>> story). For this reason, I will soon be submitting a review which 
>> incorporates the changes for the 'direct' invocation schemes.
>>
>> Cheers
>> Maurizio
>>
>> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>> [2] - https://github.com/jnr/jnr-ffi
>> [3] - https://github.com/jnr/jffi
>> [4] - https://sourceware.org/libffi/
>> [5] - 
>> https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf 
>>
>> [6] - 
>> http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
>> [7] - http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354
>>
>> (**) the benchmark also contains a 5th row in which I repeated same 
>> tests, this time using JNR [2]. JNR is built on top of libjffi [3], a 
>> JNI library in turn built on top of the popular libffi [4]. I wanted 
>> to have some numbers about JNR because that's another solution that 
>> allows for better ease to use, taking care of marshalling Java values 
>> into C and back; since the goals of JNR are similar in spirit with 
>> some of the goals of the Panama/foreign work, I thought it would be 
>> worth having a comparison of these approaches. For the records, I 
>> think the JNR numbers are very respectable given that JNR had to do 
>> all the hard work outside of the JDK!
>>
>>
>


From samuel.audet at gmail.com  Mon Sep 17 14:08:27 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Mon, 17 Sep 2018 23:08:27 +0900
Subject: [foreign] some JMH benchmarks
In-Reply-To: <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
Message-ID: <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>

Yes, the blackhole or the random number doesn't make any difference, but
not calling gcc with -O3 does. Running the compiler with optimizations on
is pretty common, but they are not enabled by default.

Samuel


2018?9?17?(?) 21:37 Maurizio Cimadamore <maurizio.cimadamore at oracle.com>:

> Hi Samuel,
> I was planning to upload the benchmark IDE project in the near future (I
> need to clean that up a bit, so that it can be opened at ease):
>
> My getpid example is like this; this is the Java decl:
>
> public class GetPid {
>
>      static {
>          System.loadLibrary("getpid");
>      }
>
>      native static long getpid();
>
>      native double exp(double base);
> }
>
> This is the JNI code:
>
> JNIEXPORT jlong JNICALL Java_org_panama_GetPid_getpid
>    (JNIEnv *env, jobject recv) {
>     return getpid();
> }
>
> JNIEXPORT jdouble JNICALL Java_org_panama_GetPid_exp
>    (JNIEnv *env, jobject recv, jdouble arg) {
>     return exp(arg);
> }
>
> And this is the benchmark:
>
> class PanamaBenchmark {
>      static GetPid pidlib = new GetPid();
>
>      @Benchmark
>      public long testJNIPid() {
>          return pidlib.getpid();
>      }
>
>      @Benchmark
>      public double testJNIExp() {
>          return pidlib.exp(10d);
>      }
> }
>
>
> I think this should be rather standard?
>
> I'm on Ubuntu 16.04.1, and using GCC 5.4.0. The command I use to compile
> the C lib is this:
>
> gcc -I<path to jni,h> -l<path to jni lib> -shared -o libgetpid.so -fPIC
> GetPid.c
>
> One difference I see between our two examples is the use of BlackHole.
> In my bench, I'm just returning the call to 'exp' - which should be
> equivalent, and, actually, preferred, as described here:
>
>
> http://hg.openjdk.java.net/code-tools/jmh/file/3769055ad883/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_09_Blackholes.java#l51
>
> Another minor difference I see is that I pass a constant argument, while
> you generate a random number on each iteration.
>
> I tried to cut and paste your benchmark and I got this:
>
> Benchmark                    Mode  Cnt         Score Error  Units
> PanamaBenchmark.testMethod  thrpt    5  26362701.827 ? 1357012.981  ops/s
>
>
> Which looks exactly the same as what I've got. So, for whatever reason,
> my machine seems to be slower than the one you are using. For what is
> worth, this website [1] seems to confirm the difference. While clock
> speeds are similar, your machine has more Ghz in Turbo boost mode and
> it's 3-4 years newer than mine, so I'd expect that to make a difference
> in terms of internal optimizations etc. Note that I'm able to beat the
> numbers of my workstation using my laptop which sports a slightly higher
> frequency and only has 2 cores and 8G of RAM.
>
> [1] -
>
> https://www.cpubenchmark.net/compare/Intel-Xeon-E5-2673-v4-vs-Intel-Xeon-E5-2665/2888vs1439
>
> Maurizio
>
>
>
> On 17/09/18 11:00, Samuel Audet wrote:
> > Thanks for the figures Maurizio! It's finally good to be speaking in
> > numbers. :)
> >
> > However, you're not providing a lot of details about how you actually
> > ran the experiments. So I've decided to run a JMH benchmark on what we
> > get by default with JavaCPP and this declaration:
> >
> >     @Platform(include = "math.h")
> >     public class MyBenchmark {
> >         static { Loader.load(); }
> >
> >         @NoException
> >         public static native double exp(double x);
> >
> >         @State(Scope.Thread)
> >         public static class MyState {
> >             double x;
> >
> >             @Setup(Level.Iteration)
> >             public void setupMethod() {
> >                 x = Math.random();
> >             }
> >         }
> >
> >         @Benchmark
> >         public void testMethod(MyState s, Blackhole bh) {
> >             bh.consume(exp(s.x));
> >         }
> >     }
> >
> > The relevant portion of generated JNI looks like this:
> >
> >     JNIEXPORT jdouble JNICALL Java_org_sample_MyBenchmark_exp(JNIEnv*
> > env, jclass cls, jdouble arg0) {
> >         jdouble rarg = 0;
> >         double rval = exp(arg0);
> >         rarg = (jdouble)rval;
> >         return rarg;
> >     }
> >
> > And with access to just 2 virtual cores of an Intel(R) Xeon(R) CPU
> > E5-2673 v4 @ 2.30GHz and 8 GB of RAM on the cloud (so probably slower
> > than your E5-2665 @ 2.40GHz) running Ubuntu 14.04 with GCC 4.9 and
> > OpenJDK 8, I get these numbers:
> > Benchmark                Mode  Cnt         Score        Error Units
> > MyBenchmark.testMethod  thrpt   25  37183556.094 ? 460795.746 ops/s
> >
> > I'm not sure how that compares with your numbers exactly, but it does
> > seem to me that what you get for JNI is a bit low. If you could
> > provide more details about how to reproduce your results, that would
> > be great.
> >
> > Samuel
> >
> >
> > On 09/14/2018 10:19 PM, Maurizio Cimadamore wrote:
> >> Hi,
> >> over the last few weeks I've been busy playing with Panama and
> >> assessing performances with JMH. For those just interested in raw
> >> numbers, the results of my explorations can be found here [1]. But as
> >> all benchmarks, I think it's better to spend few words to understand
> >> what these numbers actually _mean_.
> >>
> >> To evaluate the performances of Panama I have first created a
> >> baseline using JNI - more specifically I wanted to assess
> >> performances of three calls (all part of the C std library), namely
> >> `getpid`, `exp` and `qsort`.
> >>
> >> The first example is the de facto benchmark for FFIs - since it does
> >> relatively little computation, it is a good test to measure the
> >> 'latency' of the FFI approach (e.g. how long does it take to go to
> >> native). The second example is also relatively simple, but the this
> >> time the function takes a double argument. The third test is akin to
> >> an FFI torture test, since not only it passes substantially more
> >> arguments (4) but one of these arguments is also a callback - a
> >> pointer to a function that is used to sort the contents of the input
> >> array.
> >>
> >> As expected, the first batch of JNI results confirms our
> >> expectations: `getpid` is the fastest, followed by `exp`, and then
> >> followed by `qsort`. Note that qsort is not even close in terms of
> >> raw numbers to the other two tests - that's because, to sort the
> >> array we need to do (N * log N) upcalls into Java. In the benchmark,
> >> N = 8 and we do the upcalls using the JNI function
> >> JNIEnv::CallIntMethod.
> >>
> >> Now let's examine the second batch of results; these call `getpid`,
> >> `exp` and `qsort` using Panama. The numbers here are considerably
> >> lower than the JNI ones for all the three benchmark - although the
> >> first two seem to be the most problematic. To explain these results
> >> we need to peek under the hood. Panama implements foreign calls
> >> through a so called 'universal adapter' which, given a calling scheme
> >> and a bunch of arguments (machine words) shuffles these arguments in
> >> the right registers/stack slot and then jumps to the target native
> >> function - after which another round of adaptation must be performed
> >> (e.g. to recover the return value from the right register/memory
> >> location).
> >>
> >> Needless to say, all this generality comes at a cost - some of the
> >> cost is in Java - e.g. all arguments have to be packaged up into a
> >> long array (although this component doesn't seem to show up much in
> >> the generated JVM compiled code). A lot of the cost is in the adapter
> >> logic itself - which has to look at the 'call recipe' and move
> >> arguments around accordingly - more specifically, in order to call
> >> the native function, the adapter creates a bunch of helper C++
> >> objects and structs which model the CPU state (e.g. in the
> >> ShuffleDowncallContext struct, we find a field for each register to
> >> be modeled in the target architecture). The adapter has to first move
> >> the values coming from the Java world (stored in the aforementioned
> >> long array) into the right context fields (and it needs to do so by
> >> looking at the recipe, which involves iteration over the recipe
> >> elements). After that's done, we can jump into the assembly stub that
> >> does the native call - this stub will take as input one of those
> >> ShuffleDowncallContext structure and will load the corresponding
> >> registers/create necessary stack slots ahead of the call.
> >>
> >> As you can see, there's quite a lot of action going on here, and this
> >> explains the benchmark numbers; of course, if you are calling a
> >> native function that does a lot of computation, this adaptation cost
> >> will wash out - but for relatively quick calls such as 'getpid' and
> >> 'exp' the latency dominates the picture.
> >>
> >> Digression: the callback logic suffers pretty much from the same
> >> issues, albeit in a reversed order - this time it's the Java code
> >> which receives a 'snapshot' of the register values from a generated
> >> assembly adapter; the Java code can then read such values (using the
> >> Pointer API), turn them into Java objects, call the target Java
> >> method and store the results (after another conversion) in the right
> >> location of the snapshot. The assembly adapter will then pick up the
> >> values set onto the snapshot by the Java code, store them into the
> >> corresponding registers and return control to the native callee. In
> >> the remainder of this email we will not discuss callbacks in details
> >> - we will just posit that for any optimization technique that can be
> >> defined, there exists a _dual_ strategy that works with callbacks.
> >>
> >> How can we make sensible native calls go faster? Well, one obvious
> >> way would be to optimize the universal adapter so that we get a
> >> specialized assembly stub for each code shape. If we do that, we can
> >> move pretty much all of the computation described above from
> >> execution time to the stub generation time, so that, by the time we
> >> have to call the native function, we just have to populate the right
> >> registers (the specialized stub knows where to find them) and jump.
> >> While this sounds a good approach, it feels like there's also a move
> >> for the JIT somewhere in there - after all, the JVM knows which calls
> >> are hot and in need for optimization, so perhaps this specialization
> >> process (some or all of it) could happen dynamically. And this is
> >> indeed an approach we'd like to aim for in the long run.
> >>
> >> Now, few years ago, Vlad put together a patch which now lives in the
> >> 'linkToNative' branch [6, 7] - the goal of this patch is to implement
> >> the approach described above: generate a specialized assembly adapter
> >> for a given native signature, and then leverage the JIT to optimize
> >> it away, turning the adapter into a bare, direct, native method call.
> >> As you can see from the third batch of benchmarks, if we tweak Panama
> >> to use the linkToNative machinery, the speed up is really impressive,
> >> and we end up being much faster than JNI (up to 4x for getPid).
> >>
> >> Unfortunately, the technology in the linkToNative branch is not ready
> >> from prime time (yet) - first, it doesn't cover some useful cases
> >> (e.g. varargs, multiple returns via registers, arguments passed in
> >> memory). That is, the logic assumes there's a 1-1 mapping between a
> >> Java signature and the native function to be called - and that the
> >> arguments passed from Java will either be longs or doubles. While we
> >> can workaround this limitation and define the necessary marshalling
> >> logic in Java (as I have done to run this benchmark), some of the
> >> limitations (multiple returns, structs passed by value which are too
> >> big) cannot simply be worked around. But that's fine, we can still
> >> have a fast path for those calls which have certain characteristics
> >> and a slow path (through the universal adapter) for all the other calls.
> >>
> >> But there's a second and more serious issue lurking: as you can see
> >> in the benchmark, I was not able to get the qsort benchmark running
> >> when using the linkToNative backend. The reason is that the
> >> linkToNative code is still pretty raw, and it doesn't fully adhere to
> >> the JVM internal conventions - e.g. there are missing thread state
> >> transitions which, in the case of upcalls into Java, create issues
> >> when it comes to garbage collection, as the GC cannot parse the
> >> native stack in the correct way.
> >>
> >> This means that, while there's a clear shining path ahead of us, it
> >> is simply too early to just use the linkToNative backend from Panama.
> >> For this reason, I've been looking into some kind of stopgap solution
> >> - another way of optimizing native calls (and upcalls into Java) that
> >> doesn't require too much VM magic. Now, a crucial observation is
> >> that, in many native calls, there is indeed a 1-1 mapping between
> >> Java arguments and native arguments (and back, for return values).
> >> That is, we can think of calling a native function as a process that
> >> takes a bunch of Java arguments, turn them into native arguments
> >> (either double or longs), calls the native methods and then turns
> >> back the result into Java.
> >>
> >> The mapping between Java arguments and C values is quite simple:
> >>
> >> * primitives: either long or double, depending on whether they
> >> describe an integral value or a floating point one.
> >> * pointers: they convert to a long
> >> * callbacks: they also convert to a long
> >> * structs: they are recursively decomposed into fields and each field
> >> is marshalled separately (assuming the struct is not too big, in
> >> which case is passed in memory)
> >>
> >> So, in principle, we could define a bunch of native entry points in
> >> the VM, one per shape, which take a bunch of long and doubles and
> >> call an underlying function with those arguments. For instance, let's
> >> consider the case of a native function which is modelled in Java as:
> >>
> >> int m(Pointer<Foo>, double)
> >>
> >> To call this native function we have to first turn the Java arguments
> >> into a (long, double) pair. Then we need to call a native adapter
> >> that looks like the following:
> >>
> >> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr,
> >> jlong arg0, jdouble arg1) {
> >>      return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
> >> }
> >>
> >> And this will take care of calling the native function and returning
> >> the value back. This is, admittedly, a very simple solution; of
> >> course there are limitations: we have to define a bunch of
> >> specialized native entry point (and Java entry points, for
> >> callbacks). But here we can play a trick: most of moderns ABI pass
> >> arguments in registers; for instance System V ABI [5] uses up to 6
> >> (!!) integer registers and 7 (!!) MMXr registers for FP values - this
> >> gives us a total of 13 registers available for argument passing.
> >> Which covers quite a lot of cases. Now, if we have a call where _all_
> >> arguments are passed in registers, then the order in which these
> >> arguments are declared in the adapter doesn't matter! That is, since
> >> FP-values will always be passed in different register from integral
> >> values, we can just define entry points which look like these:
> >>
> >> invokeNative_V_DDDDD
> >> invokeNative_V_JDDDD
> >> invokeNative_V_JJDDD
> >> invokeNative_V_JJJDD
> >> invokeNative_V_JJJJD
> >> invokeNative_V_JJJJJ
> >>
> >> That is, for a given arity (5 in this case), we can just put all long
> >> arguments in front, and the double arguments after that. That is, we
> >> don't need to generate all possible permutations of J/D in all
> >> positions - as the adapter will always do the same thing (read: load
> >> from same registers) for all equivalent combinations. This keeps the
> >> number of entry points in check - and it also poses some challenges
> >> to the Java logic in charge of marshalling/unmarshalling, as there's
> >> an extra permutation step involved (although that is not something
> >> super-hard to address).
> >>
> >> You can see the performance numbers associated with this invocation
> >> scheme (which I've dubbed 'direct') in the 4th batch of the benchmark
> >> results. These numbers are on par (and slightly better) with JNI in
> >> all the three cases considered which is, I think, a very positive
> >> result, given that to write these benchmarks I did not have to write
> >> a single line of JNI code. In other words, this optimization gives
> >> you the same speed as JNI, with improved ease of use (**).
> >>
> >> Now, since the 'direct' optimization builds on top of the VM native
> >> call adapters, this approach is significantly more robust than
> >> linkToNative and I have not run into any weird VM crashes when
> >> playing with it. The downside of that, is that, for obvious reasons,
> >> this approach cannot get much faster than JNI - that is, it cannot
> >> get close to the numbers obtained with the linkToNative backend,
> >> which features much deeper optimizations. But I think that, despite
> >> its limitations, it's still a good opportunistic improvement that is
> >> worth pursuing in the short term (while we sort out the linkToNative
> >> story). For this reason, I will soon be submitting a review which
> >> incorporates the changes for the 'direct' invocation schemes.
> >>
> >> Cheers
> >> Maurizio
> >>
> >> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
> >> [2] - https://github.com/jnr/jnr-ffi
> >> [3] - https://github.com/jnr/jffi
> >> [4] - https://sourceware.org/libffi/
> >> [5] -
> >>
> https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
> >>
> >> [6] -
> >> http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
> >> [7] - http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354
> >>
> >> (**) the benchmark also contains a 5th row in which I repeated same
> >> tests, this time using JNR [2]. JNR is built on top of libjffi [3], a
> >> JNI library in turn built on top of the popular libffi [4]. I wanted
> >> to have some numbers about JNR because that's another solution that
> >> allows for better ease to use, taking care of marshalling Java values
> >> into C and back; since the goals of JNR are similar in spirit with
> >> some of the goals of the Panama/foreign work, I thought it would be
> >> worth having a comparison of these approaches. For the records, I
> >> think the JNR numbers are very respectable given that JNR had to do
> >> all the hard work outside of the JDK!
> >>
> >>
> >
>
>

From maurizio.cimadamore at oracle.com  Mon Sep 17 15:18:39 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 17 Sep 2018 16:18:39 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
Message-ID: <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>



On 17/09/18 15:08, Samuel Audet wrote:
> Yes, the blackhole or the random number doesn't make any difference, 
> but not calling gcc with -O3 does. Running the compiler with 
> optimizations on is pretty common, but they are not enabled by default.
A bit better

PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ? 8491668.248 ops/s

But not much of a difference (I did not expected much, as the body of 
the native method is extremely simple).

Maurizio
>
> Samuel
>
>
> 2018?9?17?(?) 21:37 Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com <mailto:maurizio.cimadamore at oracle.com>>:
>
>     Hi Samuel,
>     I was planning to upload the benchmark IDE project in the near
>     future (I
>     need to clean that up a bit, so that it can be opened at ease):
>
>     My getpid example is like this; this is the Java decl:
>
>     public class GetPid {
>
>     ???? static {
>     ???????? System.loadLibrary("getpid");
>     ???? }
>
>     ???? native static long getpid();
>
>     ???? native double exp(double base);
>     }
>
>     This is the JNI code:
>
>     JNIEXPORT jlong JNICALL Java_org_panama_GetPid_getpid
>     ?? (JNIEnv *env, jobject recv) {
>     ??? return getpid();
>     }
>
>     JNIEXPORT jdouble JNICALL Java_org_panama_GetPid_exp
>     ?? (JNIEnv *env, jobject recv, jdouble arg) {
>     ??? return exp(arg);
>     }
>
>     And this is the benchmark:
>
>     class PanamaBenchmark {
>     ???? static GetPid pidlib = new GetPid();
>
>     ???? @Benchmark
>     ???? public long testJNIPid() {
>     ???????? return pidlib.getpid();
>     ???? }
>
>     ???? @Benchmark
>     ???? public double testJNIExp() {
>     ???????? return pidlib.exp(10d);
>     ???? }
>     }
>
>
>     I think this should be rather standard?
>
>     I'm on Ubuntu 16.04.1, and using GCC 5.4.0. The command I use to
>     compile
>     the C lib is this:
>
>     gcc -I<path to jni,h> -l<path to jni lib> -shared -o libgetpid.so
>     -fPIC
>     GetPid.c
>
>     One difference I see between our two examples is the use of
>     BlackHole.
>     In my bench, I'm just returning the call to 'exp' - which should be
>     equivalent, and, actually, preferred, as described here:
>
>     http://hg.openjdk.java.net/code-tools/jmh/file/3769055ad883/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_09_Blackholes.java#l51
>
>     Another minor difference I see is that I pass a constant argument,
>     while
>     you generate a random number on each iteration.
>
>     I tried to cut and paste your benchmark and I got this:
>
>     Benchmark??????????????????? Mode? Cnt???????? Score Error? Units
>     PanamaBenchmark.testMethod? thrpt??? 5? 26362701.827 ?
>     1357012.981? ops/s
>
>
>     Which looks exactly the same as what I've got. So, for whatever
>     reason,
>     my machine seems to be slower than the one you are using. For what is
>     worth, this website [1] seems to confirm the difference. While clock
>     speeds are similar, your machine has more Ghz in Turbo boost mode and
>     it's 3-4 years newer than mine, so I'd expect that to make a
>     difference
>     in terms of internal optimizations etc. Note that I'm able to beat
>     the
>     numbers of my workstation using my laptop which sports a slightly
>     higher
>     frequency and only has 2 cores and 8G of RAM.
>
>     [1] -
>     https://www.cpubenchmark.net/compare/Intel-Xeon-E5-2673-v4-vs-Intel-Xeon-E5-2665/2888vs1439
>
>     Maurizio
>
>
>
>     On 17/09/18 11:00, Samuel Audet wrote:
>     > Thanks for the figures Maurizio! It's finally good to be
>     speaking in
>     > numbers. :)
>     >
>     > However, you're not providing a lot of details about how you
>     actually
>     > ran the experiments. So I've decided to run a JMH benchmark on
>     what we
>     > get by default with JavaCPP and this declaration:
>     >
>     > ??? @Platform(include = "math.h")
>     > ??? public class MyBenchmark {
>     > ??????? static { Loader.load(); }
>     >
>     > ??????? @NoException
>     > ??????? public static native double exp(double x);
>     >
>     > ??????? @State(Scope.Thread)
>     > ??????? public static class MyState {
>     > ??????????? double x;
>     >
>     > ??????????? @Setup(Level.Iteration)
>     > ??????????? public void setupMethod() {
>     > ??????????????? x = Math.random();
>     > ??????????? }
>     > ??????? }
>     >
>     > ??????? @Benchmark
>     > ??????? public void testMethod(MyState s, Blackhole bh) {
>     > ??????????? bh.consume(exp(s.x));
>     > ??????? }
>     > ??? }
>     >
>     > The relevant portion of generated JNI looks like this:
>     >
>     > ??? JNIEXPORT jdouble JNICALL
>     Java_org_sample_MyBenchmark_exp(JNIEnv*
>     > env, jclass cls, jdouble arg0) {
>     > ??????? jdouble rarg = 0;
>     > ??????? double rval = exp(arg0);
>     > ??????? rarg = (jdouble)rval;
>     > ??????? return rarg;
>     > ??? }
>     >
>     > And with access to just 2 virtual cores of an Intel(R) Xeon(R) CPU
>     > E5-2673 v4 @ 2.30GHz and 8 GB of RAM on the cloud (so probably
>     slower
>     > than your E5-2665 @ 2.40GHz) running Ubuntu 14.04 with GCC 4.9 and
>     > OpenJDK 8, I get these numbers:
>     > Benchmark??????????????? Mode? Cnt Score??????? Error Units
>     > MyBenchmark.testMethod? thrpt?? 25? 37183556.094 ? 460795.746 ops/s
>     >
>     > I'm not sure how that compares with your numbers exactly, but it
>     does
>     > seem to me that what you get for JNI is a bit low. If you could
>     > provide more details about how to reproduce your results, that
>     would
>     > be great.
>     >
>     > Samuel
>     >
>     >
>     > On 09/14/2018 10:19 PM, Maurizio Cimadamore wrote:
>     >> Hi,
>     >> over the last few weeks I've been busy playing with Panama and
>     >> assessing performances with JMH. For those just interested in raw
>     >> numbers, the results of my explorations can be found here [1].
>     But as
>     >> all benchmarks, I think it's better to spend few words to
>     understand
>     >> what these numbers actually _mean_.
>     >>
>     >> To evaluate the performances of Panama I have first created a
>     >> baseline using JNI - more specifically I wanted to assess
>     >> performances of three calls (all part of the C std library),
>     namely
>     >> `getpid`, `exp` and `qsort`.
>     >>
>     >> The first example is the de facto benchmark for FFIs - since it
>     does
>     >> relatively little computation, it is a good test to measure the
>     >> 'latency' of the FFI approach (e.g. how long does it take to go to
>     >> native). The second example is also relatively simple, but the
>     this
>     >> time the function takes a double argument. The third test is
>     akin to
>     >> an FFI torture test, since not only it passes substantially more
>     >> arguments (4) but one of these arguments is also a callback - a
>     >> pointer to a function that is used to sort the contents of the
>     input
>     >> array.
>     >>
>     >> As expected, the first batch of JNI results confirms our
>     >> expectations: `getpid` is the fastest, followed by `exp`, and then
>     >> followed by `qsort`. Note that qsort is not even close in terms of
>     >> raw numbers to the other two tests - that's because, to sort the
>     >> array we need to do (N * log N) upcalls into Java. In the
>     benchmark,
>     >> N = 8 and we do the upcalls using the JNI function
>     >> JNIEnv::CallIntMethod.
>     >>
>     >> Now let's examine the second batch of results; these call
>     `getpid`,
>     >> `exp` and `qsort` using Panama. The numbers here are considerably
>     >> lower than the JNI ones for all the three benchmark - although the
>     >> first two seem to be the most problematic. To explain these
>     results
>     >> we need to peek under the hood. Panama implements foreign calls
>     >> through a so called 'universal adapter' which, given a calling
>     scheme
>     >> and a bunch of arguments (machine words) shuffles these
>     arguments in
>     >> the right registers/stack slot and then jumps to the target native
>     >> function - after which another round of adaptation must be
>     performed
>     >> (e.g. to recover the return value from the right register/memory
>     >> location).
>     >>
>     >> Needless to say, all this generality comes at a cost - some of the
>     >> cost is in Java - e.g. all arguments have to be packaged up into a
>     >> long array (although this component doesn't seem to show up
>     much in
>     >> the generated JVM compiled code). A lot of the cost is in the
>     adapter
>     >> logic itself - which has to look at the 'call recipe' and move
>     >> arguments around accordingly - more specifically, in order to call
>     >> the native function, the adapter creates a bunch of helper C++
>     >> objects and structs which model the CPU state (e.g. in the
>     >> ShuffleDowncallContext struct, we find a field for each
>     register to
>     >> be modeled in the target architecture). The adapter has to
>     first move
>     >> the values coming from the Java world (stored in the
>     aforementioned
>     >> long array) into the right context fields (and it needs to do
>     so by
>     >> looking at the recipe, which involves iteration over the recipe
>     >> elements). After that's done, we can jump into the assembly
>     stub that
>     >> does the native call - this stub will take as input one of those
>     >> ShuffleDowncallContext structure and will load the corresponding
>     >> registers/create necessary stack slots ahead of the call.
>     >>
>     >> As you can see, there's quite a lot of action going on here,
>     and this
>     >> explains the benchmark numbers; of course, if you are calling a
>     >> native function that does a lot of computation, this adaptation
>     cost
>     >> will wash out - but for relatively quick calls such as 'getpid'
>     and
>     >> 'exp' the latency dominates the picture.
>     >>
>     >> Digression: the callback logic suffers pretty much from the same
>     >> issues, albeit in a reversed order - this time it's the Java code
>     >> which receives a 'snapshot' of the register values from a
>     generated
>     >> assembly adapter; the Java code can then read such values
>     (using the
>     >> Pointer API), turn them into Java objects, call the target Java
>     >> method and store the results (after another conversion) in the
>     right
>     >> location of the snapshot. The assembly adapter will then pick
>     up the
>     >> values set onto the snapshot by the Java code, store them into the
>     >> corresponding registers and return control to the native
>     callee. In
>     >> the remainder of this email we will not discuss callbacks in
>     details
>     >> - we will just posit that for any optimization technique that
>     can be
>     >> defined, there exists a _dual_ strategy that works with callbacks.
>     >>
>     >> How can we make sensible native calls go faster? Well, one obvious
>     >> way would be to optimize the universal adapter so that we get a
>     >> specialized assembly stub for each code shape. If we do that,
>     we can
>     >> move pretty much all of the computation described above from
>     >> execution time to the stub generation time, so that, by the
>     time we
>     >> have to call the native function, we just have to populate the
>     right
>     >> registers (the specialized stub knows where to find them) and
>     jump.
>     >> While this sounds a good approach, it feels like there's also a
>     move
>     >> for the JIT somewhere in there - after all, the JVM knows which
>     calls
>     >> are hot and in need for optimization, so perhaps this
>     specialization
>     >> process (some or all of it) could happen dynamically. And this is
>     >> indeed an approach we'd like to aim for in the long run.
>     >>
>     >> Now, few years ago, Vlad put together a patch which now lives
>     in the
>     >> 'linkToNative' branch [6, 7] - the goal of this patch is to
>     implement
>     >> the approach described above: generate a specialized assembly
>     adapter
>     >> for a given native signature, and then leverage the JIT to
>     optimize
>     >> it away, turning the adapter into a bare, direct, native method
>     call.
>     >> As you can see from the third batch of benchmarks, if we tweak
>     Panama
>     >> to use the linkToNative machinery, the speed up is really
>     impressive,
>     >> and we end up being much faster than JNI (up to 4x for getPid).
>     >>
>     >> Unfortunately, the technology in the linkToNative branch is not
>     ready
>     >> from prime time (yet) - first, it doesn't cover some useful cases
>     >> (e.g. varargs, multiple returns via registers, arguments passed in
>     >> memory). That is, the logic assumes there's a 1-1 mapping
>     between a
>     >> Java signature and the native function to be called - and that the
>     >> arguments passed from Java will either be longs or doubles.
>     While we
>     >> can workaround this limitation and define the necessary
>     marshalling
>     >> logic in Java (as I have done to run this benchmark), some of the
>     >> limitations (multiple returns, structs passed by value which
>     are too
>     >> big) cannot simply be worked around. But that's fine, we can still
>     >> have a fast path for those calls which have certain
>     characteristics
>     >> and a slow path (through the universal adapter) for all the
>     other calls.
>     >>
>     >> But there's a second and more serious issue lurking: as you can
>     see
>     >> in the benchmark, I was not able to get the qsort benchmark
>     running
>     >> when using the linkToNative backend. The reason is that the
>     >> linkToNative code is still pretty raw, and it doesn't fully
>     adhere to
>     >> the JVM internal conventions - e.g. there are missing thread state
>     >> transitions which, in the case of upcalls into Java, create issues
>     >> when it comes to garbage collection, as the GC cannot parse the
>     >> native stack in the correct way.
>     >>
>     >> This means that, while there's a clear shining path ahead of
>     us, it
>     >> is simply too early to just use the linkToNative backend from
>     Panama.
>     >> For this reason, I've been looking into some kind of stopgap
>     solution
>     >> - another way of optimizing native calls (and upcalls into
>     Java) that
>     >> doesn't require too much VM magic. Now, a crucial observation is
>     >> that, in many native calls, there is indeed a 1-1 mapping between
>     >> Java arguments and native arguments (and back, for return values).
>     >> That is, we can think of calling a native function as a process
>     that
>     >> takes a bunch of Java arguments, turn them into native arguments
>     >> (either double or longs), calls the native methods and then turns
>     >> back the result into Java.
>     >>
>     >> The mapping between Java arguments and C values is quite simple:
>     >>
>     >> * primitives: either long or double, depending on whether they
>     >> describe an integral value or a floating point one.
>     >> * pointers: they convert to a long
>     >> * callbacks: they also convert to a long
>     >> * structs: they are recursively decomposed into fields and each
>     field
>     >> is marshalled separately (assuming the struct is not too big, in
>     >> which case is passed in memory)
>     >>
>     >> So, in principle, we could define a bunch of native entry
>     points in
>     >> the VM, one per shape, which take a bunch of long and doubles and
>     >> call an underlying function with those arguments. For instance,
>     let's
>     >> consider the case of a native function which is modelled in
>     Java as:
>     >>
>     >> int m(Pointer<Foo>, double)
>     >>
>     >> To call this native function we have to first turn the Java
>     arguments
>     >> into a (long, double) pair. Then we need to call a native adapter
>     >> that looks like the following:
>     >>
>     >> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong
>     addr,
>     >> jlong arg0, jdouble arg1) {
>     >> ???? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>     >> }
>     >>
>     >> And this will take care of calling the native function and
>     returning
>     >> the value back. This is, admittedly, a very simple solution; of
>     >> course there are limitations: we have to define a bunch of
>     >> specialized native entry point (and Java entry points, for
>     >> callbacks). But here we can play a trick: most of moderns ABI pass
>     >> arguments in registers; for instance System V ABI [5] uses up to 6
>     >> (!!) integer registers and 7 (!!) MMXr registers for FP values
>     - this
>     >> gives us a total of 13 registers available for argument passing.
>     >> Which covers quite a lot of cases. Now, if we have a call where
>     _all_
>     >> arguments are passed in registers, then the order in which these
>     >> arguments are declared in the adapter doesn't matter! That is,
>     since
>     >> FP-values will always be passed in different register from
>     integral
>     >> values, we can just define entry points which look like these:
>     >>
>     >> invokeNative_V_DDDDD
>     >> invokeNative_V_JDDDD
>     >> invokeNative_V_JJDDD
>     >> invokeNative_V_JJJDD
>     >> invokeNative_V_JJJJD
>     >> invokeNative_V_JJJJJ
>     >>
>     >> That is, for a given arity (5 in this case), we can just put
>     all long
>     >> arguments in front, and the double arguments after that. That
>     is, we
>     >> don't need to generate all possible permutations of J/D in all
>     >> positions - as the adapter will always do the same thing (read:
>     load
>     >> from same registers) for all equivalent combinations. This
>     keeps the
>     >> number of entry points in check - and it also poses some
>     challenges
>     >> to the Java logic in charge of marshalling/unmarshalling, as
>     there's
>     >> an extra permutation step involved (although that is not something
>     >> super-hard to address).
>     >>
>     >> You can see the performance numbers associated with this
>     invocation
>     >> scheme (which I've dubbed 'direct') in the 4th batch of the
>     benchmark
>     >> results. These numbers are on par (and slightly better) with
>     JNI in
>     >> all the three cases considered which is, I think, a very positive
>     >> result, given that to write these benchmarks I did not have to
>     write
>     >> a single line of JNI code. In other words, this optimization gives
>     >> you the same speed as JNI, with improved ease of use (**).
>     >>
>     >> Now, since the 'direct' optimization builds on top of the VM
>     native
>     >> call adapters, this approach is significantly more robust than
>     >> linkToNative and I have not run into any weird VM crashes when
>     >> playing with it. The downside of that, is that, for obvious
>     reasons,
>     >> this approach cannot get much faster than JNI - that is, it cannot
>     >> get close to the numbers obtained with the linkToNative backend,
>     >> which features much deeper optimizations. But I think that,
>     despite
>     >> its limitations, it's still a good opportunistic improvement
>     that is
>     >> worth pursuing in the short term (while we sort out the
>     linkToNative
>     >> story). For this reason, I will soon be submitting a review which
>     >> incorporates the changes for the 'direct' invocation schemes.
>     >>
>     >> Cheers
>     >> Maurizio
>     >>
>     >> [1] -
>     http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>     <http://cr.openjdk.java.net/%7Emcimadamore/panama/foreign-jmh.txt>
>     >> [2] - https://github.com/jnr/jnr-ffi
>     >> [3] - https://github.com/jnr/jffi
>     >> [4] - https://sourceware.org/libffi/
>     >> [5] -
>     >>
>     https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
>
>     >>
>     >> [6] -
>     >>
>     http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
>     <http://cr.openjdk.java.net/%7Ejrose/panama/native-call-primitive.html>
>     >> [7] - http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354
>     >>
>     >> (**) the benchmark also contains a 5th row in which I repeated
>     same
>     >> tests, this time using JNR [2]. JNR is built on top of libjffi
>     [3], a
>     >> JNI library in turn built on top of the popular libffi [4]. I
>     wanted
>     >> to have some numbers about JNR because that's another solution
>     that
>     >> allows for better ease to use, taking care of marshalling Java
>     values
>     >> into C and back; since the goals of JNR are similar in spirit with
>     >> some of the goals of the Panama/foreign work, I thought it
>     would be
>     >> worth having a comparison of these approaches. For the records, I
>     >> think the JNR numbers are very respectable given that JNR had
>     to do
>     >> all the hard work outside of the JDK!
>     >>
>     >>
>     >
>


From maurizio.cimadamore at oracle.com  Mon Sep 17 15:58:59 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 17 Sep 2018 16:58:59 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
Message-ID: <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>

For the records, here's what I get for all the three benchmarks if I 
compile the JNI code with -O3:

Benchmark????????????????????????? Mode? Cnt Score???????? Error? Units
PanamaBenchmark.testJNIExp??????? thrpt??? 5? 28575269.294 ? 
1907726.710? ops/s
PanamaBenchmark.testJNIJavaQsort? thrpt??? 5??? 372148.433 ? 27178.529? 
ops/s
PanamaBenchmark.testJNIPid??????? thrpt??? 5? 59240069.011 ? 403881.697? 
ops/s

The first and second benchmarks get faster and very close to the 
'direct' optimization numbers in [1]. Surprisingly, the last benchmark 
(getpid) is quite slower. I've been able to reproduce across multiple 
runs; for that benchmark omitting O3 seems to be the achieve best 
results, not sure why. It starts of faster (around in the first couple 
of warmup iterations, but then it goes slower in all the other runs - 
presumably it interacts badly with the C2 generated code. For instance, 
this is a run with O3 enabled:

# Run progress: 66.67% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration?? 1: 65182202.653 ops/s
# Warmup Iteration?? 2: 64900639.094 ops/s
# Warmup Iteration?? 3: 59314945.437 ops/s 
<---------------------------------
# Warmup Iteration?? 4: 59269007.877 ops/s
# Warmup Iteration?? 5: 59239905.163 ops/s
Iteration?? 1: 59300748.074 ops/s
Iteration?? 2: 59249666.044 ops/s
Iteration?? 3: 59268597.051 ops/s
Iteration?? 4: 59322074.572 ops/s
Iteration?? 5: 59059259.317 ops/s

And this is a run with O3 disabled:

# Run progress: 0.00% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration?? 1: 55882128.787 ops/s
# Warmup Iteration?? 2: 53102361.751 ops/s
# Warmup Iteration?? 3: 66964755.699 ops/s 
<---------------------------------
# Warmup Iteration?? 4: 66414428.355 ops/s
# Warmup Iteration?? 5: 65328475.276 ops/s
Iteration?? 1: 64229192.993 ops/s
Iteration?? 2: 65191719.319 ops/s
Iteration?? 3: 65352022.471 ops/s
Iteration?? 4: 65152090.426 ops/s
Iteration?? 5: 65320545.712 ops/s


In both cases, the 3rd warmup execution sees a performance jump - with 
O3, the jump is backwards, w/o O3 the jump is forward, which is quite 
typical for a JMH benchmark as C2 optimization will start to kick in.

For these reasons, I'm reluctant to update my benchmark numbers to 
reflect the O3 behavior (although I agree that, since the Hotspot code 
is compiled with that optimization it would make more sense to use that 
as a reference).

Maurizio

[1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt



On 17/09/18 16:18, Maurizio Cimadamore wrote:
>
>
> On 17/09/18 15:08, Samuel Audet wrote:
>> Yes, the blackhole or the random number doesn't make any difference, 
>> but not calling gcc with -O3 does. Running the compiler with 
>> optimizations on is pretty common, but they are not enabled by default.
> A bit better
>
> PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ? 8491668.248 ops/s
>
> But not much of a difference (I did not expected much, as the body of 
> the native method is extremely simple).
>
> Maurizio
>>
>> Samuel
>>
>>
>> 2018?9?17?(?) 21:37 Maurizio Cimadamore 
>> <maurizio.cimadamore at oracle.com 
>> <mailto:maurizio.cimadamore at oracle.com>>:
>>
>> ??? Hi Samuel,
>> ??? I was planning to upload the benchmark IDE project in the near
>> ??? future (I
>> ??? need to clean that up a bit, so that it can be opened at ease):
>>
>> ??? My getpid example is like this; this is the Java decl:
>>
>> ??? public class GetPid {
>>
>> ??? ???? static {
>> ??? ???????? System.loadLibrary("getpid");
>> ??? ???? }
>>
>> ??? ???? native static long getpid();
>>
>> ??? ???? native double exp(double base);
>> ??? }
>>
>> ??? This is the JNI code:
>>
>> ??? JNIEXPORT jlong JNICALL Java_org_panama_GetPid_getpid
>> ??? ?? (JNIEnv *env, jobject recv) {
>> ??? ??? return getpid();
>> ??? }
>>
>> ??? JNIEXPORT jdouble JNICALL Java_org_panama_GetPid_exp
>> ??? ?? (JNIEnv *env, jobject recv, jdouble arg) {
>> ??? ??? return exp(arg);
>> ??? }
>>
>> ??? And this is the benchmark:
>>
>> ??? class PanamaBenchmark {
>> ??? ???? static GetPid pidlib = new GetPid();
>>
>> ??? ???? @Benchmark
>> ??? ???? public long testJNIPid() {
>> ??? ???????? return pidlib.getpid();
>> ??? ???? }
>>
>> ??? ???? @Benchmark
>> ??? ???? public double testJNIExp() {
>> ??? ???????? return pidlib.exp(10d);
>> ??? ???? }
>> ??? }
>>
>>
>> ??? I think this should be rather standard?
>>
>> ??? I'm on Ubuntu 16.04.1, and using GCC 5.4.0. The command I use to
>> ??? compile
>> ??? the C lib is this:
>>
>> ??? gcc -I<path to jni,h> -l<path to jni lib> -shared -o libgetpid.so
>> ??? -fPIC
>> ??? GetPid.c
>>
>> ??? One difference I see between our two examples is the use of
>> ??? BlackHole.
>> ??? In my bench, I'm just returning the call to 'exp' - which should be
>> ??? equivalent, and, actually, preferred, as described here:
>>
>> http://hg.openjdk.java.net/code-tools/jmh/file/3769055ad883/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_09_Blackholes.java#l51
>>
>> ??? Another minor difference I see is that I pass a constant argument,
>> ??? while
>> ??? you generate a random number on each iteration.
>>
>> ??? I tried to cut and paste your benchmark and I got this:
>>
>> ??? Benchmark??????????????????? Mode? Cnt???????? Score Error Units
>> ??? PanamaBenchmark.testMethod? thrpt??? 5? 26362701.827 ?
>> ??? 1357012.981? ops/s
>>
>>
>> ??? Which looks exactly the same as what I've got. So, for whatever
>> ??? reason,
>> ??? my machine seems to be slower than the one you are using. For 
>> what is
>> ??? worth, this website [1] seems to confirm the difference. While clock
>> ??? speeds are similar, your machine has more Ghz in Turbo boost mode 
>> and
>> ??? it's 3-4 years newer than mine, so I'd expect that to make a
>> ??? difference
>> ??? in terms of internal optimizations etc. Note that I'm able to beat
>> ??? the
>> ??? numbers of my workstation using my laptop which sports a slightly
>> ??? higher
>> ??? frequency and only has 2 cores and 8G of RAM.
>>
>> ??? [1] -
>> https://www.cpubenchmark.net/compare/Intel-Xeon-E5-2673-v4-vs-Intel-Xeon-E5-2665/2888vs1439
>>
>> ??? Maurizio
>>
>>
>>
>> ??? On 17/09/18 11:00, Samuel Audet wrote:
>> ??? > Thanks for the figures Maurizio! It's finally good to be
>> ??? speaking in
>> ??? > numbers. :)
>> ??? >
>> ??? > However, you're not providing a lot of details about how you
>> ??? actually
>> ??? > ran the experiments. So I've decided to run a JMH benchmark on
>> ??? what we
>> ??? > get by default with JavaCPP and this declaration:
>> ??? >
>> ??? > ??? @Platform(include = "math.h")
>> ??? > ??? public class MyBenchmark {
>> ??? > ??????? static { Loader.load(); }
>> ??? >
>> ??? > ??????? @NoException
>> ??? > ??????? public static native double exp(double x);
>> ??? >
>> ??? > ??????? @State(Scope.Thread)
>> ??? > ??????? public static class MyState {
>> ??? > ??????????? double x;
>> ??? >
>> ??? > ??????????? @Setup(Level.Iteration)
>> ??? > ??????????? public void setupMethod() {
>> ??? > ??????????????? x = Math.random();
>> ??? > ??????????? }
>> ??? > ??????? }
>> ??? >
>> ??? > ??????? @Benchmark
>> ??? > ??????? public void testMethod(MyState s, Blackhole bh) {
>> ??? > ??????????? bh.consume(exp(s.x));
>> ??? > ??????? }
>> ??? > ??? }
>> ??? >
>> ??? > The relevant portion of generated JNI looks like this:
>> ??? >
>> ??? > ??? JNIEXPORT jdouble JNICALL
>> ??? Java_org_sample_MyBenchmark_exp(JNIEnv*
>> ??? > env, jclass cls, jdouble arg0) {
>> ??? > ??????? jdouble rarg = 0;
>> ??? > ??????? double rval = exp(arg0);
>> ??? > ??????? rarg = (jdouble)rval;
>> ??? > ??????? return rarg;
>> ??? > ??? }
>> ??? >
>> ??? > And with access to just 2 virtual cores of an Intel(R) Xeon(R) CPU
>> ??? > E5-2673 v4 @ 2.30GHz and 8 GB of RAM on the cloud (so probably
>> ??? slower
>> ??? > than your E5-2665 @ 2.40GHz) running Ubuntu 14.04 with GCC 4.9 and
>> ??? > OpenJDK 8, I get these numbers:
>> ??? > Benchmark??????????????? Mode? Cnt Score??????? Error Units
>> ??? > MyBenchmark.testMethod? thrpt?? 25? 37183556.094 ? 460795.746 
>> ops/s
>> ??? >
>> ??? > I'm not sure how that compares with your numbers exactly, but it
>> ??? does
>> ??? > seem to me that what you get for JNI is a bit low. If you could
>> ??? > provide more details about how to reproduce your results, that
>> ??? would
>> ??? > be great.
>> ??? >
>> ??? > Samuel
>> ??? >
>> ??? >
>> ??? > On 09/14/2018 10:19 PM, Maurizio Cimadamore wrote:
>> ??? >> Hi,
>> ??? >> over the last few weeks I've been busy playing with Panama and
>> ??? >> assessing performances with JMH. For those just interested in raw
>> ??? >> numbers, the results of my explorations can be found here [1].
>> ??? But as
>> ??? >> all benchmarks, I think it's better to spend few words to
>> ??? understand
>> ??? >> what these numbers actually _mean_.
>> ??? >>
>> ??? >> To evaluate the performances of Panama I have first created a
>> ??? >> baseline using JNI - more specifically I wanted to assess
>> ??? >> performances of three calls (all part of the C std library),
>> ??? namely
>> ??? >> `getpid`, `exp` and `qsort`.
>> ??? >>
>> ??? >> The first example is the de facto benchmark for FFIs - since it
>> ??? does
>> ??? >> relatively little computation, it is a good test to measure the
>> ??? >> 'latency' of the FFI approach (e.g. how long does it take to 
>> go to
>> ??? >> native). The second example is also relatively simple, but the
>> ??? this
>> ??? >> time the function takes a double argument. The third test is
>> ??? akin to
>> ??? >> an FFI torture test, since not only it passes substantially more
>> ??? >> arguments (4) but one of these arguments is also a callback - a
>> ??? >> pointer to a function that is used to sort the contents of the
>> ??? input
>> ??? >> array.
>> ??? >>
>> ??? >> As expected, the first batch of JNI results confirms our
>> ??? >> expectations: `getpid` is the fastest, followed by `exp`, and 
>> then
>> ??? >> followed by `qsort`. Note that qsort is not even close in 
>> terms of
>> ??? >> raw numbers to the other two tests - that's because, to sort the
>> ??? >> array we need to do (N * log N) upcalls into Java. In the
>> ??? benchmark,
>> ??? >> N = 8 and we do the upcalls using the JNI function
>> ??? >> JNIEnv::CallIntMethod.
>> ??? >>
>> ??? >> Now let's examine the second batch of results; these call
>> ??? `getpid`,
>> ??? >> `exp` and `qsort` using Panama. The numbers here are considerably
>> ??? >> lower than the JNI ones for all the three benchmark - although 
>> the
>> ??? >> first two seem to be the most problematic. To explain these
>> ??? results
>> ??? >> we need to peek under the hood. Panama implements foreign calls
>> ??? >> through a so called 'universal adapter' which, given a calling
>> ??? scheme
>> ??? >> and a bunch of arguments (machine words) shuffles these
>> ??? arguments in
>> ??? >> the right registers/stack slot and then jumps to the target 
>> native
>> ??? >> function - after which another round of adaptation must be
>> ??? performed
>> ??? >> (e.g. to recover the return value from the right register/memory
>> ??? >> location).
>> ??? >>
>> ??? >> Needless to say, all this generality comes at a cost - some of 
>> the
>> ??? >> cost is in Java - e.g. all arguments have to be packaged up 
>> into a
>> ??? >> long array (although this component doesn't seem to show up
>> ??? much in
>> ??? >> the generated JVM compiled code). A lot of the cost is in the
>> ??? adapter
>> ??? >> logic itself - which has to look at the 'call recipe' and move
>> ??? >> arguments around accordingly - more specifically, in order to 
>> call
>> ??? >> the native function, the adapter creates a bunch of helper C++
>> ??? >> objects and structs which model the CPU state (e.g. in the
>> ??? >> ShuffleDowncallContext struct, we find a field for each
>> ??? register to
>> ??? >> be modeled in the target architecture). The adapter has to
>> ??? first move
>> ??? >> the values coming from the Java world (stored in the
>> ??? aforementioned
>> ??? >> long array) into the right context fields (and it needs to do
>> ??? so by
>> ??? >> looking at the recipe, which involves iteration over the recipe
>> ??? >> elements). After that's done, we can jump into the assembly
>> ??? stub that
>> ??? >> does the native call - this stub will take as input one of those
>> ??? >> ShuffleDowncallContext structure and will load the corresponding
>> ??? >> registers/create necessary stack slots ahead of the call.
>> ??? >>
>> ??? >> As you can see, there's quite a lot of action going on here,
>> ??? and this
>> ??? >> explains the benchmark numbers; of course, if you are calling a
>> ??? >> native function that does a lot of computation, this adaptation
>> ??? cost
>> ??? >> will wash out - but for relatively quick calls such as 'getpid'
>> ??? and
>> ??? >> 'exp' the latency dominates the picture.
>> ??? >>
>> ??? >> Digression: the callback logic suffers pretty much from the same
>> ??? >> issues, albeit in a reversed order - this time it's the Java code
>> ??? >> which receives a 'snapshot' of the register values from a
>> ??? generated
>> ??? >> assembly adapter; the Java code can then read such values
>> ??? (using the
>> ??? >> Pointer API), turn them into Java objects, call the target Java
>> ??? >> method and store the results (after another conversion) in the
>> ??? right
>> ??? >> location of the snapshot. The assembly adapter will then pick
>> ??? up the
>> ??? >> values set onto the snapshot by the Java code, store them into 
>> the
>> ??? >> corresponding registers and return control to the native
>> ??? callee. In
>> ??? >> the remainder of this email we will not discuss callbacks in
>> ??? details
>> ??? >> - we will just posit that for any optimization technique that
>> ??? can be
>> ??? >> defined, there exists a _dual_ strategy that works with 
>> callbacks.
>> ??? >>
>> ??? >> How can we make sensible native calls go faster? Well, one 
>> obvious
>> ??? >> way would be to optimize the universal adapter so that we get a
>> ??? >> specialized assembly stub for each code shape. If we do that,
>> ??? we can
>> ??? >> move pretty much all of the computation described above from
>> ??? >> execution time to the stub generation time, so that, by the
>> ??? time we
>> ??? >> have to call the native function, we just have to populate the
>> ??? right
>> ??? >> registers (the specialized stub knows where to find them) and
>> ??? jump.
>> ??? >> While this sounds a good approach, it feels like there's also a
>> ??? move
>> ??? >> for the JIT somewhere in there - after all, the JVM knows which
>> ??? calls
>> ??? >> are hot and in need for optimization, so perhaps this
>> ??? specialization
>> ??? >> process (some or all of it) could happen dynamically. And this is
>> ??? >> indeed an approach we'd like to aim for in the long run.
>> ??? >>
>> ??? >> Now, few years ago, Vlad put together a patch which now lives
>> ??? in the
>> ??? >> 'linkToNative' branch [6, 7] - the goal of this patch is to
>> ??? implement
>> ??? >> the approach described above: generate a specialized assembly
>> ??? adapter
>> ??? >> for a given native signature, and then leverage the JIT to
>> ??? optimize
>> ??? >> it away, turning the adapter into a bare, direct, native method
>> ??? call.
>> ??? >> As you can see from the third batch of benchmarks, if we tweak
>> ??? Panama
>> ??? >> to use the linkToNative machinery, the speed up is really
>> ??? impressive,
>> ??? >> and we end up being much faster than JNI (up to 4x for getPid).
>> ??? >>
>> ??? >> Unfortunately, the technology in the linkToNative branch is not
>> ??? ready
>> ??? >> from prime time (yet) - first, it doesn't cover some useful cases
>> ??? >> (e.g. varargs, multiple returns via registers, arguments 
>> passed in
>> ??? >> memory). That is, the logic assumes there's a 1-1 mapping
>> ??? between a
>> ??? >> Java signature and the native function to be called - and that 
>> the
>> ??? >> arguments passed from Java will either be longs or doubles.
>> ??? While we
>> ??? >> can workaround this limitation and define the necessary
>> ??? marshalling
>> ??? >> logic in Java (as I have done to run this benchmark), some of the
>> ??? >> limitations (multiple returns, structs passed by value which
>> ??? are too
>> ??? >> big) cannot simply be worked around. But that's fine, we can 
>> still
>> ??? >> have a fast path for those calls which have certain
>> ??? characteristics
>> ??? >> and a slow path (through the universal adapter) for all the
>> ??? other calls.
>> ??? >>
>> ??? >> But there's a second and more serious issue lurking: as you can
>> ??? see
>> ??? >> in the benchmark, I was not able to get the qsort benchmark
>> ??? running
>> ??? >> when using the linkToNative backend. The reason is that the
>> ??? >> linkToNative code is still pretty raw, and it doesn't fully
>> ??? adhere to
>> ??? >> the JVM internal conventions - e.g. there are missing thread 
>> state
>> ??? >> transitions which, in the case of upcalls into Java, create 
>> issues
>> ??? >> when it comes to garbage collection, as the GC cannot parse the
>> ??? >> native stack in the correct way.
>> ??? >>
>> ??? >> This means that, while there's a clear shining path ahead of
>> ??? us, it
>> ??? >> is simply too early to just use the linkToNative backend from
>> ??? Panama.
>> ??? >> For this reason, I've been looking into some kind of stopgap
>> ??? solution
>> ??? >> - another way of optimizing native calls (and upcalls into
>> ??? Java) that
>> ??? >> doesn't require too much VM magic. Now, a crucial observation is
>> ??? >> that, in many native calls, there is indeed a 1-1 mapping between
>> ??? >> Java arguments and native arguments (and back, for return 
>> values).
>> ??? >> That is, we can think of calling a native function as a process
>> ??? that
>> ??? >> takes a bunch of Java arguments, turn them into native arguments
>> ??? >> (either double or longs), calls the native methods and then turns
>> ??? >> back the result into Java.
>> ??? >>
>> ??? >> The mapping between Java arguments and C values is quite simple:
>> ??? >>
>> ??? >> * primitives: either long or double, depending on whether they
>> ??? >> describe an integral value or a floating point one.
>> ??? >> * pointers: they convert to a long
>> ??? >> * callbacks: they also convert to a long
>> ??? >> * structs: they are recursively decomposed into fields and each
>> ??? field
>> ??? >> is marshalled separately (assuming the struct is not too big, in
>> ??? >> which case is passed in memory)
>> ??? >>
>> ??? >> So, in principle, we could define a bunch of native entry
>> ??? points in
>> ??? >> the VM, one per shape, which take a bunch of long and doubles and
>> ??? >> call an underlying function with those arguments. For instance,
>> ??? let's
>> ??? >> consider the case of a native function which is modelled in
>> ??? Java as:
>> ??? >>
>> ??? >> int m(Pointer<Foo>, double)
>> ??? >>
>> ??? >> To call this native function we have to first turn the Java
>> ??? arguments
>> ??? >> into a (long, double) pair. Then we need to call a native adapter
>> ??? >> that looks like the following:
>> ??? >>
>> ??? >> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong
>> ??? addr,
>> ??? >> jlong arg0, jdouble arg1) {
>> ??? >> ???? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>> ??? >> }
>> ??? >>
>> ??? >> And this will take care of calling the native function and
>> ??? returning
>> ??? >> the value back. This is, admittedly, a very simple solution; of
>> ??? >> course there are limitations: we have to define a bunch of
>> ??? >> specialized native entry point (and Java entry points, for
>> ??? >> callbacks). But here we can play a trick: most of moderns ABI 
>> pass
>> ??? >> arguments in registers; for instance System V ABI [5] uses up 
>> to 6
>> ??? >> (!!) integer registers and 7 (!!) MMXr registers for FP values
>> ??? - this
>> ??? >> gives us a total of 13 registers available for argument passing.
>> ??? >> Which covers quite a lot of cases. Now, if we have a call where
>> ??? _all_
>> ??? >> arguments are passed in registers, then the order in which these
>> ??? >> arguments are declared in the adapter doesn't matter! That is,
>> ??? since
>> ??? >> FP-values will always be passed in different register from
>> ??? integral
>> ??? >> values, we can just define entry points which look like these:
>> ??? >>
>> ??? >> invokeNative_V_DDDDD
>> ??? >> invokeNative_V_JDDDD
>> ??? >> invokeNative_V_JJDDD
>> ??? >> invokeNative_V_JJJDD
>> ??? >> invokeNative_V_JJJJD
>> ??? >> invokeNative_V_JJJJJ
>> ??? >>
>> ??? >> That is, for a given arity (5 in this case), we can just put
>> ??? all long
>> ??? >> arguments in front, and the double arguments after that. That
>> ??? is, we
>> ??? >> don't need to generate all possible permutations of J/D in all
>> ??? >> positions - as the adapter will always do the same thing (read:
>> ??? load
>> ??? >> from same registers) for all equivalent combinations. This
>> ??? keeps the
>> ??? >> number of entry points in check - and it also poses some
>> ??? challenges
>> ??? >> to the Java logic in charge of marshalling/unmarshalling, as
>> ??? there's
>> ??? >> an extra permutation step involved (although that is not 
>> something
>> ??? >> super-hard to address).
>> ??? >>
>> ??? >> You can see the performance numbers associated with this
>> ??? invocation
>> ??? >> scheme (which I've dubbed 'direct') in the 4th batch of the
>> ??? benchmark
>> ??? >> results. These numbers are on par (and slightly better) with
>> ??? JNI in
>> ??? >> all the three cases considered which is, I think, a very positive
>> ??? >> result, given that to write these benchmarks I did not have to
>> ??? write
>> ??? >> a single line of JNI code. In other words, this optimization 
>> gives
>> ??? >> you the same speed as JNI, with improved ease of use (**).
>> ??? >>
>> ??? >> Now, since the 'direct' optimization builds on top of the VM
>> ??? native
>> ??? >> call adapters, this approach is significantly more robust than
>> ??? >> linkToNative and I have not run into any weird VM crashes when
>> ??? >> playing with it. The downside of that, is that, for obvious
>> ??? reasons,
>> ??? >> this approach cannot get much faster than JNI - that is, it 
>> cannot
>> ??? >> get close to the numbers obtained with the linkToNative backend,
>> ??? >> which features much deeper optimizations. But I think that,
>> ??? despite
>> ??? >> its limitations, it's still a good opportunistic improvement
>> ??? that is
>> ??? >> worth pursuing in the short term (while we sort out the
>> ??? linkToNative
>> ??? >> story). For this reason, I will soon be submitting a review which
>> ??? >> incorporates the changes for the 'direct' invocation schemes.
>> ??? >>
>> ??? >> Cheers
>> ??? >> Maurizio
>> ??? >>
>> ??? >> [1] -
>> http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>> <http://cr.openjdk.java.net/%7Emcimadamore/panama/foreign-jmh.txt>
>> ??? >> [2] - https://github.com/jnr/jnr-ffi
>> ??? >> [3] - https://github.com/jnr/jffi
>> ??? >> [4] - https://sourceware.org/libffi/
>> ??? >> [5] -
>> ??? >>
>> https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
>>
>> ??? >>
>> ??? >> [6] -
>> ??? >>
>> http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
>> <http://cr.openjdk.java.net/%7Ejrose/panama/native-call-primitive.html>
>> ??? >> [7] - http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354
>> ??? >>
>> ??? >> (**) the benchmark also contains a 5th row in which I repeated
>> ??? same
>> ??? >> tests, this time using JNR [2]. JNR is built on top of libjffi
>> ??? [3], a
>> ??? >> JNI library in turn built on top of the popular libffi [4]. I
>> ??? wanted
>> ??? >> to have some numbers about JNR because that's another solution
>> ??? that
>> ??? >> allows for better ease to use, taking care of marshalling Java
>> ??? values
>> ??? >> into C and back; since the goals of JNR are similar in spirit 
>> with
>> ??? >> some of the goals of the Panama/foreign work, I thought it
>> ??? would be
>> ??? >> worth having a comparison of these approaches. For the records, I
>> ??? >> think the JNR numbers are very respectable given that JNR had
>> ??? to do
>> ??? >> all the hard work outside of the JDK!
>> ??? >>
>> ??? >>
>> ??? >
>>
>


From shravya.rukmannagari at intel.com  Mon Sep 17 21:23:57 2018
From: shravya.rukmannagari at intel.com (shravya.rukmannagari at intel.com)
Date: Mon, 17 Sep 2018 21:23:57 +0000
Subject: hg: panama/dev: Get Fix for VectorAPI
Message-ID: <201809172123.w8HLNw3n026159@aojmv0008.oracle.com>

Changeset: b6ee9e81e362
Author:    srukmannagar
Date:      2018-09-17 14:22 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b6ee9e81e362

Get Fix for VectorAPI

! src/hotspot/cpu/x86/x86.ad


From maurizio.cimadamore at oracle.com  Mon Sep 17 21:27:02 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Mon, 17 Sep 2018 21:27:02 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809172127.w8HLR20Z028031@aojmv0008.oracle.com>

Changeset: 7e9980e0ed34
Author:    mcimadamore
Date:      2018-09-17 23:29 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/7e9980e0ed34

Automatic merge with vectorIntrinsics



From samuel.audet at gmail.com  Tue Sep 18 00:51:33 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Tue, 18 Sep 2018 09:51:33 +0900
Subject: [foreign] some JMH benchmarks
In-Reply-To: <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
 <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
Message-ID: <CAHQD+Uzx-DxLLag+YBKMchnFpKioGjRba5tK2xpsKdehPEbeCA@mail.gmail.com>

Thanks! Also, the native declaration for exp() isn't "static". That might
impose a large overhead...

2018?9?18?(?) 0:59 Maurizio Cimadamore <maurizio.cimadamore at oracle.com>:

> For the records, here's what I get for all the three benchmarks if I
> compile the JNI code with -O3:
>
> Benchmark                          Mode  Cnt Score         Error  Units
> PanamaBenchmark.testJNIExp        thrpt    5  28575269.294 ?
> 1907726.710  ops/s
> PanamaBenchmark.testJNIJavaQsort  thrpt    5    372148.433 ? 27178.529
> ops/s
> PanamaBenchmark.testJNIPid        thrpt    5  59240069.011 ? 403881.697
> ops/s
>
> The first and second benchmarks get faster and very close to the
> 'direct' optimization numbers in [1]. Surprisingly, the last benchmark
> (getpid) is quite slower. I've been able to reproduce across multiple
> runs; for that benchmark omitting O3 seems to be the achieve best
> results, not sure why. It starts of faster (around in the first couple
> of warmup iterations, but then it goes slower in all the other runs -
> presumably it interacts badly with the C2 generated code. For instance,
> this is a run with O3 enabled:
>
> # Run progress: 66.67% complete, ETA 00:01:40
> # Fork: 1 of 1
> # Warmup Iteration   1: 65182202.653 ops/s
> # Warmup Iteration   2: 64900639.094 ops/s
> # Warmup Iteration   3: 59314945.437 ops/s
> <---------------------------------
> # Warmup Iteration   4: 59269007.877 ops/s
> # Warmup Iteration   5: 59239905.163 ops/s
> Iteration   1: 59300748.074 ops/s
> Iteration   2: 59249666.044 ops/s
> Iteration   3: 59268597.051 ops/s
> Iteration   4: 59322074.572 ops/s
> Iteration   5: 59059259.317 ops/s
>
> And this is a run with O3 disabled:
>
> # Run progress: 0.00% complete, ETA 00:01:40
> # Fork: 1 of 1
> # Warmup Iteration   1: 55882128.787 ops/s
> # Warmup Iteration   2: 53102361.751 ops/s
> # Warmup Iteration   3: 66964755.699 ops/s
> <---------------------------------
> # Warmup Iteration   4: 66414428.355 ops/s
> # Warmup Iteration   5: 65328475.276 ops/s
> Iteration   1: 64229192.993 ops/s
> Iteration   2: 65191719.319 ops/s
> Iteration   3: 65352022.471 ops/s
> Iteration   4: 65152090.426 ops/s
> Iteration   5: 65320545.712 ops/s
>
>
> In both cases, the 3rd warmup execution sees a performance jump - with
> O3, the jump is backwards, w/o O3 the jump is forward, which is quite
> typical for a JMH benchmark as C2 optimization will start to kick in.
>
> For these reasons, I'm reluctant to update my benchmark numbers to
> reflect the O3 behavior (although I agree that, since the Hotspot code
> is compiled with that optimization it would make more sense to use that
> as a reference).
>
> Maurizio
>
> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>
>
>
> On 17/09/18 16:18, Maurizio Cimadamore wrote:
> >
> >
> > On 17/09/18 15:08, Samuel Audet wrote:
> >> Yes, the blackhole or the random number doesn't make any difference,
> >> but not calling gcc with -O3 does. Running the compiler with
> >> optimizations on is pretty common, but they are not enabled by default.
> > A bit better
> >
> > PanamaBenchmark.testMethod  thrpt    5  28018170.076 ? 8491668.248 ops/s
> >
> > But not much of a difference (I did not expected much, as the body of
> > the native method is extremely simple).
> >
> > Maurizio
> >>
> >> Samuel
> >>
> >>
> >> 2018?9?17?(?) 21:37 Maurizio Cimadamore
> >> <maurizio.cimadamore at oracle.com
> >> <mailto:maurizio.cimadamore at oracle.com>>:
> >>
> >>     Hi Samuel,
> >>     I was planning to upload the benchmark IDE project in the near
> >>     future (I
> >>     need to clean that up a bit, so that it can be opened at ease):
> >>
> >>     My getpid example is like this; this is the Java decl:
> >>
> >>     public class GetPid {
> >>
> >>          static {
> >>              System.loadLibrary("getpid");
> >>          }
> >>
> >>          native static long getpid();
> >>
> >>          native double exp(double base);
> >>     }
> >>
> >>     This is the JNI code:
> >>
> >>     JNIEXPORT jlong JNICALL Java_org_panama_GetPid_getpid
> >>        (JNIEnv *env, jobject recv) {
> >>         return getpid();
> >>     }
> >>
> >>     JNIEXPORT jdouble JNICALL Java_org_panama_GetPid_exp
> >>        (JNIEnv *env, jobject recv, jdouble arg) {
> >>         return exp(arg);
> >>     }
> >>
> >>     And this is the benchmark:
> >>
> >>     class PanamaBenchmark {
> >>          static GetPid pidlib = new GetPid();
> >>
> >>          @Benchmark
> >>          public long testJNIPid() {
> >>              return pidlib.getpid();
> >>          }
> >>
> >>          @Benchmark
> >>          public double testJNIExp() {
> >>              return pidlib.exp(10d);
> >>          }
> >>     }
> >>
> >>
> >>     I think this should be rather standard?
> >>
> >>     I'm on Ubuntu 16.04.1, and using GCC 5.4.0. The command I use to
> >>     compile
> >>     the C lib is this:
> >>
> >>     gcc -I<path to jni,h> -l<path to jni lib> -shared -o libgetpid.so
> >>     -fPIC
> >>     GetPid.c
> >>
> >>     One difference I see between our two examples is the use of
> >>     BlackHole.
> >>     In my bench, I'm just returning the call to 'exp' - which should be
> >>     equivalent, and, actually, preferred, as described here:
> >>
> >>
> http://hg.openjdk.java.net/code-tools/jmh/file/3769055ad883/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_09_Blackholes.java#l51
> >>
> >>     Another minor difference I see is that I pass a constant argument,
> >>     while
> >>     you generate a random number on each iteration.
> >>
> >>     I tried to cut and paste your benchmark and I got this:
> >>
> >>     Benchmark                    Mode  Cnt         Score Error Units
> >>     PanamaBenchmark.testMethod  thrpt    5  26362701.827 ?
> >>     1357012.981  ops/s
> >>
> >>
> >>     Which looks exactly the same as what I've got. So, for whatever
> >>     reason,
> >>     my machine seems to be slower than the one you are using. For
> >> what is
> >>     worth, this website [1] seems to confirm the difference. While clock
> >>     speeds are similar, your machine has more Ghz in Turbo boost mode
> >> and
> >>     it's 3-4 years newer than mine, so I'd expect that to make a
> >>     difference
> >>     in terms of internal optimizations etc. Note that I'm able to beat
> >>     the
> >>     numbers of my workstation using my laptop which sports a slightly
> >>     higher
> >>     frequency and only has 2 cores and 8G of RAM.
> >>
> >>     [1] -
> >>
> https://www.cpubenchmark.net/compare/Intel-Xeon-E5-2673-v4-vs-Intel-Xeon-E5-2665/2888vs1439
> >>
> >>     Maurizio
> >>
> >>
> >>
> >>     On 17/09/18 11:00, Samuel Audet wrote:
> >>     > Thanks for the figures Maurizio! It's finally good to be
> >>     speaking in
> >>     > numbers. :)
> >>     >
> >>     > However, you're not providing a lot of details about how you
> >>     actually
> >>     > ran the experiments. So I've decided to run a JMH benchmark on
> >>     what we
> >>     > get by default with JavaCPP and this declaration:
> >>     >
> >>     >     @Platform(include = "math.h")
> >>     >     public class MyBenchmark {
> >>     >         static { Loader.load(); }
> >>     >
> >>     >         @NoException
> >>     >         public static native double exp(double x);
> >>     >
> >>     >         @State(Scope.Thread)
> >>     >         public static class MyState {
> >>     >             double x;
> >>     >
> >>     >             @Setup(Level.Iteration)
> >>     >             public void setupMethod() {
> >>     >                 x = Math.random();
> >>     >             }
> >>     >         }
> >>     >
> >>     >         @Benchmark
> >>     >         public void testMethod(MyState s, Blackhole bh) {
> >>     >             bh.consume(exp(s.x));
> >>     >         }
> >>     >     }
> >>     >
> >>     > The relevant portion of generated JNI looks like this:
> >>     >
> >>     >     JNIEXPORT jdouble JNICALL
> >>     Java_org_sample_MyBenchmark_exp(JNIEnv*
> >>     > env, jclass cls, jdouble arg0) {
> >>     >         jdouble rarg = 0;
> >>     >         double rval = exp(arg0);
> >>     >         rarg = (jdouble)rval;
> >>     >         return rarg;
> >>     >     }
> >>     >
> >>     > And with access to just 2 virtual cores of an Intel(R) Xeon(R) CPU
> >>     > E5-2673 v4 @ 2.30GHz and 8 GB of RAM on the cloud (so probably
> >>     slower
> >>     > than your E5-2665 @ 2.40GHz) running Ubuntu 14.04 with GCC 4.9 and
> >>     > OpenJDK 8, I get these numbers:
> >>     > Benchmark                Mode  Cnt Score        Error Units
> >>     > MyBenchmark.testMethod  thrpt   25  37183556.094 ? 460795.746
> >> ops/s
> >>     >
> >>     > I'm not sure how that compares with your numbers exactly, but it
> >>     does
> >>     > seem to me that what you get for JNI is a bit low. If you could
> >>     > provide more details about how to reproduce your results, that
> >>     would
> >>     > be great.
> >>     >
> >>     > Samuel
> >>     >
> >>     >
> >>     > On 09/14/2018 10:19 PM, Maurizio Cimadamore wrote:
> >>     >> Hi,
> >>     >> over the last few weeks I've been busy playing with Panama and
> >>     >> assessing performances with JMH. For those just interested in raw
> >>     >> numbers, the results of my explorations can be found here [1].
> >>     But as
> >>     >> all benchmarks, I think it's better to spend few words to
> >>     understand
> >>     >> what these numbers actually _mean_.
> >>     >>
> >>     >> To evaluate the performances of Panama I have first created a
> >>     >> baseline using JNI - more specifically I wanted to assess
> >>     >> performances of three calls (all part of the C std library),
> >>     namely
> >>     >> `getpid`, `exp` and `qsort`.
> >>     >>
> >>     >> The first example is the de facto benchmark for FFIs - since it
> >>     does
> >>     >> relatively little computation, it is a good test to measure the
> >>     >> 'latency' of the FFI approach (e.g. how long does it take to
> >> go to
> >>     >> native). The second example is also relatively simple, but the
> >>     this
> >>     >> time the function takes a double argument. The third test is
> >>     akin to
> >>     >> an FFI torture test, since not only it passes substantially more
> >>     >> arguments (4) but one of these arguments is also a callback - a
> >>     >> pointer to a function that is used to sort the contents of the
> >>     input
> >>     >> array.
> >>     >>
> >>     >> As expected, the first batch of JNI results confirms our
> >>     >> expectations: `getpid` is the fastest, followed by `exp`, and
> >> then
> >>     >> followed by `qsort`. Note that qsort is not even close in
> >> terms of
> >>     >> raw numbers to the other two tests - that's because, to sort the
> >>     >> array we need to do (N * log N) upcalls into Java. In the
> >>     benchmark,
> >>     >> N = 8 and we do the upcalls using the JNI function
> >>     >> JNIEnv::CallIntMethod.
> >>     >>
> >>     >> Now let's examine the second batch of results; these call
> >>     `getpid`,
> >>     >> `exp` and `qsort` using Panama. The numbers here are considerably
> >>     >> lower than the JNI ones for all the three benchmark - although
> >> the
> >>     >> first two seem to be the most problematic. To explain these
> >>     results
> >>     >> we need to peek under the hood. Panama implements foreign calls
> >>     >> through a so called 'universal adapter' which, given a calling
> >>     scheme
> >>     >> and a bunch of arguments (machine words) shuffles these
> >>     arguments in
> >>     >> the right registers/stack slot and then jumps to the target
> >> native
> >>     >> function - after which another round of adaptation must be
> >>     performed
> >>     >> (e.g. to recover the return value from the right register/memory
> >>     >> location).
> >>     >>
> >>     >> Needless to say, all this generality comes at a cost - some of
> >> the
> >>     >> cost is in Java - e.g. all arguments have to be packaged up
> >> into a
> >>     >> long array (although this component doesn't seem to show up
> >>     much in
> >>     >> the generated JVM compiled code). A lot of the cost is in the
> >>     adapter
> >>     >> logic itself - which has to look at the 'call recipe' and move
> >>     >> arguments around accordingly - more specifically, in order to
> >> call
> >>     >> the native function, the adapter creates a bunch of helper C++
> >>     >> objects and structs which model the CPU state (e.g. in the
> >>     >> ShuffleDowncallContext struct, we find a field for each
> >>     register to
> >>     >> be modeled in the target architecture). The adapter has to
> >>     first move
> >>     >> the values coming from the Java world (stored in the
> >>     aforementioned
> >>     >> long array) into the right context fields (and it needs to do
> >>     so by
> >>     >> looking at the recipe, which involves iteration over the recipe
> >>     >> elements). After that's done, we can jump into the assembly
> >>     stub that
> >>     >> does the native call - this stub will take as input one of those
> >>     >> ShuffleDowncallContext structure and will load the corresponding
> >>     >> registers/create necessary stack slots ahead of the call.
> >>     >>
> >>     >> As you can see, there's quite a lot of action going on here,
> >>     and this
> >>     >> explains the benchmark numbers; of course, if you are calling a
> >>     >> native function that does a lot of computation, this adaptation
> >>     cost
> >>     >> will wash out - but for relatively quick calls such as 'getpid'
> >>     and
> >>     >> 'exp' the latency dominates the picture.
> >>     >>
> >>     >> Digression: the callback logic suffers pretty much from the same
> >>     >> issues, albeit in a reversed order - this time it's the Java code
> >>     >> which receives a 'snapshot' of the register values from a
> >>     generated
> >>     >> assembly adapter; the Java code can then read such values
> >>     (using the
> >>     >> Pointer API), turn them into Java objects, call the target Java
> >>     >> method and store the results (after another conversion) in the
> >>     right
> >>     >> location of the snapshot. The assembly adapter will then pick
> >>     up the
> >>     >> values set onto the snapshot by the Java code, store them into
> >> the
> >>     >> corresponding registers and return control to the native
> >>     callee. In
> >>     >> the remainder of this email we will not discuss callbacks in
> >>     details
> >>     >> - we will just posit that for any optimization technique that
> >>     can be
> >>     >> defined, there exists a _dual_ strategy that works with
> >> callbacks.
> >>     >>
> >>     >> How can we make sensible native calls go faster? Well, one
> >> obvious
> >>     >> way would be to optimize the universal adapter so that we get a
> >>     >> specialized assembly stub for each code shape. If we do that,
> >>     we can
> >>     >> move pretty much all of the computation described above from
> >>     >> execution time to the stub generation time, so that, by the
> >>     time we
> >>     >> have to call the native function, we just have to populate the
> >>     right
> >>     >> registers (the specialized stub knows where to find them) and
> >>     jump.
> >>     >> While this sounds a good approach, it feels like there's also a
> >>     move
> >>     >> for the JIT somewhere in there - after all, the JVM knows which
> >>     calls
> >>     >> are hot and in need for optimization, so perhaps this
> >>     specialization
> >>     >> process (some or all of it) could happen dynamically. And this is
> >>     >> indeed an approach we'd like to aim for in the long run.
> >>     >>
> >>     >> Now, few years ago, Vlad put together a patch which now lives
> >>     in the
> >>     >> 'linkToNative' branch [6, 7] - the goal of this patch is to
> >>     implement
> >>     >> the approach described above: generate a specialized assembly
> >>     adapter
> >>     >> for a given native signature, and then leverage the JIT to
> >>     optimize
> >>     >> it away, turning the adapter into a bare, direct, native method
> >>     call.
> >>     >> As you can see from the third batch of benchmarks, if we tweak
> >>     Panama
> >>     >> to use the linkToNative machinery, the speed up is really
> >>     impressive,
> >>     >> and we end up being much faster than JNI (up to 4x for getPid).
> >>     >>
> >>     >> Unfortunately, the technology in the linkToNative branch is not
> >>     ready
> >>     >> from prime time (yet) - first, it doesn't cover some useful cases
> >>     >> (e.g. varargs, multiple returns via registers, arguments
> >> passed in
> >>     >> memory). That is, the logic assumes there's a 1-1 mapping
> >>     between a
> >>     >> Java signature and the native function to be called - and that
> >> the
> >>     >> arguments passed from Java will either be longs or doubles.
> >>     While we
> >>     >> can workaround this limitation and define the necessary
> >>     marshalling
> >>     >> logic in Java (as I have done to run this benchmark), some of the
> >>     >> limitations (multiple returns, structs passed by value which
> >>     are too
> >>     >> big) cannot simply be worked around. But that's fine, we can
> >> still
> >>     >> have a fast path for those calls which have certain
> >>     characteristics
> >>     >> and a slow path (through the universal adapter) for all the
> >>     other calls.
> >>     >>
> >>     >> But there's a second and more serious issue lurking: as you can
> >>     see
> >>     >> in the benchmark, I was not able to get the qsort benchmark
> >>     running
> >>     >> when using the linkToNative backend. The reason is that the
> >>     >> linkToNative code is still pretty raw, and it doesn't fully
> >>     adhere to
> >>     >> the JVM internal conventions - e.g. there are missing thread
> >> state
> >>     >> transitions which, in the case of upcalls into Java, create
> >> issues
> >>     >> when it comes to garbage collection, as the GC cannot parse the
> >>     >> native stack in the correct way.
> >>     >>
> >>     >> This means that, while there's a clear shining path ahead of
> >>     us, it
> >>     >> is simply too early to just use the linkToNative backend from
> >>     Panama.
> >>     >> For this reason, I've been looking into some kind of stopgap
> >>     solution
> >>     >> - another way of optimizing native calls (and upcalls into
> >>     Java) that
> >>     >> doesn't require too much VM magic. Now, a crucial observation is
> >>     >> that, in many native calls, there is indeed a 1-1 mapping between
> >>     >> Java arguments and native arguments (and back, for return
> >> values).
> >>     >> That is, we can think of calling a native function as a process
> >>     that
> >>     >> takes a bunch of Java arguments, turn them into native arguments
> >>     >> (either double or longs), calls the native methods and then turns
> >>     >> back the result into Java.
> >>     >>
> >>     >> The mapping between Java arguments and C values is quite simple:
> >>     >>
> >>     >> * primitives: either long or double, depending on whether they
> >>     >> describe an integral value or a floating point one.
> >>     >> * pointers: they convert to a long
> >>     >> * callbacks: they also convert to a long
> >>     >> * structs: they are recursively decomposed into fields and each
> >>     field
> >>     >> is marshalled separately (assuming the struct is not too big, in
> >>     >> which case is passed in memory)
> >>     >>
> >>     >> So, in principle, we could define a bunch of native entry
> >>     points in
> >>     >> the VM, one per shape, which take a bunch of long and doubles and
> >>     >> call an underlying function with those arguments. For instance,
> >>     let's
> >>     >> consider the case of a native function which is modelled in
> >>     Java as:
> >>     >>
> >>     >> int m(Pointer<Foo>, double)
> >>     >>
> >>     >> To call this native function we have to first turn the Java
> >>     arguments
> >>     >> into a (long, double) pair. Then we need to call a native adapter
> >>     >> that looks like the following:
> >>     >>
> >>     >> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong
> >>     addr,
> >>     >> jlong arg0, jdouble arg1) {
> >>     >>      return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
> >>     >> }
> >>     >>
> >>     >> And this will take care of calling the native function and
> >>     returning
> >>     >> the value back. This is, admittedly, a very simple solution; of
> >>     >> course there are limitations: we have to define a bunch of
> >>     >> specialized native entry point (and Java entry points, for
> >>     >> callbacks). But here we can play a trick: most of moderns ABI
> >> pass
> >>     >> arguments in registers; for instance System V ABI [5] uses up
> >> to 6
> >>     >> (!!) integer registers and 7 (!!) MMXr registers for FP values
> >>     - this
> >>     >> gives us a total of 13 registers available for argument passing.
> >>     >> Which covers quite a lot of cases. Now, if we have a call where
> >>     _all_
> >>     >> arguments are passed in registers, then the order in which these
> >>     >> arguments are declared in the adapter doesn't matter! That is,
> >>     since
> >>     >> FP-values will always be passed in different register from
> >>     integral
> >>     >> values, we can just define entry points which look like these:
> >>     >>
> >>     >> invokeNative_V_DDDDD
> >>     >> invokeNative_V_JDDDD
> >>     >> invokeNative_V_JJDDD
> >>     >> invokeNative_V_JJJDD
> >>     >> invokeNative_V_JJJJD
> >>     >> invokeNative_V_JJJJJ
> >>     >>
> >>     >> That is, for a given arity (5 in this case), we can just put
> >>     all long
> >>     >> arguments in front, and the double arguments after that. That
> >>     is, we
> >>     >> don't need to generate all possible permutations of J/D in all
> >>     >> positions - as the adapter will always do the same thing (read:
> >>     load
> >>     >> from same registers) for all equivalent combinations. This
> >>     keeps the
> >>     >> number of entry points in check - and it also poses some
> >>     challenges
> >>     >> to the Java logic in charge of marshalling/unmarshalling, as
> >>     there's
> >>     >> an extra permutation step involved (although that is not
> >> something
> >>     >> super-hard to address).
> >>     >>
> >>     >> You can see the performance numbers associated with this
> >>     invocation
> >>     >> scheme (which I've dubbed 'direct') in the 4th batch of the
> >>     benchmark
> >>     >> results. These numbers are on par (and slightly better) with
> >>     JNI in
> >>     >> all the three cases considered which is, I think, a very positive
> >>     >> result, given that to write these benchmarks I did not have to
> >>     write
> >>     >> a single line of JNI code. In other words, this optimization
> >> gives
> >>     >> you the same speed as JNI, with improved ease of use (**).
> >>     >>
> >>     >> Now, since the 'direct' optimization builds on top of the VM
> >>     native
> >>     >> call adapters, this approach is significantly more robust than
> >>     >> linkToNative and I have not run into any weird VM crashes when
> >>     >> playing with it. The downside of that, is that, for obvious
> >>     reasons,
> >>     >> this approach cannot get much faster than JNI - that is, it
> >> cannot
> >>     >> get close to the numbers obtained with the linkToNative backend,
> >>     >> which features much deeper optimizations. But I think that,
> >>     despite
> >>     >> its limitations, it's still a good opportunistic improvement
> >>     that is
> >>     >> worth pursuing in the short term (while we sort out the
> >>     linkToNative
> >>     >> story). For this reason, I will soon be submitting a review which
> >>     >> incorporates the changes for the 'direct' invocation schemes.
> >>     >>
> >>     >> Cheers
> >>     >> Maurizio
> >>     >>
> >>     >> [1] -
> >> http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
> >> <http://cr.openjdk.java.net/%7Emcimadamore/panama/foreign-jmh.txt>
> >>     >> [2] - https://github.com/jnr/jnr-ffi
> >>     >> [3] - https://github.com/jnr/jffi
> >>     >> [4] - https://sourceware.org/libffi/
> >>     >> [5] -
> >>     >>
> >>
> https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
> >>
> >>     >>
> >>     >> [6] -
> >>     >>
> >> http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
> >> <http://cr.openjdk.java.net/%7Ejrose/panama/native-call-primitive.html>
> >>     >> [7] -
> http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354
> >>     >>
> >>     >> (**) the benchmark also contains a 5th row in which I repeated
> >>     same
> >>     >> tests, this time using JNR [2]. JNR is built on top of libjffi
> >>     [3], a
> >>     >> JNI library in turn built on top of the popular libffi [4]. I
> >>     wanted
> >>     >> to have some numbers about JNR because that's another solution
> >>     that
> >>     >> allows for better ease to use, taking care of marshalling Java
> >>     values
> >>     >> into C and back; since the goals of JNR are similar in spirit
> >> with
> >>     >> some of the goals of the Panama/foreign work, I thought it
> >>     would be
> >>     >> worth having a comparison of these approaches. For the records, I
> >>     >> think the JNR numbers are very respectable given that JNR had
> >>     to do
> >>     >> all the hard work outside of the JDK!
> >>     >>
> >>     >>
> >>     >
> >>
> >
>
>

From samuel.audet at gmail.com  Tue Sep 18 04:48:58 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Tue, 18 Sep 2018 13:48:58 +0900
Subject: [foreign] some JMH benchmarks
In-Reply-To: <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
 <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
Message-ID: <25be0579-eb55-65e7-b379-206b8713a687@gmail.com>

Anyway, I've put online an updated version of my benchmark files here:
https://gist.github.com/saudet/1bf14a000e64c245675cf5d4e9ad6e69
Just run "git clone" on the URL and run "mvn package" on the pom.xml.

With the 2 virtual cores of an Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz 
running Ubuntu 14.04 on the cloud with GCC 4.9 and OpenJDK 8, I get 
these numbers:

Benchmark                         Mode  Cnt          Score         Error 
  Units
NativeBenchmark.expBenchmark     thrpt   25   37460540.440 ?  393299.974 
  ops/s
NativeBenchmark.getpidBenchmark  thrpt   25  100323188.451 ? 1254197.449 
  ops/s

While on my laptop, an Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz running 
Fedora 27, GCC 7.3, and OpenJDK 9, I get the following:

Benchmark                         Mode  Cnt         Score        Error 
Units
NativeBenchmark.expBenchmark     thrpt   25  50047147.099 ? 924366.937 
ops/s
NativeBenchmark.getpidBenchmark  thrpt   25   4825508.193 ?  21662.633 
ops/s

Now, it looks like getpid() is really slow on Fedora 27 for some reason, 
but as Linus puts it, we should not be using that for benchmarking:
https://yarchive.net/comp/linux/getpid_caching.html

What do you get on your machines?

Samuel


On 09/18/2018 12:58 AM, Maurizio Cimadamore wrote:
> For the records, here's what I get for all the three benchmarks if I 
> compile the JNI code with -O3:
> 
> Benchmark????????????????????????? Mode? Cnt Score???????? Error? Units
> PanamaBenchmark.testJNIExp??????? thrpt??? 5? 28575269.294 ? 1907726.710? ops/s
> PanamaBenchmark.testJNIJavaQsort? thrpt??? 5??? 372148.433 ? 27178.529  ops/s
> PanamaBenchmark.testJNIPid??????? thrpt??? 5? 59240069.011 ? 403881.697  ops/s
> 
> The first and second benchmarks get faster and very close to the 
> 'direct' optimization numbers in [1]. Surprisingly, the last benchmark 
> (getpid) is quite slower. I've been able to reproduce across multiple 
> runs; for that benchmark omitting O3 seems to be the achieve best 
> results, not sure why. It starts of faster (around in the first couple 
> of warmup iterations, but then it goes slower in all the other runs - 
> presumably it interacts badly with the C2 generated code. For instance, 
> this is a run with O3 enabled:
> 
> # Run progress: 66.67% complete, ETA 00:01:40
> # Fork: 1 of 1
> # Warmup Iteration?? 1: 65182202.653 ops/s
> # Warmup Iteration?? 2: 64900639.094 ops/s
> # Warmup Iteration?? 3: 59314945.437 ops/s 
> <---------------------------------
> # Warmup Iteration?? 4: 59269007.877 ops/s
> # Warmup Iteration?? 5: 59239905.163 ops/s
> Iteration?? 1: 59300748.074 ops/s
> Iteration?? 2: 59249666.044 ops/s
> Iteration?? 3: 59268597.051 ops/s
> Iteration?? 4: 59322074.572 ops/s
> Iteration?? 5: 59059259.317 ops/s
> 
> And this is a run with O3 disabled:
> 
> # Run progress: 0.00% complete, ETA 00:01:40
> # Fork: 1 of 1
> # Warmup Iteration?? 1: 55882128.787 ops/s
> # Warmup Iteration?? 2: 53102361.751 ops/s
> # Warmup Iteration?? 3: 66964755.699 ops/s 
> <---------------------------------
> # Warmup Iteration?? 4: 66414428.355 ops/s
> # Warmup Iteration?? 5: 65328475.276 ops/s
> Iteration?? 1: 64229192.993 ops/s
> Iteration?? 2: 65191719.319 ops/s
> Iteration?? 3: 65352022.471 ops/s
> Iteration?? 4: 65152090.426 ops/s
> Iteration?? 5: 65320545.712 ops/s
> 
> 
> In both cases, the 3rd warmup execution sees a performance jump - with 
> O3, the jump is backwards, w/o O3 the jump is forward, which is quite 
> typical for a JMH benchmark as C2 optimization will start to kick in.
> 
> For these reasons, I'm reluctant to update my benchmark numbers to 
> reflect the O3 behavior (although I agree that, since the Hotspot code 
> is compiled with that optimization it would make more sense to use that 
> as a reference).
> 
> Maurizio
> 
> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
> 
> 
> 
> On 17/09/18 16:18, Maurizio Cimadamore wrote:
>>
>>
>> On 17/09/18 15:08, Samuel Audet wrote:
>>> Yes, the blackhole or the random number doesn't make any difference, 
>>> but not calling gcc with -O3 does. Running the compiler with 
>>> optimizations on is pretty common, but they are not enabled by default.
>> A bit better
>>
>> PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ? 8491668.248 ops/s
>>
>> But not much of a difference (I did not expected much, as the body of 
>> the native method is extremely simple).
>>
>> Maurizio 


From maurizio.cimadamore at oracle.com  Tue Sep 18 09:43:43 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 18 Sep 2018 10:43:43 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <CAHQD+Uzx-DxLLag+YBKMchnFpKioGjRba5tK2xpsKdehPEbeCA@mail.gmail.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
 <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
 <CAHQD+Uzx-DxLLag+YBKMchnFpKioGjRba5tK2xpsKdehPEbeCA@mail.gmail.com>
Message-ID: <8fea0a87-f0ad-61a0-403f-5e9bf385bc89@oracle.com>

No discernible difference with static (in fact getpid was already using 
static but I forgot that in getpid). I wouldn't expect the JIT not to 
optimize that (given that the receiver was always the same value created 
once at the beginning of the class' lifecycle).

Maurizio


On 18/09/18 01:51, Samuel Audet wrote:
> Thanks! Also, the native declaration for exp() isn't "static". That 
> might impose a large overhead...
>
> 2018?9?18?(?) 0:59 Maurizio Cimadamore <maurizio.cimadamore at oracle.com 
> <mailto:maurizio.cimadamore at oracle.com>>:
>
>     For the records, here's what I get for all the three benchmarks if I
>     compile the JNI code with -O3:
>
>     Benchmark????????????????????????? Mode? Cnt Score Error? Units
>     PanamaBenchmark.testJNIExp??????? thrpt??? 5? 28575269.294 ?
>     1907726.710? ops/s
>     PanamaBenchmark.testJNIJavaQsort? thrpt??? 5??? 372148.433 ?
>     27178.529
>     ops/s
>     PanamaBenchmark.testJNIPid??????? thrpt??? 5? 59240069.011 ?
>     403881.697
>     ops/s
>
>     The first and second benchmarks get faster and very close to the
>     'direct' optimization numbers in [1]. Surprisingly, the last
>     benchmark
>     (getpid) is quite slower. I've been able to reproduce across multiple
>     runs; for that benchmark omitting O3 seems to be the achieve best
>     results, not sure why. It starts of faster (around in the first
>     couple
>     of warmup iterations, but then it goes slower in all the other runs -
>     presumably it interacts badly with the C2 generated code. For
>     instance,
>     this is a run with O3 enabled:
>
>     # Run progress: 66.67% complete, ETA 00:01:40
>     # Fork: 1 of 1
>     # Warmup Iteration?? 1: 65182202.653 ops/s
>     # Warmup Iteration?? 2: 64900639.094 ops/s
>     # Warmup Iteration?? 3: 59314945.437 ops/s
>     <---------------------------------
>     # Warmup Iteration?? 4: 59269007.877 ops/s
>     # Warmup Iteration?? 5: 59239905.163 ops/s
>     Iteration?? 1: 59300748.074 ops/s
>     Iteration?? 2: 59249666.044 ops/s
>     Iteration?? 3: 59268597.051 ops/s
>     Iteration?? 4: 59322074.572 ops/s
>     Iteration?? 5: 59059259.317 ops/s
>
>     And this is a run with O3 disabled:
>
>     # Run progress: 0.00% complete, ETA 00:01:40
>     # Fork: 1 of 1
>     # Warmup Iteration?? 1: 55882128.787 ops/s
>     # Warmup Iteration?? 2: 53102361.751 ops/s
>     # Warmup Iteration?? 3: 66964755.699 ops/s
>     <---------------------------------
>     # Warmup Iteration?? 4: 66414428.355 ops/s
>     # Warmup Iteration?? 5: 65328475.276 ops/s
>     Iteration?? 1: 64229192.993 ops/s
>     Iteration?? 2: 65191719.319 ops/s
>     Iteration?? 3: 65352022.471 ops/s
>     Iteration?? 4: 65152090.426 ops/s
>     Iteration?? 5: 65320545.712 ops/s
>
>
>     In both cases, the 3rd warmup execution sees a performance jump -
>     with
>     O3, the jump is backwards, w/o O3 the jump is forward, which is quite
>     typical for a JMH benchmark as C2 optimization will start to kick in.
>
>     For these reasons, I'm reluctant to update my benchmark numbers to
>     reflect the O3 behavior (although I agree that, since the Hotspot
>     code
>     is compiled with that optimization it would make more sense to use
>     that
>     as a reference).
>
>     Maurizio
>
>     [1] -
>     http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>     <http://cr.openjdk.java.net/%7Emcimadamore/panama/foreign-jmh.txt>
>
>
>
>     On 17/09/18 16:18, Maurizio Cimadamore wrote:
>     >
>     >
>     > On 17/09/18 15:08, Samuel Audet wrote:
>     >> Yes, the blackhole or the random number doesn't make any
>     difference,
>     >> but not calling gcc with -O3 does. Running the compiler with
>     >> optimizations on is pretty common, but they are not enabled by
>     default.
>     > A bit better
>     >
>     > PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ?
>     8491668.248 ops/s
>     >
>     > But not much of a difference (I did not expected much, as the
>     body of
>     > the native method is extremely simple).
>     >
>     > Maurizio
>     >>
>     >> Samuel
>     >>
>     >>
>     >> 2018?9?17?(?) 21:37 Maurizio Cimadamore
>     >> <maurizio.cimadamore at oracle.com
>     <mailto:maurizio.cimadamore at oracle.com>
>     >> <mailto:maurizio.cimadamore at oracle.com
>     <mailto:maurizio.cimadamore at oracle.com>>>:
>     >>
>     >> ??? Hi Samuel,
>     >> ??? I was planning to upload the benchmark IDE project in the near
>     >> ??? future (I
>     >> ??? need to clean that up a bit, so that it can be opened at ease):
>     >>
>     >> ??? My getpid example is like this; this is the Java decl:
>     >>
>     >> ??? public class GetPid {
>     >>
>     >> ??? ???? static {
>     >> ??? ???????? System.loadLibrary("getpid");
>     >> ??? ???? }
>     >>
>     >> ??? ???? native static long getpid();
>     >>
>     >> ??? ???? native double exp(double base);
>     >> ??? }
>     >>
>     >> ??? This is the JNI code:
>     >>
>     >> ??? JNIEXPORT jlong JNICALL Java_org_panama_GetPid_getpid
>     >> ??? ?? (JNIEnv *env, jobject recv) {
>     >> ??? ??? return getpid();
>     >> ??? }
>     >>
>     >> ??? JNIEXPORT jdouble JNICALL Java_org_panama_GetPid_exp
>     >> ??? ?? (JNIEnv *env, jobject recv, jdouble arg) {
>     >> ??? ??? return exp(arg);
>     >> ??? }
>     >>
>     >> ??? And this is the benchmark:
>     >>
>     >> ??? class PanamaBenchmark {
>     >> ??? ???? static GetPid pidlib = new GetPid();
>     >>
>     >> ??? ???? @Benchmark
>     >> ??? ???? public long testJNIPid() {
>     >> ??? ???????? return pidlib.getpid();
>     >> ??? ???? }
>     >>
>     >> ??? ???? @Benchmark
>     >> ??? ???? public double testJNIExp() {
>     >> ??? ???????? return pidlib.exp(10d);
>     >> ??? ???? }
>     >> ??? }
>     >>
>     >>
>     >> ??? I think this should be rather standard?
>     >>
>     >> ??? I'm on Ubuntu 16.04.1, and using GCC 5.4.0. The command I
>     use to
>     >> ??? compile
>     >> ??? the C lib is this:
>     >>
>     >> ??? gcc -I<path to jni,h> -l<path to jni lib> -shared -o
>     libgetpid.so
>     >> ??? -fPIC
>     >> ??? GetPid.c
>     >>
>     >> ??? One difference I see between our two examples is the use of
>     >> ??? BlackHole.
>     >> ??? In my bench, I'm just returning the call to 'exp' - which
>     should be
>     >> ??? equivalent, and, actually, preferred, as described here:
>     >>
>     >>
>     http://hg.openjdk.java.net/code-tools/jmh/file/3769055ad883/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_09_Blackholes.java#l51
>     >>
>     >> ??? Another minor difference I see is that I pass a constant
>     argument,
>     >> ??? while
>     >> ??? you generate a random number on each iteration.
>     >>
>     >> ??? I tried to cut and paste your benchmark and I got this:
>     >>
>     >> ??? Benchmark??????????????????? Mode Cnt???????? Score Error Units
>     >> ??? PanamaBenchmark.testMethod? thrpt??? 5 26362701.827 ?
>     >> ??? 1357012.981? ops/s
>     >>
>     >>
>     >> ??? Which looks exactly the same as what I've got. So, for whatever
>     >> ??? reason,
>     >> ??? my machine seems to be slower than the one you are using. For
>     >> what is
>     >> ??? worth, this website [1] seems to confirm the difference.
>     While clock
>     >> ??? speeds are similar, your machine has more Ghz in Turbo
>     boost mode
>     >> and
>     >> ??? it's 3-4 years newer than mine, so I'd expect that to make a
>     >> ??? difference
>     >> ??? in terms of internal optimizations etc. Note that I'm able
>     to beat
>     >> ??? the
>     >> ??? numbers of my workstation using my laptop which sports a
>     slightly
>     >> ??? higher
>     >> ??? frequency and only has 2 cores and 8G of RAM.
>     >>
>     >> ??? [1] -
>     >>
>     https://www.cpubenchmark.net/compare/Intel-Xeon-E5-2673-v4-vs-Intel-Xeon-E5-2665/2888vs1439
>     >>
>     >> ??? Maurizio
>     >>
>     >>
>     >>
>     >> ??? On 17/09/18 11:00, Samuel Audet wrote:
>     >> ??? > Thanks for the figures Maurizio! It's finally good to be
>     >> ??? speaking in
>     >> ??? > numbers. :)
>     >> ??? >
>     >> ??? > However, you're not providing a lot of details about how you
>     >> ??? actually
>     >> ??? > ran the experiments. So I've decided to run a JMH
>     benchmark on
>     >> ??? what we
>     >> ??? > get by default with JavaCPP and this declaration:
>     >> ??? >
>     >> ??? > ??? @Platform(include = "math.h")
>     >> ??? > ??? public class MyBenchmark {
>     >> ??? > ??????? static { Loader.load(); }
>     >> ??? >
>     >> ??? > ??????? @NoException
>     >> ??? > ??????? public static native double exp(double x);
>     >> ??? >
>     >> ??? > ??????? @State(Scope.Thread)
>     >> ??? > ??????? public static class MyState {
>     >> ??? > ??????????? double x;
>     >> ??? >
>     >> ??? > ??????????? @Setup(Level.Iteration)
>     >> ??? > ??????????? public void setupMethod() {
>     >> ??? > ??????????????? x = Math.random();
>     >> ??? > ??????????? }
>     >> ??? > ??????? }
>     >> ??? >
>     >> ??? > ??????? @Benchmark
>     >> ??? > ??????? public void testMethod(MyState s, Blackhole bh) {
>     >> ??? > ??????????? bh.consume(exp(s.x));
>     >> ??? > ??????? }
>     >> ??? > ??? }
>     >> ??? >
>     >> ??? > The relevant portion of generated JNI looks like this:
>     >> ??? >
>     >> ??? > ??? JNIEXPORT jdouble JNICALL
>     >> ??? Java_org_sample_MyBenchmark_exp(JNIEnv*
>     >> ??? > env, jclass cls, jdouble arg0) {
>     >> ??? > ??????? jdouble rarg = 0;
>     >> ??? > ??????? double rval = exp(arg0);
>     >> ??? > ??????? rarg = (jdouble)rval;
>     >> ??? > ??????? return rarg;
>     >> ??? > ??? }
>     >> ??? >
>     >> ??? > And with access to just 2 virtual cores of an Intel(R)
>     Xeon(R) CPU
>     >> ??? > E5-2673 v4 @ 2.30GHz and 8 GB of RAM on the cloud (so
>     probably
>     >> ??? slower
>     >> ??? > than your E5-2665 @ 2.40GHz) running Ubuntu 14.04 with
>     GCC 4.9 and
>     >> ??? > OpenJDK 8, I get these numbers:
>     >> ??? > Benchmark??????????????? Mode? Cnt Score??????? Error Units
>     >> ??? > MyBenchmark.testMethod? thrpt?? 25 37183556.094 ? 460795.746
>     >> ops/s
>     >> ??? >
>     >> ??? > I'm not sure how that compares with your numbers exactly,
>     but it
>     >> ??? does
>     >> ??? > seem to me that what you get for JNI is a bit low. If you
>     could
>     >> ??? > provide more details about how to reproduce your results,
>     that
>     >> ??? would
>     >> ??? > be great.
>     >> ??? >
>     >> ??? > Samuel
>     >> ??? >
>     >> ??? >
>     >> ??? > On 09/14/2018 10:19 PM, Maurizio Cimadamore wrote:
>     >> ??? >> Hi,
>     >> ??? >> over the last few weeks I've been busy playing with
>     Panama and
>     >> ??? >> assessing performances with JMH. For those just
>     interested in raw
>     >> ??? >> numbers, the results of my explorations can be found
>     here [1].
>     >> ??? But as
>     >> ??? >> all benchmarks, I think it's better to spend few words to
>     >> ??? understand
>     >> ??? >> what these numbers actually _mean_.
>     >> ??? >>
>     >> ??? >> To evaluate the performances of Panama I have first
>     created a
>     >> ??? >> baseline using JNI - more specifically I wanted to assess
>     >> ??? >> performances of three calls (all part of the C std library),
>     >> ??? namely
>     >> ??? >> `getpid`, `exp` and `qsort`.
>     >> ??? >>
>     >> ??? >> The first example is the de facto benchmark for FFIs -
>     since it
>     >> ??? does
>     >> ??? >> relatively little computation, it is a good test to
>     measure the
>     >> ??? >> 'latency' of the FFI approach (e.g. how long does it
>     take to
>     >> go to
>     >> ??? >> native). The second example is also relatively simple,
>     but the
>     >> ??? this
>     >> ??? >> time the function takes a double argument. The third test is
>     >> ??? akin to
>     >> ??? >> an FFI torture test, since not only it passes
>     substantially more
>     >> ??? >> arguments (4) but one of these arguments is also a
>     callback - a
>     >> ??? >> pointer to a function that is used to sort the contents
>     of the
>     >> ??? input
>     >> ??? >> array.
>     >> ??? >>
>     >> ??? >> As expected, the first batch of JNI results confirms our
>     >> ??? >> expectations: `getpid` is the fastest, followed by
>     `exp`, and
>     >> then
>     >> ??? >> followed by `qsort`. Note that qsort is not even close in
>     >> terms of
>     >> ??? >> raw numbers to the other two tests - that's because, to
>     sort the
>     >> ??? >> array we need to do (N * log N) upcalls into Java. In the
>     >> ??? benchmark,
>     >> ??? >> N = 8 and we do the upcalls using the JNI function
>     >> ??? >> JNIEnv::CallIntMethod.
>     >> ??? >>
>     >> ??? >> Now let's examine the second batch of results; these call
>     >> ??? `getpid`,
>     >> ??? >> `exp` and `qsort` using Panama. The numbers here are
>     considerably
>     >> ??? >> lower than the JNI ones for all the three benchmark -
>     although
>     >> the
>     >> ??? >> first two seem to be the most problematic. To explain these
>     >> ??? results
>     >> ??? >> we need to peek under the hood. Panama implements
>     foreign calls
>     >> ??? >> through a so called 'universal adapter' which, given a
>     calling
>     >> ??? scheme
>     >> ??? >> and a bunch of arguments (machine words) shuffles these
>     >> ??? arguments in
>     >> ??? >> the right registers/stack slot and then jumps to the target
>     >> native
>     >> ??? >> function - after which another round of adaptation must be
>     >> ??? performed
>     >> ??? >> (e.g. to recover the return value from the right
>     register/memory
>     >> ??? >> location).
>     >> ??? >>
>     >> ??? >> Needless to say, all this generality comes at a cost -
>     some of
>     >> the
>     >> ??? >> cost is in Java - e.g. all arguments have to be packaged up
>     >> into a
>     >> ??? >> long array (although this component doesn't seem to show up
>     >> ??? much in
>     >> ??? >> the generated JVM compiled code). A lot of the cost is
>     in the
>     >> ??? adapter
>     >> ??? >> logic itself - which has to look at the 'call recipe'
>     and move
>     >> ??? >> arguments around accordingly - more specifically, in
>     order to
>     >> call
>     >> ??? >> the native function, the adapter creates a bunch of
>     helper C++
>     >> ??? >> objects and structs which model the CPU state (e.g. in the
>     >> ??? >> ShuffleDowncallContext struct, we find a field for each
>     >> ??? register to
>     >> ??? >> be modeled in the target architecture). The adapter has to
>     >> ??? first move
>     >> ??? >> the values coming from the Java world (stored in the
>     >> ??? aforementioned
>     >> ??? >> long array) into the right context fields (and it needs
>     to do
>     >> ??? so by
>     >> ??? >> looking at the recipe, which involves iteration over the
>     recipe
>     >> ??? >> elements). After that's done, we can jump into the assembly
>     >> ??? stub that
>     >> ??? >> does the native call - this stub will take as input one
>     of those
>     >> ??? >> ShuffleDowncallContext structure and will load the
>     corresponding
>     >> ??? >> registers/create necessary stack slots ahead of the call.
>     >> ??? >>
>     >> ??? >> As you can see, there's quite a lot of action going on here,
>     >> ??? and this
>     >> ??? >> explains the benchmark numbers; of course, if you are
>     calling a
>     >> ??? >> native function that does a lot of computation, this
>     adaptation
>     >> ??? cost
>     >> ??? >> will wash out - but for relatively quick calls such as
>     'getpid'
>     >> ??? and
>     >> ??? >> 'exp' the latency dominates the picture.
>     >> ??? >>
>     >> ??? >> Digression: the callback logic suffers pretty much from
>     the same
>     >> ??? >> issues, albeit in a reversed order - this time it's the
>     Java code
>     >> ??? >> which receives a 'snapshot' of the register values from a
>     >> ??? generated
>     >> ??? >> assembly adapter; the Java code can then read such values
>     >> ??? (using the
>     >> ??? >> Pointer API), turn them into Java objects, call the
>     target Java
>     >> ??? >> method and store the results (after another conversion)
>     in the
>     >> ??? right
>     >> ??? >> location of the snapshot. The assembly adapter will then
>     pick
>     >> ??? up the
>     >> ??? >> values set onto the snapshot by the Java code, store
>     them into
>     >> the
>     >> ??? >> corresponding registers and return control to the native
>     >> ??? callee. In
>     >> ??? >> the remainder of this email we will not discuss callbacks in
>     >> ??? details
>     >> ??? >> - we will just posit that for any optimization technique
>     that
>     >> ??? can be
>     >> ??? >> defined, there exists a _dual_ strategy that works with
>     >> callbacks.
>     >> ??? >>
>     >> ??? >> How can we make sensible native calls go faster? Well, one
>     >> obvious
>     >> ??? >> way would be to optimize the universal adapter so that
>     we get a
>     >> ??? >> specialized assembly stub for each code shape. If we do
>     that,
>     >> ??? we can
>     >> ??? >> move pretty much all of the computation described above from
>     >> ??? >> execution time to the stub generation time, so that, by the
>     >> ??? time we
>     >> ??? >> have to call the native function, we just have to
>     populate the
>     >> ??? right
>     >> ??? >> registers (the specialized stub knows where to find
>     them) and
>     >> ??? jump.
>     >> ??? >> While this sounds a good approach, it feels like there's
>     also a
>     >> ??? move
>     >> ??? >> for the JIT somewhere in there - after all, the JVM
>     knows which
>     >> ??? calls
>     >> ??? >> are hot and in need for optimization, so perhaps this
>     >> ??? specialization
>     >> ??? >> process (some or all of it) could happen dynamically.
>     And this is
>     >> ??? >> indeed an approach we'd like to aim for in the long run.
>     >> ??? >>
>     >> ??? >> Now, few years ago, Vlad put together a patch which now
>     lives
>     >> ??? in the
>     >> ??? >> 'linkToNative' branch [6, 7] - the goal of this patch is to
>     >> ??? implement
>     >> ??? >> the approach described above: generate a specialized
>     assembly
>     >> ??? adapter
>     >> ??? >> for a given native signature, and then leverage the JIT to
>     >> ??? optimize
>     >> ??? >> it away, turning the adapter into a bare, direct, native
>     method
>     >> ??? call.
>     >> ??? >> As you can see from the third batch of benchmarks, if we
>     tweak
>     >> ??? Panama
>     >> ??? >> to use the linkToNative machinery, the speed up is really
>     >> ??? impressive,
>     >> ??? >> and we end up being much faster than JNI (up to 4x for
>     getPid).
>     >> ??? >>
>     >> ??? >> Unfortunately, the technology in the linkToNative branch
>     is not
>     >> ??? ready
>     >> ??? >> from prime time (yet) - first, it doesn't cover some
>     useful cases
>     >> ??? >> (e.g. varargs, multiple returns via registers, arguments
>     >> passed in
>     >> ??? >> memory). That is, the logic assumes there's a 1-1 mapping
>     >> ??? between a
>     >> ??? >> Java signature and the native function to be called -
>     and that
>     >> the
>     >> ??? >> arguments passed from Java will either be longs or doubles.
>     >> ??? While we
>     >> ??? >> can workaround this limitation and define the necessary
>     >> ??? marshalling
>     >> ??? >> logic in Java (as I have done to run this benchmark),
>     some of the
>     >> ??? >> limitations (multiple returns, structs passed by value which
>     >> ??? are too
>     >> ??? >> big) cannot simply be worked around. But that's fine, we
>     can
>     >> still
>     >> ??? >> have a fast path for those calls which have certain
>     >> ??? characteristics
>     >> ??? >> and a slow path (through the universal adapter) for all the
>     >> ??? other calls.
>     >> ??? >>
>     >> ??? >> But there's a second and more serious issue lurking: as
>     you can
>     >> ??? see
>     >> ??? >> in the benchmark, I was not able to get the qsort benchmark
>     >> ??? running
>     >> ??? >> when using the linkToNative backend. The reason is that the
>     >> ??? >> linkToNative code is still pretty raw, and it doesn't fully
>     >> ??? adhere to
>     >> ??? >> the JVM internal conventions - e.g. there are missing
>     thread
>     >> state
>     >> ??? >> transitions which, in the case of upcalls into Java, create
>     >> issues
>     >> ??? >> when it comes to garbage collection, as the GC cannot
>     parse the
>     >> ??? >> native stack in the correct way.
>     >> ??? >>
>     >> ??? >> This means that, while there's a clear shining path ahead of
>     >> ??? us, it
>     >> ??? >> is simply too early to just use the linkToNative backend
>     from
>     >> ??? Panama.
>     >> ??? >> For this reason, I've been looking into some kind of stopgap
>     >> ??? solution
>     >> ??? >> - another way of optimizing native calls (and upcalls into
>     >> ??? Java) that
>     >> ??? >> doesn't require too much VM magic. Now, a crucial
>     observation is
>     >> ??? >> that, in many native calls, there is indeed a 1-1
>     mapping between
>     >> ??? >> Java arguments and native arguments (and back, for return
>     >> values).
>     >> ??? >> That is, we can think of calling a native function as a
>     process
>     >> ??? that
>     >> ??? >> takes a bunch of Java arguments, turn them into native
>     arguments
>     >> ??? >> (either double or longs), calls the native methods and
>     then turns
>     >> ??? >> back the result into Java.
>     >> ??? >>
>     >> ??? >> The mapping between Java arguments and C values is quite
>     simple:
>     >> ??? >>
>     >> ??? >> * primitives: either long or double, depending on
>     whether they
>     >> ??? >> describe an integral value or a floating point one.
>     >> ??? >> * pointers: they convert to a long
>     >> ??? >> * callbacks: they also convert to a long
>     >> ??? >> * structs: they are recursively decomposed into fields
>     and each
>     >> ??? field
>     >> ??? >> is marshalled separately (assuming the struct is not too
>     big, in
>     >> ??? >> which case is passed in memory)
>     >> ??? >>
>     >> ??? >> So, in principle, we could define a bunch of native entry
>     >> ??? points in
>     >> ??? >> the VM, one per shape, which take a bunch of long and
>     doubles and
>     >> ??? >> call an underlying function with those arguments. For
>     instance,
>     >> ??? let's
>     >> ??? >> consider the case of a native function which is modelled in
>     >> ??? Java as:
>     >> ??? >>
>     >> ??? >> int m(Pointer<Foo>, double)
>     >> ??? >>
>     >> ??? >> To call this native function we have to first turn the Java
>     >> ??? arguments
>     >> ??? >> into a (long, double) pair. Then we need to call a
>     native adapter
>     >> ??? >> that looks like the following:
>     >> ??? >>
>     >> ??? >> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused,
>     jlong
>     >> ??? addr,
>     >> ??? >> jlong arg0, jdouble arg1) {
>     >> ??? >> ???? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>     >> ??? >> }
>     >> ??? >>
>     >> ??? >> And this will take care of calling the native function and
>     >> ??? returning
>     >> ??? >> the value back. This is, admittedly, a very simple
>     solution; of
>     >> ??? >> course there are limitations: we have to define a bunch of
>     >> ??? >> specialized native entry point (and Java entry points, for
>     >> ??? >> callbacks). But here we can play a trick: most of
>     moderns ABI
>     >> pass
>     >> ??? >> arguments in registers; for instance System V ABI [5]
>     uses up
>     >> to 6
>     >> ??? >> (!!) integer registers and 7 (!!) MMXr registers for FP
>     values
>     >> ??? - this
>     >> ??? >> gives us a total of 13 registers available for argument
>     passing.
>     >> ??? >> Which covers quite a lot of cases. Now, if we have a
>     call where
>     >> ??? _all_
>     >> ??? >> arguments are passed in registers, then the order in
>     which these
>     >> ??? >> arguments are declared in the adapter doesn't matter!
>     That is,
>     >> ??? since
>     >> ??? >> FP-values will always be passed in different register from
>     >> ??? integral
>     >> ??? >> values, we can just define entry points which look like
>     these:
>     >> ??? >>
>     >> ??? >> invokeNative_V_DDDDD
>     >> ??? >> invokeNative_V_JDDDD
>     >> ??? >> invokeNative_V_JJDDD
>     >> ??? >> invokeNative_V_JJJDD
>     >> ??? >> invokeNative_V_JJJJD
>     >> ??? >> invokeNative_V_JJJJJ
>     >> ??? >>
>     >> ??? >> That is, for a given arity (5 in this case), we can just put
>     >> ??? all long
>     >> ??? >> arguments in front, and the double arguments after that.
>     That
>     >> ??? is, we
>     >> ??? >> don't need to generate all possible permutations of J/D
>     in all
>     >> ??? >> positions - as the adapter will always do the same thing
>     (read:
>     >> ??? load
>     >> ??? >> from same registers) for all equivalent combinations. This
>     >> ??? keeps the
>     >> ??? >> number of entry points in check - and it also poses some
>     >> ??? challenges
>     >> ??? >> to the Java logic in charge of marshalling/unmarshalling, as
>     >> ??? there's
>     >> ??? >> an extra permutation step involved (although that is not
>     >> something
>     >> ??? >> super-hard to address).
>     >> ??? >>
>     >> ??? >> You can see the performance numbers associated with this
>     >> ??? invocation
>     >> ??? >> scheme (which I've dubbed 'direct') in the 4th batch of the
>     >> ??? benchmark
>     >> ??? >> results. These numbers are on par (and slightly better) with
>     >> ??? JNI in
>     >> ??? >> all the three cases considered which is, I think, a very
>     positive
>     >> ??? >> result, given that to write these benchmarks I did not
>     have to
>     >> ??? write
>     >> ??? >> a single line of JNI code. In other words, this
>     optimization
>     >> gives
>     >> ??? >> you the same speed as JNI, with improved ease of use (**).
>     >> ??? >>
>     >> ??? >> Now, since the 'direct' optimization builds on top of the VM
>     >> ??? native
>     >> ??? >> call adapters, this approach is significantly more
>     robust than
>     >> ??? >> linkToNative and I have not run into any weird VM
>     crashes when
>     >> ??? >> playing with it. The downside of that, is that, for obvious
>     >> ??? reasons,
>     >> ??? >> this approach cannot get much faster than JNI - that is, it
>     >> cannot
>     >> ??? >> get close to the numbers obtained with the linkToNative
>     backend,
>     >> ??? >> which features much deeper optimizations. But I think that,
>     >> ??? despite
>     >> ??? >> its limitations, it's still a good opportunistic improvement
>     >> ??? that is
>     >> ??? >> worth pursuing in the short term (while we sort out the
>     >> ??? linkToNative
>     >> ??? >> story). For this reason, I will soon be submitting a
>     review which
>     >> ??? >> incorporates the changes for the 'direct' invocation
>     schemes.
>     >> ??? >>
>     >> ??? >> Cheers
>     >> ??? >> Maurizio
>     >> ??? >>
>     >> ??? >> [1] -
>     >> http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>     <http://cr.openjdk.java.net/%7Emcimadamore/panama/foreign-jmh.txt>
>     >> <http://cr.openjdk.java.net/%7Emcimadamore/panama/foreign-jmh.txt>
>     >> ??? >> [2] - https://github.com/jnr/jnr-ffi
>     >> ??? >> [3] - https://github.com/jnr/jffi
>     >> ??? >> [4] - https://sourceware.org/libffi/
>     >> ??? >> [5] -
>     >> ??? >>
>     >>
>     https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
>     >>
>     >> ??? >>
>     >> ??? >> [6] -
>     >> ??? >>
>     >>
>     http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html
>     <http://cr.openjdk.java.net/%7Ejrose/panama/native-call-primitive.html>
>     >>
>     <http://cr.openjdk.java.net/%7Ejrose/panama/native-call-primitive.html>
>     >> ??? >> [7] -
>     http://hg.openjdk.java.net/panama/dev/shortlog/b9ebb1bb8354
>     >> ??? >>
>     >> ??? >> (**) the benchmark also contains a 5th row in which I
>     repeated
>     >> ??? same
>     >> ??? >> tests, this time using JNR [2]. JNR is built on top of
>     libjffi
>     >> ??? [3], a
>     >> ??? >> JNI library in turn built on top of the popular libffi
>     [4]. I
>     >> ??? wanted
>     >> ??? >> to have some numbers about JNR because that's another
>     solution
>     >> ??? that
>     >> ??? >> allows for better ease to use, taking care of
>     marshalling Java
>     >> ??? values
>     >> ??? >> into C and back; since the goals of JNR are similar in
>     spirit
>     >> with
>     >> ??? >> some of the goals of the Panama/foreign work, I thought it
>     >> ??? would be
>     >> ??? >> worth having a comparison of these approaches. For the
>     records, I
>     >> ??? >> think the JNR numbers are very respectable given that
>     JNR had
>     >> ??? to do
>     >> ??? >> all the hard work outside of the JDK!
>     >> ??? >>
>     >> ??? >>
>     >> ??? >
>     >>
>     >
>


From maurizio.cimadamore at oracle.com  Tue Sep 18 10:03:46 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 18 Sep 2018 11:03:46 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <25be0579-eb55-65e7-b379-206b8713a687@gmail.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
 <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
 <25be0579-eb55-65e7-b379-206b8713a687@gmail.com>
Message-ID: <d6a908bb-7a31-5e6a-37b8-106149ffac29@oracle.com>

These are the numbers I get

Benchmark???????????????????????? Mode? Cnt???????? Score Error? Units
NativeBenchmark.expBenchmark???? thrpt??? 5? 30542590.094 ? 44126.434? ops/s
NativeBenchmark.getpidBenchmark? thrpt??? 5? 61764677.092 ? 21102.236? ops/s

They are in the same ballpark, but exp() is a bit faster; byw, I tried 
to repeat my benchmark with JNI exp() _and_ O3 and I've got very similar 
numbers (yesterday I did a very quick test and there was probably some 
other job running on the machine and brining down the figures a bit).

But overall, the results in your bench seem to match what I got: exp is 
faster, pid is slower, the difference is mostly caused by O3. If no O3 
is used, then the numbers should match what I included in my numbers 
(and getpid should be a bit faster).

Maurizio


On 18/09/18 05:48, Samuel Audet wrote:
> Anyway, I've put online an updated version of my benchmark files here:
> https://gist.github.com/saudet/1bf14a000e64c245675cf5d4e9ad6e69
> Just run "git clone" on the URL and run "mvn package" on the pom.xml.
>
> With the 2 virtual cores of an Intel(R) Xeon(R) CPU E5-2673 v4 @ 
> 2.30GHz running Ubuntu 14.04 on the cloud with GCC 4.9 and OpenJDK 8, 
> I get these numbers:
>
> Benchmark???????????????????????? Mode? Cnt????????? Score Error ?Units
> NativeBenchmark.expBenchmark???? thrpt?? 25?? 37460540.440 ? 
> 393299.974 ?ops/s
> NativeBenchmark.getpidBenchmark? thrpt?? 25? 100323188.451 ? 
> 1254197.449 ?ops/s
>
> While on my laptop, an Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz 
> running Fedora 27, GCC 7.3, and OpenJDK 9, I get the following:
>
> Benchmark???????????????????????? Mode? Cnt???????? Score Error Units
> NativeBenchmark.expBenchmark???? thrpt?? 25? 50047147.099 ? 924366.937 
> ops/s
> NativeBenchmark.getpidBenchmark? thrpt?? 25?? 4825508.193 ? 21662.633 
> ops/s
>
> Now, it looks like getpid() is really slow on Fedora 27 for some 
> reason, but as Linus puts it, we should not be using that for 
> benchmarking:
> https://yarchive.net/comp/linux/getpid_caching.html
>
> What do you get on your machines?
>
> Samuel
>
>
> On 09/18/2018 12:58 AM, Maurizio Cimadamore wrote:
>> For the records, here's what I get for all the three benchmarks if I 
>> compile the JNI code with -O3:
>>
>> Benchmark????????????????????????? Mode? Cnt Score Error? Units
>> PanamaBenchmark.testJNIExp??????? thrpt??? 5? 28575269.294 ? 
>> 1907726.710? ops/s
>> PanamaBenchmark.testJNIJavaQsort? thrpt??? 5??? 372148.433 ? 
>> 27178.529? ops/s
>> PanamaBenchmark.testJNIPid??????? thrpt??? 5? 59240069.011 ? 
>> 403881.697? ops/s
>>
>> The first and second benchmarks get faster and very close to the 
>> 'direct' optimization numbers in [1]. Surprisingly, the last 
>> benchmark (getpid) is quite slower. I've been able to reproduce 
>> across multiple runs; for that benchmark omitting O3 seems to be the 
>> achieve best results, not sure why. It starts of faster (around in 
>> the first couple of warmup iterations, but then it goes slower in all 
>> the other runs - presumably it interacts badly with the C2 generated 
>> code. For instance, this is a run with O3 enabled:
>>
>> # Run progress: 66.67% complete, ETA 00:01:40
>> # Fork: 1 of 1
>> # Warmup Iteration?? 1: 65182202.653 ops/s
>> # Warmup Iteration?? 2: 64900639.094 ops/s
>> # Warmup Iteration?? 3: 59314945.437 ops/s 
>> <---------------------------------
>> # Warmup Iteration?? 4: 59269007.877 ops/s
>> # Warmup Iteration?? 5: 59239905.163 ops/s
>> Iteration?? 1: 59300748.074 ops/s
>> Iteration?? 2: 59249666.044 ops/s
>> Iteration?? 3: 59268597.051 ops/s
>> Iteration?? 4: 59322074.572 ops/s
>> Iteration?? 5: 59059259.317 ops/s
>>
>> And this is a run with O3 disabled:
>>
>> # Run progress: 0.00% complete, ETA 00:01:40
>> # Fork: 1 of 1
>> # Warmup Iteration?? 1: 55882128.787 ops/s
>> # Warmup Iteration?? 2: 53102361.751 ops/s
>> # Warmup Iteration?? 3: 66964755.699 ops/s 
>> <---------------------------------
>> # Warmup Iteration?? 4: 66414428.355 ops/s
>> # Warmup Iteration?? 5: 65328475.276 ops/s
>> Iteration?? 1: 64229192.993 ops/s
>> Iteration?? 2: 65191719.319 ops/s
>> Iteration?? 3: 65352022.471 ops/s
>> Iteration?? 4: 65152090.426 ops/s
>> Iteration?? 5: 65320545.712 ops/s
>>
>>
>> In both cases, the 3rd warmup execution sees a performance jump - 
>> with O3, the jump is backwards, w/o O3 the jump is forward, which is 
>> quite typical for a JMH benchmark as C2 optimization will start to 
>> kick in.
>>
>> For these reasons, I'm reluctant to update my benchmark numbers to 
>> reflect the O3 behavior (although I agree that, since the Hotspot 
>> code is compiled with that optimization it would make more sense to 
>> use that as a reference).
>>
>> Maurizio
>>
>> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>>
>>
>>
>> On 17/09/18 16:18, Maurizio Cimadamore wrote:
>>>
>>>
>>> On 17/09/18 15:08, Samuel Audet wrote:
>>>> Yes, the blackhole or the random number doesn't make any 
>>>> difference, but not calling gcc with -O3 does. Running the compiler 
>>>> with optimizations on is pretty common, but they are not enabled by 
>>>> default.
>>> A bit better
>>>
>>> PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ? 8491668.248 
>>> ops/s
>>>
>>> But not much of a difference (I did not expected much, as the body 
>>> of the native method is extremely simple).
>>>
>>> Maurizio 
>


From sundararajan.athijegannathan at oracle.com  Tue Sep 18 15:35:30 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Tue, 18 Sep 2018 21:05:30 +0530
Subject: [foreign] RFR 8210860 Tree visitors should be used to handle empty
 names, to do symbol filtering and to do typedef name propagation
Message-ID: <5BA11B42.5060402@oracle.com>

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8210860
Webrev: http://cr.openjdk.java.net/~sundar/8210860/webrev.00/

Thanks,
-Sundar

From maurizio.cimadamore at oracle.com  Tue Sep 18 16:22:49 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 18 Sep 2018 17:22:49 +0100
Subject: [foreign] RFR 8210860 Tree visitors should be used to handle
 empty names, to do symbol filtering and to do typedef name propagation
In-Reply-To: <5BA11B42.5060402@oracle.com>
References: <5BA11B42.5060402@oracle.com>
Message-ID: <2380bcdd-7090-0685-c8eb-ccbaee8ab624@oracle.com>

Looks really good,
the tree-based approach is starting to shine here, as we can clearly see 
the kind of manipulation that were already taking place in the code, but 
were previously scattered all over the place; now they are dealt with in 
a 50 LoC tree pass; nice!

A bit unfortunate that typedefs are handled so poorly by the clang API, 
so much that we need to build a dictionary of replacements, but as we 
have discussed, not much can be done about that.

Moving forward, I'd like to see those main methods turned into real 
tests: if we added ways to 'dump the tree after step XYZ', we could 
easily write unit tests which compare the output of jextract against a 
golden file.

Maurizio


On 18/09/18 16:35, Sundararajan Athijegannathan wrote:
> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8210860
> Webrev: http://cr.openjdk.java.net/~sundar/8210860/webrev.00/
>
> Thanks,
> -Sundar


From sundararajan.athijegannathan at oracle.com  Tue Sep 18 16:55:45 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Tue, 18 Sep 2018 22:25:45 +0530
Subject: [foreign] RFR 8210860 Tree visitors should be used to handle
 empty names, to do symbol filtering and to do typedef name propagation
In-Reply-To: <2380bcdd-7090-0685-c8eb-ccbaee8ab624@oracle.com>
References: <5BA11B42.5060402@oracle.com>
 <2380bcdd-7090-0685-c8eb-ccbaee8ab624@oracle.com>
Message-ID: <5BA12E11.8040805@oracle.com>

Thanks for your review Maurizio. I'll add tests for individual passes in 
a later changeset.

-Sundar


On 18/09/18, 9:52 PM, Maurizio Cimadamore wrote:
> Looks really good,
> the tree-based approach is starting to shine here, as we can clearly 
> see the kind of manipulation that were already taking place in the 
> code, but were previously scattered all over the place; now they are 
> dealt with in a 50 LoC tree pass; nice!
>
> A bit unfortunate that typedefs are handled so poorly by the clang 
> API, so much that we need to build a dictionary of replacements, but 
> as we have discussed, not much can be done about that.
>
> Moving forward, I'd like to see those main methods turned into real 
> tests: if we added ways to 'dump the tree after step XYZ', we could 
> easily write unit tests which compare the output of jextract against a 
> golden file.
>
> Maurizio
>
>
> On 18/09/18 16:35, Sundararajan Athijegannathan wrote:
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210860
>> Webrev: http://cr.openjdk.java.net/~sundar/8210860/webrev.00/
>>
>> Thanks,
>> -Sundar
>

From sundararajan.athijegannathan at oracle.com  Tue Sep 18 17:09:45 2018
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Tue, 18 Sep 2018 17:09:45 +0000
Subject: hg: panama/dev: 8210860: Tree visitors should be used to handle empty
 names, to do symbol filtering and to do typedef name propagation
Message-ID: <201809181709.w8IH9jp2021462@aojmv0008.oracle.com>

Changeset: d2d130f4b312
Author:    sundar
Date:      2018-09-18 22:47 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/d2d130f4b312

8210860: Tree visitors should be used to handle empty names, to do symbol filtering and to do typedef name propagation
Reviewed-by: mcimadamore

! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/Context.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/EmptyNameHandler.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/HeaderFile.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/TreeFilter.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/TypedefHandler.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/Utils.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/FindSymbol.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/Parser.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/EnumTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/FieldTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/FunctionTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/HeaderTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/MacroTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/StructTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/Tree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreeMaker.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreePrinter.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TypedefTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/VarTree.java


From maurizio.cimadamore at oracle.com  Tue Sep 18 17:11:48 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Tue, 18 Sep 2018 17:11:48 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809181711.w8IHBmGU023045@aojmv0008.oracle.com>

Changeset: cd31afbf3e78
Author:    mcimadamore
Date:      2018-09-18 19:14 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/cd31afbf3e78

Automatic merge with foreign



From shravya.rukmannagari at intel.com  Tue Sep 18 21:50:50 2018
From: shravya.rukmannagari at intel.com (shravya.rukmannagari at intel.com)
Date: Tue, 18 Sep 2018 21:50:50 +0000
Subject: hg: panama/dev: Build Fix for get
Message-ID: <201809182150.w8ILooiH011775@aojmv0008.oracle.com>

Changeset: ffe207cf3fd0
Author:    srukmannagar
Date:      2018-09-18 14:48 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/ffe207cf3fd0

Build Fix for get

! src/hotspot/cpu/x86/x86.ad


From maurizio.cimadamore at oracle.com  Tue Sep 18 21:56:50 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Tue, 18 Sep 2018 21:56:50 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809182156.w8ILupWi014746@aojmv0008.oracle.com>

Changeset: e98cc9487657
Author:    mcimadamore
Date:      2018-09-18 23:59 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/e98cc9487657

Automatic merge with vectorIntrinsics



From samuel.audet at gmail.com  Wed Sep 19 00:13:27 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Wed, 19 Sep 2018 09:13:27 +0900
Subject: [foreign] some JMH benchmarks
In-Reply-To: <d6a908bb-7a31-5e6a-37b8-106149ffac29@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
 <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
 <25be0579-eb55-65e7-b379-206b8713a687@gmail.com>
 <d6a908bb-7a31-5e6a-37b8-106149ffac29@oracle.com>
Message-ID: <580516c9-0ef2-0ceb-7c8c-c67d6621c074@gmail.com>

Thanks! You haven't mentioned the version of the JDK you're using 
though. I'm starting to get the impression that JNI in newer versions of 
OpenJDK will be slower... ?

On 09/18/2018 07:03 PM, Maurizio Cimadamore wrote:
> These are the numbers I get
> 
> Benchmark???????????????????????? Mode? Cnt???????? Score Error? Units
> NativeBenchmark.expBenchmark???? thrpt??? 5? 30542590.094 ? 44126.434  
> ops/s
> NativeBenchmark.getpidBenchmark? thrpt??? 5? 61764677.092 ? 21102.236  
> ops/s
> 
> They are in the same ballpark, but exp() is a bit faster; byw, I tried 
> to repeat my benchmark with JNI exp() _and_ O3 and I've got very similar 
> numbers (yesterday I did a very quick test and there was probably some 
> other job running on the machine and brining down the figures a bit).
> 
> But overall, the results in your bench seem to match what I got: exp is 
> faster, pid is slower, the difference is mostly caused by O3. If no O3 
> is used, then the numbers should match what I included in my numbers 
> (and getpid should be a bit faster).
> 
> Maurizio
> 
> 
> On 18/09/18 05:48, Samuel Audet wrote:
>> Anyway, I've put online an updated version of my benchmark files here:
>> https://gist.github.com/saudet/1bf14a000e64c245675cf5d4e9ad6e69
>> Just run "git clone" on the URL and run "mvn package" on the pom.xml.
>>
>> With the 2 virtual cores of an Intel(R) Xeon(R) CPU E5-2673 v4 @ 
>> 2.30GHz running Ubuntu 14.04 on the cloud with GCC 4.9 and OpenJDK 8, 
>> I get these numbers:
>>
>> Benchmark???????????????????????? Mode? Cnt????????? Score Error ?Units
>> NativeBenchmark.expBenchmark???? thrpt?? 25?? 37460540.440 ? 
>> 393299.974 ?ops/s
>> NativeBenchmark.getpidBenchmark? thrpt?? 25? 100323188.451 ? 
>> 1254197.449 ?ops/s
>>
>> While on my laptop, an Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz 
>> running Fedora 27, GCC 7.3, and OpenJDK 9, I get the following:
>>
>> Benchmark???????????????????????? Mode? Cnt???????? Score Error Units
>> NativeBenchmark.expBenchmark???? thrpt?? 25? 50047147.099 ? 924366.937 
>> ops/s
>> NativeBenchmark.getpidBenchmark? thrpt?? 25?? 4825508.193 ? 21662.633 
>> ops/s
>>
>> Now, it looks like getpid() is really slow on Fedora 27 for some 
>> reason, but as Linus puts it, we should not be using that for 
>> benchmarking:
>> https://yarchive.net/comp/linux/getpid_caching.html
>>
>> What do you get on your machines?
>>
>> Samuel
>>
>>
>> On 09/18/2018 12:58 AM, Maurizio Cimadamore wrote:
>>> For the records, here's what I get for all the three benchmarks if I 
>>> compile the JNI code with -O3:
>>>
>>> Benchmark????????????????????????? Mode? Cnt Score Error? Units
>>> PanamaBenchmark.testJNIExp??????? thrpt??? 5? 28575269.294 ? 
>>> 1907726.710? ops/s
>>> PanamaBenchmark.testJNIJavaQsort? thrpt??? 5??? 372148.433 ? 
>>> 27178.529? ops/s
>>> PanamaBenchmark.testJNIPid??????? thrpt??? 5? 59240069.011 ? 
>>> 403881.697? ops/s
>>>
>>> The first and second benchmarks get faster and very close to the 
>>> 'direct' optimization numbers in [1]. Surprisingly, the last 
>>> benchmark (getpid) is quite slower. I've been able to reproduce 
>>> across multiple runs; for that benchmark omitting O3 seems to be the 
>>> achieve best results, not sure why. It starts of faster (around in 
>>> the first couple of warmup iterations, but then it goes slower in all 
>>> the other runs - presumably it interacts badly with the C2 generated 
>>> code. For instance, this is a run with O3 enabled:
>>>
>>> # Run progress: 66.67% complete, ETA 00:01:40
>>> # Fork: 1 of 1
>>> # Warmup Iteration?? 1: 65182202.653 ops/s
>>> # Warmup Iteration?? 2: 64900639.094 ops/s
>>> # Warmup Iteration?? 3: 59314945.437 ops/s 
>>> <---------------------------------
>>> # Warmup Iteration?? 4: 59269007.877 ops/s
>>> # Warmup Iteration?? 5: 59239905.163 ops/s
>>> Iteration?? 1: 59300748.074 ops/s
>>> Iteration?? 2: 59249666.044 ops/s
>>> Iteration?? 3: 59268597.051 ops/s
>>> Iteration?? 4: 59322074.572 ops/s
>>> Iteration?? 5: 59059259.317 ops/s
>>>
>>> And this is a run with O3 disabled:
>>>
>>> # Run progress: 0.00% complete, ETA 00:01:40
>>> # Fork: 1 of 1
>>> # Warmup Iteration?? 1: 55882128.787 ops/s
>>> # Warmup Iteration?? 2: 53102361.751 ops/s
>>> # Warmup Iteration?? 3: 66964755.699 ops/s 
>>> <---------------------------------
>>> # Warmup Iteration?? 4: 66414428.355 ops/s
>>> # Warmup Iteration?? 5: 65328475.276 ops/s
>>> Iteration?? 1: 64229192.993 ops/s
>>> Iteration?? 2: 65191719.319 ops/s
>>> Iteration?? 3: 65352022.471 ops/s
>>> Iteration?? 4: 65152090.426 ops/s
>>> Iteration?? 5: 65320545.712 ops/s
>>>
>>>
>>> In both cases, the 3rd warmup execution sees a performance jump - 
>>> with O3, the jump is backwards, w/o O3 the jump is forward, which is 
>>> quite typical for a JMH benchmark as C2 optimization will start to 
>>> kick in.
>>>
>>> For these reasons, I'm reluctant to update my benchmark numbers to 
>>> reflect the O3 behavior (although I agree that, since the Hotspot 
>>> code is compiled with that optimization it would make more sense to 
>>> use that as a reference).
>>>
>>> Maurizio
>>>
>>> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>>>
>>>
>>>
>>> On 17/09/18 16:18, Maurizio Cimadamore wrote:
>>>>
>>>>
>>>> On 17/09/18 15:08, Samuel Audet wrote:
>>>>> Yes, the blackhole or the random number doesn't make any 
>>>>> difference, but not calling gcc with -O3 does. Running the compiler 
>>>>> with optimizations on is pretty common, but they are not enabled by 
>>>>> default.
>>>> A bit better
>>>>
>>>> PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ? 8491668.248 
>>>> ops/s
>>>>
>>>> But not much of a difference (I did not expected much, as the body 
>>>> of the native method is extremely simple).
>>>>
>>>> Maurizio 
>>
> 


From jbvernee at xs4all.nl  Wed Sep 19 12:00:38 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 19 Sep 2018 14:00:38 +0200
Subject: [foreign] Running tests on Windows
Message-ID: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>

Hello guys,

I'm making good progress with getting things to work on windows and now 
have mixed arguments and float varargs working.

I'm currently running the jdk_foreign tests (of which 31 are being run) 
to see which ones are failing. I'm getting currently 14 out of 31.

One of the problems I'm running into is that the tests depend on POSIX 
functions, which are not in the C standard library on windows. The main 
culprit being getpid (although it is in the C++ standard library under 
the mangled name _getpid, but name mangling in C++ is non-portable 
either). I'm wondering how you'd like to handle these? I don't know if 
there is a way to make platform specific tests?

There is also a use of alloca, for which there is a replacement in C99, 
but I don't know which C and C++ standards the jdk is on? (iirc C++ was 
before std11, which is pretty old). I wanted to change alloca to using a 
variable sized array, but the compiler is rejecting that. (currently 
using _alloca to test with)

I was seeing the warning 'Not building java.foreign tests' coming from 
make/test/native/JtregNativeJdk.gmk, so I've added a filter and flags 
for windows as well. But, it doesn't look like these filter were doing 
anything in the first place, since `make test` was already outputting a 
bunch of dlls, just through some other path it seems. I'm seeing a lot 
of warnings like

     JtregNativeJdk.gmk:96: warning: ignoring old recipe for target 
'/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll'

So I don't know if that filter is still relevant? What is it actually 
(supposed to be) doing?

Another thing with the native test files is that dlls require you to 
explicitly export symbols. There are 4 different ways to do this, either 
through a command line option, a side file, using a decorator 
`__declspec(dllexport)`, or using a #pragma comment. I wonder if there 
is an established way of doing that for tests that you know of?

And then of course one of the biggest things is getting jextract to 
build and run. Currently the build system is not picking up libclang. 
When explicitly specifying it with --with-libclang='/cygdrive/j/LLVM' I 
still get the "Cannot locate libclang!" warning. I wanted to ask what 
value and files it's actually looking for? (I've also tried some of the 
additional flags from makeautoconf/lib-clang.m4, but to no avail).

Thanks,
Jorn



From sundararajan.athijegannathan at oracle.com  Wed Sep 19 14:13:54 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Wed, 19 Sep 2018 19:43:54 +0530
Subject: [foreign] RFR 8210911 jextract does not handle redundant forward,
 backward declarations of struct, union, enum properly
Message-ID: <5BA259A2.5050106@oracle.com>

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8210911
Webrev: http://cr.openjdk.java.net/~sundar/8210911/webrev.00/

PS. Piggybacking to add a TreePhase interface for visitors.

Thanks
-Sundar

From henry.jen at oracle.com  Wed Sep 19 15:23:37 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 19 Sep 2018 08:23:37 -0700
Subject: [foreign] Running tests on Windows
In-Reply-To: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
Message-ID: <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>

On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
> 
> Hello guys,
> 
> I'm making good progress with getting things to work on windows and now have mixed arguments and float varargs working.
> 

Nice, do you have a reference to Win32 ABI/calling convention?

> I'm currently running the jdk_foreign tests (of which 31 are being run) to see which ones are failing. I'm getting currently 14 out of 31.
> 
> One of the problems I'm running into is that the tests depend on POSIX functions, which are not in the C standard library on windows. The main culprit being getpid (although it is in the C++ standard library under the mangled name _getpid, but name mangling in C++ is non-portable either). I'm wondering how you'd like to handle these? I don't know if there is a way to make platform specific tests?
> 

Use jtreg @requires tag, something like @requires (os.family == "windows")

> There is also a use of alloca, for which there is a replacement in C99, but I don't know which C and C++ standards the jdk is on? (iirc C++ was before std11, which is pretty old). I wanted to change alloca to using a variable sized array, but the compiler is rejecting that. (currently using _alloca to test with)
> 
> I was seeing the warning 'Not building java.foreign tests' coming from make/test/native/JtregNativeJdk.gmk, so I've added a filter and flags for windows as well. But, it doesn't look like these filter were doing anything in the first place, since `make test` was already outputting a bunch of dlls, just through some other path it seems. I'm seeing a lot of warnings like
> 
>    JtregNativeJdk.gmk:96: warning: ignoring old recipe for target '/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll'
> 
> So I don't know if that filter is still relevant? What is it actually (supposed to be) doing?
> 
> Another thing with the native test files is that dlls require you to explicitly export symbols. There are 4 different ways to do this, either through a command line option, a side file, using a decorator `__declspec(dllexport)`, or using a #pragma comment. I wonder if there is an established way of doing that for tests that you know of?
> 

We haven?t. But a common practice is use #ifdef to make a decoration portable, and that would serve as a test case for jextract as well. I would try to avoid mess with build system unless there is an established way to do what we needed.

> And then of course one of the biggest things is getting jextract to build and run. Currently the build system is not picking up libclang. When explicitly specifying it with --with-libclang='/cygdrive/j/LLVM' I still get the "Cannot locate libclang!" warning. I wanted to ask what value and files it's actually looking for? (I've also tried some of the additional flags from makeautoconf/lib-clang.m4, but to no avail).
> 

?with-libclang should point to a directory has include/clang-c and lib/libclang.dll. If include and lib are not under the same directory, you can use ?with-libclang-include and ?with-libclang-lib instead.

HTH,
Henry


From bob.vandette at oracle.com  Wed Sep 19 15:25:50 2018
From: bob.vandette at oracle.com (Bob Vandette)
Date: Wed, 19 Sep 2018 11:25:50 -0400
Subject: [foreign] Running tests on Windows
In-Reply-To: <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
Message-ID: <62C8C518-F877-46C5-938B-A3B562354B9A@oracle.com>


> On Sep 19, 2018, at 11:23 AM, Henry Jen <henry.jen at oracle.com> wrote:
> 
> On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>> 
>> Hello guys,
>> 
>> I'm making good progress with getting things to work on windows and now have mixed arguments and float varargs working.
>> 
> 
> Nice, do you have a reference to Win32 ABI/calling convention?

I used this as reference for the SVM Windows calling convention work I just completed.

https://msdn.microsoft.com/en-us/library/ms235286.aspx

Bob.

> 
>> I'm currently running the jdk_foreign tests (of which 31 are being run) to see which ones are failing. I'm getting currently 14 out of 31.
>> 
>> One of the problems I'm running into is that the tests depend on POSIX functions, which are not in the C standard library on windows. The main culprit being getpid (although it is in the C++ standard library under the mangled name _getpid, but name mangling in C++ is non-portable either). I'm wondering how you'd like to handle these? I don't know if there is a way to make platform specific tests?
>> 
> 
> Use jtreg @requires tag, something like @requires (os.family == "windows")
> 
>> There is also a use of alloca, for which there is a replacement in C99, but I don't know which C and C++ standards the jdk is on? (iirc C++ was before std11, which is pretty old). I wanted to change alloca to using a variable sized array, but the compiler is rejecting that. (currently using _alloca to test with)
>> 
>> I was seeing the warning 'Not building java.foreign tests' coming from make/test/native/JtregNativeJdk.gmk, so I've added a filter and flags for windows as well. But, it doesn't look like these filter were doing anything in the first place, since `make test` was already outputting a bunch of dlls, just through some other path it seems. I'm seeing a lot of warnings like
>> 
>>   JtregNativeJdk.gmk:96: warning: ignoring old recipe for target '/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll'
>> 
>> So I don't know if that filter is still relevant? What is it actually (supposed to be) doing?
>> 
>> Another thing with the native test files is that dlls require you to explicitly export symbols. There are 4 different ways to do this, either through a command line option, a side file, using a decorator `__declspec(dllexport)`, or using a #pragma comment. I wonder if there is an established way of doing that for tests that you know of?
>> 
> 
> We haven?t. But a common practice is use #ifdef to make a decoration portable, and that would serve as a test case for jextract as well. I would try to avoid mess with build system unless there is an established way to do what we needed.
> 
>> And then of course one of the biggest things is getting jextract to build and run. Currently the build system is not picking up libclang. When explicitly specifying it with --with-libclang='/cygdrive/j/LLVM' I still get the "Cannot locate libclang!" warning. I wanted to ask what value and files it's actually looking for? (I've also tried some of the additional flags from makeautoconf/lib-clang.m4, but to no avail).
>> 
> 
> ?with-libclang should point to a directory has include/clang-c and lib/libclang.dll. If include and lib are not under the same directory, you can use ?with-libclang-include and ?with-libclang-lib instead.
> 
> HTH,
> Henry
> 


From henry.jen at oracle.com  Wed Sep 19 15:44:42 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 19 Sep 2018 08:44:42 -0700
Subject: [foreign] RFR 8210911 jextract does not handle redundant forward, 
 backward declarations of struct, union, enum properly
In-Reply-To: <5BA259A2.5050106@oracle.com>
References: <5BA259A2.5050106@oracle.com>
Message-ID: <A035092A-4E11-4039-ADCD-958B9A4A0017@oracle.com>

Change looks reasonable to me.

Cheers,
Henry


> On Sep 19, 2018, at 7:13 AM, Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com> wrote:
> 
> Please review.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8210911
> Webrev: http://cr.openjdk.java.net/~sundar/8210911/webrev.00/
> 
> PS. Piggybacking to add a TreePhase interface for visitors.
> 
> Thanks
> -Sundar


From henry.jen at oracle.com  Wed Sep 19 15:45:47 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 19 Sep 2018 08:45:47 -0700
Subject: [foreign] Running tests on Windows
In-Reply-To: <62C8C518-F877-46C5-938B-A3B562354B9A@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <62C8C518-F877-46C5-938B-A3B562354B9A@oracle.com>
Message-ID: <0CF29DED-EBEC-4B39-967F-C54C874335AA@oracle.com>

Thanks for the pointer, Bob.

Cheers,
Henry


> On Sep 19, 2018, at 8:25 AM, Bob Vandette <bob.vandette at oracle.com> wrote:
> 
> 
>> On Sep 19, 2018, at 11:23 AM, Henry Jen <henry.jen at oracle.com> wrote:
>> 
>> On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>>> 
>>> Hello guys,
>>> 
>>> I'm making good progress with getting things to work on windows and now have mixed arguments and float varargs working.
>>> 
>> 
>> Nice, do you have a reference to Win32 ABI/calling convention?
> 
> I used this as reference for the SVM Windows calling convention work I just completed.
> 
> https://msdn.microsoft.com/en-us/library/ms235286.aspx
> 
> Bob.
> 
>> 
>>> I'm currently running the jdk_foreign tests (of which 31 are being run) to see which ones are failing. I'm getting currently 14 out of 31.
>>> 
>>> One of the problems I'm running into is that the tests depend on POSIX functions, which are not in the C standard library on windows. The main culprit being getpid (although it is in the C++ standard library under the mangled name _getpid, but name mangling in C++ is non-portable either). I'm wondering how you'd like to handle these? I don't know if there is a way to make platform specific tests?
>>> 
>> 
>> Use jtreg @requires tag, something like @requires (os.family == "windows")
>> 
>>> There is also a use of alloca, for which there is a replacement in C99, but I don't know which C and C++ standards the jdk is on? (iirc C++ was before std11, which is pretty old). I wanted to change alloca to using a variable sized array, but the compiler is rejecting that. (currently using _alloca to test with)
>>> 
>>> I was seeing the warning 'Not building java.foreign tests' coming from make/test/native/JtregNativeJdk.gmk, so I've added a filter and flags for windows as well. But, it doesn't look like these filter were doing anything in the first place, since `make test` was already outputting a bunch of dlls, just through some other path it seems. I'm seeing a lot of warnings like
>>> 
>>>  JtregNativeJdk.gmk:96: warning: ignoring old recipe for target '/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll'
>>> 
>>> So I don't know if that filter is still relevant? What is it actually (supposed to be) doing?
>>> 
>>> Another thing with the native test files is that dlls require you to explicitly export symbols. There are 4 different ways to do this, either through a command line option, a side file, using a decorator `__declspec(dllexport)`, or using a #pragma comment. I wonder if there is an established way of doing that for tests that you know of?
>>> 
>> 
>> We haven?t. But a common practice is use #ifdef to make a decoration portable, and that would serve as a test case for jextract as well. I would try to avoid mess with build system unless there is an established way to do what we needed.
>> 
>>> And then of course one of the biggest things is getting jextract to build and run. Currently the build system is not picking up libclang. When explicitly specifying it with --with-libclang='/cygdrive/j/LLVM' I still get the "Cannot locate libclang!" warning. I wanted to ask what value and files it's actually looking for? (I've also tried some of the additional flags from makeautoconf/lib-clang.m4, but to no avail).
>>> 
>> 
>> ?with-libclang should point to a directory has include/clang-c and lib/libclang.dll. If include and lib are not under the same directory, you can use ?with-libclang-include and ?with-libclang-lib instead.
>> 
>> HTH,
>> Henry
>> 
> 


From sundararajan.athijegannathan at oracle.com  Wed Sep 19 16:03:40 2018
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Wed, 19 Sep 2018 16:03:40 +0000
Subject: hg: panama/dev: 8210911: jextract does not handle redundant forward, 
 backward declarations of struct, union, enum properly
Message-ID: <201809191603.w8JG3flv008330@aojmv0008.oracle.com>

Changeset: 690e0bdeaddf
Author:    sundar
Date:      2018-09-19 21:41 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/690e0bdeaddf

8210911: jextract does not handle redundant forward, backward declarations of struct, union, enum properly
Reviewed-by: henryjen

! src/jdk.jextract/share/classes/com/sun/tools/jextract/Context.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/EmptyNameHandler.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/TreeFilter.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/TypedefHandler.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/EnumTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/StructTree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/Tree.java
! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreeMaker.java
+ src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreePhase.java
+ test/jdk/com/sun/tools/jextract/RedundantDeclsTest.java
+ test/jdk/com/sun/tools/jextract/redundantDecls.h


From maurizio.cimadamore at oracle.com  Wed Sep 19 16:06:55 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 19 Sep 2018 16:06:55 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809191607.w8JG70aX010064@aojmv0008.oracle.com>

Changeset: 05e8e49ff9af
Author:    mcimadamore
Date:      2018-09-19 18:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/05e8e49ff9af

Automatic merge with foreign



From jbvernee at xs4all.nl  Wed Sep 19 16:07:00 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 19 Sep 2018 18:07:00 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
Message-ID: <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>

Henry Jen schreef op 2018-09-19 17:23:
> On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>> 
>> Hello guys,
>> 
>> I'm making good progress with getting things to work on windows and 
>> now have mixed arguments and float varargs working.
>> 
> 
> Nice, do you have a reference to Win32 ABI/calling convention?

Yeah, I'm using the same one Bob is, but the one for VS2017 [1], though 
I think that only covers x64, not 32 bit stuff. There are no more Oracle 
builds of 32bit jdks but I'm not sure what the OpenJDK consensus is for 
supporting 32 bits? It certainly makes things easier to only have to 
deal with 1 calling convention.

>> I'm currently running the jdk_foreign tests (of which 31 are being 
>> run) to see which ones are failing. I'm getting currently 14 out of 
>> 31.
>> 
>> One of the problems I'm running into is that the tests depend on POSIX 
>> functions, which are not in the C standard library on windows. The 
>> main culprit being getpid (although it is in the C++ standard library 
>> under the mangled name _getpid, but name mangling in C++ is 
>> non-portable either). I'm wondering how you'd like to handle these? I 
>> don't know if there is a way to make platform specific tests?
>> 
> 
> Use jtreg @requires tag, something like @requires (os.family == 
> "windows")

Thanks!

>> There is also a use of alloca, for which there is a replacement in 
>> C99, but I don't know which C and C++ standards the jdk is on? (iirc 
>> C++ was before std11, which is pretty old). I wanted to change alloca 
>> to using a variable sized array, but the compiler is rejecting that. 
>> (currently using _alloca to test with)
>> 
>> I was seeing the warning 'Not building java.foreign tests' coming from 
>> make/test/native/JtregNativeJdk.gmk, so I've added a filter and flags 
>> for windows as well. But, it doesn't look like these filter were doing 
>> anything in the first place, since `make test` was already outputting 
>> a bunch of dlls, just through some other path it seems. I'm seeing a 
>> lot of warnings like
>> 
>>    JtregNativeJdk.gmk:96: warning: ignoring old recipe for target 
>> '/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll'
>> 
>> So I don't know if that filter is still relevant? What is it actually 
>> (supposed to be) doing?
>> 
>> Another thing with the native test files is that dlls require you to 
>> explicitly export symbols. There are 4 different ways to do this, 
>> either through a command line option, a side file, using a decorator 
>> `__declspec(dllexport)`, or using a #pragma comment. I wonder if there 
>> is an established way of doing that for tests that you know of?
>> 
> 
> We haven?t. But a common practice is use #ifdef to make a decoration
> portable, and that would serve as a test case for jextract as well. I
> would try to avoid mess with build system unless there is an
> established way to do what we needed.

That should work, I think I can just copy what JNI is doing with 
JNIEXPORT.

>> And then of course one of the biggest things is getting jextract to 
>> build and run. Currently the build system is not picking up libclang. 
>> When explicitly specifying it with --with-libclang='/cygdrive/j/LLVM' 
>> I still get the "Cannot locate libclang!" warning. I wanted to ask 
>> what value and files it's actually looking for? (I've also tried some 
>> of the additional flags from makeautoconf/lib-clang.m4, but to no 
>> avail).
>> 
> 
> ?with-libclang should point to a directory has include/clang-c and
> lib/libclang.dll. If include and lib are not under the same directory,
> you can use ?with-libclang-include and ?with-libclang-lib instead.

I have tried all of these flags together:

--with-libclang=/cygdrive/j/LLVM 
--with-libclang-include=/cygdrive/j/LLVM/include 
--with-libclang-include-aux=/cygdrive/j/LLVM/lib 
--with-libclang-lib=/cygdrive/j/LLVM/bin

On windows the dll is under /bin, the lib folder just has an archive 
file (filled with symbols) [2]. There is a folder include/clang-c but it 
is just not being picked up. I'm seeing:

checking clang-c/Index.h usability... no
checking clang-c/Index.h presence... no
checking for clang-c/Index.h... no
configure: Cannot locate libclang! You can download pre-built llvm
         binary from http://llvm.org/releases/download.html, then specify 
the
         location using --with-libclang

Sorry, the build system is pretty opaque from my point of view, so I 
just don't know how to deal with this.

> HTH,
> Henry

Thanks,
Jorn

[1] : 
https://docs.microsoft.com/en-us/cpp/build/overview-of-x64-calling-conventions?view=vs-2017
[2] : 
https://gist.github.com/JornVernee/dae92a19f4d2036f30153e747b55d874

From henry.jen at oracle.com  Wed Sep 19 17:11:30 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 19 Sep 2018 10:11:30 -0700
Subject: [foreign] Running tests on Windows
In-Reply-To: <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
Message-ID: <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>

On Sep 19, 2018, at 9:07 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
> 
> Henry Jen schreef op 2018-09-19 17:23:
>> On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
> 
>>> And then of course one of the biggest things is getting jextract to build and run. Currently the build system is not picking up libclang. When explicitly specifying it with --with-libclang='/cygdrive/j/LLVM' I still get the "Cannot locate libclang!" warning. I wanted to ask what value and files it's actually looking for? (I've also tried some of the additional flags from makeautoconf/lib-clang.m4, but to no avail).
>> ?with-libclang should point to a directory has include/clang-c and
>> lib/libclang.dll. If include and lib are not under the same directory,
>> you can use ?with-libclang-include and ?with-libclang-lib instead.
> 
> I have tried all of these flags together:
> 
> --with-libclang=/cygdrive/j/LLVM --with-libclang-include=/cygdrive/j/LLVM/include --with-libclang-include-aux=/cygdrive/j/LLVM/lib --with-libclang-lib=/cygdrive/j/LLVM/bin
> 

Try remove ?with-libclang and use the rest three. Explicit setting ?with-libclang cause the other options to be ignored.

Cheers,
Henry

> On windows the dll is under /bin, the lib folder just has an archive file (filled with symbols) [2]. There is a folder include/clang-c but it is just not being picked up. I'm seeing:
> 
> checking clang-c/Index.h usability... no
> checking clang-c/Index.h presence... no
> checking for clang-c/Index.h... no
> configure: Cannot locate libclang! You can download pre-built llvm
>        binary from http://llvm.org/releases/download.html, then specify the
>        location using --with-libclang
> 
> Sorry, the build system is pretty opaque from my point of view, so I just don't know how to deal with this.
> 
>> HTH,
>> Henry
> 
> Thanks,
> Jorn
> 
> [1] : https://docs.microsoft.com/en-us/cpp/build/overview-of-x64-calling-conventions?view=vs-2017
> [2] : https://gist.github.com/JornVernee/dae92a19f4d2036f30153e747b55d874


From jbvernee at xs4all.nl  Wed Sep 19 17:50:15 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 19 Sep 2018 19:50:15 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
Message-ID: <19d34989f673a8399bfe96f4bea29834@xs4all.nl>

Henry Jen schreef op 2018-09-19 19:11:
> On Sep 19, 2018, at 9:07 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>> 
>> Henry Jen schreef op 2018-09-19 17:23:
>>> On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>> 
>>>> And then of course one of the biggest things is getting jextract to 
>>>> build and run. Currently the build system is not picking up 
>>>> libclang. When explicitly specifying it with 
>>>> --with-libclang='/cygdrive/j/LLVM' I still get the "Cannot locate 
>>>> libclang!" warning. I wanted to ask what value and files it's 
>>>> actually looking for? (I've also tried some of the additional flags 
>>>> from makeautoconf/lib-clang.m4, but to no avail).
>>> ?with-libclang should point to a directory has include/clang-c and
>>> lib/libclang.dll. If include and lib are not under the same 
>>> directory,
>>> you can use ?with-libclang-include and ?with-libclang-lib instead.
>> 
>> I have tried all of these flags together:
>> 
>> --with-libclang=/cygdrive/j/LLVM 
>> --with-libclang-include=/cygdrive/j/LLVM/include 
>> --with-libclang-include-aux=/cygdrive/j/LLVM/lib 
>> --with-libclang-lib=/cygdrive/j/LLVM/bin
>> 
> 
> Try remove ?with-libclang and use the rest three. Explicit setting
> ?with-libclang cause the other options to be ignored.

Still the same problem I'm afraid.

It's probably kind of hard to debug this remotely, so for now I'll focus 
on getting the other tests to work, and come back to this later.

Thanks,
Jorn

From maurizio.cimadamore at oracle.com  Wed Sep 19 20:03:04 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 19 Sep 2018 20:03:04 +0000
Subject: hg: panama/dev: 91 new changesets
Message-ID: <201809192003.w8JK3AwB008567@aojmv0008.oracle.com>

Changeset: 75261571c13d
Author:    jlaskey
Date:      2018-09-12 17:14 -0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/75261571c13d

8210671: CheckExamples.java fail after Raw String Literals checkin
Reviewed-by: vromero, darcy

! test/langtools/tools/javac/diags/examples.not-yet.txt

Changeset: 2b7ff77c1496
Author:    jwilhelm
Date:      2018-09-13 01:41 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/2b7ff77c1496

Added tag jdk-12+11 for changeset f0f5d23449d3

! .hgtags

Changeset: e3411e5e473d
Author:    vromero
Date:      2018-09-12 16:28 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e3411e5e473d

8207160: ClassReader::adjustMethodParams can potentially return null if the args list is empty
Reviewed-by: mcimadamore, cushon

! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java
+ test/langtools/tools/javac/AvoidNPEAtClassReader/AvoidNPEAtClassReaderTest.java
+ test/langtools/tools/javac/AvoidNPEAtClassReader/pkg/Outer$Inner.jasm
+ test/langtools/tools/javac/AvoidNPEAtClassReader/pkg/Outer.jasm

Changeset: b7bfd64e43a6
Author:    iklam
Date:      2018-09-12 17:45 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b7bfd64e43a6

8210523: runtime/appcds/cacheObject/DifferentHeapSizes.java crash
Reviewed-by: jiangli, ccheung

! src/hotspot/share/classfile/compactHashtable.cpp
! src/hotspot/share/classfile/compactHashtable.inline.hpp
! src/hotspot/share/memory/filemap.cpp
! src/hotspot/share/memory/filemap.hpp
! src/hotspot/share/memory/heapShared.cpp
! src/hotspot/share/memory/heapShared.hpp
! src/hotspot/share/memory/heapShared.inline.hpp
! src/hotspot/share/memory/metaspaceShared.cpp
! test/hotspot/jtreg/runtime/appcds/cacheObject/DifferentHeapSizes.java
! test/lib/jdk/test/lib/cds/CDSTestUtils.java

Changeset: 96b76dca2be8
Author:    cushon
Date:      2018-09-10 16:59 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/96b76dca2be8

8210483: AssertionError in DeferredAttr at setOverloadKind caused by JDK-8203679
Reviewed-by: mcimadamore, vromero

! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/ArgumentAttr.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/DeferredAttr.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/JCTree.java
+ test/langtools/tools/javac/lambda/methodReference/MethodRefStuck.java
+ test/langtools/tools/javac/lambda/methodReference/MethodRefStuck.out

Changeset: 49efbe629159
Author:    iignatyev
Date:      2018-09-12 21:56 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/49efbe629159

8210699: Problem list tests which times out in Xcomp mode
Reviewed-by: kvn

+ test/hotspot/jtreg/ProblemList-Xcomp.txt
+ test/jdk/ProblemList-Xcomp.txt

Changeset: d3ada4479724
Author:    ihse
Date:      2018-09-13 12:41 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d3ada4479724

8210704: Remove dead build tools
Reviewed-by: alanb

! make/ToolsJdk.gmk
- make/jdk/src/classes/build/tools/hasher/Hasher.java
- make/jdk/src/classes/build/tools/jarreorder/JarReorder.java

Changeset: 8abb0fa2c334
Author:    michaelm
Date:      2018-09-13 12:07 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/8abb0fa2c334

8210311: IllegalArgumentException in CookieManager - Comparison method violates its general contract
Reviewed-by: chegar, dfuchs

! src/java.base/share/classes/java/net/CookieManager.java
! src/java.base/share/classes/java/net/HttpCookie.java
! test/jdk/java/net/CookieHandler/CookieManagerTest.java
+ test/jdk/java/net/whitebox/CookieTestDriver.java
+ test/jdk/java/net/whitebox/TEST.properties
+ test/jdk/java/net/whitebox/java.base/java/net/CookieManagerTest.java

Changeset: ccea318862ae
Author:    jlaskey
Date:      2018-09-13 14:15 -0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/ccea318862ae

8210674: Need to add examples for use of javac properties introduced by Raw String Literals
Reviewed-by: vromero, jjg

! test/langtools/tools/javac/diags/examples.not-yet.txt
+ test/langtools/tools/javac/diags/examples/RawStringLiteral.java

Changeset: a3989376ade2
Author:    jiangli
Date:      2018-09-13 13:30 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/a3989376ade2

8210193: [TESTBUG]gc/g1/mixedgc/TestOldGenCollectionUsage.java fails intermittently with OutOfMemoryError in CDS mode.
Summary: Increase java heap size in TestOldGenCollectionUsage. Catch OOM in tests.
Reviewed-by: phh, iklam

! test/hotspot/jtreg/gc/g1/mixedgc/TestLogging.java
! test/hotspot/jtreg/gc/g1/mixedgc/TestOldGenCollectionUsage.java

Changeset: 49b885c1711a
Author:    ihse
Date:      2018-09-13 21:12 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/49b885c1711a

8210702: Remove dtrace mapfiles
Reviewed-by: erikj

! make/hotspot/lib/CompileDtraceLibraries.gmk
- make/mapfiles/libjsound/mapfile-vers
- make/mapfiles/libjvm_db/mapfile-vers
- make/mapfiles/libjvm_dtrace/mapfile-vers
! src/java.base/solaris/native/libjvm_db/libjvm_db.h
! src/java.base/solaris/native/libjvm_dtrace/jvm_dtrace.h

Changeset: 506e9b592f7b
Author:    ihse
Date:      2018-09-13 21:14 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/506e9b592f7b

8207264: solaris-sparcv9-cmp-baseline fails
Reviewed-by: erikj, prr

! make/scripts/compare_exceptions.sh.incl

Changeset: 1ebe04845112
Author:    jcbeyler
Date:      2018-09-13 13:03 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/1ebe04845112

8210665: Clean up JNI_ENV_ARG and factorize the macros for vmTestbase/jvmti[R-U] tests
Summary: Remove JNI_ENV and JVMTI_ENV macros from jvmti/[R-U] tests
Reviewed-by: cjplummer, sspitsyn

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass020/redefclass020.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass021/redefclass021.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass022/redefclass022.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass023/redefclass023.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass024/redefclass024.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass025/redefclass025.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass026/redefclass026.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass027/redefclass027.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass031/redefclass031.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RunAgentThread/agentthr001/agentthr001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RunAgentThread/agentthr002/agentthr002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RunAgentThread/agentthr003/agentthr003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetBreakpoint/setbrk002/setbrk002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetBreakpoint/setbrk003/setbrk003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetBreakpoint/setbrk005/setbrk005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetBreakpoint/setbrk007/setbrk007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetBreakpoint/setbrk008/setbrk008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetEventNotificationMode/setnotif001/setnotif001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/setfldw001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw002/setfldw002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw003/setfldw003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw004/setfldw004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw005/setfldw005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw006/setfldw006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldModificationWatch/setfmodw001/setfmodw001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldModificationWatch/setfmodw002/setfmodw002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldModificationWatch/setfmodw003/setfmodw003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldModificationWatch/setfmodw004/setfmodw004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldModificationWatch/setfmodw005/setfmodw005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldModificationWatch/setfmodw006/setfmodw006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetJNIFunctionTable/setjniftab001/setjniftab001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetJNIFunctionTable/setjniftab002/setjniftab002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetLocalVariable/setlocal001/setlocal001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetLocalVariable/setlocal002/setlocal002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetLocalVariable/setlocal003/setlocal003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetLocalVariable/setlocal004/setlocal004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/StopThread/stopthrd006/stopthrd006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ThreadEnd/threadend001/threadend001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ThreadStart/threadstart001/threadstart001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ThreadStart/threadstart003/threadstart003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/VMDeath/vmdeath001/vmdeath001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/VMInit/vminit001/vminit001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF06/gf06t001/gf06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI03/ji03t001/ji03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI03/ji03t002/ji03t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI03/ji03t003/ji03t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI03/ji03t004/ji03t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/ji05t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI06/ji06t001/ji06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/earlyretbase.cpp

Changeset: cd0775f67ab0
Author:    naoto
Date:      2018-09-13 13:41 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/cd0775f67ab0

8209167: Use CLDR's time zone mappings for Windows
Reviewed-by: erikj, rriggs, ihse

! make/copy/Copy-java.base.gmk
! make/gensrc/GensrcCLDR.gmk
! make/jdk/src/classes/build/tools/cldrconverter/CLDRConverter.java
+ make/jdk/src/classes/build/tools/cldrconverter/WinZonesParseHandler.java
! make/lib/Lib-java.base.gmk
- src/java.base/windows/conf/tzmappings
! src/java.base/windows/native/libjava/TimeZone_md.c

Changeset: 8745f8f1b0f8
Author:    kvn
Date:      2018-09-13 15:27 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/8745f8f1b0f8

8210220: [AOT] jdwp test cases are failing with error # ERROR: TEST FAILED: Cought IOException while receiving event packet
Summary: don't register AOT method if corresponding java method has breakpoints.
Reviewed-by: dlong

! src/hotspot/share/aot/aotCodeHeap.cpp
! src/hotspot/share/aot/aotCodeHeap.hpp
! src/hotspot/share/aot/aotCompiledMethod.cpp
! src/hotspot/share/aot/aotCompiledMethod.hpp
! src/hotspot/share/aot/aotLoader.cpp
! src/hotspot/share/aot/aotLoader.hpp

Changeset: dc68380e6497
Author:    gadams
Date:      2018-09-13 07:54 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/dc68380e6497

8210252: com/sun/jdi/DebuggerThreadTest.java fails with NPE
Reviewed-by: cjplummer, sspitsyn

! test/jdk/com/sun/jdi/DebuggerThreadTest.java

Changeset: a929ad0569ee
Author:    xuelei
Date:      2018-09-13 17:11 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/a929ad0569ee

8209916: NPE in SupportedGroupsExtension
Reviewed-by: jnimeh, wetmore

! src/java.base/share/classes/sun/security/ssl/SupportedGroupsExtension.java

Changeset: e6b524cdcc34
Author:    cushon
Date:      2018-09-13 15:29 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e6b524cdcc34

8193037: package-info annotations are not reported when annotation processing is enabled
Reviewed-by: jjg

! src/jdk.compiler/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java
+ test/langtools/tools/javac/processing/PackageInfo/ClassAnnotations/ClassAnnotations.java
+ test/langtools/tools/javac/processing/PackageInfo/ClassAnnotations/Processor.java
+ test/langtools/tools/javac/processing/PackageInfo/ClassAnnotations/package-info.java
+ test/langtools/tools/javac/processing/PackageInfo/Overwrite/Overwrite.java
+ test/langtools/tools/javac/processing/PackageInfo/Overwrite/Processor.java
+ test/langtools/tools/javac/processing/PackageInfo/Overwrite/package-info.java

Changeset: 6ebcec186980
Author:    amlu
Date:      2018-09-14 13:18 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/6ebcec186980

8209772: Refactor shell test java/util/ServiceLoader/basic/basic.sh to java
Reviewed-by: alanb

+ test/jdk/java/util/ServiceLoader/basic/ServiceLoaderBasicTest.java
- test/jdk/java/util/ServiceLoader/basic/basic.sh

Changeset: 42d99cb7f50f
Author:    iveresov
Date:      2018-09-13 22:45 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/42d99cb7f50f

8210478: Update Graal
Reviewed-by: kvn

! src/jdk.aot/share/classes/jdk.tools.jaotc/src/jdk/tools/jaotc/AOTCompiledClass.java
! src/jdk.internal.vm.compiler.management/share/classes/org.graalvm.compiler.hotspot.management/src/org/graalvm/compiler/hotspot/management/HotSpotGraalRuntimeMBean.java
! src/jdk.internal.vm.compiler/share/classes/module-info.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.asm.amd64/src/org/graalvm/compiler/asm/amd64/AMD64Assembler.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.asm.amd64/src/org/graalvm/compiler/asm/amd64/AMD64BaseAssembler.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/CheckGraalInvariants.java
+ src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/NewInstanceTest.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/ea/EscapeAnalysisTest.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core/src/org/graalvm/compiler/core/CompilationWrapper.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core/src/org/graalvm/compiler/core/GraalCompilerOptions.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.debug.test/src/org/graalvm/compiler/debug/test/DebugContextTest.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.debug/src/org/graalvm/compiler/debug/DiagnosticsOutputDirectory.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.debug/src/org/graalvm/compiler/debug/PathUtilities.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/CompilationWrapperTest.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/GraalOSRTest.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/HotSpotGraalManagementTest.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/HotSpotBackend.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/HotSpotGraalCompiler.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/HotSpotGraalRuntimeProvider.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/JVMCIVersionCheck.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/meta/HotSpotHostForeignCallsProvider.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/replacements/NewObjectSnippets.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewArrayStub.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewInstanceStub.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/SnippetStub.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/StubOptions.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.java/src/org/graalvm/compiler/java/BciBlockMapping.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.java/src/org/graalvm/compiler/java/BytecodeParser.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/GraphEncoder.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/StructuredGraph.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/java/InstanceOfNode.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/java/NewInstanceNode.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/java/StoreIndexedNode.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.options/src/org/graalvm/compiler/options/OptionsParser.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.phases.common/src/org/graalvm/compiler/phases/common/ConditionalEliminationPhase.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.phases.common/src/org/graalvm/compiler/phases/common/inlining/InliningUtil.java
+ src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.phases.common/src/org/graalvm/compiler/phases/common/jmx/HotSpotMBeanOperationProvider.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.phases/src/org/graalvm/compiler/phases/tiers/PhaseContext.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.test/src/org/graalvm/compiler/replacements/test/PEGraphDecoderTest.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/CachingPEGraphDecoder.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/IntrinsicGraphBuilder.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/PEGraphDecoder.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/ReplacementsImpl.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/SnippetTemplate.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/nodes/MacroNode.java
! src/jdk.internal.vm.compiler/share/classes/org.graalvm.graphio/src/org/graalvm/graphio/GraphOutput.java

Changeset: 30e6a0b9d691
Author:    ihse
Date:      2018-09-14 09:16 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/30e6a0b9d691

8210731: PropertiesParser does not produce reproducible output
Reviewed-by: mchung, jjg, erikj

! make/langtools/tools/propertiesparser/PropertiesParser.java

Changeset: 606e039bd655
Author:    sgehwolf
Date:      2018-09-13 11:07 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/606e039bd655

8210703: vmStructs.cpp compiled with -O0
Reviewed-by: erikj, ihse

! make/hotspot/lib/JvmOverrideFiles.gmk

Changeset: 7bed934d439e
Author:    dpochepk
Date:      2018-09-14 14:24 +0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/7bed934d439e

8210461: AArch64: Math.cos intrinsic gives incorrect results
Reviewed-by: aph

! src/hotspot/cpu/aarch64/macroAssembler_aarch64_trig.cpp
+ test/hotspot/jtreg/compiler/intrinsics/math/Test8210461.java

Changeset: f79cfb07f13f
Author:    hannesw
Date:      2018-09-14 14:45 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f79cfb07f13f

8209914: javadoc search sometimes generates bad URIs
Reviewed-by: jjg

! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/search.js

Changeset: ed9b1200dd81
Author:    pliden
Date:      2018-09-14 14:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/ed9b1200dd81

8209163: SA: Show Object Histogram asserts with ZGC
Reviewed-by: ysuenaga, jcbeyler

! src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/ObjectHeap.java
! src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/ProgressiveHeapVisitor.java

Changeset: 3dd95a83791b
Author:    pliden
Date:      2018-09-14 14:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/3dd95a83791b

8210710: Rename ThreadLocalAllocBuffer::myThread() to thread()
Reviewed-by: rkennke, tschatzl

! src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp
! src/hotspot/share/gc/shared/threadLocalAllocBuffer.hpp
! src/hotspot/share/gc/shared/threadLocalAllocBuffer.inline.hpp

Changeset: 47466ac8dcf0
Author:    pliden
Date:      2018-09-14 14:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/47466ac8dcf0

8210711: Remove unused offset getters in ThreadLocalAllocBuffer
Reviewed-by: rkennke, tschatzl, mdoerr

! src/hotspot/share/gc/shared/threadLocalAllocBuffer.hpp
! src/hotspot/share/runtime/thread.hpp

Changeset: 1cb25b6589e9
Author:    pliden
Date:      2018-09-14 14:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/1cb25b6589e9

8210714: ZGC: ZWeakRootsIterator should no longer call reset/finish_dead_counter()
Reviewed-by: eosterlund

! src/hotspot/share/gc/z/zRootsIterator.cpp

Changeset: 90c1dcdebc64
Author:    alanb
Date:      2018-09-14 16:56 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/90c1dcdebc64

8208780: (se) test SelectWithConsumer.testReadableAndWriteable(): failure
Reviewed-by: bpb

! test/jdk/java/nio/channels/Selector/SelectWithConsumer.java

Changeset: 07ae9da7a230
Author:    bpb
Date:      2018-09-14 09:00 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/07ae9da7a230

8210741: Typo in Java API documentation of java.nio.file.Paths
Reviewed-by: alanb, rriggs

! src/java.base/share/classes/java/nio/file/Paths.java

Changeset: 9bf5205655ee
Author:    coleenp
Date:      2018-09-14 12:10 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/9bf5205655ee

8210559: ClassLoaderData Symbols can leak
Summary: unrefcount the symbol names when the CLD is destroyed
Reviewed-by: lfoltan, jiangli, iklam

! src/hotspot/share/classfile/classLoaderData.cpp
! src/hotspot/share/prims/whitebox.cpp
! test/hotspot/jtreg/runtime/ClassUnload/UnloadTest.java
! test/hotspot/jtreg/runtime/testlibrary/ClassUnloadCommon.java
! test/lib/sun/hotspot/WhiteBox.java

Changeset: 13d6be5fbfa5
Author:    rriggs
Date:      2018-09-14 12:53 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/13d6be5fbfa5

8098798: Thread.join(ms) on Linux still affected by changes to the time-of-day clock
8210004: Thread.sleep(millis, nanos) timeout returns early
Reviewed-by: martin, igerasim

! src/java.base/share/classes/java/lang/Thread.java

Changeset: a0f0da2c2719
Author:    akolarkunnu
Date:      2018-09-11 22:16 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/a0f0da2c2719

8210056: Enable different look and feel tests in SwingSet3 demo test TextFieldDemoTest
Reviewed-by: serb
Contributed-by: abdul.kolarkunnu at oracle.com

! test/jdk/sanity/client/SwingSet/src/TextFieldDemoTest.java

Changeset: 9151fde080e6
Author:    ccheung
Date:      2018-09-14 11:17 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/9151fde080e6

8190737: use unicode version of the canonicalize() function to handle long path on windows
Summary: also calling CreateFileW in zip_util.c to handle long path
Reviewed-by: sherman, iklam

! src/hotspot/os/windows/include/jvm_md.h
! src/java.base/share/native/libzip/zip_util.c
! src/java.base/windows/native/libjava/canonicalize_md.c
! src/java.base/windows/native/libjava/io_util_md.h
! test/hotspot/jtreg/runtime/LoadClass/LongBCP.java

Changeset: b5921376ff2c
Author:    gadams
Date:      2018-09-13 07:46 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/b5921376ff2c

8208468: [TESTBUG] nsk/jdb/locals/locals002: fails with "Prompt is not received during ... milliseconds"
Reviewed-by: cjplummer, amenkov

! test/hotspot/jtreg/vmTestbase/nsk/jdb/locals/locals002/locals002a.java

Changeset: 43323ced5e40
Author:    jcbeyler
Date:      2018-09-14 08:48 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/43323ced5e40

8210724: Change the verbosity threshold of logging for [oopstorage,ref]
Summary: Improve logging verbosity levels in oopStorage.cpp
Reviewed-by: kbarrett, sjohanss
Contributed-by: manc at google.com

! src/hotspot/share/gc/shared/oopStorage.cpp

Changeset: 763aa4d1d596
Author:    gromero
Date:      2018-09-14 15:32 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/763aa4d1d596

8209972: [GRAAL] Don't run RTM tests with Graal
Reviewed-by: kvn, goetz

! test/hotspot/jtreg/TEST.ROOT
! test/hotspot/jtreg/compiler/rtm/cli/TestPrintPreciseRTMLockingStatisticsOptionOnSupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestPrintPreciseRTMLockingStatisticsOptionOnUnsupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestRTMAbortThresholdOption.java
! test/hotspot/jtreg/compiler/rtm/cli/TestRTMLockingCalculationDelayOption.java
! test/hotspot/jtreg/compiler/rtm/cli/TestRTMLockingThresholdOption.java
! test/hotspot/jtreg/compiler/rtm/cli/TestRTMRetryCountOption.java
! test/hotspot/jtreg/compiler/rtm/cli/TestRTMSpinLoopCountOption.java
! test/hotspot/jtreg/compiler/rtm/cli/TestRTMTotalCountIncrRateOptionOnSupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMDeoptOptionOnSupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMDeoptOptionOnUnsupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMForStackLocksOptionOnSupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMForStackLocksOptionOnUnsupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMLockingOptionOnSupportedConfig.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMLockingOptionOnUnsupportedCPU.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMLockingOptionWithBiasedLocking.java
! test/hotspot/jtreg/compiler/rtm/cli/TestUseRTMXendForLockBusyOption.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMAbortRatio.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMAbortThreshold.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMAfterNonRTMDeopt.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMDeoptOnHighAbortRatio.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMDeoptOnLowAbortRatio.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMLockingCalculationDelay.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMLockingThreshold.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMRetryCount.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMSpinLoopCount.java
! test/hotspot/jtreg/compiler/rtm/locking/TestRTMTotalCountIncrRate.java
! test/hotspot/jtreg/compiler/rtm/locking/TestUseRTMAfterLockInflation.java
! test/hotspot/jtreg/compiler/rtm/locking/TestUseRTMDeopt.java
! test/hotspot/jtreg/compiler/rtm/locking/TestUseRTMForInflatedLocks.java
! test/hotspot/jtreg/compiler/rtm/locking/TestUseRTMForStackLocks.java
! test/hotspot/jtreg/compiler/rtm/locking/TestUseRTMXendForLockBusy.java
! test/hotspot/jtreg/compiler/rtm/method_options/TestNoRTMLockElidingOption.java
! test/hotspot/jtreg/compiler/rtm/method_options/TestUseRTMLockElidingOption.java
! test/hotspot/jtreg/compiler/rtm/print/TestPrintPreciseRTMLockingStatistics.java
! test/jtreg-ext/requires/VMProps.java

Changeset: 594919232b8f
Author:    iignatyev
Date:      2018-09-14 14:02 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/594919232b8f

8210732: remove jdk.testlibrary.Utils
Reviewed-by: alanb, jcbeyler

! test/hotspot/jtreg/runtime/Dictionary/ProtectionDomainCacheTest.java
! test/jdk/com/sun/jdi/BadHandshakeTest.java
! test/jdk/com/sun/jdi/BasicJDWPConnectionTest.java
! test/jdk/com/sun/jdi/DoubleAgentTest.java
! test/jdk/com/sun/jdi/ExclusiveBind.java
! test/jdk/com/sun/tools/attach/RunnerUtil.java
! test/jdk/com/sun/tools/attach/StartManagementAgent.java
! test/jdk/java/awt/datatransfer/ClipboardInterVMTest/ClipboardInterVMTest.java
! test/jdk/java/lang/ClassLoader/forNameLeak/ClassForNameLeak.java
! test/jdk/java/lang/Thread/ThreadStateController.java
! test/jdk/java/lang/instrument/PremainClass/NoPremainAgentTest.java
! test/jdk/java/lang/instrument/PremainClass/PremainClassTest.java
! test/jdk/java/lang/instrument/PremainClass/ZeroArgPremainAgentTest.java
! test/jdk/java/lang/invoke/LFCaching/LambdaFormTestCase.java
! test/jdk/java/lang/invoke/MethodHandles/CatchExceptionTest.java
! test/jdk/java/lang/invoke/MethodHandlesAsCollectorTest.java
! test/jdk/java/lang/invoke/MethodHandlesCastFailureTest.java
! test/jdk/java/lang/invoke/MethodHandlesGeneralTest.java
! test/jdk/java/lang/invoke/MethodHandlesInsertArgumentsTest.java
! test/jdk/java/lang/invoke/MethodHandlesInvokersTest.java
! test/jdk/java/lang/invoke/MethodHandlesPermuteArgumentsTest.java
! test/jdk/java/lang/invoke/MethodHandlesSpreadArgumentsTest.java
! test/jdk/java/lang/invoke/PermuteArgsTest.java
! test/jdk/java/lang/invoke/TestCatchExceptionWithVarargs.java
! test/jdk/java/lang/invoke/VarargsArrayTest.java
! test/jdk/java/lang/invoke/common/test/java/lang/invoke/lib/CodeCacheOverflowProcessor.java
! test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java
! test/jdk/java/lang/management/MemoryMXBean/RunUtil.java
! test/jdk/java/net/MulticastSocket/MultiDead.java
! test/jdk/java/net/httpclient/security/Driver.java
! test/jdk/java/nio/channels/FileChannel/LoopingTruncate.java
! test/jdk/java/nio/channels/Selector/Wakeup.java
! test/jdk/java/util/concurrent/BlockingQueue/CancelledProducerConsumerLoops.java
! test/jdk/java/util/concurrent/BlockingQueue/Interrupt.java
! test/jdk/java/util/concurrent/BlockingQueue/MultipleProducersSingleConsumerLoops.java
! test/jdk/java/util/concurrent/BlockingQueue/ProducerConsumerLoops.java
! test/jdk/java/util/concurrent/BlockingQueue/SingleProducerMultipleConsumerLoops.java
! test/jdk/java/util/concurrent/CompletableFuture/Basic.java
! test/jdk/java/util/concurrent/ConcurrentHashMap/MapLoops.java
! test/jdk/java/util/concurrent/ConcurrentQueues/ConcurrentQueueLoops.java
! test/jdk/java/util/concurrent/CyclicBarrier/Basic.java
! test/jdk/java/util/concurrent/Exchanger/ExchangeLoops.java
! test/jdk/java/util/concurrent/ExecutorCompletionService/ExecutorCompletionServiceLoops.java
! test/jdk/java/util/concurrent/Executors/AutoShutdown.java
! test/jdk/java/util/concurrent/FutureTask/BlockingTaskExecutor.java
! test/jdk/java/util/concurrent/FutureTask/CancelledFutureLoops.java
! test/jdk/java/util/concurrent/FutureTask/DoneMeansDone.java
! test/jdk/java/util/concurrent/ScheduledThreadPoolExecutor/DelayOverflow.java
! test/jdk/java/util/concurrent/ScheduledThreadPoolExecutor/ZeroCorePoolSize.java
! test/jdk/java/util/concurrent/ScheduledThreadPoolExecutor/ZeroCoreThreads.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/CoreThreadTimeOut.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/Custom.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/FlakyThreadFactory.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/SelfInterrupt.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/ThreadRestarts.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/TimeOutShrink.java
! test/jdk/java/util/concurrent/forkjoin/SubmissionTest.java
! test/jdk/java/util/concurrent/locks/Lock/CheckedLockLoops.java
! test/jdk/java/util/concurrent/locks/Lock/TimedAcquireLeak.java
! test/jdk/java/util/concurrent/locks/ReentrantLock/LockOncePerThreadLoops.java
! test/jdk/java/util/concurrent/locks/ReentrantLock/SimpleReentrantLockLoops.java
! test/jdk/java/util/concurrent/locks/ReentrantLock/TimeoutLockLoops.java
! test/jdk/java/util/concurrent/locks/ReentrantReadWriteLock/Count.java
! test/jdk/java/util/concurrent/locks/ReentrantReadWriteLock/MapLoops.java
! test/jdk/java/util/concurrent/locks/StampedLock/Basic.java
! test/jdk/javax/management/monitor/GaugeMonitorDeadlockTest.java
! test/jdk/javax/management/monitor/StartStopTest.java
! test/jdk/javax/management/mxbean/MXBeanWeirdParamTest.java
! test/jdk/javax/management/remote/mandatory/connection/DefaultAgentFilterTest.java
! test/jdk/javax/management/remote/mandatory/loading/MethodResultTest.java
! test/jdk/javax/management/security/AuthorizationTest.java
! test/jdk/javax/management/security/SecurityTest.java
- test/jdk/lib/testlibrary/jdk/testlibrary/Utils.java
! test/jdk/sun/jvmstat/monitor/MonitoredVm/TestPollingInterval.java
! test/jdk/sun/management/jdp/DynamicLauncher.java
! test/jdk/sun/management/jdp/PortAlreadyInUseTest.java
! test/jdk/sun/management/jmxremote/bootstrap/AbstractFilePermissionTest.java
! test/jdk/sun/management/jmxremote/bootstrap/LocalManagementTest.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.sh
! test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTestApp.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh
! test/jdk/sun/management/jmxremote/startstop/JMXStartStopTest.java
! test/jdk/sun/security/ssl/SSLEngineImpl/SSLEngineKeyLimit.java
! test/jdk/sun/security/ssl/SSLSocketImpl/SSLSocketKeyLimit.java
! test/jdk/sun/tools/jcmd/TestJcmdDefaults.java
! test/jdk/sun/tools/jcmd/TestJcmdSanity.java
! test/jdk/sun/tools/jhsdb/BasicLauncherTest.java
! test/jdk/sun/tools/jhsdb/heapconfig/JMapHeapConfigTest.java
! test/jdk/sun/tools/jhsdb/heapconfig/TmtoolTestScenario.java
! test/jdk/sun/tools/jps/JpsHelper.java
! test/jdk/sun/tools/jstack/DeadlockDetectionTest.java
! test/jdk/sun/tools/jstatd/JstatGCUtilParser.java
! test/jdk/sun/tools/jstatd/JstatdTest.java

Changeset: 6c394ed56b07
Author:    xuelei
Date:      2018-09-14 20:30 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/6c394ed56b07

8210785: Trivial typo fix in X509ExtendedKeyManager javadoc
Reviewed-by: xuelei
Contributed-by: Jaikiran Pai <jaikiran.pai at gmail.com>

! src/java.base/share/classes/javax/net/ssl/X509ExtendedKeyManager.java

Changeset: 4bd35a5ec694
Author:    mikael
Date:      2018-09-14 22:35 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/4bd35a5ec694

8210676: Remove some unused Label variables
Reviewed-by: kvn, dholmes, njian, aph

! src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.cpp
! src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp
! src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp
! src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
! src/hotspot/cpu/aarch64/stubGenerator_aarch64.cpp
! src/hotspot/cpu/aarch64/templateInterpreterGenerator_aarch64.cpp
! src/hotspot/cpu/arm/arm.ad
! src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp
! src/hotspot/cpu/arm/macroAssembler_arm.cpp
! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/arm/sharedRuntime_arm.cpp
! src/hotspot/cpu/ppc/stubGenerator_ppc.cpp
! src/hotspot/cpu/ppc/templateTable_ppc_64.cpp
! src/hotspot/cpu/s390/macroAssembler_s390.cpp
! src/hotspot/cpu/s390/stubGenerator_s390.cpp
! src/hotspot/cpu/s390/templateTable_s390.cpp
! src/hotspot/cpu/sparc/macroAssembler_sparc.cpp
! src/hotspot/cpu/sparc/vtableStubs_sparc.cpp
! src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp
! src/hotspot/cpu/x86/c1_Runtime1_x86.cpp
! src/hotspot/cpu/x86/interp_masm_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86_cos.cpp
! src/hotspot/cpu/x86/macroAssembler_x86_exp.cpp
! src/hotspot/cpu/x86/macroAssembler_x86_log.cpp
! src/hotspot/cpu/x86/macroAssembler_x86_log10.cpp
! src/hotspot/cpu/x86/macroAssembler_x86_sin.cpp
! src/hotspot/cpu/x86/macroAssembler_x86_tan.cpp
! src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp
! src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
! src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp
! src/hotspot/cpu/x86/templateTable_x86.cpp

Changeset: 5b033f23aced
Author:    bsrbnd
Date:      2018-09-15 22:16 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/5b033f23aced

8183548: Comma-expressions shouldn't use any temporary variable
Summary: Uses enhanced let-expressions allowing multiple statements
Reviewed-by: vromero

! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java
! test/langtools/tools/javac/desugar/BoxingAndSuper.java

Changeset: 6c956c883137
Author:    igerasim
Date:      2018-09-15 13:53 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/6c956c883137

8210787: Object.wait(long, int) throws inappropriate IllegalArgumentException
Reviewed-by: martin, rriggs

! src/java.base/share/classes/java/lang/Object.java
+ test/jdk/java/lang/Object/WaitTooLong.java

Changeset: ac6e9a2ebc04
Author:    igerasim
Date:      2018-09-15 22:02 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/ac6e9a2ebc04

8210786: Typo s/overriden/overridden/ in several places
Reviewed-by: weijun

! src/java.base/share/classes/java/security/Identity.java
! src/java.base/share/classes/java/security/Provider.java
! src/java.base/share/classes/java/security/Security.java
! src/java.base/share/classes/java/text/RuleBasedCollator.java
! src/java.base/share/classes/java/util/Random.java
! src/java.base/share/classes/java/util/StringJoiner.java
! src/java.base/share/classes/javax/crypto/CipherInputStream.java
! src/java.base/share/classes/javax/crypto/CipherOutputStream.java
! src/java.base/share/classes/javax/net/ssl/HttpsURLConnection.java
! src/java.base/share/classes/javax/net/ssl/SSLServerSocket.java
! src/java.base/unix/classes/sun/net/www/protocol/file/Handler.java
! src/java.base/windows/classes/sun/net/www/protocol/file/Handler.java

Changeset: caac55d48dc3
Author:    xyin
Date:      2018-09-17 13:49 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/caac55d48dc3

8210695: Create test to cover JDK-8205330 InitialDirContext ctor sometimes throws NPE if the server has sent a disconnection
Reviewed-by: vtewari, dfuchs, chegar

+ test/jdk/com/sun/jndi/ldap/DisconnectNPETest.java

Changeset: 5f868838bc5f
Author:    weijun
Date:      2018-09-17 14:52 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/5f868838bc5f

8210736: jdk/javax/xml/crypto/dsig/GenerationTests.java slow on linux
Reviewed-by: alanb

! test/jdk/javax/xml/crypto/dsig/GenerationTests.java

Changeset: c608b2190460
Author:    sgehwolf
Date:      2018-09-14 14:47 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/c608b2190460

8210647: libsaproc is being compiled without optimization.
Summary: Compile with high optimization instead of none.
Reviewed-by: erikj, jgeorge, sballal

! make/lib/Lib-jdk.hotspot.agent.gmk

Changeset: caef940517be
Author:    rgoel
Date:      2018-09-17 14:16 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/caef940517be

8210490: TimeZone.getDisplayName given Locale.US doesn't always honor the Locale.
Summary: specified locale for formatting..
Reviewed-by: naoto

! src/java.base/share/classes/sun/util/cldr/CLDRTimeZoneNameProviderImpl.java
! test/jdk/java/util/TimeZone/CLDRDisplayNamesTest.java

Changeset: d883f528689d
Author:    jlahoda
Date:      2018-09-17 11:49 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d883f528689d

8193561: Cyclic hierarchy causes a NullPointerException when setting DEFAULT flag
Summary: When marking interface as having default methods, use .owner as it is always defined.
Reviewed-by: mcimadamore

! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/MemberEnter.java
+ test/langtools/tools/javac/cycle/T8193561.java
+ test/langtools/tools/javac/cycle/T8193561.out

Changeset: e10ade04afe5
Author:    sgehwolf
Date:      2018-09-17 10:53 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/e10ade04afe5

8210416: [linux] Poor StrictMath performance due to non-optimized compilation
Summary: Compile fdlibm with -O2 -ffp-contract=off on gcc/clang arches.
Reviewed-by: aph, erikj, dholmes, darcy, ihse

! make/autoconf/flags-cflags.m4
! make/autoconf/spec.gmk.in
! make/lib/CoreLibraries.gmk

Changeset: 3ca7d5385653
Author:    rfield
Date:      2018-09-17 08:37 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/3ca7d5385653

8210596: jshell does not support raw string literals
Reviewed-by: jlahoda, jlaskey

! src/jdk.jshell/share/classes/jdk/jshell/CompletenessAnalyzer.java
! src/jdk.jshell/share/classes/jdk/jshell/MaskCommentsAndModifiers.java
! src/jdk.jshell/share/classes/jdk/jshell/SourceCodeAnalysisImpl.java
! test/langtools/jdk/jshell/ToolLocalSimpleTest.java
! test/langtools/jdk/jshell/ToolSimpleTest.java

Changeset: 497950fd69a7
Author:    jcbeyler
Date:      2018-09-17 09:07 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/497950fd69a7

8210726: Fix up a few minor nits forgotten by JDK-8210665
Summary: Minor changes to 4 tests to make one-liners
Reviewed-by: cjplummer, iignatyev, sspitsyn

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw005/setfldw005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw006/setfldw006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetFieldModificationWatch/setfmodw006/setfmodw006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetLocalVariable/setlocal002/setlocal002.cpp

Changeset: 9884b717f2ed
Author:    iris
Date:      2018-09-17 10:14 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/9884b717f2ed

8210775: JVM TI Spec missing copyright
Reviewed-by: dholmes, mchung, sspitsyn

! src/hotspot/share/prims/jvmti.xml
! src/hotspot/share/prims/jvmti.xsl

Changeset: 0ae80830256e
Author:    iignatyev
Date:      2018-09-17 11:50 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/0ae80830256e

8210779: 8182404 and 8210732 haven't updated copyright years
Reviewed-by: bchristi

! test/jdk/com/sun/jdi/BasicJDWPConnectionTest.java
! test/jdk/java/awt/datatransfer/ClipboardInterVMTest/ClipboardInterVMTest.java
! test/jdk/java/lang/Thread/ThreadStateController.java
! test/jdk/java/lang/invoke/LFCaching/LambdaFormTestCase.java
! test/jdk/java/lang/invoke/MethodHandles/CatchExceptionTest.java
! test/jdk/java/lang/invoke/PermuteArgsTest.java
! test/jdk/java/lang/invoke/TestCatchExceptionWithVarargs.java
! test/jdk/java/lang/invoke/VarargsArrayTest.java
! test/jdk/java/lang/invoke/common/test/java/lang/invoke/lib/CodeCacheOverflowProcessor.java
! test/jdk/java/net/MulticastSocket/MultiDead.java
! test/jdk/java/net/spi/URLStreamHandlerProvider/Basic.java
! test/jdk/java/nio/channels/FileChannel/LoopingTruncate.java
! test/jdk/java/nio/channels/Selector/Wakeup.java
! test/jdk/java/security/KeyStore/PKCS12/EntryProtectionTest.java
! test/jdk/java/util/concurrent/BlockingQueue/Interrupt.java
! test/jdk/java/util/concurrent/CyclicBarrier/Basic.java
! test/jdk/java/util/concurrent/Executors/AutoShutdown.java
! test/jdk/java/util/concurrent/FutureTask/BlockingTaskExecutor.java
! test/jdk/java/util/concurrent/ScheduledThreadPoolExecutor/ZeroCorePoolSize.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/CoreThreadTimeOut.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/Custom.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/SelfInterrupt.java
! test/jdk/java/util/concurrent/ThreadPoolExecutor/TimeOutShrink.java
! test/jdk/java/util/concurrent/forkjoin/SubmissionTest.java
! test/jdk/java/util/concurrent/locks/Lock/TimedAcquireLeak.java
! test/jdk/java/util/concurrent/locks/ReentrantReadWriteLock/Count.java
! test/jdk/javax/management/monitor/GaugeMonitorDeadlockTest.java
! test/jdk/javax/management/monitor/StartStopTest.java
! test/jdk/javax/management/remote/mandatory/loading/MethodResultTest.java
! test/jdk/sun/jvmstat/monitor/MonitoredVm/TestPollingInterval.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.sh
! test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTestApp.java
! test/jdk/sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh
! test/jdk/sun/tools/jhsdb/heapconfig/TmtoolTestScenario.java
! test/jdk/tools/jar/compat/CLICompatibility.java

Changeset: b19734760ed3
Author:    kvn
Date:      2018-09-17 13:43 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b19734760ed3

8209574: [AOT] breakpoint events are generated in different threads does not meet expected count
Summary: Disable AOT when debugger is attached.
Reviewed-by: dlong

! src/hotspot/share/aot/aotLoader.cpp
! src/hotspot/share/aot/aotLoader.hpp
! test/hotspot/jtreg/compiler/aot/AotCompiler.java

Changeset: 1f805481d8de
Author:    ascarpino
Date:      2018-09-17 14:04 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/1f805481d8de

8209031: SSLSocket should throw an exception when configuring DTLS
Reviewed-by: xuelei

! src/java.base/share/classes/javax/net/ssl/SSLServerSocketFactory.java
! src/java.base/share/classes/javax/net/ssl/SSLSocketFactory.java
! src/java.base/share/classes/sun/security/ssl/SSLContextImpl.java
! test/jdk/sun/security/ssl/SSLContextImpl/CustomizedDTLSDefaultProtocols.java
! test/jdk/sun/security/ssl/SSLContextImpl/CustomizedDTLSServerDefaultProtocols.java
! test/jdk/sun/security/ssl/SSLContextImpl/DefaultDTLSEnabledProtocols.java

Changeset: 5432cebf6627
Author:    mchung
Date:      2018-09-17 15:22 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/5432cebf6627

8210721: Replace legacy serial exception field with Throwable::cause
Reviewed-by: dfuchs, lancea

! src/java.base/share/classes/java/lang/ClassNotFoundException.java
! src/java.base/share/classes/java/lang/ExceptionInInitializerError.java
! src/java.base/share/classes/java/lang/System.java
! src/java.base/share/classes/java/lang/Throwable.java
! src/java.base/share/classes/java/lang/reflect/InvocationTargetException.java
! src/java.base/share/classes/java/lang/reflect/UndeclaredThrowableException.java
! src/java.base/share/classes/java/security/PrivilegedActionException.java
! src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java
! test/jdk/java/lang/Throwable/LegacyChainedExceptionSerialization.java

Changeset: 720fd6544b03
Author:    jnimeh
Date:      2018-09-17 15:25 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/720fd6544b03

8140466: ChaCha20 and Poly1305 TLS Cipher Suites
Reviewed-by: xuelei, mullan

! src/java.base/share/classes/sun/security/ssl/CipherSuite.java
! src/java.base/share/classes/sun/security/ssl/JsseJce.java
! src/java.base/share/classes/sun/security/ssl/SSLCipher.java
! test/jdk/javax/net/ssl/TLSCommon/CipherSuite.java
! test/jdk/javax/net/ssl/TLSCommon/SSLEngineTestCase.java

Changeset: 79dc492c00ab
Author:    jcbeyler
Date:      2018-09-17 19:36 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/79dc492c00ab

8210481: Remove #ifdef cplusplus from vmTestbase
Summary: Remove all cplusplus ifdefs from vmTestbase
Reviewed-by: dholmes, iignatyev, cjplummer, sspitsyn

! test/hotspot/jtreg/vmTestbase/gc/gctests/mallocWithGC1/libmallocWithGC1.cpp
! test/hotspot/jtreg/vmTestbase/gc/gctests/mallocWithGC2/libmallocWithGC2.cpp
! test/hotspot/jtreg/vmTestbase/gc/gctests/mallocWithGC3/libmallocWithGC3.cpp
! test/hotspot/jtreg/vmTestbase/gc/gctests/nativeGC01/libnativeGC01.cpp
! test/hotspot/jtreg/vmTestbase/gc/gctests/nativeGC02/libnativeGC02.cpp
! test/hotspot/jtreg/vmTestbase/gc/gctests/nativeGC03/libnativeGC03.cpp
! test/hotspot/jtreg/vmTestbase/gc/gctests/nativeGC05/libnativeGC05.cpp
! test/hotspot/jtreg/vmTestbase/nsk/aod/VirtualMachine/VirtualMachine07/agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/aod/VirtualMachine/VirtualMachine07/agent01.cpp
! test/hotspot/jtreg/vmTestbase/nsk/aod/VirtualMachine/VirtualMachine07/agent02.cpp
! test/hotspot/jtreg/vmTestbase/nsk/aod/VirtualMachine/VirtualMachine07/agent03.cpp
! test/hotspot/jtreg/vmTestbase/nsk/aod/VirtualMachine/VirtualMachine09/agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AddCapabilities/addcaps001/addcaps001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AddCapabilities/addcaps002/addcaps002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AddCapabilities/addcaps003/addcaps003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Agent_OnLoad/agentonload001/agentonload001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Agent_OnLoad/agentonload002/agentonload002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Agent_OnLoad/agentonload003/agentonload003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Agent_OnUnload/agentonunload001/agentonunload001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach002/attach002Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach002a/attach002aAgent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach008/attach008Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach009/attach009Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach012/attach012Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach014/attach014Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach015/attach015Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach015/attach015Agent01.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach015/attach015Target.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach020/attach020Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach022/attach022Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach037/attach037Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach038/attach038Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach039/attach039Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach040/attach040Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach041/attach041Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach042/attach042Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach045/attach045Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach045/attach045Agent01.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach045/attach045Agent02.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach045/attach045Agent03.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach046/attach046Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach050/attach050Agent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/sharedAgents/simpleAgent00.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Breakpoint/breakpoint001/breakpoint001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk001/classfloadhk001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk002/classfloadhk002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk003/classfloadhk003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk004/classfloadhk004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk005/classfloadhk005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk006/classfloadhk006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk007/classfloadhk007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk008/classfloadhk008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassFileLoadHook/classfloadhk009/classfloadhk009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassLoad/classload001/classload001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/CompiledMethodLoad/compmethload001/compmethload001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/CompiledMethodUnload/compmethunload001/compmethunload001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/DataDumpRequest/datadumpreq001/datadumpreq001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/DisposeEnvironment/disposeenv001/disposeenv001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/DisposeEnvironment/disposeenv002/disposeenv002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/DynamicCodeGenerated/dyncodgen001/dyncodgen001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ForceEarlyReturn/ForceEarlyReturn001/ForceEarlyReturn001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ForceGarbageCollection/forcegc001/forcegc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ForceGarbageCollection/forcegc002/forcegc002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GarbageCollectionFinish/gcfinish001/gcfinish001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GarbageCollectionStart/gcstart001/gcstart001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GarbageCollectionStart/gcstart002/gcstart002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GenerateEvents/genevents001/genevents001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/InterruptThread/intrpthrd001/intrpthrd001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IsMethodObsolete/isobsolete001/isobsolete001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverHeap/iterheap001/iterheap001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverHeap/iterheap002/iterheap002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverHeap/iterheap003/iterheap003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverHeap/iterheap004/iterheap004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverHeap/iterheap005/iterheap005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverHeap/iterheap006/iterheap006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverHeap/iterheap007/iterheap007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverInstancesOfClass/iterinstcls001/iterinstcls001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverInstancesOfClass/iterinstcls002/iterinstcls002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverInstancesOfClass/iterinstcls003/iterinstcls003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverInstancesOfClass/iterinstcls004/iterinstcls004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverInstancesOfClass/iterinstcls005/iterinstcls005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverInstancesOfClass/iterinstcls006/iterinstcls006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverInstancesOfClass/iterinstcls007/iterinstcls007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverObjectsReachableFromObject/iterobjreachobj001/iterobjreachobj001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverObjectsReachableFromObject/iterobjreachobj002/iterobjreachobj002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverObjectsReachableFromObject/iterobjreachobj003/iterobjreachobj003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverObjectsReachableFromObject/iterobjreachobj004/iterobjreachobj004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverObjectsReachableFromObject/iterobjreachobj005/iterobjreachobj005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverReachableObjects/iterreachobj001/iterreachobj001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverReachableObjects/iterreachobj002/iterreachobj002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverReachableObjects/iterreachobj003/iterreachobj003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverReachableObjects/iterreachobj004/iterreachobj004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateOverReachableObjects/iterreachobj005/iterreachobj005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateThroughHeap/abort/Abort.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateThroughHeap/callbacks/Callbacks.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateThroughHeap/concrete-klass-filter/ConcreteKlassFilter.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateThroughHeap/filter-tagged/HeapFilter.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateThroughHeap/non-concrete-klass-filter/NonConcreteKlassFilter.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MonitorContendedEnter/mcontenter001/mcontenter001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MonitorContendedEntered/mcontentered001/mcontentered001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MonitorWait/monitorwait001/monitorwait001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/MonitorWaited/monitorwaited001/monitorwaited001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind001/nativemethbind001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind002/nativemethbind002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind003/nativemethbind003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind004/nativemethbind004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ObjectFree/objfree001/objfree001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ObjectFree/objfree002/objfree002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe005/popframe005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass028/redefclass028.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass029/redefclass029.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass030/redefclass030.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RelinquishCapabilities/relcaps001/relcaps001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RelinquishCapabilities/relcaps002/relcaps002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ResumeThread/resumethrd001/resumethrd001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ResumeThread/resumethrd002/resumethrd002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ResumeThreadList/resumethrdlst001/resumethrdlst001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ResumeThreadList/resumethrdlst002/resumethrdlst002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RetransformClasses/retransform002/retransform002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RetransformClasses/retransform003/retransform003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RetransformClasses/retransform004/retransform004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetEnvironmentLocalStorage/setenvstor001/setenvstor001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetEnvironmentLocalStorage/setenvstor002/setenvstor002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetEnvironmentLocalStorage/setenvstor003/setenvstor003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetEventCallbacks/setevntcallb001/setevntcallb001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetEventCallbacks/setevntcallb002/setevntcallb002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetEventCallbacks/setevntcallb003/setevntcallb003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetExtensionEventCallback/setextevent001/setextevent001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix001/SetNativeMethodPrefix001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/SetNativeMethodPrefix002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/SetNativeMethodPrefix002Main.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetSystemProperty/setsysprop002/setsysprop002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetSystemProperty/setsysprop003/setsysprop003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetTag/settag001/settag001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetThreadLocalStorage/setthrdstor001/setthrdstor001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetThreadLocalStorage/setthrdstor002/setthrdstor002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetThreadLocalStorage/setthrdstor003/setthrdstor003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetVerboseFlag/setvrbflag001/setvrbflag001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetVerboseFlag/setvrbflag002/setvrbflag002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SingleStep/singlestep001/singlestep001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SingleStep/singlestep002/singlestep002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SingleStep/singlestep003/singlestep003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/StopThread/stopthrd007/stopthrd007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SuspendThread/suspendthrd001/suspendthrd001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SuspendThread/suspendthrd002/suspendthrd002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SuspendThread/suspendthrd003/suspendthrd003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SuspendThreadList/suspendthrdlst001/suspendthrdlst001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SuspendThreadList/suspendthrdlst002/suspendthrdlst002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ThreadEnd/threadend002/threadend002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ThreadStart/threadstart002/threadstart002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/VMObjectAlloc/vmobjalloc001/vmobjalloc001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP01/ap01t001/ap01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP02/ap02t001/ap02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP03/ap03t001/ap03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t001/ap04t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t002/ap04t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP05/ap05t001/ap05t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP05/ap05t002/ap05t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP06/ap06t001/ap06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP07/ap07t001/ap07t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP07/ap07t002/ap07t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP09/ap09t001/ap09t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP10/ap10t001/ap10t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP11/ap11t001/ap11t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP12/ap12t001/ap12t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/bcinstr/BI01/bi01t001/bi01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/bcinstr/BI01/bi01t002/bi01t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/bcinstr/BI02/bi02t001/bi02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/bcinstr/BI02/bi02t002/bi02t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/bcinstr/BI03/bi03t001/bi03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/bcinstr/BI03/bi03t002/bi03t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/bcinstr/BI04/bi04t002/bi04t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t001/cm01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t002/cm01t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t003/cm01t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t004/cm01t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t005/cm01t005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t006/cm01t006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t007/cm01t007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t008/cm01t008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t009/cm01t009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t010/cm01t010.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t011/cm01t011.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t012/cm01t012.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t013/cm01t013.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t014/cm01t014.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t015/cm01t015.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t016/cm01t016.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t017/cm01t017.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t018/cm01t018.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t019/cm01t019.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t020/cm01t020.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM01/cm01t021/cm01t021.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM03/cm03t001/cm03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/contention/TC01/tc01t001/tc01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/contention/TC02/tc02t001/tc02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/contention/TC03/tc03t001/tc03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/contention/TC03/tc03t002/tc03t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/contention/TC04/tc04t001/tc04t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/contention/TC05/tc05t001/tc05t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM01/em01t001/em01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM01/em01t002/em01t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t001/em02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t002/em02t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t003/em02t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t004/em02t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t005/em02t005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t006/em02t006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t007/em02t007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t008/em02t008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t009/em02t009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t010/em02t010.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t011/em02t011.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM02/em02t012/em02t012.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM04/em04t001/em04t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM05/em05t001/em05t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM05/em05t002/em05t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM06/em06t001/em06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM07/em07t001/em07t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/events/EM07/em07t002/em07t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/extension/EX03/ex03t001/ex03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF01/gf01t001/gf01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF04/gf04t001/gf04t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF08/gf08t001/gf08t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF08/gf08t002/gf08t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/general_functions/GF08/gf08t003/gf08t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS103/hs103t002/hs103t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS104/hs104t001/hs104t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS104/hs104t002/hs104t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS201/hs201t001/hs201t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS201/hs201t002/hs201t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS201/hs201t003/hs201t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS202/hs202t001/hs202t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS202/hs202t002/hs202t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t001/hs203t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t002/hs203t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t003/hs203t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t004/hs203t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t001/hs204t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t002/hs204t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t002/hs204t002.h
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t003/hs204t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t004/hs204t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS301/hs301t001/hs301t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS301/hs301t002/hs301t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS301/hs301t003/hs301t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS301/hs301t004/hs301t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS301/hs301t005/hs301t005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t001/hs302t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t002/hs302t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t003/hs302t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t004/hs302t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t005/hs302t005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t006/hs302t006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t007/hs302t007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t008/hs302t008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t009/hs302t009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t010/hs302t010.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t011/hs302t011.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS302/hs302t012/hs302t012.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI01/ji01t001/ji01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA01/ma01t001/ma01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA01/ma01t001/ma01t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA02/ma02t001/ma02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA02/ma02t001/ma02t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA03/ma03t001/ma03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA03/ma03t001/ma03t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA04/ma04t001/ma04t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA04/ma04t001/ma04t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA04/ma04t002/ma04t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA04/ma04t002/ma04t002a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA04/ma04t003/ma04t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA04/ma04t003/ma04t003a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA05/ma05t001/ma05t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA05/ma05t001/ma05t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA06/ma06t001/ma06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA06/ma06t001/ma06t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA07/ma07t001/ma07t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA07/ma07t001/ma07t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA08/ma08t001/ma08t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA08/ma08t001/ma08t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t001/ma10t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t001/ma10t001a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t002/ma10t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t002/ma10t002a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t003/ma10t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t003/ma10t003a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t004/ma10t004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t004/ma10t004a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t005/ma10t005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t005/ma10t005a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t006/ma10t006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t006/ma10t006a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t007/ma10t007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t007/ma10t007a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t008/ma10t008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/multienv/MA10/ma10t008/ma10t008a.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP01/sp01t001/sp01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP01/sp01t002/sp01t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP01/sp01t003/sp01t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP02/sp02t001/sp02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP02/sp02t002/sp02t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP02/sp02t003/sp02t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP03/sp03t001/sp03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP03/sp03t002/sp03t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP04/sp04t001/sp04t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP04/sp04t002/sp04t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP05/sp05t002/sp05t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP05/sp05t003/sp05t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t001/sp06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t002/sp06t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t003/sp06t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP07/sp07t001/sp07t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/sampling/SP07/sp07t002/sp07t002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref001/followref001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref002/followref002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref003/followref003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref004/followref004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref005/followref005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/FollowReferences/followref006/followref006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/extmech/extmech.cpp
! test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/StackTraceController.cpp
! test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/ThreadController.cpp
! test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/thread/Deadlock.cpp
! test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/thread/LockingThreads.cpp
! test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/thread/RecursiveMonitoringThread.cpp
! test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/thread/libNativeBlockedThread.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/JVMDITools.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/JVMDITools.h
! test/hotspot/jtreg/vmTestbase/nsk/share/JVMTIagent.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/aod/aod.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/aod/aod.h
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/BooleanArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/ByteArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/CharArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/DoubleArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/FloatArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/IntArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/LongArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/ShortArrayCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jni/StringCriticalLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jniref/JNIGlobalRefLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jniref/JNILocalRefLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jniref/JNIRefLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jniref/JNIWeakGlobalRefLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/jvmti/JVMTIAllocLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/gc/lock/malloc/MallocLocker.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jni/jni_tools.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jni/jni_tools.h
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/AddToBootstrapClassLoaderSearch/bootclssearch_agent.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/AddToSystemClassLoaderSearch/systemclssearch_agent.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/Injector.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/Injector.h
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/JVMTITools.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/JVMTITools.h
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/agent_common/agent_common.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/agent_common/agent_common.h
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/agent_tools.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/aod/jvmti_aod.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/aod/jvmti_aod.h
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/hotswap/HotSwap.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_FollowRefObjects.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_FollowRefObjects.h
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h
! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/unit/Heap.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/native/native_thread.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/native/native_thread.h
! test/hotspot/jtreg/vmTestbase/nsk/share/native/native_utils.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/native/nsk_list.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/native/nsk_list.h
! test/hotspot/jtreg/vmTestbase/nsk/share/native/nsk_mutex.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/native/nsk_mutex.h
! test/hotspot/jtreg/vmTestbase/nsk/share/native/nsk_tools.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/native/nsk_tools.h
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/gclocker/libgcl001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/libjnistress001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/libjnistress002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/libjnistress003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/libjnistress004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/libjnistress005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/libjnistress006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/jni/libjnistress007.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace011.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace012.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace014.cpp
! test/hotspot/jtreg/vmTestbase/nsk/stress/strace/strace015.cpp
! test/hotspot/jtreg/vmTestbase/vm/jit/LongTransitions/libLTTest.cpp
! test/hotspot/jtreg/vmTestbase/vm/mlvm/indy/func/jvmti/share/IndyRedefineClass.cpp
! test/hotspot/jtreg/vmTestbase/vm/mlvm/indy/func/jvmti/stepBreakPopReturn/stepBreakPopReturn.cpp
! test/hotspot/jtreg/vmTestbase/vm/mlvm/meth/stress/jni/nativeAndMH/nativeAndMH.cpp
! test/hotspot/jtreg/vmTestbase/vm/mlvm/share/mlvmJvmtiUtils.cpp
! test/hotspot/jtreg/vmTestbase/vm/mlvm/share/mlvmJvmtiUtils.h
! test/hotspot/jtreg/vmTestbase/vm/share/ProcessUtils.cpp

Changeset: dfed97156841
Author:    jcbeyler
Date:      2018-09-17 19:48 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/dfed97156841

8210700: Clean up JNI_ENV_ARG and factorize the macros for vmTestbase/jvmti/unit tests
Summary: Remove JNI_ENV macros from the remaining vmTestbase tests
Reviewed-by: amenkov, sspitsyn, dholmes, cjplummer

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretfp/earlyretfp.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretint/earlyretint.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretlong/earlyretlong.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretobj/earlyretobj.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretstr/earlyretstr.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretvoid/earlyretvoid.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001/getallstktr001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/GetConstantPool/getcpool001/getcpool001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/GetLineNumberTable/linetab004/linetab004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/GetLocalVariable/getlocal003/getlocal003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/GetLocalVariable/getlocal004/getlocal004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/IsSynthetic/issynth001/issynth001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/MethodBind/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/OnUnload/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/StackTrace/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/agentthr/agentthr.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/clsldrclss00x/clsldrclss00x.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/events/redefineCFLH/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/functions/AddToBootstrapClassLoaderSearch/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/functions/Dispose/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/functions/ForceGarbageCollection/gc/gc.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/functions/environment/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/functions/nosuspendMonitorInfo/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/functions/nosuspendStackTrace/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/functions/rawmonitor/rawmonitor.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/heapref/heapref.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/refignore/refignore.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/setNullVMInit/JvmtiTest/JvmtiTest.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/timers/JvmtiTest/JvmtiTest.cpp

Changeset: 925d79f56c05
Author:    fyuan
Date:      2018-09-18 11:09 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/925d79f56c05

8210819: Update the host name in CNameTest.java
Reviewed-by: chegar

! test/jdk/sun/net/InetAddress/nameservice/dns/CNameTest.java

Changeset: 9c6d5e31618e
Author:    mhorie
Date:      2018-09-17 23:35 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/9c6d5e31618e

8210660: PPC64: Mapping floating point registers to vsx registers in ppc.ad
Reviewed-by: mdoerr, gromero

! src/hotspot/cpu/ppc/assembler_ppc.hpp
! src/hotspot/cpu/ppc/assembler_ppc.inline.hpp
! src/hotspot/cpu/ppc/ppc.ad
! src/hotspot/cpu/ppc/register_ppc.cpp
! src/hotspot/cpu/ppc/register_ppc.hpp

Changeset: 300523d8b7b3
Author:    mli
Date:      2018-09-18 13:24 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/300523d8b7b3

8210802: temp files left by tests in jdk/java/net/httpclient
Reviewed-by: chegar, clanger

! test/jdk/java/net/httpclient/EchoHandler.java

Changeset: 1740b162dc0e
Author:    mchung
Date:      2018-09-17 22:56 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/1740b162dc0e

8210841: test/jdk/vm/runtime/ReflectStackOverflow.java fails with NoClassDefFoundError
Reviewed-by: dholmes, mikael

! src/java.base/share/classes/java/lang/reflect/InvocationTargetException.java

Changeset: 23e7cd5a9c54
Author:    ihse
Date:      2018-09-18 10:29 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/23e7cd5a9c54

8210723: Better information in configure for invalid Xcode
Reviewed-by: erikj

! make/autoconf/toolchain.m4

Changeset: dd737bf6abde
Author:    ihse
Date:      2018-09-18 10:35 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/dd737bf6abde

8210750: Clean up compare.sh exceptions
Reviewed-by: erikj

! make/scripts/compare.sh
! make/scripts/compare_exceptions.sh.incl

Changeset: a50956e2368c
Author:    sgehwolf
Date:      2018-09-11 18:18 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/a50956e2368c

8210425: [x86] sharedRuntimeTrig/sharedRuntimeTrans compiled without optimization
Summary: Compile with -O2 and -ffp-contract=off as for fdlibm.
Reviewed-by: erikj, lucy, aph

! make/hotspot/lib/JvmOverrideFiles.gmk

Changeset: 4482acfef2a5
Author:    ihse
Date:      2018-09-18 13:32 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/4482acfef2a5

8210729: Clean up macosx static library handling
Reviewed-by: erikj

! make/autoconf/flags-other.m4
! make/autoconf/flags.m4
! make/launcher/LauncherCommon.gmk
! make/lib/CoreLibraries.gmk
! make/lib/Lib-java.instrument.gmk

Changeset: fbec908e2783
Author:    coleenp
Date:      2018-09-18 08:27 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/fbec908e2783

8203466: intermittent crash at jdk.internal.misc.Unsafe::getObjectVolatile (native)
Summary: Store rsi, rdi on thread local memory, store r15 in r9, for the stubs that have gc barriers
Reviewed-by: dlong, eosterlund

! src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
! src/hotspot/os_cpu/windows_x86/thread_windows_x86.hpp

Changeset: 05b05af6c160
Author:    jiangli
Date:      2018-09-18 11:55 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/05b05af6c160

8210237: [TESTBUG]gc/stress/TestStressIHOPMultiThread.java fails with 'Unexpected exit from test [exit code: 1]' in CDS mode
Summary: Fix TestStressIHOPMultiThread to handle possible OutOfMemoryError.
Reviewed-by: ccheung, gziemski

! test/hotspot/jtreg/gc/stress/TestStressIHOPMultiThread.java

Changeset: c93f14a4ae29
Author:    joehw
Date:      2018-09-18 09:44 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/c93f14a4ae29

8207760: SAXException: Invalid UTF-16 surrogate detected: d83c ?
Reviewed-by: lancea, dfuchs

! src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/ToHTMLStream.java
! src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/ToStream.java
! src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/ToTextStream.java
+ test/jaxp/javax/xml/jaxp/unittest/transform/JDK8207760.java

Changeset: ba51515b64e5
Author:    sherman
Date:      2018-09-18 10:43 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/ba51515b64e5

8034802: (zipfs) newFileSystem throws UOE when the zip file is located in a custom file system
Reviewed-by: xiaofeya, clanger

+ src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java
! src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java
! src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java
! test/jdk/jdk/nio/zipfs/ZipFSTester.java

Changeset: c0f9161f591e
Author:    jgeorge
Date:      2018-09-18 23:20 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/c0f9161f591e

8210836: Build fails with warn_unused_result in openjdk/src/jdk.hotspot.agent/linux/native/libsaproc/ps_core.c
Summary: Read in and process the return value of pread() while dealing with the PT_INTERP segment
Reviewed-by: stuefe, jcbeyler

! src/jdk.hotspot.agent/linux/native/libsaproc/ps_core.c

Changeset: 0535b5a54b83
Author:    naoto
Date:      2018-09-18 12:42 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/0535b5a54b83

8209880: tzdb.dat is not reproducibly built
Reviewed-by: erikj, rriggs

! make/jdk/src/classes/build/tools/tzdb/TzdbZoneRulesCompiler.java
! make/jdk/src/classes/build/tools/tzdb/TzdbZoneRulesProvider.java

Changeset: 62a8579bb6f1
Author:    bpb
Date:      2018-09-18 13:18 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/62a8579bb6f1

8210817: Minor typo in java.nio.file.attribute package summary
Reviewed-by: dfuchs, alanb

! src/java.base/share/classes/java/nio/file/attribute/package-info.java

Changeset: 4f48bad8220f
Author:    joehw
Date:      2018-09-18 13:31 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/4f48bad8220f

8210874: Test for JDK-8209615
Reviewed-by: lancea

+ test/jaxp/javax/xml/jaxp/unittest/stream/XMLEventReaderTest/JDK8209615.java

Changeset: d0dfb9775c08
Author:    dnsimon
Date:      2018-09-18 22:32 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d0dfb9775c08

8210793: [JVMCI] AllocateCompileIdTest.java failed to find DiagnosticCommand.class
Reviewed-by: thartmann, kvn

! test/hotspot/jtreg/compiler/jvmci/compilerToVM/AllocateCompileIdTest.java

Changeset: 6cd1ee2b4fa1
Author:    coleenp
Date:      2018-09-18 16:11 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/6cd1ee2b4fa1

8210861: Move assert to help diagnose rare RedefineStress crash
Summary: assert that Method being marked on stack hasn't been missed by previous metadata walk
Reviewed-by: lfoltan

! src/hotspot/share/oops/method.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption/TestDescription.java

Changeset: 4129f43607cb
Author:    pliden
Date:      2018-09-18 22:46 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/4129f43607cb

8210753: Make ThreadLocalAllocBuffer::resize() public
Reviewed-by: eosterlund, jcbeyler

! src/hotspot/share/gc/shared/collectedHeap.cpp
! src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp
! src/hotspot/share/gc/shared/threadLocalAllocBuffer.hpp

Changeset: feb4c9e03aed
Author:    sherman
Date:      2018-09-18 19:44 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/feb4c9e03aed

8210899: (zipfs) ZipFileSystem.EntryOutputStreamCRC32 mistakenly set the crc32 value into size field
Reviewed-by: bpb

! src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java
! test/jdk/jdk/nio/zipfs/ZipFSTester.java

Changeset: 9d3a00c8c047
Author:    iklam
Date:      2018-09-18 21:46 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/9d3a00c8c047

8210864: Reduce the use of metaspaceShared.hpp
Reviewed-by: coleenp, lfoltan

! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/stringTable.cpp
! src/hotspot/share/gc/g1/g1CollectedHeap.cpp
! src/hotspot/share/gc/serial/markSweep.inline.hpp
! src/hotspot/share/memory/heapShared.cpp
! src/hotspot/share/memory/metaspaceShared.cpp
! src/hotspot/share/memory/metaspaceShared.hpp
+ src/hotspot/share/memory/metaspaceShared.inline.hpp
! src/hotspot/share/oops/oop.cpp
! src/hotspot/share/oops/oop.hpp
! src/hotspot/share/oops/oop.inline.hpp
! src/hotspot/share/prims/whitebox.cpp
! src/hotspot/share/runtime/flags/jvmFlagRangeList.hpp

Changeset: 3efead10e303
Author:    pmuthuswamy
Date:      2018-09-19 12:14 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/3efead10e303

8210047: some pages contain content outside of landmark region
Reviewed-by: jjg

! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractModuleIndexWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractPackageIndexWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AllClassesFrameWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDocletWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModuleIndexFrameWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModuleIndexWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModulePackageIndexFrameWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PackageIndexFrameWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PackageIndexWriter.java
+ test/langtools/jdk/javadoc/doclet/testHtmlLandmarkRegions/TestHtmlLankmarkRegions.java
! test/langtools/jdk/javadoc/doclet/testHtmlVersion/TestHtmlVersion.java
! test/langtools/jdk/javadoc/doclet/testOverview/TestOverview.java

Changeset: f55a4bc91ef4
Author:    alanb
Date:      2018-09-19 08:49 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/f55a4bc91ef4

8210496: Improve filtering for classes with security sensitive fields
Reviewed-by: plevart, mchung

! src/java.base/share/classes/java/lang/invoke/MethodHandles.java
! src/java.base/share/classes/jdk/internal/reflect/ConstantPool.java
! src/java.base/share/classes/jdk/internal/reflect/Reflection.java
! src/java.base/share/classes/jdk/internal/reflect/UnsafeStaticFieldAccessorImpl.java
! src/jdk.unsupported/share/classes/sun/misc/Unsafe.java
! test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java
! test/jdk/java/lang/reflect/callerCache/AccessTest.java
+ test/jdk/jdk/internal/reflect/Reflection/Filtering.java

Changeset: 3fabe59fe4de
Author:    jlahoda
Date:      2018-09-19 10:50 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/3fabe59fe4de

8207954: Data for --release 11
Summary: Adding support for --release 11
Reviewed-by: erikj, ihse, jjg

+ make/data/symbols/java.activation-B.sym.txt
+ make/data/symbols/java.base-B.sym.txt
+ make/data/symbols/java.compiler-B.sym.txt
+ make/data/symbols/java.corba-B.sym.txt
+ make/data/symbols/java.datatransfer-B.sym.txt
+ make/data/symbols/java.desktop-B.sym.txt
+ make/data/symbols/java.instrument-B.sym.txt
+ make/data/symbols/java.logging-B.sym.txt
+ make/data/symbols/java.management-B.sym.txt
+ make/data/symbols/java.management.rmi-B.sym.txt
+ make/data/symbols/java.naming-B.sym.txt
+ make/data/symbols/java.net.http-B.sym.txt
+ make/data/symbols/java.prefs-B.sym.txt
+ make/data/symbols/java.rmi-B.sym.txt
+ make/data/symbols/java.scripting-B.sym.txt
+ make/data/symbols/java.se-B.sym.txt
+ make/data/symbols/java.se.ee-B.sym.txt
+ make/data/symbols/java.security.jgss-B.sym.txt
+ make/data/symbols/java.security.sasl-B.sym.txt
+ make/data/symbols/java.smartcardio-B.sym.txt
+ make/data/symbols/java.sql-B.sym.txt
+ make/data/symbols/java.sql.rowset-B.sym.txt
+ make/data/symbols/java.transaction-B.sym.txt
+ make/data/symbols/java.transaction.xa-B.sym.txt
+ make/data/symbols/java.xml-B.sym.txt
+ make/data/symbols/java.xml.bind-B.sym.txt
+ make/data/symbols/java.xml.crypto-B.sym.txt
+ make/data/symbols/java.xml.ws-B.sym.txt
+ make/data/symbols/java.xml.ws.annotation-B.sym.txt
+ make/data/symbols/jdk.accessibility-B.sym.txt
+ make/data/symbols/jdk.attach-B.sym.txt
+ make/data/symbols/jdk.charsets-B.sym.txt
+ make/data/symbols/jdk.compiler-B.sym.txt
+ make/data/symbols/jdk.crypto.cryptoki-B.sym.txt
+ make/data/symbols/jdk.crypto.ec-B.sym.txt
+ make/data/symbols/jdk.dynalink-B.sym.txt
+ make/data/symbols/jdk.editpad-B.sym.txt
+ make/data/symbols/jdk.hotspot.agent-B.sym.txt
+ make/data/symbols/jdk.httpserver-B.sym.txt
+ make/data/symbols/jdk.incubator.httpclient-B.sym.txt
+ make/data/symbols/jdk.jartool-B.sym.txt
+ make/data/symbols/jdk.javadoc-B.sym.txt
+ make/data/symbols/jdk.jcmd-B.sym.txt
+ make/data/symbols/jdk.jconsole-B.sym.txt
+ make/data/symbols/jdk.jdeps-B.sym.txt
+ make/data/symbols/jdk.jdi-B.sym.txt
+ make/data/symbols/jdk.jdwp.agent-B.sym.txt
+ make/data/symbols/jdk.jfr-B.sym.txt
+ make/data/symbols/jdk.jlink-B.sym.txt
+ make/data/symbols/jdk.jshell-B.sym.txt
+ make/data/symbols/jdk.jsobject-B.sym.txt
+ make/data/symbols/jdk.jstatd-B.sym.txt
+ make/data/symbols/jdk.localedata-B.sym.txt
+ make/data/symbols/jdk.management-B.sym.txt
+ make/data/symbols/jdk.management.agent-B.sym.txt
+ make/data/symbols/jdk.management.jfr-B.sym.txt
+ make/data/symbols/jdk.naming.dns-B.sym.txt
+ make/data/symbols/jdk.naming.rmi-B.sym.txt
+ make/data/symbols/jdk.net-B.sym.txt
+ make/data/symbols/jdk.pack-B.sym.txt
+ make/data/symbols/jdk.rmic-B.sym.txt
+ make/data/symbols/jdk.scripting.nashorn-B.sym.txt
+ make/data/symbols/jdk.sctp-B.sym.txt
+ make/data/symbols/jdk.security.auth-B.sym.txt
+ make/data/symbols/jdk.security.jgss-B.sym.txt
+ make/data/symbols/jdk.unsupported-B.sym.txt
+ make/data/symbols/jdk.xml.dom-B.sym.txt
+ make/data/symbols/jdk.zipfs-B.sym.txt
! make/data/symbols/symbols
! make/gendata/Gendata-jdk.compiler.gmk
! make/langtools/src/classes/build/tools/symbolgenerator/CreateSymbols.java
+ test/langtools/tools/javac/platform/CanHandleClassFilesTest.java

Changeset: bccd9966f1ed
Author:    mbalao
Date:      2018-09-12 13:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/bccd9966f1ed

8029661: Support TLS v1.2 algorithm in SunPKCS11 provider
Summary: TLS v1.2 algorithms for key and MAC derivation added to SunPKCS11 crypto provider.
Reviewed-by: valeriep

! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsKeyMaterialGenerator.java
! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsMasterSecretGenerator.java
! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsPrfGenerator.java
! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsRsaPremasterSecretGenerator.java
! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/SunPKCS11.java
! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/wrapper/CK_MECHANISM.java
+ src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/wrapper/CK_TLS12_KEY_MAT_PARAMS.java
+ src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/wrapper/CK_TLS12_MASTER_KEY_DERIVE_PARAMS.java
+ src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/wrapper/CK_TLS_MAC_PARAMS.java
! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/wrapper/Functions.java
! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/wrapper/PKCS11Constants.java
! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/p11_convert.c
! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/p11_keymgmt.c
! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/pkcs11t.h
! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/pkcs11wrapper.h
+ test/jdk/sun/security/pkcs11/fips/TestTLS12.java
! test/jdk/sun/security/pkcs11/fips/cert8.db
! test/jdk/sun/security/pkcs11/fips/key3.db
! test/jdk/sun/security/pkcs11/fips/keystore

Changeset: 09e8e51c948a
Author:    pliden
Date:      2018-09-19 14:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/09e8e51c948a

8210713: Let CollectedHeap::ensure_parsability() take care of TLAB statistics gathering
Reviewed-by: eosterlund, sjohanss

! src/hotspot/share/gc/g1/g1CollectedHeap.cpp
! src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp
! src/hotspot/share/gc/parallel/parallelScavengeHeap.hpp
! src/hotspot/share/gc/parallel/psMarkSweep.cpp
! src/hotspot/share/gc/parallel/psParallelCompact.cpp
! src/hotspot/share/gc/parallel/psScavenge.cpp
! src/hotspot/share/gc/shared/collectedHeap.cpp
! src/hotspot/share/gc/shared/collectedHeap.hpp
! src/hotspot/share/gc/shared/genCollectedHeap.cpp
! src/hotspot/share/gc/z/zCollectedHeap.hpp
! src/hotspot/share/gc/z/zObjectAllocator.cpp

Changeset: 15094d12a632
Author:    mikael
Date:      2018-09-19 09:24 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/15094d12a632

8210912: Build error in src/jdk.crypto.cryptoki/share/native/libj2pkcs11/p11_convert.c after JDK-8029661
Reviewed-by: mullan

! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/p11_convert.c

Changeset: a16777c0a6c5
Author:    pliden
Date:      2018-09-19 19:12 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/a16777c0a6c5

8210881: ZGC: Introduce ZRootsIteratorClosure
Reviewed-by: eosterlund

! src/hotspot/share/gc/z/zHeap.cpp
! src/hotspot/share/gc/z/zHeapIterator.cpp
! src/hotspot/share/gc/z/zMark.cpp
! src/hotspot/share/gc/z/zOopClosures.hpp
! src/hotspot/share/gc/z/zOopClosures.inline.hpp
! src/hotspot/share/gc/z/zRelocate.cpp
! src/hotspot/share/gc/z/zRootsIterator.cpp
! src/hotspot/share/gc/z/zRootsIterator.hpp

Changeset: eef954e29714
Author:    pliden
Date:      2018-09-19 19:12 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/eef954e29714

8210884: ZGC: Remove insertion of filler objects
Reviewed-by: eosterlund

! src/hotspot/share/gc/z/zPage.inline.hpp
! src/hotspot/share/gc/z/zUtils.cpp
! src/hotspot/share/gc/z/zUtils.hpp

Changeset: eb2adb0a9b09
Author:    martin
Date:      2018-09-19 10:51 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/eb2adb0a9b09

8209817: stack is executable when building with Clang on Linux
Reviewed-by: dholmes, martin, mikael, ihse
Contributed-by: Arthur Eubanks <aeubanks at google.com>

! make/autoconf/flags-ldflags.m4
! make/autoconf/toolchain.m4
! test/hotspot/jtreg/runtime/execstack/TestCheckJDK.java

Changeset: 1ecc914fb707
Author:    rkennke
Date:      2018-09-19 21:31 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/1ecc914fb707

8210829: Modularize allocations in C2
Reviewed-by: kvn, roland

! src/hotspot/share/gc/shared/c2/barrierSetC2.cpp
! src/hotspot/share/gc/shared/c2/barrierSetC2.hpp
! src/hotspot/share/opto/macro.cpp
! src/hotspot/share/opto/macro.hpp

Changeset: 3221f5e14866
Author:    coleenp
Date:      2018-09-19 15:25 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/3221f5e14866

8198717: Remove compute_optional_offset
Summary: remove optional_offset computation and related unused code from javaClasses for reflection.
Reviewed-by: redestad, lfoltan

! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/runtime/reflection.cpp


From maurizio.cimadamore at oracle.com  Wed Sep 19 20:05:31 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 19 Sep 2018 21:05:31 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
Message-ID: <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>

First off - thanks for doing this; it's immensely appreciated.

as for this:

JtregNativeJdk.gmk:96: warning: ignoring old recipe for target 
'/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll'

I'm getting similar warnings too, but for Hello.so. I think that's how 
supposed to work but then again I'm not a build guru and I haven't 
digged too deep (yet) to figure out what's causing that. But I get that 
too, so I think we can ignore that for the time being.

I agree that we should focus on Win x64.

About the 'cannot locate clang' that's the first hurdle that we should 
be able to solve. First question: what's in your /cygdrive/j/LLVM 
folder? Is that a straight download of the LLVM binaries?

Maurizio




On 19/09/18 18:50, Jorn Vernee wrote:
> Henry Jen schreef op 2018-09-19 19:11:
>> On Sep 19, 2018, at 9:07 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>>>
>>> Henry Jen schreef op 2018-09-19 17:23:
>>>> On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>>>
>>>>> And then of course one of the biggest things is getting jextract 
>>>>> to build and run. Currently the build system is not picking up 
>>>>> libclang. When explicitly specifying it with 
>>>>> --with-libclang='/cygdrive/j/LLVM' I still get the "Cannot locate 
>>>>> libclang!" warning. I wanted to ask what value and files it's 
>>>>> actually looking for? (I've also tried some of the additional 
>>>>> flags from makeautoconf/lib-clang.m4, but to no avail).
>>>> ?with-libclang should point to a directory has include/clang-c and
>>>> lib/libclang.dll. If include and lib are not under the same directory,
>>>> you can use ?with-libclang-include and ?with-libclang-lib instead.
>>>
>>> I have tried all of these flags together:
>>>
>>> --with-libclang=/cygdrive/j/LLVM 
>>> --with-libclang-include=/cygdrive/j/LLVM/include 
>>> --with-libclang-include-aux=/cygdrive/j/LLVM/lib 
>>> --with-libclang-lib=/cygdrive/j/LLVM/bin
>>>
>>
>> Try remove ?with-libclang and use the rest three. Explicit setting
>> ?with-libclang cause the other options to be ignored.
>
> Still the same problem I'm afraid.
>
> It's probably kind of hard to debug this remotely, so for now I'll 
> focus on getting the other tests to work, and come back to this later.
>
> Thanks,
> Jorn


From maurizio.cimadamore at oracle.com  Wed Sep 19 20:06:27 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 19 Sep 2018 21:06:27 +0100
Subject: [foreign] RFR 8210911 jextract does not handle redundant forward, 
 backward declarations of struct, union, enum properly
In-Reply-To: <5BA259A2.5050106@oracle.com>
References: <5BA259A2.5050106@oracle.com>
Message-ID: <3053630f-4e31-59c1-2795-65a96ceb2419@oracle.com>

Looks good!

Maurizio


On 19/09/18 15:13, Sundararajan Athijegannathan wrote:
> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8210911
> Webrev: http://cr.openjdk.java.net/~sundar/8210911/webrev.00/
>
> PS. Piggybacking to add a TreePhase interface for visitors.
>
> Thanks
> -Sundar


From maurizio.cimadamore at oracle.com  Wed Sep 19 20:06:37 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 19 Sep 2018 20:06:37 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809192006.w8JK6cLd009755@aojmv0008.oracle.com>

Changeset: 0cc7a6ad0591
Author:    mcimadamore
Date:      2018-09-19 22:08 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/0cc7a6ad0591

Automatic merge with default

! make/autoconf/toolchain.m4
- make/jdk/src/classes/build/tools/hasher/Hasher.java
- make/jdk/src/classes/build/tools/jarreorder/JarReorder.java
- make/mapfiles/libjsound/mapfile-vers
- make/mapfiles/libjvm_db/mapfile-vers
- make/mapfiles/libjvm_dtrace/mapfile-vers
! src/hotspot/cpu/arm/arm.ad
! src/hotspot/cpu/arm/sharedRuntime_arm.cpp
! src/hotspot/cpu/ppc/ppc.ad
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp
! src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
! src/hotspot/share/opto/macro.cpp
! src/hotspot/share/opto/macro.hpp
- src/java.base/windows/conf/tzmappings
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewArrayStub.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewInstanceStub.java
- test/jdk/java/util/ServiceLoader/basic/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Utils.java


From maurizio.cimadamore at oracle.com  Wed Sep 19 20:07:03 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 19 Sep 2018 20:07:03 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809192007.w8JK74JQ010127@aojmv0008.oracle.com>

Changeset: d874c711ca95
Author:    mcimadamore
Date:      2018-09-19 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d874c711ca95

Automatic merge with default

- make/jdk/src/classes/build/tools/hasher/Hasher.java
- make/jdk/src/classes/build/tools/jarreorder/JarReorder.java
- make/mapfiles/libjsound/mapfile-vers
- make/mapfiles/libjvm_db/mapfile-vers
- make/mapfiles/libjvm_dtrace/mapfile-vers
- src/java.base/windows/conf/tzmappings
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewArrayStub.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewInstanceStub.java
- test/jdk/java/util/ServiceLoader/basic/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Utils.java


From maurizio.cimadamore at oracle.com  Wed Sep 19 20:07:25 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 19 Sep 2018 20:07:25 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809192007.w8JK7PfU010454@aojmv0008.oracle.com>

Changeset: a6fd68a623c8
Author:    mcimadamore
Date:      2018-09-19 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/a6fd68a623c8

Automatic merge with default

- make/jdk/src/classes/build/tools/hasher/Hasher.java
- make/jdk/src/classes/build/tools/jarreorder/JarReorder.java
- make/mapfiles/libjsound/mapfile-vers
- make/mapfiles/libjvm_db/mapfile-vers
- make/mapfiles/libjvm_dtrace/mapfile-vers
! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/arm/sharedRuntime_arm.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/oops/method.cpp
! src/hotspot/share/opto/macro.cpp
! src/hotspot/share/opto/macro.hpp
! src/java.base/share/classes/java/lang/invoke/MethodHandles.java
- src/java.base/windows/conf/tzmappings
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewArrayStub.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewInstanceStub.java
- test/jdk/java/util/ServiceLoader/basic/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Utils.java


From maurizio.cimadamore at oracle.com  Wed Sep 19 20:07:47 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 19 Sep 2018 20:07:47 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809192007.w8JK7lNm010854@aojmv0008.oracle.com>

Changeset: f76783e51259
Author:    mcimadamore
Date:      2018-09-19 22:10 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f76783e51259

Automatic merge with default

- make/jdk/src/classes/build/tools/hasher/Hasher.java
- make/jdk/src/classes/build/tools/jarreorder/JarReorder.java
- make/mapfiles/libjsound/mapfile-vers
- make/mapfiles/libjvm_db/mapfile-vers
- make/mapfiles/libjvm_dtrace/mapfile-vers
! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/oops/method.cpp
! src/java.base/share/classes/java/lang/invoke/MethodHandles.java
- src/java.base/windows/conf/tzmappings
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewArrayStub.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewInstanceStub.java
- test/jdk/java/util/ServiceLoader/basic/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Utils.java


From jbvernee at xs4all.nl  Wed Sep 19 20:51:51 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 19 Sep 2018 22:51:51 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <849B98C3-4E66-4EF3-A66C-D91A2023410E@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <849B98C3-4E66-4EF3-A66C-D91A2023410E@oracle.com>
Message-ID: <dfea0fd67a60efcdb4334cb3064bc1ab@xs4all.nl>

Henry Jen schreef op 2018-09-19 20:49:
>> On Sep 19, 2018, at 10:50 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>> 
>> Henry Jen schreef op 2018-09-19 19:11:
>>> On Sep 19, 2018, at 9:07 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>>>> Henry Jen schreef op 2018-09-19 17:23:
>>>>> On Sep 19, 2018, at 5:00 AM, Jorn Vernee <jbvernee at xs4all.nl> 
>>>>> wrote:
>>>>>> And then of course one of the biggest things is getting jextract 
>>>>>> to build and run. Currently the build system is not picking up 
>>>>>> libclang. When explicitly specifying it with 
>>>>>> --with-libclang='/cygdrive/j/LLVM' I still get the "Cannot locate 
>>>>>> libclang!" warning. I wanted to ask what value and files it's 
>>>>>> actually looking for? (I've also tried some of the additional 
>>>>>> flags from makeautoconf/lib-clang.m4, but to no avail).
>>>>> ?with-libclang should point to a directory has include/clang-c and
>>>>> lib/libclang.dll. If include and lib are not under the same 
>>>>> directory,
>>>>> you can use ?with-libclang-include and ?with-libclang-lib instead.
>>>> I have tried all of these flags together:
>>>> --with-libclang=/cygdrive/j/LLVM 
>>>> --with-libclang-include=/cygdrive/j/LLVM/include 
>>>> --with-libclang-include-aux=/cygdrive/j/LLVM/lib 
>>>> --with-libclang-lib=/cygdrive/j/LLVM/bin
>>> Try remove ?with-libclang and use the rest three. Explicit setting
>>> ?with-libclang cause the other options to be ignored.
>> 
>> Still the same problem I'm afraid.
>> 
>> It's probably kind of hard to debug this remotely, so for now I'll 
>> focus on getting the other tests to work, and come back to this later.
>> 
> 
> I believe you still need to point ?with-libclang-lib tp
> /cygdrive/j/LLVM/lib to link with the library.
> Just to test the theory, if you can try following,
> 
> 1. Make sure j:\LLVM\bin is in your PATH environment variable,
> 2. Use only ?with-libclang=/cygdrive/j/LLVM
> 
> Above should be enough to get the build detect clang and build.
> However, we have a problem in build system assuming .dll can be copied
> from the lib folder, that we will have to fix.

Sorry, still not detecting it. I'm still seeing:

checking clang-c/Index.h usability... no
checking clang-c/Index.h presence... no
checking for clang-c/Index.h... no
configure: Cannot locate libclang! You can download pre-built llvm
         binary from http://llvm.org/releases/download.html, then specify 
the
         location using --with-libclang

So it seems like it can't find the header files?

I think I've found at least part of the problem though, looking at 
config.log, it looks like:

configure:59831: result: no
configure:60239: checking clang-c/Index.h usability
configure:60239: 
/cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
-c    -I/cygdrive/j/LLVM/include conftest.cpp >&5
conftest.cpp
conftest.cpp(52): fatal error C1083: Cannot open include file: 
'clang-c/Index.h': No such file or directory
Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

I've tested that command with a test file that just has `#include 
"clang-c/Index.h"` in it and it's working when I'm using window's native 
terminal, PowerShell, but not when I use cygwin and using either a unix 
style path `-I/cygdrive/j/LLVM/include` or a windows style path `-I 
J:\LLVM\include`. I thought maybe it had to do with the `-` in the 
folder name, but if I remove that and try again I still get a failing 
result.

So it seems to be a problem with cygwin.

Jorn

From jbvernee at xs4all.nl  Wed Sep 19 20:58:49 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 19 Sep 2018 22:58:49 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
Message-ID: <d71266d2ef2469661f8819d19132678e@xs4all.nl>

Maurizio Cimadamore schreef op 2018-09-19 22:05:
> First off - thanks for doing this; it's immensely appreciated.
> 
> as for this:
> 
> JtregNativeJdk.gmk:96: warning: ignoring old recipe for target
> '/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll'
> 
> I'm getting similar warnings too, but for Hello.so. I think that's how
> supposed to work but then again I'm not a build guru and I haven't
> digged too deep (yet) to figure out what's causing that. But I get
> that too, so I think we can ignore that for the time being.
> 
> I agree that we should focus on Win x64.
> 
> About the 'cannot locate clang' that's the first hurdle that we should
> be able to solve. First question: what's in your /cygdrive/j/LLVM
> folder? Is that a straight download of the LLVM binaries?
> 
> Maurizio

Hi Maurizo,

I just downloaded the pre-built binary of LLVM 6.0.1 for Windows 
(64-bit) from http://releases.llvm.org/download.html which comes as an 
installer that creates the LLVM folder.

I've made a gist with the folder structure here 
https://gist.github.com/JornVernee/dae92a19f4d2036f30153e747b55d874 So 
you can take a look if you want. Everything seems to be in the right 
place, except for the DLL. But the error I'm seeing seems to be from the 
compiler not finding the header files.

Jorn


From maurizio.cimadamore at oracle.com  Wed Sep 19 21:11:27 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 19 Sep 2018 22:11:27 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <d71266d2ef2469661f8819d19132678e@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
Message-ID: <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>



On 19/09/18 21:58, Jorn Vernee wrote:
> Maurizio Cimadamore schreef op 2018-09-19 22:05:
>> First off - thanks for doing this; it's immensely appreciated.
>>
>> as for this:
>>
>> JtregNativeJdk.gmk:96: warning: ignoring old recipe for target
>> '/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll' 
>>
>>
>> I'm getting similar warnings too, but for Hello.so. I think that's how
>> supposed to work but then again I'm not a build guru and I haven't
>> digged too deep (yet) to figure out what's causing that. But I get
>> that too, so I think we can ignore that for the time being.
>>
>> I agree that we should focus on Win x64.
>>
>> About the 'cannot locate clang' that's the first hurdle that we should
>> be able to solve. First question: what's in your /cygdrive/j/LLVM
>> folder? Is that a straight download of the LLVM binaries?
>>
>> Maurizio
>
> Hi Maurizo,
>
> I just downloaded the pre-built binary of LLVM 6.0.1 for Windows 
> (64-bit) from http://releases.llvm.org/download.html which comes as an 
> installer that creates the LLVM folder.
>
> I've made a gist with the folder structure here 
> https://gist.github.com/JornVernee/dae92a19f4d2036f30153e747b55d874 So 
> you can take a look if you want. Everything seems to be in the right 
> place, except for the DLL. But the error I'm seeing seems to be from 
> the compiler not finding the header files.
That seems ok.

On this:

conftest.cpp(52): fatal error C1083: Cannot open include file: 
'clang-c/Index.h': No such file or directory

I also thought that the problem could lie in the forward slash in the 
#include directive, but that seems accepted by MSVC and converted 
accordingly.

So it seems like your issue is: if you try to compile a c/cpp file with 
an include to clang-c/Index.h using MSVC from cygwin, that fails, 
correct? Which in turns trips up the build configure script.

Maurizio



>
> Jorn
>


From maurizio.cimadamore at oracle.com  Wed Sep 19 21:29:12 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 19 Sep 2018 22:29:12 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
Message-ID: <245df6be-fd20-089c-e08c-07130b075b44@oracle.com>

Maybe try with the /showInclude option - to see where cl.exe is looking 
for things? Perhaps some odd path will pop out and point you in the 
right direction?

https://msdn.microsoft.com/en-us/library/hdkef6tk.aspx

Maurizio


On 19/09/18 22:11, Maurizio Cimadamore wrote:
>
>
> On 19/09/18 21:58, Jorn Vernee wrote:
>> Maurizio Cimadamore schreef op 2018-09-19 22:05:
>>> First off - thanks for doing this; it's immensely appreciated.
>>>
>>> as for this:
>>>
>>> JtregNativeJdk.gmk:96: warning: ignoring old recipe for target
>>> '/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/test/jdk/jtreg/native/lib/Hello.dll' 
>>>
>>>
>>> I'm getting similar warnings too, but for Hello.so. I think that's how
>>> supposed to work but then again I'm not a build guru and I haven't
>>> digged too deep (yet) to figure out what's causing that. But I get
>>> that too, so I think we can ignore that for the time being.
>>>
>>> I agree that we should focus on Win x64.
>>>
>>> About the 'cannot locate clang' that's the first hurdle that we should
>>> be able to solve. First question: what's in your /cygdrive/j/LLVM
>>> folder? Is that a straight download of the LLVM binaries?
>>>
>>> Maurizio
>>
>> Hi Maurizo,
>>
>> I just downloaded the pre-built binary of LLVM 6.0.1 for Windows 
>> (64-bit) from http://releases.llvm.org/download.html which comes as 
>> an installer that creates the LLVM folder.
>>
>> I've made a gist with the folder structure here 
>> https://gist.github.com/JornVernee/dae92a19f4d2036f30153e747b55d874 
>> So you can take a look if you want. Everything seems to be in the 
>> right place, except for the DLL. But the error I'm seeing seems to be 
>> from the compiler not finding the header files.
> That seems ok.
>
> On this:
>
> conftest.cpp(52): fatal error C1083: Cannot open include file: 
> 'clang-c/Index.h': No such file or directory
>
> I also thought that the problem could lie in the forward slash in the 
> #include directive, but that seems accepted by MSVC and converted 
> accordingly.
>
> So it seems like your issue is: if you try to compile a c/cpp file 
> with an include to clang-c/Index.h using MSVC from cygwin, that fails, 
> correct? Which in turns trips up the build configure script.
>
> Maurizio
>
>
>
>>
>> Jorn
>>
>


From jbvernee at xs4all.nl  Wed Sep 19 21:44:58 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 19 Sep 2018 23:44:58 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
Message-ID: <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>

Maurizio Cimadamore schreef op 2018-09-19 23:11:
> So it seems like your issue is: if you try to compile a c/cpp file
> with an include to clang-c/Index.h using MSVC from cygwin, that fails,
> correct? Which in turns trips up the build configure script.

Yes, I've tried with both .c and .cpp files.

I've also tested with renaming the LLVM folder (since I read there can 
be naming conflicts with cygwin), no success.

I've also tried using a dummy header file `dummy.h` in a J:\test folder, 
and also putting it in a folder somewhere in the cygwin file system and 
that is not being picked up either, so it does not seem to be a problem 
with the LLVM binary.

I have found a way that it does work, if I use a windows style path, but 
also escape the the slashes i.e. :

$ ~/cl.exe -c -I J:\\LLVM\\include /showIncludes main.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

main.c
Note: including file: J:\LLVM\include\clang-c/Index.h
J:\LLVM\include\clang-c/Index.h(19): fatal error C1083: Cannot open 
include file: 'time.h': No such file or directory

It seems like cygwin is not translating the unix style path, since that 
should work according to cygpath:

$ cygpath -w /cygdrive/j/LLVM/include/clang-c/Index.h
J:\LLVM\include\clang-c\Index.h

But it _is_ trying to translate the windows style path, which doesn't 
work:

$ cygpath -w J:\LLVM\include\clang-c\Index.h
J:LLVMincludeclang-cIndex.h

So adding escapes for the slashes seems to work:

$ cygpath -w \J:\\LLVM\\include\\clang-c\\Index.h
J:\LLVM\include\clang-c\Index.h

I noticed that the -I option is also being used in other places in 
config, so maybe there is a missing configuration step of cygwin path 
mangling before trying to find the header files.

Jorn

From jbvernee at xs4all.nl  Wed Sep 19 21:48:41 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 19 Sep 2018 23:48:41 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <245df6be-fd20-089c-e08c-07130b075b44@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <245df6be-fd20-089c-e08c-07130b075b44@oracle.com>
Message-ID: <e5b1488e8f4a4763248566bc48f5eb2e@xs4all.nl>

Maurizio Cimadamore schreef op 2018-09-19 23:29:
> Maybe try with the /showInclude option - to see where cl.exe is
> looking for things? Perhaps some odd path will pop out and point you
> in the right direction?
> 
> https://msdn.microsoft.com/en-us/library/hdkef6tk.aspx
> 
> Maurizio

Yep, I tried that :)

Unfortunately it looks like that is not a 'list all files in the include 
directories you have', but rather a 'log whenever you process a #include 
successfully' option.

Jorn

From jbvernee at xs4all.nl  Wed Sep 19 22:12:33 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 20 Sep 2018 00:12:33 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
Message-ID: <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>

If I use the following flags:

$ bash configure --disable-warnings-as-errors 
--with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' 
--with-libclang-include='J:\\LLVM\\include' 
--with-libclang-include-aux='J:\\LLVM\\lib' 
--with-libclang-lib='J:\\LLVM\\lib'

(Notice that I'm having to use different path styles)

It's detecting the header files, but it's failing on being passed the 
wrong flags. from config.log (see at the bottom):

configure:60248: checking for clang_getClangVersion in -lclang
configure:60273: 
/cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
-o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp 
-lclang   >&5
conftest.cpp
Microsoft (R) Incremental Linker Version 14.14.26430.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:conftest.exe
/out:conftest.exe
conftest.obj
conftest.obj : error LNK2019: unresolved external symbol 
clang_getClangVersion referenced in function main
conftest.exe : fatal error LNK1120: 1 unresolved externals
Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and 
will be removed in a future release
cl : Command line warning D9002 : ignoring unknown option 
'-LJ:\\LLVM\\lib'
cl : Command line warning D9002 : ignoring unknown option '-lclang'

That seems to be a simple problem of casing-off windows and passing the 
right flags, but I call it a night here :)

Jorn


From maurizio.cimadamore at oracle.com  Wed Sep 19 22:57:39 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 19 Sep 2018 23:57:39 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
Message-ID: <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>

Looks like good progress; tomorrow I'll take a look at some of our build 
files and see if something special is done for mangling windows include 
paths.

Cheers
Maurizio


On 19/09/18 23:12, Jorn Vernee wrote:
> If I use the following flags:
>
> $ bash configure --disable-warnings-as-errors 
> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' 
> --with-libclang-include='J:\\LLVM\\include' 
> --with-libclang-include-aux='J:\\LLVM\\lib' 
> --with-libclang-lib='J:\\LLVM\\lib'
>
> (Notice that I'm having to use different path styles)
>
> It's detecting the header files, but it's failing on being passed the 
> wrong flags. from config.log (see at the bottom):
>
> configure:60248: checking for clang_getClangVersion in -lclang
> configure:60273: 
> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp 
> -lclang?? >&5
> conftest.cpp
> Microsoft (R) Incremental Linker Version 14.14.26430.0
> Copyright (C) Microsoft Corporation.? All rights reserved.
>
> /out:conftest.exe
> /out:conftest.exe
> conftest.obj
> conftest.obj : error LNK2019: unresolved external symbol 
> clang_getClangVersion referenced in function main
> conftest.exe : fatal error LNK1120: 1 unresolved externals
> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
> Copyright (C) Microsoft Corporation.? All rights reserved.
>
> cl : Command line warning D9035 : option 'o' has been deprecated and 
> will be removed in a future release
> cl : Command line warning D9002 : ignoring unknown option 
> '-LJ:\\LLVM\\lib'
> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>
> That seems to be a simple problem of casing-off windows and passing 
> the right flags, but I call it a night here :)
>
> Jorn
>


From henry.jen at oracle.com  Wed Sep 19 23:17:37 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 19 Sep 2018 16:17:37 -0700
Subject: [foreign] Running tests on Windows
In-Reply-To: <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
Message-ID: <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>

Haven?t test it, but try this,

diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
--- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018 -0700
+++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018 -0700
@@ -63,12 +63,20 @@
     fi

     if test "x$CLANG_INCLUDE_PATH" != "x"; then
+      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
         LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
+      else
+        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
+      fi
     else
         LIBCLANG_CPPFLAGS=""
     fi
     if test "x$CLANG_LIB_PATH" != "x"; then
+      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
         LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
+      else
+        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
+      fi
     else
         LIBCLANG_LDFLAGS=""
     fi

We still need to fix the copy of DLL.

Cheers,
Henry


> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> Looks like good progress; tomorrow I'll take a look at some of our build files and see if something special is done for mangling windows include paths.
> 
> Cheers
> Maurizio
> 
> 
> On 19/09/18 23:12, Jorn Vernee wrote:
>> If I use the following flags:
>> 
>> $ bash configure --disable-warnings-as-errors --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' --with-libclang-include='J:\\LLVM\\include' --with-libclang-include-aux='J:\\LLVM\\lib' --with-libclang-lib='J:\\LLVM\\lib'
>> 
>> (Notice that I'm having to use different path styles)
>> 
>> It's detecting the header files, but it's failing on being passed the wrong flags. from config.log (see at the bottom):
>> 
>> configure:60248: checking for clang_getClangVersion in -lclang
>> configure:60273: /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl -o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp -lclang   >&5
>> conftest.cpp
>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>> 
>> /out:conftest.exe
>> /out:conftest.exe
>> conftest.obj
>> conftest.obj : error LNK2019: unresolved external symbol clang_getClangVersion referenced in function main
>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>> 
>> cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release
>> cl : Command line warning D9002 : ignoring unknown option '-LJ:\\LLVM\\lib'
>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>> 
>> That seems to be a simple problem of casing-off windows and passing the right flags, but I call it a night here :)
>> 
>> Jorn
>> 
> 


From henry.jen at oracle.com  Wed Sep 19 23:41:24 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Wed, 19 Sep 2018 16:41:24 -0700
Subject: [foreign] Running tests on Windows
In-Reply-To: <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
Message-ID: <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>

Actually, -I works, and link option need to be passed with /link, but you got the idea?

However, -lclang is added by the AC_CHECK_LIB macro, so I am not sure it would work. Need to find a better way to check the lib.

diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
--- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018 -0700
+++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018 -0700
@@ -68,7 +68,11 @@
         LIBCLANG_CPPFLAGS=""
     fi
     if test "x$CLANG_LIB_PATH" != "x"; then
+      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
         LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
+      else
+        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
+      fi
     else
         LIBCLANG_LDFLAGS=""
     fi

Cheers,
Henry


> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
> 
> Haven?t test it, but try this,
> 
> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018 -0700
> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018 -0700
> @@ -63,12 +63,20 @@
>     fi
> 
>     if test "x$CLANG_INCLUDE_PATH" != "x"; then
> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>         LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
> +      else
> +        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
> +      fi
>     else
>         LIBCLANG_CPPFLAGS=""
>     fi
>     if test "x$CLANG_LIB_PATH" != "x"; then
> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>         LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
> +      else
> +        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
> +      fi
>     else
>         LIBCLANG_LDFLAGS=""
>     fi
> 
> We still need to fix the copy of DLL.
> 
> Cheers,
> Henry
> 
> 
>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>> 
>> Looks like good progress; tomorrow I'll take a look at some of our build files and see if something special is done for mangling windows include paths.
>> 
>> Cheers
>> Maurizio
>> 
>> 
>> On 19/09/18 23:12, Jorn Vernee wrote:
>>> If I use the following flags:
>>> 
>>> $ bash configure --disable-warnings-as-errors --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' --with-libclang-include='J:\\LLVM\\include' --with-libclang-include-aux='J:\\LLVM\\lib' --with-libclang-lib='J:\\LLVM\\lib'
>>> 
>>> (Notice that I'm having to use different path styles)
>>> 
>>> It's detecting the header files, but it's failing on being passed the wrong flags. from config.log (see at the bottom):
>>> 
>>> configure:60248: checking for clang_getClangVersion in -lclang
>>> configure:60273: /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl -o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp -lclang   >&5
>>> conftest.cpp
>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>> 
>>> /out:conftest.exe
>>> /out:conftest.exe
>>> conftest.obj
>>> conftest.obj : error LNK2019: unresolved external symbol clang_getClangVersion referenced in function main
>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>> 
>>> cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release
>>> cl : Command line warning D9002 : ignoring unknown option '-LJ:\\LLVM\\lib'
>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>> 
>>> That seems to be a simple problem of casing-off windows and passing the right flags, but I call it a night here :)
>>> 
>>> Jorn
>>> 
>> 
> 


From sundararajan.athijegannathan at oracle.com  Thu Sep 20 05:15:40 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Thu, 20 Sep 2018 10:45:40 +0530
Subject: [foreign] RFR 8210935 C enum constants should be mapped to interface
 methods instead of static final int constants
Message-ID: <5BA32CFC.5060802@oracle.com>

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/

Thanks,
-Sundar

From maurizio.cimadamore at oracle.com  Thu Sep 20 08:44:18 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 09:44:18 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
Message-ID: <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>

FTR, as I'm looking at other configure files, I've seen

 ?"xwindows" used not "xmicrosoft"

As for the need for double backslash, I don't see any special processing 
done in other configure files prior to the AC_CHECK_HEADER call.

The build guide [1] is very explicit that forward slashes should be used 
in options (unlike what you did) and mentions some 'fixpath' tool which 
is used by configure to convert Unix paths into Windows ones. I wonder 
if this could be a bug in that tool?

Seems like this tool is compiled in make/autoconf/basic_windows.m4 - it 
could be worth chasing down as to where the compiled 'fixpath' 
executable is put (should be somewhere in 
build/<conf>/configure-support/) call it with the clang include folder 
with Unix-style forward slash and see whether it spits the correct path.

The source code for this tool can be found here:

http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c

Maurizio

[1] - 
http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows


On 20/09/18 00:41, Henry Jen wrote:
> Actually, -I works, and link option need to be passed with /link, but you got the idea?
>
> However, -lclang is added by the AC_CHECK_LIB macro, so I am not sure it would work. Need to find a better way to check the lib.
>
> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018 -0700
> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018 -0700
> @@ -68,7 +68,11 @@
>           LIBCLANG_CPPFLAGS=""
>       fi
>       if test "x$CLANG_LIB_PATH" != "x"; then
> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>           LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
> +      else
> +        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
> +      fi
>       else
>           LIBCLANG_LDFLAGS=""
>       fi
>
> Cheers,
> Henry
>
>
>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>
>> Haven?t test it, but try this,
>>
>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018 -0700
>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018 -0700
>> @@ -63,12 +63,20 @@
>>      fi
>>
>>      if test "x$CLANG_INCLUDE_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>          LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>> +      else
>> +        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>> +      fi
>>      else
>>          LIBCLANG_CPPFLAGS=""
>>      fi
>>      if test "x$CLANG_LIB_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>          LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>> +      else
>> +        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>> +      fi
>>      else
>>          LIBCLANG_LDFLAGS=""
>>      fi
>>
>> We still need to fix the copy of DLL.
>>
>> Cheers,
>> Henry
>>
>>
>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>>
>>> Looks like good progress; tomorrow I'll take a look at some of our build files and see if something special is done for mangling windows include paths.
>>>
>>> Cheers
>>> Maurizio
>>>
>>>
>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>> If I use the following flags:
>>>>
>>>> $ bash configure --disable-warnings-as-errors --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' --with-libclang-include='J:\\LLVM\\include' --with-libclang-include-aux='J:\\LLVM\\lib' --with-libclang-lib='J:\\LLVM\\lib'
>>>>
>>>> (Notice that I'm having to use different path styles)
>>>>
>>>> It's detecting the header files, but it's failing on being passed the wrong flags. from config.log (see at the bottom):
>>>>
>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>> configure:60273: /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl -o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp -lclang   >&5
>>>> conftest.cpp
>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>
>>>> /out:conftest.exe
>>>> /out:conftest.exe
>>>> conftest.obj
>>>> conftest.obj : error LNK2019: unresolved external symbol clang_getClangVersion referenced in function main
>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>
>>>> cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release
>>>> cl : Command line warning D9002 : ignoring unknown option '-LJ:\\LLVM\\lib'
>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>
>>>> That seems to be a simple problem of casing-off windows and passing the right flags, but I call it a night here :)
>>>>
>>>> Jorn
>>>>


From maurizio.cimadamore at oracle.com  Thu Sep 20 09:02:46 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 10:02:46 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
Message-ID: <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>

Try this:

diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
--- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
+++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
@@ -60,6 +60,9 @@
 ?????? VER=`ls $with_libclang/lib/clang/`
CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
 ?????? CLANG_LIB_PATH="$with_libclang/lib"
+??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
+??? ? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
+??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
 ???? fi

 ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then


Now, if you use only the --with-libclang option with a Unix-style path 
(the thing you were trying at first), I believe it should do the right 
thing.

Note that we call this BASIC_FIXUP_PATH thingie on all incoming paths 
which can possibly contain forward slashes - e.g.

http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49

Maurizio



On 20/09/18 09:44, Maurizio Cimadamore wrote:
> FTR, as I'm looking at other configure files, I've seen
>
> ?"xwindows" used not "xmicrosoft"
>
> As for the need for double backslash, I don't see any special 
> processing done in other configure files prior to the AC_CHECK_HEADER 
> call.
>
> The build guide [1] is very explicit that forward slashes should be 
> used in options (unlike what you did) and mentions some 'fixpath' tool 
> which is used by configure to convert Unix paths into Windows ones. I 
> wonder if this could be a bug in that tool?
>
> Seems like this tool is compiled in make/autoconf/basic_windows.m4 - 
> it could be worth chasing down as to where the compiled 'fixpath' 
> executable is put (should be somewhere in 
> build/<conf>/configure-support/) call it with the clang include folder 
> with Unix-style forward slash and see whether it spits the correct path.
>
> The source code for this tool can be found here:
>
> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>
>
> Maurizio
>
> [1] - 
> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>
>
> On 20/09/18 00:41, Henry Jen wrote:
>> Actually, -I works, and link option need to be passed with /link, but 
>> you got the idea?
>>
>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not sure 
>> it would work. Need to find a better way to check the lib.
>>
>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 -0700
>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018 -0700
>> @@ -68,7 +68,11 @@
>> ????????? LIBCLANG_CPPFLAGS=""
>> ????? fi
>> ????? if test "x$CLANG_LIB_PATH" != "x"; then
>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>> ????????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>> +????? else
>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>> +????? fi
>> ????? else
>> ????????? LIBCLANG_LDFLAGS=""
>> ????? fi
>>
>> Cheers,
>> Henry
>>
>>
>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>
>>> Haven?t test it, but try this,
>>>
>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 -0700
>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018 -0700
>>> @@ -63,12 +63,20 @@
>>> ???? fi
>>>
>>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> ???????? LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>> +????? else
>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>> +????? fi
>>> ???? else
>>> ???????? LIBCLANG_CPPFLAGS=""
>>> ???? fi
>>> ???? if test "x$CLANG_LIB_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> ???????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>> +????? else
>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>> +????? fi
>>> ???? else
>>> ???????? LIBCLANG_LDFLAGS=""
>>> ???? fi
>>>
>>> We still need to fix the copy of DLL.
>>>
>>> Cheers,
>>> Henry
>>>
>>>
>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore 
>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>
>>>> Looks like good progress; tomorrow I'll take a look at some of our 
>>>> build files and see if something special is done for mangling 
>>>> windows include paths.
>>>>
>>>> Cheers
>>>> Maurizio
>>>>
>>>>
>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>> If I use the following flags:
>>>>>
>>>>> $ bash configure --disable-warnings-as-errors 
>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' 
>>>>> --with-libclang-include='J:\\LLVM\\include' 
>>>>> --with-libclang-include-aux='J:\\LLVM\\lib' 
>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>
>>>>> (Notice that I'm having to use different path styles)
>>>>>
>>>>> It's detecting the header files, but it's failing on being passed 
>>>>> the wrong flags. from config.log (see at the bottom):
>>>>>
>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>> configure:60273: 
>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib 
>>>>> conftest.cpp -lclang?? >&5
>>>>> conftest.cpp
>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>
>>>>> /out:conftest.exe
>>>>> /out:conftest.exe
>>>>> conftest.obj
>>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>>> clang_getClangVersion referenced in function main
>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>
>>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>>> and will be removed in a future release
>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>> '-LJ:\\LLVM\\lib'
>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>>
>>>>> That seems to be a simple problem of casing-off windows and 
>>>>> passing the right flags, but I call it a night here :)
>>>>>
>>>>> Jorn
>>>>>
>


From magnus.ihse.bursie at oracle.com  Thu Sep 20 09:29:10 2018
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 20 Sep 2018 11:29:10 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
Message-ID: <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>

On 2018-09-20 11:02, Maurizio Cimadamore wrote:
> Try this:
>
> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
> @@ -60,6 +60,9 @@
> ?????? VER=`ls $with_libclang/lib/clang/`
> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
> ?????? CLANG_LIB_PATH="$with_libclang/lib"
> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
> +??? ? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
> ???? fi
>
> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>
>
> Now, if you use only the --with-libclang option with a Unix-style path 
> (the thing you were trying at first), I believe it should do the right 
> thing.

I have not followed the entire conversation, but this part looks sane. 
As Maurizio says, we should use BASIC_FIXUP_PATH on all paths from 
configure argument. (And these should, yes indeed, be in unix style). 
However, this might not be all fixes that are needed. I can help take a 
look at it, if someone points me to where the problematic code resides.

/Magnus


>
> Note that we call this BASIC_FIXUP_PATH thingie on all incoming paths 
> which can possibly contain forward slashes - e.g.
>
> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>
>
> Maurizio
>
>
>
> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>> FTR, as I'm looking at other configure files, I've seen
>>
>> ?"xwindows" used not "xmicrosoft"
>>
>> As for the need for double backslash, I don't see any special 
>> processing done in other configure files prior to the AC_CHECK_HEADER 
>> call.
>>
>> The build guide [1] is very explicit that forward slashes should be 
>> used in options (unlike what you did) and mentions some 'fixpath' 
>> tool which is used by configure to convert Unix paths into Windows 
>> ones. I wonder if this could be a bug in that tool?
>>
>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 - 
>> it could be worth chasing down as to where the compiled 'fixpath' 
>> executable is put (should be somewhere in 
>> build/<conf>/configure-support/) call it with the clang include 
>> folder with Unix-style forward slash and see whether it spits the 
>> correct path.
>>
>> The source code for this tool can be found here:
>>
>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>
>>
>> Maurizio
>>
>> [1] - 
>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>>
>>
>> On 20/09/18 00:41, Henry Jen wrote:
>>> Actually, -I works, and link option need to be passed with /link, 
>>> but you got the idea?
>>>
>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not 
>>> sure it would work. Need to find a better way to check the lib.
>>>
>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 -0700
>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018 -0700
>>> @@ -68,7 +68,11 @@
>>> ????????? LIBCLANG_CPPFLAGS=""
>>> ????? fi
>>> ????? if test "x$CLANG_LIB_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> ????????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>> +????? else
>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>> +????? fi
>>> ????? else
>>> ????????? LIBCLANG_LDFLAGS=""
>>> ????? fi
>>>
>>> Cheers,
>>> Henry
>>>
>>>
>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>
>>>> Haven?t test it, but try this,
>>>>
>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 -0700
>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018 -0700
>>>> @@ -63,12 +63,20 @@
>>>> ???? fi
>>>>
>>>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>> ???????? LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>> +????? else
>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>> +????? fi
>>>> ???? else
>>>> ???????? LIBCLANG_CPPFLAGS=""
>>>> ???? fi
>>>> ???? if test "x$CLANG_LIB_PATH" != "x"; then
>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>> ???????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>> +????? else
>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>> +????? fi
>>>> ???? else
>>>> ???????? LIBCLANG_LDFLAGS=""
>>>> ???? fi
>>>>
>>>> We still need to fix the copy of DLL.
>>>>
>>>> Cheers,
>>>> Henry
>>>>
>>>>
>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore 
>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>
>>>>> Looks like good progress; tomorrow I'll take a look at some of our 
>>>>> build files and see if something special is done for mangling 
>>>>> windows include paths.
>>>>>
>>>>> Cheers
>>>>> Maurizio
>>>>>
>>>>>
>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>> If I use the following flags:
>>>>>>
>>>>>> $ bash configure --disable-warnings-as-errors 
>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' 
>>>>>> --with-libclang-include='J:\\LLVM\\include' 
>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib' 
>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>>
>>>>>> (Notice that I'm having to use different path styles)
>>>>>>
>>>>>> It's detecting the header files, but it's failing on being passed 
>>>>>> the wrong flags. from config.log (see at the bottom):
>>>>>>
>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>> configure:60273: 
>>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib 
>>>>>> conftest.cpp -lclang?? >&5
>>>>>> conftest.cpp
>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>
>>>>>> /out:conftest.exe
>>>>>> /out:conftest.exe
>>>>>> conftest.obj
>>>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>>>> clang_getClangVersion referenced in function main
>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>
>>>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>>>> and will be removed in a future release
>>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>>> '-LJ:\\LLVM\\lib'
>>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>>>
>>>>>> That seems to be a simple problem of casing-off windows and 
>>>>>> passing the right flags, but I call it a night here :)
>>>>>>
>>>>>> Jorn
>>>>>>
>>
>


From jbvernee at xs4all.nl  Thu Sep 20 10:52:47 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 20 Sep 2018 12:52:47 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
Message-ID: <3ee797f0bd81349c1e9be581f424d7cc@xs4all.nl>

Thanks for the help!

Using BASIC_FIXUP_PATH did not work unfortunately, it doesn't seem like 
it affected the path, since config.log still shows:

configure:60635: checking clang-c/Index.h usability
configure:60635: 
/cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
-c    -I/cygdrive/j/LLVM/include conftest.cpp >&5
conftest.cpp
conftest.cpp(52): fatal error C1083: Cannot open include file: 
'clang-c/Index.h': No such file or directory
Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

Magnus, if you want to have a look, the problematic file is 
/make/autoconf/lib-clang.m4 which we're trying to adapt to work on 
windows as well.

The build file is referencing the lib-clang library that you can get 
here: http://releases.llvm.org/download.html getting one of the 
pre-built binaries should be enough, and then using `bash configure 
--with-libclang=/path/to/installation` is the use case we're trying to 
get to work.

Currently I keep running into the warning:

configure: Cannot locate libclang! You can download pre-built llvm
         binary from http://llvm.org/releases/download.html, then specify 
the
         location using --with-libclang

Here is a gist of config.log : 
https://gist.github.com/JornVernee/bbcf78d0ebe399dcad82f6551c16f04b

---

Using escaped paths together with Henry's linker flag patch I'm now 
seeing this error in config.log:

configure:60648: checking for clang_getClangVersion in -lclang
configure:60673: 
/cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
-o conftest.exe    -IJ:\\LLVM\\include /link /LIBPATH J:\\LLVM\\lib 
conftest.cpp -lclang   >&5
Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and 
will be removed in a future release
cl : Command line error D8003 : missing source filename

It seems like the '/link' flag is eating up all the other inputs?

Cheers,
Jorn

[1] : 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002711.html

Magnus Ihse Bursie schreef op 2018-09-20 11:29:
> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
> 
>> Try this:
>> 
>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4    Tue Sep 11 18:39:11 2018 +0100
>> +++ b/make/autoconf/lib-clang.m4    Thu Sep 20 09:59:08 2018 +0100
>> @@ -60,6 +60,9 @@
>> VER=`ls $with_libclang/lib/clang/`
>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>> CLANG_LIB_PATH="$with_libclang/lib"
>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>> +      BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>> fi
>> 
>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>> 
>> Now, if you use only the --with-libclang option with a Unix-style
>> path (the thing you were trying at first), I believe it should do
>> the right thing.
> 
> I have not followed the entire conversation, but this part looks sane.
> As Maurizio says, we should use BASIC_FIXUP_PATH on all paths from
> configure argument. (And these should, yes indeed, be in unix style).
> However, this might not be all fixes that are needed. I can help take
> a look at it, if someone points me to where the problematic code
> resides.
> 
> /Magnus
> 
>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>> paths which can possibly contain forward slashes - e.g.
>> 
>> 
> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49
>> 
>> 
>> Maurizio
>> 
>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>> FTR, as I'm looking at other configure files, I've seen
>> 
>> "xwindows" used not "xmicrosoft"
>> 
>> As for the need for double backslash, I don't see any special
>> processing done in other configure files prior to the
>> AC_CHECK_HEADER call.
>> 
>> The build guide [1] is very explicit that forward slashes should be
>> used in options (unlike what you did) and mentions some 'fixpath'
>> tool which is used by configure to convert Unix paths into Windows
>> ones. I wonder if this could be a bug in that tool?
>> 
>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>> it could be worth chasing down as to where the compiled 'fixpath'
>> executable is put (should be somewhere in
>> build/<conf>/configure-support/) call it with the clang include
>> folder with Unix-style forward slash and see whether it spits the
>> correct path.
>> 
>> The source code for this tool can be found here:
>> 
>> 
> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c
>> 
>> 
>> Maurizio
>> 
>> [1] -
>> 
> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>> 
>> On 20/09/18 00:41, Henry Jen wrote:
>> Actually, -I works, and link option need to be passed with /link,
>> but you got the idea?
>> 
>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>> sure it would work. Need to find a better way to check the lib.
>> 
>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>> -0700
>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018
>> -0700
>> @@ -68,7 +68,11 @@
>> LIBCLANG_CPPFLAGS=""
>> fi
>> if test "x$CLANG_LIB_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>> +      else
>> +        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>> +      fi
>> else
>> LIBCLANG_LDFLAGS=""
>> fi
>> 
>> Cheers,
>> Henry
>> 
>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>> 
>> 
>> Haven?t test it, but try this,
>> 
>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>> -0700
>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018
>> -0700
>> @@ -63,12 +63,20 @@
>> fi
>> 
>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>> +      else
>> +        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>> +      fi
>> else
>> LIBCLANG_CPPFLAGS=""
>> fi
>> if test "x$CLANG_LIB_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>> +      else
>> +        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>> +      fi
>> else
>> LIBCLANG_LDFLAGS=""
>> fi
>> 
>> We still need to fix the copy of DLL.
>> 
>> Cheers,
>> Henry
>> 
>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com> wrote:
>> 
>> Looks like good progress; tomorrow I'll take a look at some of our
>> build files and see if something special is done for mangling
>> windows include paths.
>> 
>> Cheers
>> Maurizio
>> 
>> On 19/09/18 23:12, Jorn Vernee wrote:
>> If I use the following flags:
>> 
>> $ bash configure --disable-warnings-as-errors
>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>> --with-libclang-include='J:\\LLVM\\include'
>> --with-libclang-include-aux='J:\\LLVM\\lib'
>> --with-libclang-lib='J:\\LLVM\\lib'
>> 
>> (Notice that I'm having to use different path styles)
>> 
>> It's detecting the header files, but it's failing on being passed
>> the wrong flags. from config.log (see at the bottom):
>> 
>> configure:60248: checking for clang_getClangVersion in -lclang
>> configure:60273:
>> 
> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl
>> -o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>> -lclang   >&5
>> conftest.cpp
>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>> 
>> /out:conftest.exe
>> /out:conftest.exe
>> conftest.obj
>> conftest.obj : error LNK2019: unresolved external symbol
>> clang_getClangVersion referenced in function main
>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>> 
>> cl : Command line warning D9035 : option 'o' has been deprecated and
>> will be removed in a future release
>> cl : Command line warning D9002 : ignoring unknown option
>> '-LJ:\\LLVM\\lib'
>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>> 
>> That seems to be a simple problem of casing-off windows and passing
>> the right flags, but I call it a night here :)
>> 
>> Jorn

From magnus.ihse.bursie at oracle.com  Thu Sep 20 11:26:48 2018
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 20 Sep 2018 13:26:48 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <3ee797f0bd81349c1e9be581f424d7cc@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <3ee797f0bd81349c1e9be581f424d7cc@xs4all.nl>
Message-ID: <30c3128a-d59d-5cfc-f466-cbf2b1c6283f@oracle.com>

On 2018-09-20 12:52, Jorn Vernee wrote:
> Thanks for the help!
>
> Using BASIC_FIXUP_PATH did not work unfortunately, it doesn't seem 
> like it affected the path, since config.log still shows:
>
> configure:60635: checking clang-c/Index.h usability
> configure:60635: 
> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
> -c??? -I/cygdrive/j/LLVM/include conftest.cpp >&5
> conftest.cpp
> conftest.cpp(52): fatal error C1083: Cannot open include file: 
> 'clang-c/Index.h': No such file or directory
> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
> Copyright (C) Microsoft Corporation.? All rights reserved.
>
> Magnus, if you want to have a look, the problematic file is 
> /make/autoconf/lib-clang.m4 which we're trying to adapt to work on 
> windows as well.
And this is in http://hg.openjdk.java.net/panama/dev/? Which branch?

/Magnus
>
> The build file is referencing the lib-clang library that you can get 
> here: http://releases.llvm.org/download.html getting one of the 
> pre-built binaries should be enough, and then using `bash configure 
> --with-libclang=/path/to/installation` is the use case we're trying to 
> get to work.
>
> Currently I keep running into the warning:
>
> configure: Cannot locate libclang! You can download pre-built llvm
> ??????? binary from http://llvm.org/releases/download.html, then 
> specify the
> ??????? location using --with-libclang
>
> Here is a gist of config.log : 
> https://gist.github.com/JornVernee/bbcf78d0ebe399dcad82f6551c16f04b
>
> ---
>
> Using escaped paths together with Henry's linker flag patch I'm now 
> seeing this error in config.log:
>
> configure:60648: checking for clang_getClangVersion in -lclang
> configure:60673: 
> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
> -o conftest.exe??? -IJ:\\LLVM\\include /link /LIBPATH J:\\LLVM\\lib 
> conftest.cpp -lclang?? >&5
> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
> Copyright (C) Microsoft Corporation.? All rights reserved.
>
> cl : Command line warning D9035 : option 'o' has been deprecated and 
> will be removed in a future release
> cl : Command line error D8003 : missing source filename
>
> It seems like the '/link' flag is eating up all the other inputs?
>
> Cheers,
> Jorn
>
> [1] : 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002711.html
>
> Magnus Ihse Bursie schreef op 2018-09-20 11:29:
>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>
>>> Try this:
>>>
>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>> @@ -60,6 +60,9 @@
>>> VER=`ls $with_libclang/lib/clang/`
>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>> CLANG_LIB_PATH="$with_libclang/lib"
>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>> +????? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>> fi
>>>
>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>
>>> Now, if you use only the --with-libclang option with a Unix-style
>>> path (the thing you were trying at first), I believe it should do
>>> the right thing.
>>
>> I have not followed the entire conversation, but this part looks sane.
>> As Maurizio says, we should use BASIC_FIXUP_PATH on all paths from
>> configure argument. (And these should, yes indeed, be in unix style).
>> However, this might not be all fixes that are needed. I can help take
>> a look at it, if someone points me to where the problematic code
>> resides.
>>
>> /Magnus
>>
>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>> paths which can possibly contain forward slashes - e.g.
>>>
>>>
>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>
>>>
>>>
>>> Maurizio
>>>
>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>> FTR, as I'm looking at other configure files, I've seen
>>>
>>> "xwindows" used not "xmicrosoft"
>>>
>>> As for the need for double backslash, I don't see any special
>>> processing done in other configure files prior to the
>>> AC_CHECK_HEADER call.
>>>
>>> The build guide [1] is very explicit that forward slashes should be
>>> used in options (unlike what you did) and mentions some 'fixpath'
>>> tool which is used by configure to convert Unix paths into Windows
>>> ones. I wonder if this could be a bug in that tool?
>>>
>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>>> it could be worth chasing down as to where the compiled 'fixpath'
>>> executable is put (should be somewhere in
>>> build/<conf>/configure-support/) call it with the clang include
>>> folder with Unix-style forward slash and see whether it spits the
>>> correct path.
>>>
>>> The source code for this tool can be found here:
>>>
>>>
>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>
>>>
>>>
>>> Maurizio
>>>
>>> [1] -
>>>
>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows 
>>
>>>
>>> On 20/09/18 00:41, Henry Jen wrote:
>>> Actually, -I works, and link option need to be passed with /link,
>>> but you got the idea?
>>>
>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>> sure it would work. Need to find a better way to check the lib.
>>>
>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>> -0700
>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018
>>> -0700
>>> @@ -68,7 +68,11 @@
>>> LIBCLANG_CPPFLAGS=""
>>> fi
>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>> +????? else
>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>> +????? fi
>>> else
>>> LIBCLANG_LDFLAGS=""
>>> fi
>>>
>>> Cheers,
>>> Henry
>>>
>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>
>>>
>>> Haven?t test it, but try this,
>>>
>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>> -0700
>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018
>>> -0700
>>> @@ -63,12 +63,20 @@
>>> fi
>>>
>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>> +????? else
>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>> +????? fi
>>> else
>>> LIBCLANG_CPPFLAGS=""
>>> fi
>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>> +????? else
>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>> +????? fi
>>> else
>>> LIBCLANG_LDFLAGS=""
>>> fi
>>>
>>> We still need to fix the copy of DLL.
>>>
>>> Cheers,
>>> Henry
>>>
>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>> <maurizio.cimadamore at oracle.com> wrote:
>>>
>>> Looks like good progress; tomorrow I'll take a look at some of our
>>> build files and see if something special is done for mangling
>>> windows include paths.
>>>
>>> Cheers
>>> Maurizio
>>>
>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>> If I use the following flags:
>>>
>>> $ bash configure --disable-warnings-as-errors
>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>> --with-libclang-include='J:\\LLVM\\include'
>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>
>>> (Notice that I'm having to use different path styles)
>>>
>>> It's detecting the header files, but it's failing on being passed
>>> the wrong flags. from config.log (see at the bottom):
>>>
>>> configure:60248: checking for clang_getClangVersion in -lclang
>>> configure:60273:
>>>
>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>
>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>> -lclang?? >&5
>>> conftest.cpp
>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>
>>> /out:conftest.exe
>>> /out:conftest.exe
>>> conftest.obj
>>> conftest.obj : error LNK2019: unresolved external symbol
>>> clang_getClangVersion referenced in function main
>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>
>>> cl : Command line warning D9035 : option 'o' has been deprecated and
>>> will be removed in a future release
>>> cl : Command line warning D9002 : ignoring unknown option
>>> '-LJ:\\LLVM\\lib'
>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>
>>> That seems to be a simple problem of casing-off windows and passing
>>> the right flags, but I call it a night here :)
>>>
>>> Jorn


From jbvernee at xs4all.nl  Thu Sep 20 11:34:18 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 20 Sep 2018 13:34:18 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <30c3128a-d59d-5cfc-f466-cbf2b1c6283f@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <3ee797f0bd81349c1e9be581f424d7cc@xs4all.nl>
 <30c3128a-d59d-5cfc-f466-cbf2b1c6283f@oracle.com>
Message-ID: <b8fee703ec7261d9569f88106269009a@xs4all.nl>

Magnus Ihse Bursie schreef op 2018-09-20 13:26:
> On 2018-09-20 12:52, Jorn Vernee wrote:
> 
>> Thanks for the help!
>> 
>> Using BASIC_FIXUP_PATH did not work unfortunately, it doesn't seem
>> like it affected the path, since config.log still shows:
>> 
>> configure:60635: checking clang-c/Index.h usability
>> configure:60635:
>> 
> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl
>> -c    -I/cygdrive/j/LLVM/include conftest.cpp >&5
>> conftest.cpp
>> conftest.cpp(52): fatal error C1083: Cannot open include file:
>> 'clang-c/Index.h': No such file or directory
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>> 
>> Magnus, if you want to have a look, the problematic file is
>> /make/autoconf/lib-clang.m4 which we're trying to adapt to work on
>> windows as well.
>  And this is in http://hg.openjdk.java.net/panama/dev/? Which branch?
> 
> /Magnus

Sorry, yes this is in http://hg.openjdk.java.net/panama/dev/ on the 
foreign branch.

Jorn

From maurizio.cimadamore at oracle.com  Thu Sep 20 14:56:29 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 15:56:29 +0100
Subject: [foreign] RFR 8210935 C enum constants should be mapped to
 interface methods instead of static final int constants
In-Reply-To: <5BA32CFC.5060802@oracle.com>
References: <5BA32CFC.5060802@oracle.com>
Message-ID: <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>

Looks good.

Maybe in the test you could also check that e.g. combining enum 
constants in Java and C yields the same - e.g.

R | G

?

Maurizio



On 20/09/18 06:15, Sundararajan Athijegannathan wrote:
> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
> Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/
>
> Thanks,
> -Sundar


From henry.jen at oracle.com  Thu Sep 20 15:04:17 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Thu, 20 Sep 2018 08:04:17 -0700
Subject: [foreign] Running tests on Windows
In-Reply-To: <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
Message-ID: <670A5A18-EA25-4150-A89E-D3D0E79B2B0F@oracle.com>



> On Sep 20, 2018, at 1:44 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> FTR, as I'm looking at other configure files, I've seen
> 
>  "xwindows" used not ?xmicrosoft"
> 

windows is platform, microsoft is toolchain last time I check.

Cheers,
Henry



From sundararajan.athijegannathan at oracle.com  Thu Sep 20 15:22:45 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Thu, 20 Sep 2018 20:52:45 +0530
Subject: [foreign] RFR 8210935 C enum constants should be mapped to
 interface methods instead of static final int constants
In-Reply-To: <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>
References: <5BA32CFC.5060802@oracle.com>
 <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>
Message-ID: <5BA3BB45.5090806@oracle.com>

Thanks. I've added few more tests to combine enum constants.

Pushing this updated changes: 
http://cr.openjdk.java.net/~sundar/8210935/webrev.01/

-Sundar

On 20/09/18, 8:26 PM, Maurizio Cimadamore wrote:
> Looks good.
>
> Maybe in the test you could also check that e.g. combining enum 
> constants in Java and C yields the same - e.g.
>
> R | G
>
> ?
>
> Maurizio
>
>
>
> On 20/09/18 06:15, Sundararajan Athijegannathan wrote:
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
>> Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/
>>
>> Thanks,
>> -Sundar
>

From sundararajan.athijegannathan at oracle.com  Thu Sep 20 15:15:50 2018
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Thu, 20 Sep 2018 15:15:50 +0000
Subject: hg: panama/dev: 8201935: C enum constants should be mapped to
 interface methods instead of static final int constants
Message-ID: <201809201515.w8KFFo7Y018863@aojmv0008.oracle.com>

Changeset: 54e97bdd90c3
Author:    sundar
Date:      2018-09-20 20:54 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/54e97bdd90c3

8201935: C enum constants should be mapped to interface methods instead of static final int constants
Reviewed-by: mcimadamore

! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java
! test/jdk/com/sun/tools/jextract/JextractToolProviderTest.java
! test/jdk/com/sun/tools/jextract/JextractToolRunner.java
! test/jdk/com/sun/tools/jextract/RedundantDeclsTest.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Cursor.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Index.java
+ test/jdk/com/sun/tools/jextract/testEnum/LibEnumsTest.java
+ test/jdk/com/sun/tools/jextract/testEnum/enums.h
+ test/jdk/com/sun/tools/jextract/testEnum/libEnums.c


From maurizio.cimadamore at oracle.com  Thu Sep 20 15:22:11 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 20 Sep 2018 15:22:11 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809201522.w8KFMCUN021127@aojmv0008.oracle.com>

Changeset: 1ff19c0322e0
Author:    mcimadamore
Date:      2018-09-20 17:24 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/1ff19c0322e0

Automatic merge with foreign



From maurizio.cimadamore at oracle.com  Thu Sep 20 15:57:40 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 16:57:40 +0100
Subject: [foreign] RFR 8210967: Consolidate Scope API
Message-ID: <4a281813-ca26-8b92-f280-bf2ba385ad37@oracle.com>

Hi,
this is a simple webrev with some cosmetic changes to Scope.

First, it renames Scope::toCStr to Scope::allocateString (to bring it in 
sync with remaining allocateXYZ methods)

Second, it removes the Scope::toCStrArray and moves it where it is used 
(clang/FFI test). The rationale for the removal is that I don't think 
that, as a primitive, this is 'hot enough' to deserve a place in the 
API. Note also that 'string array' is a tad ambiguous in that some users 
might infer that the method is allocating a chunk of memory big enough 
to hold all the strings of the source array. For these reasons, I've 
decided to drop it for now - if use cases suggest otherwise, we can add 
it back in some form.

Third, it add the javadoc to allocateString, which was missing.

Fourth, the implementation of allocateString is now much simpler, and 
just piggy backs on allocateArray (as it should).

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8210967/

Maurizio





From sundararajan.athijegannathan at oracle.com  Thu Sep 20 16:23:09 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Thu, 20 Sep 2018 21:53:09 +0530
Subject: [foreign] RFR 8210967: Consolidate Scope API
In-Reply-To: <4a281813-ca26-8b92-f280-bf2ba385ad37@oracle.com>
References: <4a281813-ca26-8b92-f280-bf2ba385ad37@oracle.com>
Message-ID: <5BA3C96D.9020800@oracle.com>

Looks good

PS. allocateString -> allocateCString? It is null terminated C char* ...

-Sundar

On 20/09/18, 9:27 PM, Maurizio Cimadamore wrote:
> Hi,
> this is a simple webrev with some cosmetic changes to Scope.
>
> First, it renames Scope::toCStr to Scope::allocateString (to bring it 
> in sync with remaining allocateXYZ methods)
>
> Second, it removes the Scope::toCStrArray and moves it where it is 
> used (clang/FFI test). The rationale for the removal is that I don't 
> think that, as a primitive, this is 'hot enough' to deserve a place in 
> the API. Note also that 'string array' is a tad ambiguous in that some 
> users might infer that the method is allocating a chunk of memory big 
> enough to hold all the strings of the source array. For these reasons, 
> I've decided to drop it for now - if use cases suggest otherwise, we 
> can add it back in some form.
>
> Third, it add the javadoc to allocateString, which was missing.
>
> Fourth, the implementation of allocateString is now much simpler, and 
> just piggy backs on allocateArray (as it should).
>
> Webrev:
>
> http://cr.openjdk.java.net/~mcimadamore/panama/8210967/
>
> Maurizio
>
>
>
>

From maurizio.cimadamore at oracle.com  Thu Sep 20 17:08:41 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 18:08:41 +0100
Subject: [foreign] RFR 8210967: Consolidate Scope API
In-Reply-To: <5BA3C96D.9020800@oracle.com>
References: <4a281813-ca26-8b92-f280-bf2ba385ad37@oracle.com>
 <5BA3C96D.9020800@oracle.com>
Message-ID: <aa942264-64eb-e948-eb24-63b0f061c00d@oracle.com>

Yeah, I can do that - I'll push with allocateCString

Cheers
Maurizio


On 20/09/18 17:23, Sundararajan Athijegannathan wrote:
> Looks good
>
> PS. allocateString -> allocateCString? It is null terminated C char* ...
>
> -Sundar
>
> On 20/09/18, 9:27 PM, Maurizio Cimadamore wrote:
>> Hi,
>> this is a simple webrev with some cosmetic changes to Scope.
>>
>> First, it renames Scope::toCStr to Scope::allocateString (to bring it 
>> in sync with remaining allocateXYZ methods)
>>
>> Second, it removes the Scope::toCStrArray and moves it where it is 
>> used (clang/FFI test). The rationale for the removal is that I don't 
>> think that, as a primitive, this is 'hot enough' to deserve a place 
>> in the API. Note also that 'string array' is a tad ambiguous in that 
>> some users might infer that the method is allocating a chunk of 
>> memory big enough to hold all the strings of the source array. For 
>> these reasons, I've decided to drop it for now - if use cases suggest 
>> otherwise, we can add it back in some form.
>>
>> Third, it add the javadoc to allocateString, which was missing.
>>
>> Fourth, the implementation of allocateString is now much simpler, and 
>> just piggy backs on allocateArray (as it should).
>>
>> Webrev:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/8210967/
>>
>> Maurizio
>>
>>
>>
>>


From maurizio.cimadamore at oracle.com  Thu Sep 20 17:16:09 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 18:16:09 +0100
Subject: [foreign] RFR 8210935 C enum constants should be mapped to
 interface methods instead of static final int constants
In-Reply-To: <5BA3BB45.5090806@oracle.com>
References: <5BA32CFC.5060802@oracle.com>
 <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>
 <5BA3BB45.5090806@oracle.com>
Message-ID: <211c6f8d-ce93-dbad-e014-5528653f94f0@oracle.com>

Looks good!

Maurizio


On 20/09/18 16:22, Sundararajan Athijegannathan wrote:
> Thanks. I've added few more tests to combine enum constants.
>
> Pushing this updated changes: 
> http://cr.openjdk.java.net/~sundar/8210935/webrev.01/
>
> -Sundar
>
> On 20/09/18, 8:26 PM, Maurizio Cimadamore wrote:
>> Looks good.
>>
>> Maybe in the test you could also check that e.g. combining enum 
>> constants in Java and C yields the same - e.g.
>>
>> R | G
>>
>> ?
>>
>> Maurizio
>>
>>
>>
>> On 20/09/18 06:15, Sundararajan Athijegannathan wrote:
>>> Please review.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
>>> Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/
>>>
>>> Thanks,
>>> -Sundar
>>


From maurizio.cimadamore at oracle.com  Thu Sep 20 17:23:16 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 18:23:16 +0100
Subject: [foreign] RFR 8210935 C enum constants should be mapped to
 interface methods instead of static final int constants
In-Reply-To: <211c6f8d-ce93-dbad-e014-5528653f94f0@oracle.com>
References: <5BA32CFC.5060802@oracle.com>
 <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>
 <5BA3BB45.5090806@oracle.com>
 <211c6f8d-ce93-dbad-e014-5528653f94f0@oracle.com>
Message-ID: <1caedbc9-05a7-f536-a377-116fb49a9179@oracle.com>

I have an issue with this test when running on Linux:

/usr/include/limits.h:123:16: fatal error: 'limits.h' file not found
java.lang.RuntimeException: jextract returns non-zero value
 ??? at JtregJextract.jextract(JtregJextract.java:85)
 ??? at JtregJextract.main(JtregJextract.java:96)
 ??? at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
 ??? at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 ??? at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ??? at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 ??? at 
com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:229)
 ??? at java.base/java.lang.Thread.run(Thread.java:834)

I think the culprit is that we are not copying the clang version of 
limits.h into the JDK; from make/copy/Copy-jdk.jextract.gmk:

JEXTRACT_SRC_FILES := $(wildcard $(JEXTRACT_CONF_SRC)/*std*.h)

That is we are filtering with 'std'. I think we need limits.h too (and 
possibly others).

Maurizio


On 20/09/18 18:16, Maurizio Cimadamore wrote:
> Looks good!
>
> Maurizio
>
>
> On 20/09/18 16:22, Sundararajan Athijegannathan wrote:
>> Thanks. I've added few more tests to combine enum constants.
>>
>> Pushing this updated changes: 
>> http://cr.openjdk.java.net/~sundar/8210935/webrev.01/
>>
>> -Sundar
>>
>> On 20/09/18, 8:26 PM, Maurizio Cimadamore wrote:
>>> Looks good.
>>>
>>> Maybe in the test you could also check that e.g. combining enum 
>>> constants in Java and C yields the same - e.g.
>>>
>>> R | G
>>>
>>> ?
>>>
>>> Maurizio
>>>
>>>
>>>
>>> On 20/09/18 06:15, Sundararajan Athijegannathan wrote:
>>>> Please review.
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
>>>> Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/
>>>>
>>>> Thanks,
>>>> -Sundar
>>>
>


From maurizio.cimadamore at oracle.com  Thu Sep 20 17:25:32 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 20 Sep 2018 17:25:32 +0000
Subject: hg: panama/dev: 8210967: Consolidate Scope API
Message-ID: <201809201725.w8KHPWV4004234@aojmv0008.oracle.com>

Changeset: 7fbd302c1e8e
Author:    mcimadamore
Date:      2018-09-20 18:25 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/7fbd302c1e8e

8210967: Consolidate Scope API
Reviewed-by: sundar

! src/java.base/share/classes/java/foreign/Scope.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Index.java
! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Type.java
! test/jdk/java/foreign/StdLibTest.java
! test/jdk/java/foreign/System/UnixSystem.java
! test/jdk/java/foreign/printf/Printf.java


From razvan.a.lupusoru at intel.com  Thu Sep 20 17:27:01 2018
From: razvan.a.lupusoru at intel.com (razvan.a.lupusoru at intel.com)
Date: Thu, 20 Sep 2018 17:27:01 +0000
Subject: hg: panama/dev: Fix vpshuf* assembler compiler warning
Message-ID: <201809201727.w8KHR2G6005179@aojmv0008.oracle.com>

Changeset: 841b527c55de
Author:    rlupusoru
Date:      2018-09-20 10:27 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/841b527c55de

Fix vpshuf* assembler compiler warning

! src/hotspot/cpu/x86/assembler_x86.cpp


From maurizio.cimadamore at oracle.com  Thu Sep 20 17:31:52 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 20 Sep 2018 17:31:52 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809201731.w8KHVrvT007051@aojmv0008.oracle.com>

Changeset: 1ad899bd002f
Author:    mcimadamore
Date:      2018-09-20 19:34 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/1ad899bd002f

Automatic merge with foreign



From maurizio.cimadamore at oracle.com  Thu Sep 20 17:39:59 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 20 Sep 2018 17:39:59 +0000
Subject: hg: panama/dev: manual merge
Message-ID: <201809201739.w8KHdxJ9012300@aojmv0008.oracle.com>

Changeset: 748f0ad08c18
Author:    mcimadamore
Date:      2018-09-20 18:39 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/748f0ad08c18

manual merge

! make/autoconf/flags.m4
! make/autoconf/spec.gmk.in
- make/jdk/src/classes/build/tools/hasher/Hasher.java
- make/jdk/src/classes/build/tools/jarreorder/JarReorder.java
- make/mapfiles/libjsound/mapfile-vers
- make/mapfiles/libjvm_db/mapfile-vers
- make/mapfiles/libjvm_dtrace/mapfile-vers
! src/java.base/share/classes/java/lang/System.java
! src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java
- src/java.base/windows/conf/tzmappings
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewArrayStub.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewInstanceStub.java
- test/jdk/java/util/ServiceLoader/basic/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Utils.java


From maurizio.cimadamore at oracle.com  Thu Sep 20 17:41:57 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 20 Sep 2018 17:41:57 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809201741.w8KHfwAk013847@aojmv0008.oracle.com>

Changeset: dc2eb8201b76
Author:    mcimadamore
Date:      2018-09-20 19:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/dc2eb8201b76

Automatic merge with foreign

! make/autoconf/toolchain.m4
- make/jdk/src/classes/build/tools/hasher/Hasher.java
- make/jdk/src/classes/build/tools/jarreorder/JarReorder.java
- make/mapfiles/libjsound/mapfile-vers
- make/mapfiles/libjvm_db/mapfile-vers
- make/mapfiles/libjvm_dtrace/mapfile-vers
! src/hotspot/cpu/arm/arm.ad
! src/hotspot/cpu/arm/sharedRuntime_arm.cpp
! src/hotspot/cpu/ppc/ppc.ad
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp
! src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
! src/hotspot/share/opto/macro.cpp
! src/hotspot/share/opto/macro.hpp
- src/java.base/windows/conf/tzmappings
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewArrayStub.java
- src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/stubs/NewInstanceStub.java
- test/jdk/java/util/ServiceLoader/basic/basic.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Utils.java


From maurizio.cimadamore at oracle.com  Thu Sep 20 17:42:17 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 20 Sep 2018 17:42:17 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809201742.w8KHgIKg014204@aojmv0008.oracle.com>

Changeset: 8b7263b165f4
Author:    mcimadamore
Date:      2018-09-20 19:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/8b7263b165f4

Automatic merge with vectorIntrinsics



From maurizio.cimadamore at oracle.com  Thu Sep 20 17:53:12 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 18:53:12 +0100
Subject: [foreign] RFR 8210935 C enum constants should be mapped to
 interface methods instead of static final int constants
In-Reply-To: <1caedbc9-05a7-f536-a377-116fb49a9179@oracle.com>
References: <5BA32CFC.5060802@oracle.com>
 <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>
 <5BA3BB45.5090806@oracle.com>
 <211c6f8d-ce93-dbad-e014-5528653f94f0@oracle.com>
 <1caedbc9-05a7-f536-a377-116fb49a9179@oracle.com>
Message-ID: <60c52bcc-973b-ef14-f092-d5bae11d314a@oracle.com>

More on this; consider this test:

$ cat testLimits.h

#include <limits.h>

void f() { }


If I extract this I get an error:

$ jextract testLimits.h
/usr/include/limits.h:123:16: fatal error: 'limits.h' file not found
/usr/include/limits.h:123:16: fatal error: 'limits.h' file not found


With some more debugging sprinkled:

$ jextract -C-v testLimits.h
clang version 4.0.1 (tags/RELEASE_401/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir:
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/5
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/6
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/6.0.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.5
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.3
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
ignoring nonexistent directory "../lib/clang/4.0.1/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 ?/usr/local/include
 ?/usr/include/x86_64-linux-gnu
 ?/usr/include
End of search list.
/usr/include/limits.h:123:16: fatal error: 'limits.h' file not found
/usr/include/limits.h:123:16: fatal error: 'limits.h' file not found

So, /usr/include is there, but the problem is that it does some trick to 
include an additional limits.h header which is GCC-specific:

 ?/* Get the compiler's limits.h, which defines almost all the ISO 
constants.

 ??? We put this #include_next outside the double inclusion check because
 ??? it should be possible to include this file more than once and still get
 ??? the definitions from gcc's header.? */
#if defined __GNUC__ && !defined _GCC_LIMITS_H_
/* `_GCC_LIMITS_H_' is what GCC's file defines.? */
# include_next <limits.h>
#endif

So, in order to use limits.h in my platform I have to resort to clang 
trickery - e.g.

$ jextract -C-U__GNUC__ testLimits.h

the above passes - but feels dirty (a lot).



My feeling is that clang's header should always be picked up in these 
cases - as those are the ones that get picked up if I do a standalone 
call to clang:

$ clang -M testLimits.h

testLimits.o: testLimits.h \
 ? /usr/lib/llvm-3.8/bin/../lib/clang/3.8.0/include/limits.h \
 ? /usr/include/limits.h /usr/include/features.h \
 ? /usr/include/stdc-predef.h /usr/include/x86_64-linux-gnu/sys/cdefs.h \
 ? /usr/include/x86_64-linux-gnu/bits/wordsize.h \
 ? /usr/include/x86_64-linux-gnu/gnu/stubs.h \
 ? /usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
 ? /usr/include/x86_64-linux-gnu/bits/posix1_lim.h \
 ? /usr/include/x86_64-linux-gnu/bits/local_lim.h \
 ? /usr/include/linux/limits.h \
 ? /usr/include/x86_64-linux-gnu/bits/posix2_lim.h

As you can see, it's getting LLVM limits.h; jextract should behave the same.

Maurizio

On 20/09/18 18:23, Maurizio Cimadamore wrote:
> I have an issue with this test when running on Linux:
>
> /usr/include/limits.h:123:16: fatal error: 'limits.h' file not found
> java.lang.RuntimeException: jextract returns non-zero value
> ??? at JtregJextract.jextract(JtregJextract.java:85)
> ??? at JtregJextract.main(JtregJextract.java:96)
> ??? at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
> ??? at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ??? at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ??? at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> ??? at 
> com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:229)
> ??? at java.base/java.lang.Thread.run(Thread.java:834)
>
> I think the culprit is that we are not copying the clang version of 
> limits.h into the JDK; from make/copy/Copy-jdk.jextract.gmk:
>
> JEXTRACT_SRC_FILES := $(wildcard $(JEXTRACT_CONF_SRC)/*std*.h)
>
> That is we are filtering with 'std'. I think we need limits.h too (and 
> possibly others).
>
> Maurizio
>
>
> On 20/09/18 18:16, Maurizio Cimadamore wrote:
>> Looks good!
>>
>> Maurizio
>>
>>
>> On 20/09/18 16:22, Sundararajan Athijegannathan wrote:
>>> Thanks. I've added few more tests to combine enum constants.
>>>
>>> Pushing this updated changes: 
>>> http://cr.openjdk.java.net/~sundar/8210935/webrev.01/
>>>
>>> -Sundar
>>>
>>> On 20/09/18, 8:26 PM, Maurizio Cimadamore wrote:
>>>> Looks good.
>>>>
>>>> Maybe in the test you could also check that e.g. combining enum 
>>>> constants in Java and C yields the same - e.g.
>>>>
>>>> R | G
>>>>
>>>> ?
>>>>
>>>> Maurizio
>>>>
>>>>
>>>>
>>>> On 20/09/18 06:15, Sundararajan Athijegannathan wrote:
>>>>> Please review.
>>>>>
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
>>>>> Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/
>>>>>
>>>>> Thanks,
>>>>> -Sundar
>>>>
>>
>


From maurizio.cimadamore at oracle.com  Thu Sep 20 18:14:25 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 20 Sep 2018 19:14:25 +0100
Subject: [foreign] some JMH benchmarks
In-Reply-To: <580516c9-0ef2-0ceb-7c8c-c67d6621c074@gmail.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
 <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
 <25be0579-eb55-65e7-b379-206b8713a687@gmail.com>
 <d6a908bb-7a31-5e6a-37b8-106149ffac29@oracle.com>
 <580516c9-0ef2-0ceb-7c8c-c67d6621c074@gmail.com>
Message-ID: <55e13144-ad01-dd24-5081-864ebe8e7524@oracle.com>

Sorry for the delay in getting back at you. There's indeed something 
fishy going on here, and I have spotted a regression in JNI perf since 
JDK 11. This could be caused by update in compiler toolchain introduced 
in same version, but I have filed an issue for our hotspot team to 
investigate:

https://bugs.openjdk.java.net/browse/JDK-8210975

In the context of this discussion, it's likely that the rtegression is 
affecting the numbers of both Panama (which is built on top of JNI at 
the moment) and the JNI benchmarks.

Thanks
Maurizio


On 19/09/18 01:13, Samuel Audet wrote:
> Thanks! You haven't mentioned the version of the JDK you're using 
> though. I'm starting to get the impression that JNI in newer versions 
> of OpenJDK will be slower... ?
>
> On 09/18/2018 07:03 PM, Maurizio Cimadamore wrote:
>> These are the numbers I get
>>
>> Benchmark???????????????????????? Mode? Cnt???????? Score Error Units
>> NativeBenchmark.expBenchmark???? thrpt??? 5? 30542590.094 ? 
>> 44126.434? ops/s
>> NativeBenchmark.getpidBenchmark? thrpt??? 5? 61764677.092 ? 
>> 21102.236? ops/s
>>
>> They are in the same ballpark, but exp() is a bit faster; byw, I 
>> tried to repeat my benchmark with JNI exp() _and_ O3 and I've got 
>> very similar numbers (yesterday I did a very quick test and there was 
>> probably some other job running on the machine and brining down the 
>> figures a bit).
>>
>> But overall, the results in your bench seem to match what I got: exp 
>> is faster, pid is slower, the difference is mostly caused by O3. If 
>> no O3 is used, then the numbers should match what I included in my 
>> numbers (and getpid should be a bit faster).
>>
>> Maurizio
>>
>>
>> On 18/09/18 05:48, Samuel Audet wrote:
>>> Anyway, I've put online an updated version of my benchmark files here:
>>> https://gist.github.com/saudet/1bf14a000e64c245675cf5d4e9ad6e69
>>> Just run "git clone" on the URL and run "mvn package" on the pom.xml.
>>>
>>> With the 2 virtual cores of an Intel(R) Xeon(R) CPU E5-2673 v4 @ 
>>> 2.30GHz running Ubuntu 14.04 on the cloud with GCC 4.9 and OpenJDK 
>>> 8, I get these numbers:
>>>
>>> Benchmark???????????????????????? Mode? Cnt????????? Score Error ?Units
>>> NativeBenchmark.expBenchmark???? thrpt?? 25?? 37460540.440 ? 
>>> 393299.974 ?ops/s
>>> NativeBenchmark.getpidBenchmark? thrpt?? 25? 100323188.451 ? 
>>> 1254197.449 ?ops/s
>>>
>>> While on my laptop, an Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz 
>>> running Fedora 27, GCC 7.3, and OpenJDK 9, I get the following:
>>>
>>> Benchmark???????????????????????? Mode? Cnt???????? Score Error Units
>>> NativeBenchmark.expBenchmark???? thrpt?? 25? 50047147.099 ? 
>>> 924366.937 ops/s
>>> NativeBenchmark.getpidBenchmark? thrpt?? 25?? 4825508.193 ? 
>>> 21662.633 ops/s
>>>
>>> Now, it looks like getpid() is really slow on Fedora 27 for some 
>>> reason, but as Linus puts it, we should not be using that for 
>>> benchmarking:
>>> https://yarchive.net/comp/linux/getpid_caching.html
>>>
>>> What do you get on your machines?
>>>
>>> Samuel
>>>
>>>
>>> On 09/18/2018 12:58 AM, Maurizio Cimadamore wrote:
>>>> For the records, here's what I get for all the three benchmarks if 
>>>> I compile the JNI code with -O3:
>>>>
>>>> Benchmark????????????????????????? Mode? Cnt Score Error Units
>>>> PanamaBenchmark.testJNIExp??????? thrpt??? 5? 28575269.294 ? 
>>>> 1907726.710? ops/s
>>>> PanamaBenchmark.testJNIJavaQsort? thrpt??? 5??? 372148.433 ? 
>>>> 27178.529? ops/s
>>>> PanamaBenchmark.testJNIPid??????? thrpt??? 5? 59240069.011 ? 
>>>> 403881.697? ops/s
>>>>
>>>> The first and second benchmarks get faster and very close to the 
>>>> 'direct' optimization numbers in [1]. Surprisingly, the last 
>>>> benchmark (getpid) is quite slower. I've been able to reproduce 
>>>> across multiple runs; for that benchmark omitting O3 seems to be 
>>>> the achieve best results, not sure why. It starts of faster (around 
>>>> in the first couple of warmup iterations, but then it goes slower 
>>>> in all the other runs - presumably it interacts badly with the C2 
>>>> generated code. For instance, this is a run with O3 enabled:
>>>>
>>>> # Run progress: 66.67% complete, ETA 00:01:40
>>>> # Fork: 1 of 1
>>>> # Warmup Iteration?? 1: 65182202.653 ops/s
>>>> # Warmup Iteration?? 2: 64900639.094 ops/s
>>>> # Warmup Iteration?? 3: 59314945.437 ops/s 
>>>> <---------------------------------
>>>> # Warmup Iteration?? 4: 59269007.877 ops/s
>>>> # Warmup Iteration?? 5: 59239905.163 ops/s
>>>> Iteration?? 1: 59300748.074 ops/s
>>>> Iteration?? 2: 59249666.044 ops/s
>>>> Iteration?? 3: 59268597.051 ops/s
>>>> Iteration?? 4: 59322074.572 ops/s
>>>> Iteration?? 5: 59059259.317 ops/s
>>>>
>>>> And this is a run with O3 disabled:
>>>>
>>>> # Run progress: 0.00% complete, ETA 00:01:40
>>>> # Fork: 1 of 1
>>>> # Warmup Iteration?? 1: 55882128.787 ops/s
>>>> # Warmup Iteration?? 2: 53102361.751 ops/s
>>>> # Warmup Iteration?? 3: 66964755.699 ops/s 
>>>> <---------------------------------
>>>> # Warmup Iteration?? 4: 66414428.355 ops/s
>>>> # Warmup Iteration?? 5: 65328475.276 ops/s
>>>> Iteration?? 1: 64229192.993 ops/s
>>>> Iteration?? 2: 65191719.319 ops/s
>>>> Iteration?? 3: 65352022.471 ops/s
>>>> Iteration?? 4: 65152090.426 ops/s
>>>> Iteration?? 5: 65320545.712 ops/s
>>>>
>>>>
>>>> In both cases, the 3rd warmup execution sees a performance jump - 
>>>> with O3, the jump is backwards, w/o O3 the jump is forward, which 
>>>> is quite typical for a JMH benchmark as C2 optimization will start 
>>>> to kick in.
>>>>
>>>> For these reasons, I'm reluctant to update my benchmark numbers to 
>>>> reflect the O3 behavior (although I agree that, since the Hotspot 
>>>> code is compiled with that optimization it would make more sense to 
>>>> use that as a reference).
>>>>
>>>> Maurizio
>>>>
>>>> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>>>>
>>>>
>>>>
>>>> On 17/09/18 16:18, Maurizio Cimadamore wrote:
>>>>>
>>>>>
>>>>> On 17/09/18 15:08, Samuel Audet wrote:
>>>>>> Yes, the blackhole or the random number doesn't make any 
>>>>>> difference, but not calling gcc with -O3 does. Running the 
>>>>>> compiler with optimizations on is pretty common, but they are not 
>>>>>> enabled by default.
>>>>> A bit better
>>>>>
>>>>> PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ? 8491668.248 
>>>>> ops/s
>>>>>
>>>>> But not much of a difference (I did not expected much, as the body 
>>>>> of the native method is extremely simple).
>>>>>
>>>>> Maurizio 
>>>
>>
>


From magnus.ihse.bursie at oracle.com  Thu Sep 20 18:47:52 2018
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 20 Sep 2018 20:47:52 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <670A5A18-EA25-4150-A89E-D3D0E79B2B0F@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <670A5A18-EA25-4150-A89E-D3D0E79B2B0F@oracle.com>
Message-ID: <bd68de1d-e34e-7bfa-70b5-20df64716253@oracle.com>


On 2018-09-20 17:04, Henry Jen wrote:
>
>> On Sep 20, 2018, at 1:44 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>
>> FTR, as I'm looking at other configure files, I've seen
>>
>>   "xwindows" used not ?xmicrosoft"
>>
> windows is platform, microsoft is toolchain last time I check.
In general, we try to decide if we're doing a certain thing because 
we're running on Windows as OS, or using the Microsoft toolchain to 
compile. At times, it's hard to tell the difference, not at least since 
we only support a single toolchain on Windows. :-) Also, some old code 
written before the toolchain concept was available for testing might 
test for Windows when the Microsoft toolchain was what really should 
have been tested. (Such code should really be updated.)

In general, when dealing with compilation flags etc, it's more likely 
that you should test for "microsoft" as toolchain than "windows" as os. 
(I'm just talking generally here; I have not yet had time to study the 
code in question.)

/Magnus

>
> Cheers,
> Henry
>
>


From samuel.audet at gmail.com  Fri Sep 21 00:51:13 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Fri, 21 Sep 2018 09:51:13 +0900
Subject: [foreign] some JMH benchmarks
In-Reply-To: <55e13144-ad01-dd24-5081-864ebe8e7524@oracle.com>
References: <f4489307-f9ef-ee41-61da-71646f38c8d2@oracle.com>
 <d77cf3fe-c402-0e95-af11-fd4a683e03b2@gmail.com>
 <03fd2280-50a7-7772-7100-317a199ecf78@oracle.com>
 <CAHQD+UxUv1RpK=d9ifPLNCneaONX+ugfRWX7h2bL3OWD9TgAzw@mail.gmail.com>
 <34d05b5e-cae7-a82e-5f7b-a84d904cc8b7@oracle.com>
 <106fc755-1f1a-1fba-5de2-03b0e21e9a86@oracle.com>
 <25be0579-eb55-65e7-b379-206b8713a687@gmail.com>
 <d6a908bb-7a31-5e6a-37b8-106149ffac29@oracle.com>
 <580516c9-0ef2-0ceb-7c8c-c67d6621c074@gmail.com>
 <55e13144-ad01-dd24-5081-864ebe8e7524@oracle.com>
Message-ID: <26e88428-a2fc-5833-8433-ca394ef19f96@gmail.com>

Sounds good, thanks for testing this and for filing the bug report!

Samuel

On 09/21/2018 03:14 AM, Maurizio Cimadamore wrote:
> Sorry for the delay in getting back at you. There's indeed something 
> fishy going on here, and I have spotted a regression in JNI perf since 
> JDK 11. This could be caused by update in compiler toolchain introduced 
> in same version, but I have filed an issue for our hotspot team to 
> investigate:
> 
> https://bugs.openjdk.java.net/browse/JDK-8210975
> 
> In the context of this discussion, it's likely that the rtegression is 
> affecting the numbers of both Panama (which is built on top of JNI at 
> the moment) and the JNI benchmarks.
> 
> Thanks
> Maurizio
> 
> 
> On 19/09/18 01:13, Samuel Audet wrote:
>> Thanks! You haven't mentioned the version of the JDK you're using 
>> though. I'm starting to get the impression that JNI in newer versions 
>> of OpenJDK will be slower... ?
>>
>> On 09/18/2018 07:03 PM, Maurizio Cimadamore wrote:
>>> These are the numbers I get
>>>
>>> Benchmark???????????????????????? Mode? Cnt???????? Score Error Units
>>> NativeBenchmark.expBenchmark???? thrpt??? 5? 30542590.094 ? 
>>> 44126.434? ops/s
>>> NativeBenchmark.getpidBenchmark? thrpt??? 5? 61764677.092 ? 
>>> 21102.236? ops/s
>>>
>>> They are in the same ballpark, but exp() is a bit faster; byw, I 
>>> tried to repeat my benchmark with JNI exp() _and_ O3 and I've got 
>>> very similar numbers (yesterday I did a very quick test and there was 
>>> probably some other job running on the machine and brining down the 
>>> figures a bit).
>>>
>>> But overall, the results in your bench seem to match what I got: exp 
>>> is faster, pid is slower, the difference is mostly caused by O3. If 
>>> no O3 is used, then the numbers should match what I included in my 
>>> numbers (and getpid should be a bit faster).
>>>
>>> Maurizio
>>>
>>>
>>> On 18/09/18 05:48, Samuel Audet wrote:
>>>> Anyway, I've put online an updated version of my benchmark files here:
>>>> https://gist.github.com/saudet/1bf14a000e64c245675cf5d4e9ad6e69
>>>> Just run "git clone" on the URL and run "mvn package" on the pom.xml.
>>>>
>>>> With the 2 virtual cores of an Intel(R) Xeon(R) CPU E5-2673 v4 @ 
>>>> 2.30GHz running Ubuntu 14.04 on the cloud with GCC 4.9 and OpenJDK 
>>>> 8, I get these numbers:
>>>>
>>>> Benchmark???????????????????????? Mode? Cnt????????? Score Error ?Units
>>>> NativeBenchmark.expBenchmark???? thrpt?? 25?? 37460540.440 ? 
>>>> 393299.974 ?ops/s
>>>> NativeBenchmark.getpidBenchmark? thrpt?? 25? 100323188.451 ? 
>>>> 1254197.449 ?ops/s
>>>>
>>>> While on my laptop, an Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz 
>>>> running Fedora 27, GCC 7.3, and OpenJDK 9, I get the following:
>>>>
>>>> Benchmark???????????????????????? Mode? Cnt???????? Score Error Units
>>>> NativeBenchmark.expBenchmark???? thrpt?? 25? 50047147.099 ? 
>>>> 924366.937 ops/s
>>>> NativeBenchmark.getpidBenchmark? thrpt?? 25?? 4825508.193 ? 
>>>> 21662.633 ops/s
>>>>
>>>> Now, it looks like getpid() is really slow on Fedora 27 for some 
>>>> reason, but as Linus puts it, we should not be using that for 
>>>> benchmarking:
>>>> https://yarchive.net/comp/linux/getpid_caching.html
>>>>
>>>> What do you get on your machines?
>>>>
>>>> Samuel
>>>>
>>>>
>>>> On 09/18/2018 12:58 AM, Maurizio Cimadamore wrote:
>>>>> For the records, here's what I get for all the three benchmarks if 
>>>>> I compile the JNI code with -O3:
>>>>>
>>>>> Benchmark????????????????????????? Mode? Cnt Score Error Units
>>>>> PanamaBenchmark.testJNIExp??????? thrpt??? 5? 28575269.294 ? 
>>>>> 1907726.710? ops/s
>>>>> PanamaBenchmark.testJNIJavaQsort? thrpt??? 5??? 372148.433 ? 
>>>>> 27178.529? ops/s
>>>>> PanamaBenchmark.testJNIPid??????? thrpt??? 5? 59240069.011 ? 
>>>>> 403881.697? ops/s
>>>>>
>>>>> The first and second benchmarks get faster and very close to the 
>>>>> 'direct' optimization numbers in [1]. Surprisingly, the last 
>>>>> benchmark (getpid) is quite slower. I've been able to reproduce 
>>>>> across multiple runs; for that benchmark omitting O3 seems to be 
>>>>> the achieve best results, not sure why. It starts of faster (around 
>>>>> in the first couple of warmup iterations, but then it goes slower 
>>>>> in all the other runs - presumably it interacts badly with the C2 
>>>>> generated code. For instance, this is a run with O3 enabled:
>>>>>
>>>>> # Run progress: 66.67% complete, ETA 00:01:40
>>>>> # Fork: 1 of 1
>>>>> # Warmup Iteration?? 1: 65182202.653 ops/s
>>>>> # Warmup Iteration?? 2: 64900639.094 ops/s
>>>>> # Warmup Iteration?? 3: 59314945.437 ops/s 
>>>>> <---------------------------------
>>>>> # Warmup Iteration?? 4: 59269007.877 ops/s
>>>>> # Warmup Iteration?? 5: 59239905.163 ops/s
>>>>> Iteration?? 1: 59300748.074 ops/s
>>>>> Iteration?? 2: 59249666.044 ops/s
>>>>> Iteration?? 3: 59268597.051 ops/s
>>>>> Iteration?? 4: 59322074.572 ops/s
>>>>> Iteration?? 5: 59059259.317 ops/s
>>>>>
>>>>> And this is a run with O3 disabled:
>>>>>
>>>>> # Run progress: 0.00% complete, ETA 00:01:40
>>>>> # Fork: 1 of 1
>>>>> # Warmup Iteration?? 1: 55882128.787 ops/s
>>>>> # Warmup Iteration?? 2: 53102361.751 ops/s
>>>>> # Warmup Iteration?? 3: 66964755.699 ops/s 
>>>>> <---------------------------------
>>>>> # Warmup Iteration?? 4: 66414428.355 ops/s
>>>>> # Warmup Iteration?? 5: 65328475.276 ops/s
>>>>> Iteration?? 1: 64229192.993 ops/s
>>>>> Iteration?? 2: 65191719.319 ops/s
>>>>> Iteration?? 3: 65352022.471 ops/s
>>>>> Iteration?? 4: 65152090.426 ops/s
>>>>> Iteration?? 5: 65320545.712 ops/s
>>>>>
>>>>>
>>>>> In both cases, the 3rd warmup execution sees a performance jump - 
>>>>> with O3, the jump is backwards, w/o O3 the jump is forward, which 
>>>>> is quite typical for a JMH benchmark as C2 optimization will start 
>>>>> to kick in.
>>>>>
>>>>> For these reasons, I'm reluctant to update my benchmark numbers to 
>>>>> reflect the O3 behavior (although I agree that, since the Hotspot 
>>>>> code is compiled with that optimization it would make more sense to 
>>>>> use that as a reference).
>>>>>
>>>>> Maurizio
>>>>>
>>>>> [1] - http://cr.openjdk.java.net/~mcimadamore/panama/foreign-jmh.txt
>>>>>
>>>>>
>>>>>
>>>>> On 17/09/18 16:18, Maurizio Cimadamore wrote:
>>>>>>
>>>>>>
>>>>>> On 17/09/18 15:08, Samuel Audet wrote:
>>>>>>> Yes, the blackhole or the random number doesn't make any 
>>>>>>> difference, but not calling gcc with -O3 does. Running the 
>>>>>>> compiler with optimizations on is pretty common, but they are not 
>>>>>>> enabled by default.
>>>>>> A bit better
>>>>>>
>>>>>> PanamaBenchmark.testMethod? thrpt??? 5? 28018170.076 ? 8491668.248 
>>>>>> ops/s
>>>>>>
>>>>>> But not much of a difference (I did not expected much, as the body 
>>>>>> of the native method is extremely simple).
>>>>>>
>>>>>> Maurizio 
>>>>
>>>
>>
> 


From vladimir.x.ivanov at oracle.com  Fri Sep 21 01:23:29 2018
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Thu, 20 Sep 2018 18:23:29 -0700
Subject: [vector] RFC: Add scalable shapes for Arm Scalable Vector
 Extension
In-Reply-To: <AM0PR08MB3523229FCD23E33A488724E790000@AM0PR08MB3523.eurprd08.prod.outlook.com>
References: <AM0PR08MB3523229FCD23E33A488724E790000@AM0PR08MB3523.eurprd08.prod.outlook.com>
Message-ID: <5a9d8c0d-dfd9-efb0-0198-f5d84e8ea925@oracle.com>

Hi Ningsheng,

First of all, it's great to see Arm looking into Vector API! Welcome!

> As you may know that Arm SVE architecturally supports scalable vector length, from 128 to 2048bits, multiple of 128bits. Currently Vector API only supports 64, 128, 256 and 512 bit vector shapes. It's inappropriate to pre-generate all SVE supported shapes, hence, we propose a new type of vector for scalable length (or length agnostic) shapes.
> 
> Webrev:
> http://cr.openjdk.java.net/~njian/sve/vectorapi.webrev00/
> 
> Could you please help to take a look at this? The *ScalableVector are the new scalable length vectors and to be compatible with existing architectures, the preferredSpecies() will return those known species first.

Overall, I like how it shapes out. There are some aspects I'd like to 
clarify though.

The name (*ScalableVector) is slightly misleading to me, but I'm fine 
with it.  I interpret new shapes as representing vectors of maximally 
supported size at runtime without specifying the actual size.

New shapes don't interoperate well with existing ones, so unless you 
change how vector shapes are checked (==), at runtime scalable and fixed 
shape variants shouldn't meet. That leads to the next question: how do 
you expect vector shape changing operation working (resize(), 
rebracket(), cast()) with new variants?

> We are working on SVE support on OpenJDK, and current Vector API's intrinsics can really help us on our SVE backend work. We have an initial prototype which adds SVE backend and updates a bit for current register allocator to support scalable length vector types. Our WIP SVE backend shows that this scalable vector proposal works fine with current Vector API intrinsics. We would like to collect your comments about this scalable vector patch so that we can move forward for the SVE backend support. I notice that Vladimir is also thinking about alternative loop shapes, that's really helpful for SVE! We will also take a look at that further.

It's very encouraging to hear existing intrinsics generalize well to 
SVE. Looking forward to seeing the results of SVE backend work. Feel 
free to share your pain points while implementing it.

Best regards,
Vladimir Ivanov

From sundararajan.athijegannathan at oracle.com  Fri Sep 21 04:11:11 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Fri, 21 Sep 2018 09:41:11 +0530
Subject: [foreign] RFR 8210935 C enum constants should be mapped to
 interface methods instead of static final int constants
In-Reply-To: <1caedbc9-05a7-f536-a377-116fb49a9179@oracle.com>
References: <5BA32CFC.5060802@oracle.com>
 <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>
 <5BA3BB45.5090806@oracle.com>
 <211c6f8d-ce93-dbad-e014-5528653f94f0@oracle.com>
 <1caedbc9-05a7-f536-a377-116fb49a9179@oracle.com>
Message-ID: <5BA46F5F.3090605@oracle.com>

I filed two issues:

1) https://bugs.openjdk.java.net/browse/JDK-8210992

This is for the test failure on Linux. That test need not use limits.h 
at all.

2) https://bugs.openjdk.java.net/browse/JDK-8210993

This is to handle limits.h & other related files as compiler built-ins.

Thanks,
-Sundar

On 20/09/18, 10:53 PM, Maurizio Cimadamore wrote:
> I have an issue with this test when running on Linux:
>
> /usr/include/limits.h:123:16: fatal error: 'limits.h' file not found
> java.lang.RuntimeException: jextract returns non-zero value
>     at JtregJextract.jextract(JtregJextract.java:85)
>     at JtregJextract.main(JtregJextract.java:96)
>     at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
>     at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>     at 
> com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:229)
>     at java.base/java.lang.Thread.run(Thread.java:834)
>
> I think the culprit is that we are not copying the clang version of 
> limits.h into the JDK; from make/copy/Copy-jdk.jextract.gmk:
>
> JEXTRACT_SRC_FILES := $(wildcard $(JEXTRACT_CONF_SRC)/*std*.h)
>
> That is we are filtering with 'std'. I think we need limits.h too (and 
> possibly others).
>
> Maurizio
>
>
> On 20/09/18 18:16, Maurizio Cimadamore wrote:
>> Looks good!
>>
>> Maurizio
>>
>>
>> On 20/09/18 16:22, Sundararajan Athijegannathan wrote:
>>> Thanks. I've added few more tests to combine enum constants.
>>>
>>> Pushing this updated changes: 
>>> http://cr.openjdk.java.net/~sundar/8210935/webrev.01/
>>>
>>> -Sundar
>>>
>>> On 20/09/18, 8:26 PM, Maurizio Cimadamore wrote:
>>>> Looks good.
>>>>
>>>> Maybe in the test you could also check that e.g. combining enum 
>>>> constants in Java and C yields the same - e.g.
>>>>
>>>> R | G
>>>>
>>>> ?
>>>>
>>>> Maurizio
>>>>
>>>>
>>>>
>>>> On 20/09/18 06:15, Sundararajan Athijegannathan wrote:
>>>>> Please review.
>>>>>
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
>>>>> Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/
>>>>>
>>>>> Thanks,
>>>>> -Sundar
>>>>
>>
>

From sundararajan.athijegannathan at oracle.com  Fri Sep 21 04:13:48 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Fri, 21 Sep 2018 09:43:48 +0530
Subject: [foreign] RFR 8210992: LibEnumsTest fails on Linux with "fatal error:
 'limits.h' file not found"
Message-ID: <5BA46FFC.2090805@oracle.com>

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8210992
Webrev: http://cr.openjdk.java.net/~sundar/8210992/webrev.00/

Background:

http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002740.html

Thanks,
-Sundar


From maurizio.cimadamore at oracle.com  Fri Sep 21 09:44:47 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 21 Sep 2018 10:44:47 +0100
Subject: [foreign] RFR 8210992: LibEnumsTest fails on Linux with "fatal
 error: 'limits.h' file not found"
In-Reply-To: <5BA46FFC.2090805@oracle.com>
References: <5BA46FFC.2090805@oracle.com>
Message-ID: <f4473d55-5570-cae6-61dc-62942c06c04c@oracle.com>

Looks good

Maurizio


On 21/09/18 05:13, Sundararajan Athijegannathan wrote:
> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8210992
> Webrev: http://cr.openjdk.java.net/~sundar/8210992/webrev.00/
>
> Background:
>
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002740.html 
>
>
> Thanks,
> -Sundar
>


From maurizio.cimadamore at oracle.com  Fri Sep 21 09:46:08 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 21 Sep 2018 10:46:08 +0100
Subject: [foreign] RFR 8210935 C enum constants should be mapped to
 interface methods instead of static final int constants
In-Reply-To: <5BA46F5F.3090605@oracle.com>
References: <5BA32CFC.5060802@oracle.com>
 <2bc346b1-fa23-3c4b-2833-57534794d4bb@oracle.com>
 <5BA3BB45.5090806@oracle.com>
 <211c6f8d-ce93-dbad-e014-5528653f94f0@oracle.com>
 <1caedbc9-05a7-f536-a377-116fb49a9179@oracle.com>
 <5BA46F5F.3090605@oracle.com>
Message-ID: <e452af08-8475-af80-e6f8-f9efffd5fc82@oracle.com>

Thanks! I've been experiencing (2) a lot when working on posix libraries 
few months ago, so at least on Linux this is something that will come 
up, I think, quite frequently.

Maurizio


On 21/09/18 05:11, Sundararajan Athijegannathan wrote:
> I filed two issues:
>
> 1) https://bugs.openjdk.java.net/browse/JDK-8210992
>
> This is for the test failure on Linux. That test need not use limits.h 
> at all.
>
> 2) https://bugs.openjdk.java.net/browse/JDK-8210993
>
> This is to handle limits.h & other related files as compiler built-ins.
>
> Thanks,
> -Sundar
>
> On 20/09/18, 10:53 PM, Maurizio Cimadamore wrote:
>> I have an issue with this test when running on Linux:
>>
>> /usr/include/limits.h:123:16: fatal error: 'limits.h' file not found
>> java.lang.RuntimeException: jextract returns non-zero value
>> ??? at JtregJextract.jextract(JtregJextract.java:85)
>> ??? at JtregJextract.main(JtregJextract.java:96)
>> ??? at 
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
>> Method)
>> ??? at 
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> ??? at 
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> ??? at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>> ??? at 
>> com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:229)
>> ??? at java.base/java.lang.Thread.run(Thread.java:834)
>>
>> I think the culprit is that we are not copying the clang version of 
>> limits.h into the JDK; from make/copy/Copy-jdk.jextract.gmk:
>>
>> JEXTRACT_SRC_FILES := $(wildcard $(JEXTRACT_CONF_SRC)/*std*.h)
>>
>> That is we are filtering with 'std'. I think we need limits.h too 
>> (and possibly others).
>>
>> Maurizio
>>
>>
>> On 20/09/18 18:16, Maurizio Cimadamore wrote:
>>> Looks good!
>>>
>>> Maurizio
>>>
>>>
>>> On 20/09/18 16:22, Sundararajan Athijegannathan wrote:
>>>> Thanks. I've added few more tests to combine enum constants.
>>>>
>>>> Pushing this updated changes: 
>>>> http://cr.openjdk.java.net/~sundar/8210935/webrev.01/
>>>>
>>>> -Sundar
>>>>
>>>> On 20/09/18, 8:26 PM, Maurizio Cimadamore wrote:
>>>>> Looks good.
>>>>>
>>>>> Maybe in the test you could also check that e.g. combining enum 
>>>>> constants in Java and C yields the same - e.g.
>>>>>
>>>>> R | G
>>>>>
>>>>> ?
>>>>>
>>>>> Maurizio
>>>>>
>>>>>
>>>>>
>>>>> On 20/09/18 06:15, Sundararajan Athijegannathan wrote:
>>>>>> Please review.
>>>>>>
>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210935
>>>>>> Webrev: http://cr.openjdk.java.net/~sundar/8210935/webrev.00/
>>>>>>
>>>>>> Thanks,
>>>>>> -Sundar
>>>>>
>>>
>>


From sundararajan.athijegannathan at oracle.com  Fri Sep 21 09:55:24 2018
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Fri, 21 Sep 2018 09:55:24 +0000
Subject: hg: panama/dev: 8210992: LibEnumsTest fails on Linux with "fatal
 error: 'limits.h' file not found"
Message-ID: <201809210955.w8L9tPSv021991@aojmv0008.oracle.com>

Changeset: 77d4d11c5a4f
Author:    sundar
Date:      2018-09-21 15:31 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/77d4d11c5a4f

8210992: LibEnumsTest fails on Linux with "fatal error: 'limits.h' file not found"
Reviewed-by: mcimadamore

! test/jdk/com/sun/tools/jextract/testEnum/LibEnumsTest.java
! test/jdk/com/sun/tools/jextract/testEnum/enums.h
! test/jdk/com/sun/tools/jextract/testEnum/libEnums.c


From maurizio.cimadamore at oracle.com  Fri Sep 21 09:58:15 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 21 Sep 2018 09:58:15 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809210958.w8L9wGs4023879@aojmv0008.oracle.com>

Changeset: 3db6509289ec
Author:    mcimadamore
Date:      2018-09-21 12:00 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/3db6509289ec

Automatic merge with foreign



From maurizio.cimadamore at oracle.com  Fri Sep 21 11:47:19 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 21 Sep 2018 12:47:19 +0100
Subject: [foreign] RFR 8210998: Missing pointer scope checks
Message-ID: <41bc2150-d7dd-3acb-4c6e-08ea9c15065b@oracle.com>

Hi,
while playing around with the Panama API in jshell, I realized that we 
don't always enforce scope liveness in all contexts. One of the most 
important cases was:

Pointer::addr

But there were other cases left out too (see Reference subclasses).

This patch adds the missing checks in the right places and also add a 
comprehensive test which check:

* Pointer dereference (get/set) after Scope::close() with various 
pointer shapes

* Struct/array/callback access after Scope::close

* Passing pointer/struct/callback after Scope::close

In writing this tests I uncovered few issues:

1) The signature of LayoutType.ofFunction was wrong - it had a 
Callback<Z> bound on the type variable which was bogus. In fact there's 
no bound here (this is a leftover from previous code). This also 
required tweaks to LayoutTypeImpl

2) Scope::allocateCallback was NOT associating the callback entry point 
pointer with the current scope. As a result the callback pointer was 
accessible even AFTER the owning scope was closed. I fixed it by moving 
the pointer creation logic inside Scope - now UpcallHandler just has a 
'long' entry point, which I think makes the code even clearer.

3) CallbackImplGenerator - this code was going out of his way in order 
to enforce checks on pointer scope; however, on a closer inspection, 
since this code relied on 'addr()' and Pointer::addr is now checked 
properly, there's no need to add special cruft.

4) The logic for 'setting' arrays into array references was overly 
convoluted: with Array we can now just do the operation with a bulk 
pointer copy.

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8210998/

Cheers
Maurizio




From maurizio.cimadamore at oracle.com  Fri Sep 21 11:52:22 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 21 Sep 2018 12:52:22 +0100
Subject: [foreign] RFR 8210992: LibEnumsTest fails on Linux with "fatal
 error: 'limits.h' file not found"
In-Reply-To: <f4473d55-5570-cae6-61dc-62942c06c04c@oracle.com>
References: <5BA46FFC.2090805@oracle.com>
 <f4473d55-5570-cae6-61dc-62942c06c04c@oracle.com>
Message-ID: <21954ecb-0245-0121-9c73-9cfb41a61d11@oracle.com>

For the records, the test passes now :-)

Maurizio


On 21/09/18 10:44, Maurizio Cimadamore wrote:
> Looks good
>
> Maurizio
>
>
> On 21/09/18 05:13, Sundararajan Athijegannathan wrote:
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210992
>> Webrev: http://cr.openjdk.java.net/~sundar/8210992/webrev.00/
>>
>> Background:
>>
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002740.html 
>>
>>
>> Thanks,
>> -Sundar
>>
>


From sundararajan.athijegannathan at oracle.com  Fri Sep 21 12:26:06 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Fri, 21 Sep 2018 17:56:06 +0530
Subject: [foreign] RFR 8210992: LibEnumsTest fails on Linux with "fatal
 error: 'limits.h' file not found"
In-Reply-To: <21954ecb-0245-0121-9c73-9cfb41a61d11@oracle.com>
References: <5BA46FFC.2090805@oracle.com>
 <f4473d55-5570-cae6-61dc-62942c06c04c@oracle.com>
 <21954ecb-0245-0121-9c73-9cfb41a61d11@oracle.com>
Message-ID: <5BA4E35E.5080604@oracle.com>

Thanks for the confirmation!

-Sundar

On 21/09/18, 5:22 PM, Maurizio Cimadamore wrote:
> For the records, the test passes now :-)
>
> Maurizio
>
>
> On 21/09/18 10:44, Maurizio Cimadamore wrote:
>> Looks good
>>
>> Maurizio
>>
>>
>> On 21/09/18 05:13, Sundararajan Athijegannathan wrote:
>>> Please review.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8210992
>>> Webrev: http://cr.openjdk.java.net/~sundar/8210992/webrev.00/
>>>
>>> Background:
>>>
>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002740.html 
>>>
>>>
>>> Thanks,
>>> -Sundar
>>>
>>
>

From sundararajan.athijegannathan at oracle.com  Fri Sep 21 12:36:22 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Fri, 21 Sep 2018 18:06:22 +0530
Subject: [foreign] RFR 8210998: Missing pointer scope checks
In-Reply-To: <41bc2150-d7dd-3acb-4c6e-08ea9c15065b@oracle.com>
References: <41bc2150-d7dd-3acb-4c6e-08ea9c15065b@oracle.com>
Message-ID: <5BA4E5C6.1090508@oracle.com>

Looks good

-Sundar

On 21/09/18, 5:17 PM, Maurizio Cimadamore wrote:
> Hi,
> while playing around with the Panama API in jshell, I realized that we 
> don't always enforce scope liveness in all contexts. One of the most 
> important cases was:
>
> Pointer::addr
>
> But there were other cases left out too (see Reference subclasses).
>
> This patch adds the missing checks in the right places and also add a 
> comprehensive test which check:
>
> * Pointer dereference (get/set) after Scope::close() with various 
> pointer shapes
>
> * Struct/array/callback access after Scope::close
>
> * Passing pointer/struct/callback after Scope::close
>
> In writing this tests I uncovered few issues:
>
> 1) The signature of LayoutType.ofFunction was wrong - it had a 
> Callback<Z> bound on the type variable which was bogus. In fact 
> there's no bound here (this is a leftover from previous code). This 
> also required tweaks to LayoutTypeImpl
>
> 2) Scope::allocateCallback was NOT associating the callback entry 
> point pointer with the current scope. As a result the callback pointer 
> was accessible even AFTER the owning scope was closed. I fixed it by 
> moving the pointer creation logic inside Scope - now UpcallHandler 
> just has a 'long' entry point, which I think makes the code even clearer.
>
> 3) CallbackImplGenerator - this code was going out of his way in order 
> to enforce checks on pointer scope; however, on a closer inspection, 
> since this code relied on 'addr()' and Pointer::addr is now checked 
> properly, there's no need to add special cruft.
>
> 4) The logic for 'setting' arrays into array references was overly 
> convoluted: with Array we can now just do the operation with a bulk 
> pointer copy.
>
> Webrev:
>
> http://cr.openjdk.java.net/~mcimadamore/panama/8210998/
>
> Cheers
> Maurizio
>
>
>

From maurizio.cimadamore at oracle.com  Fri Sep 21 14:27:14 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 21 Sep 2018 14:27:14 +0000
Subject: hg: panama/dev: 8210998: Missing pointer scope checks
Message-ID: <201809211427.w8LERFUM026458@aojmv0008.oracle.com>

Changeset: acbf4c4be734
Author:    mcimadamore
Date:      2018-09-21 15:26 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/acbf4c4be734

8210998: Missing pointer scope checks
Reviewed-by: sundar

! src/java.base/share/classes/java/foreign/memory/LayoutType.java
! src/java.base/share/classes/jdk/internal/foreign/CallbackImplGenerator.java
! src/java.base/share/classes/jdk/internal/foreign/ScopeImpl.java
! src/java.base/share/classes/jdk/internal/foreign/UpcallHandler.java
! src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java
! src/java.base/share/classes/jdk/internal/foreign/memory/LayoutTypeImpl.java
! src/java.base/share/classes/jdk/internal/foreign/memory/References.java
+ test/jdk/java/foreign/types/PointerScopeTest.java
+ test/jdk/java/foreign/types/libPointerScope.c


From maurizio.cimadamore at oracle.com  Fri Sep 21 14:31:48 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 21 Sep 2018 14:31:48 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809211431.w8LEVmBY028398@aojmv0008.oracle.com>

Changeset: 01ecea66dd99
Author:    mcimadamore
Date:      2018-09-21 16:34 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/01ecea66dd99

Automatic merge with foreign



From maurizio.cimadamore at oracle.com  Fri Sep 21 15:17:02 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 21 Sep 2018 16:17:02 +0100
Subject: [foreign] RFR 8211006: Add API points to support by-value semantics
 for structs and arrays
Message-ID: <45aafb81-8fb9-2b89-c194-4cea63274837@oracle.com>

As written in the issue summary (see for more details):

https://bugs.openjdk.java.net/browse/JDK-8211006

We need API points to perform deep copy of structs/arrays, which would 
allow developers to easily mimic what happens when operating on 
struct/array struct fields.

I also did a minor tweak on References.ofArray, as the setter was 
redundantly creating an Array object instead of just copying.

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8211006/

Cheers
Maurizio



From jbvernee at xs4all.nl  Fri Sep 21 19:44:29 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Fri, 21 Sep 2018 21:44:29 +0200
Subject: [foreign] test_time timezone trouble (TZ)
Message-ID: <548afe90ca8727551817bd4a3b215345@xs4all.nl>

Hello guys,

I was running into a problem with the test_time test in StdLibTest. The 
generator for test values was apparently generating invalid values. 
After fixing that, all the test iterations were failing because the 
hours were off by one (the output of LocalDateTime.getHours() is being 
compared to the output of localtime().hour()). Good thing somebody else 
on the internet seems to have had the same problem [1]. TL;DR when using 
the MSVC version of localtime, and when running in cygwin, the function 
tries to interpret the TZ environment variable, but since that has a 
unix format (courtesy of cygwin), the interpretation fails and defaults 
to GMT. When TZ is not set, it defaults to the system default timezone, 
which is also what's being tested against.

I can get the tests to pass by using `unset TZ` in the cygwin terminal 
before running them, but I'd have to do that every time I reboot it. I 
was trying to unset TZ automatically by using jtreq `@run shell 
unsettz.sh` where unsettz.sh is a file containing just the command 
`unset TZ`. It seems to be running successfully according to the test 
output:

ACTION: shell -- Passed. Execution successful
REASON: User specified action: run shell unsettz.sh
TIME:   0.126 seconds
messages:
command: shell unsettz.sh
reason: User specified action: run shell unsettz.sh
elapsed time (seconds): 0.126
STDOUT:
STDERR:

But it doesn't seem to affect the tests itself, and they still fail 
(still off by 1 hour).

I was wondering if there is a way to let jtreg control environment 
variables? Or maybe you can suggest a different solution?

The jtreg guide [2] mentions that TZ will be propagated from Windows 'if 
set', but I need it to be not set, or automatically set to the system's 
default time zone (by default it's blank).

(other than that, tests are looking good: `passed: 24; failed: 4; error: 
1`. I just need to fix structs by value, which on Windows cheats and 
just passes a pointer. 2 failing tests are from jextract missing)

Thanks,
Jorn

[1] : https://stackoverflow.com/q/11655003
[2] : http://openjdk.java.net/jtreg/tag-spec.html

From michel.trudeau at oracle.com  Fri Sep 21 21:12:08 2018
From: michel.trudeau at oracle.com (Michel Trudeau)
Date: Fri, 21 Sep 2018 14:12:08 -0700
Subject: [foreign] test_time timezone trouble (TZ)
In-Reply-To: <548afe90ca8727551817bd4a3b215345@xs4all.nl>
References: <548afe90ca8727551817bd4a3b215345@xs4all.nl>
Message-ID: <9FDB2BFC-2A7C-4485-9EAE-401669F86BF0@oracle.com>

[adding jtreg mailing list to seek out answer about jtreg and TZ on Windows]

On Sep 21, 2018, at 12:44 PM, Jorn Vernee <jbvernee at xs4all.nl> wrote:

Hello guys,

I was running into a problem with the test_time test in StdLibTest. The generator for test values was apparently generating invalid values. After fixing that, all the test iterations were failing because the hours were off by one (the output of LocalDateTime.getHours() is being compared to the output of localtime().hour()). Good thing somebody else on the internet seems to have had the same problem [1]. TL;DR when using the MSVC version of localtime, and when running in cygwin, the function tries to interpret the TZ environment variable, but since that has a unix format (courtesy of cygwin), the interpretation fails and defaults to GMT. When TZ is not set, it defaults to the system default timezone, which is also what's being tested against.

I can get the tests to pass by using `unset TZ` in the cygwin terminal before running them, but I'd have to do that every time I reboot it. I was trying to unset TZ automatically by using jtreq `@run shell unsettz.sh` where unsettz.sh is a file containing just the command `unset TZ`. It seems to be running successfully according to the test output:

ACTION: shell -- Passed. Execution successful
REASON: User specified action: run shell unsettz.sh
TIME:   0.126 seconds
messages:
command: shell unsettz.sh
reason: User specified action: run shell unsettz.sh
elapsed time (seconds): 0.126
STDOUT:
STDERR:

But it doesn't seem to affect the tests itself, and they still fail (still off by 1 hour).

I was wondering if there is a way to let jtreg control environment variables? Or maybe you can suggest a different solution?

The jtreg guide [2] mentions that TZ will be propagated from Windows 'if set', but I need it to be not set, or automatically set to the system's default time zone (by default it's blank).

(other than that, tests are looking good: `passed: 24; failed: 4; error: 1`. I just need to fix structs by value, which on Windows cheats and just passes a pointer. 2 failing tests are from jextract missing)

Thanks,
Jorn

[1] : https://stackoverflow.com/q/11655003
[2] : http://openjdk.java.net/jtreg/tag-spec.html


From maurizio.cimadamore at oracle.com  Fri Sep 21 23:18:14 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Sat, 22 Sep 2018 00:18:14 +0100
Subject: [foreign] test_time timezone trouble (TZ)
In-Reply-To: <548afe90ca8727551817bd4a3b215345@xs4all.nl>
References: <548afe90ca8727551817bd4a3b215345@xs4all.nl>
Message-ID: <ffa2d039-6a61-f1c1-d396-34747088dbc1@oracle.com>

Maybe an easier solution would be to tweak the test to use gmtime() 
instead of localtime() and then stick to GMT? That ought to be more robust?

Maurizio


On 21/09/18 20:44, Jorn Vernee wrote:
> Hello guys,
>
> I was running into a problem with the test_time test in StdLibTest. 
> The generator for test values was apparently generating invalid 
> values. After fixing that, all the test iterations were failing 
> because the hours were off by one (the output of 
> LocalDateTime.getHours() is being compared to the output of 
> localtime().hour()). Good thing somebody else on the internet seems to 
> have had the same problem [1]. TL;DR when using the MSVC version of 
> localtime, and when running in cygwin, the function tries to interpret 
> the TZ environment variable, but since that has a unix format 
> (courtesy of cygwin), the interpretation fails and defaults to GMT. 
> When TZ is not set, it defaults to the system default timezone, which 
> is also what's being tested against.
>
> I can get the tests to pass by using `unset TZ` in the cygwin terminal 
> before running them, but I'd have to do that every time I reboot it. I 
> was trying to unset TZ automatically by using jtreq `@run shell 
> unsettz.sh` where unsettz.sh is a file containing just the command 
> `unset TZ`. It seems to be running successfully according to the test 
> output:
>
> ACTION: shell -- Passed. Execution successful
> REASON: User specified action: run shell unsettz.sh
> TIME:?? 0.126 seconds
> messages:
> command: shell unsettz.sh
> reason: User specified action: run shell unsettz.sh
> elapsed time (seconds): 0.126
> STDOUT:
> STDERR:
>
> But it doesn't seem to affect the tests itself, and they still fail 
> (still off by 1 hour).
>
> I was wondering if there is a way to let jtreg control environment 
> variables? Or maybe you can suggest a different solution?
>
> The jtreg guide [2] mentions that TZ will be propagated from Windows 
> 'if set', but I need it to be not set, or automatically set to the 
> system's default time zone (by default it's blank).
>
> (other than that, tests are looking good: `passed: 24; failed: 4; 
> error: 1`. I just need to fix structs by value, which on Windows 
> cheats and just passes a pointer. 2 failing tests are from jextract 
> missing)
>
> Thanks,
> Jorn
>
> [1] : https://stackoverflow.com/q/11655003
> [2] : http://openjdk.java.net/jtreg/tag-spec.html


From jbvernee at xs4all.nl  Fri Sep 21 23:41:58 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Sat, 22 Sep 2018 01:41:58 +0200
Subject: [foreign] test_time timezone trouble (TZ)
In-Reply-To: <ffa2d039-6a61-f1c1-d396-34747088dbc1@oracle.com>
References: <548afe90ca8727551817bd4a3b215345@xs4all.nl>
 <ffa2d039-6a61-f1c1-d396-34747088dbc1@oracle.com>
Message-ID: <926de2fb006aa0657f929a761f887576@xs4all.nl>

Maurizio Cimadamore schreef op 2018-09-22 01:18:
> Maybe an easier solution would be to tweak the test to use gmtime()
> instead of localtime() and then stick to GMT? That ought to be more
> robust?
> 
> Maurizio

That's a good idea, I didn't know if you specifically wanted to test 
localtime(). I've changed the test to use gmtime(), and am testing 
against `ZoneOffset.UTC`. StdLibTest is now passing, I'll try to get the 
last few tests passing as well before submitting as a patch!

Jorn

From sandhya.viswanathan at intel.com  Sat Sep 22 00:44:36 2018
From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya)
Date: Sat, 22 Sep 2018 00:44:36 +0000
Subject: [vector] Linux SVML gcc version string handling
Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EEF6086@FMSMSX126.amr.corp.intel.com>


Please find below a patch which fixes the gcc version string handling for the Vector API Linux SVML assembly files:
http://cr.openjdk.java.net/~svkamath/webrev.00/

For the preprocessor to work in Linux assembly, the filenames need to have .S as extension instead of .s.
Also corrected the required major minor version string and the header file placement.

Best Regards,
Sandhya


From maurizio.cimadamore at oracle.com  Sun Sep 23 19:57:16 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Sun, 23 Sep 2018 20:57:16 +0100
Subject: [foreign] test_time timezone trouble (TZ)
In-Reply-To: <926de2fb006aa0657f929a761f887576@xs4all.nl>
References: <548afe90ca8727551817bd4a3b215345@xs4all.nl>
 <ffa2d039-6a61-f1c1-d396-34747088dbc1@oracle.com>
 <926de2fb006aa0657f929a761f887576@xs4all.nl>
Message-ID: <59840e41-cc6a-873d-fd25-e9edc97d6e51@oracle.com>



On 22/09/18 00:41, Jorn Vernee wrote:
> Maurizio Cimadamore schreef op 2018-09-22 01:18:
>> Maybe an easier solution would be to tweak the test to use gmtime()
>> instead of localtime() and then stick to GMT? That ought to be more
>> robust?
>>
>> Maurizio
>
> That's a good idea, I didn't know if you specifically wanted to test 
> localtime(). I've changed the test to use gmtime(), and am testing 
> against `ZoneOffset.UTC`. StdLibTest is now passing, I'll try to get 
> the last few tests passing as well before submitting as a patch!
>
There was no specific need to call localtime() - I just wanted something 
that returned a Tm struct back and I overlooked the gmtime function 
which should be far more reliable. So I'm all for replacing that.

Maurizio
> Jorn


From jbvernee at xs4all.nl  Mon Sep 24 10:52:54 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 24 Sep 2018 12:52:54 +0200
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on Windows
Message-ID: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>

Hello,

I'd like to contribute a patch that adds support for the default library 
on windows.

Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
Diff: 
https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8

As mentioned before [1], this fixes 2 bugs:

1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
threw an NPE (at least on windows). That is fixed by returning
defaultLibrary.fromClass when the nativeLibraryContext is empty.

2.) The default library search was not working on windows. It was using
a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not on
windows (see https://stackoverflow.com/q/23437007). I have changed the
implementation from using a default handle to using a new native
function findEntryInProcess, which does the right thing for Windows
(iterate over all loaded modules), and does the same thing it used to
for unix. findEntry is now a Java method, and the original findEntry is
renamed to findEntry0. The NativeLibrary implementation of findEntry
forwards to findEntry0, and the anonymous class of the default library
overrides to forward to findEntryInProcess.

Please test this patch on unix as well, since I don't have the ability 
to do so.

Jorn

[1] : 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html

From maurizio.cimadamore at oracle.com  Mon Sep 24 11:10:07 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 24 Sep 2018 12:10:07 +0100
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
Message-ID: <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>

Hi Jorn,
thanks for the patch. As mentioned last time we have two options here: 
one is to follow the approach forward in your patch; another would be to 
ditch Library.getDefault() entirely and adopt a more explicit approach - 
that is to always require 'implicit' libraries to be mentioned - either 
by jextract artifacts or by API points.

A note on the latter - when you do:

Libraries.bindRaw(lookup, Foo.class)

the code lookup the @NativeHeader annotation on Foo.class and tries to 
extract required library names from there. Currently, we do not add 
library names for standard libraries such as "c" or "m" (math), which is 
what led us down the (slippery?) slope of having a Library.getDefault 
somewhere.

Another option would be to have jextract to always generate the names of 
the libraries an artifact depends on, and then the API should throw an 
exception if a @NativeHeader is found with no libraries. More 
specifically, jextract should always add "c" to the set of used 
libraries in the @NativeHeader annotation (other libraries can be 
explicitly supplied using the -l flag).

Note that I'm not saying "the binder should always add in 'clib'" for 
you, because that's kind of a lame assumption: it works obviously well 
for C but it doesn't make a lot of sense if you want to use Panama for 
other purposes/languages. Which seems to suggest that, as far as the 
binder is concerned, library dependencies should always be explicit.

Thoughts?

Maurizio



On 24/09/18 11:52, Jorn Vernee wrote:
> Hello,
>
> I'd like to contribute a patch that adds support for the default 
> library on windows.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
> Diff: https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>
> As mentioned before [1], this fixes 2 bugs:
>
> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
> threw an NPE (at least on windows). That is fixed by returning
> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>
> 2.) The default library search was not working on windows. It was using
> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not on
> windows (see https://stackoverflow.com/q/23437007). I have changed the
> implementation from using a default handle to using a new native
> function findEntryInProcess, which does the right thing for Windows
> (iterate over all loaded modules), and does the same thing it used to
> for unix. findEntry is now a Java method, and the original findEntry is
> renamed to findEntry0. The NativeLibrary implementation of findEntry
> forwards to findEntry0, and the anonymous class of the default library
> overrides to forward to findEntryInProcess.
>
> Please test this patch on unix as well, since I don't have the ability 
> to do so.
>
> Jorn
>
> [1] : 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html


From jbvernee at xs4all.nl  Mon Sep 24 11:32:02 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 24 Sep 2018 13:32:02 +0200
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
Message-ID: <e9daf33dc915c4916ee9156ad87a2d12@xs4all.nl>

I also think the approach of removing the default library is more 
robust, since a symbol can be loaded from several libraries at once, but 
when using the default library you don't know which one you're gonna 
get. I can imagine a big project using several decencies which try to 
load the same symbol from different libraries through the default 
library and things breaking when the wrong function address is loaded.

On the other hand, library names for the standard library are platform 
specific. e.g. the standard C library dll is named msvcrt on windows, so 
writing platform agnostic code that just depends on the portable 
standard library, like the tests we have, would be more difficult.

Maybe that problem could be solved by shipping pre-computed library 
interfaces for the standard library with the JDK, where the value of 
NativeHeader#libraries would be platform specific. That would also 
decrease the time-to-helloworld for foreign, and I think that at least 
the standard C library is small enough that that won't be problematic.

Jorn

Maurizio Cimadamore schreef op 2018-09-24 13:10:
> Hi Jorn,
> thanks for the patch. As mentioned last time we have two options here:
> one is to follow the approach forward in your patch; another would be
> to ditch Library.getDefault() entirely and adopt a more explicit
> approach - that is to always require 'implicit' libraries to be
> mentioned - either by jextract artifacts or by API points.
> 
> A note on the latter - when you do:
> 
> Libraries.bindRaw(lookup, Foo.class)
> 
> the code lookup the @NativeHeader annotation on Foo.class and tries to
> extract required library names from there. Currently, we do not add
> library names for standard libraries such as "c" or "m" (math), which
> is what led us down the (slippery?) slope of having a
> Library.getDefault somewhere.
> 
> Another option would be to have jextract to always generate the names
> of the libraries an artifact depends on, and then the API should throw
> an exception if a @NativeHeader is found with no libraries. More
> specifically, jextract should always add "c" to the set of used
> libraries in the @NativeHeader annotation (other libraries can be
> explicitly supplied using the -l flag).
> 
> Note that I'm not saying "the binder should always add in 'clib'" for
> you, because that's kind of a lame assumption: it works obviously well
> for C but it doesn't make a lot of sense if you want to use Panama for
> other purposes/languages. Which seems to suggest that, as far as the
> binder is concerned, library dependencies should always be explicit.
> 
> Thoughts?
> 
> Maurizio
> 
> 
> 
> On 24/09/18 11:52, Jorn Vernee wrote:
>> Hello,
>> 
>> I'd like to contribute a patch that adds support for the default 
>> library on windows.
>> 
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>> Diff: 
>> https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>> 
>> As mentioned before [1], this fixes 2 bugs:
>> 
>> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
>> threw an NPE (at least on windows). That is fixed by returning
>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>> 
>> 2.) The default library search was not working on windows. It was 
>> using
>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not 
>> on
>> windows (see https://stackoverflow.com/q/23437007). I have changed the
>> implementation from using a default handle to using a new native
>> function findEntryInProcess, which does the right thing for Windows
>> (iterate over all loaded modules), and does the same thing it used to
>> for unix. findEntry is now a Java method, and the original findEntry 
>> is
>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>> forwards to findEntry0, and the anonymous class of the default library
>> overrides to forward to findEntryInProcess.
>> 
>> Please test this patch on unix as well, since I don't have the ability 
>> to do so.
>> 
>> Jorn
>> 
>> [1] : 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html

From samuel.audet at gmail.com  Mon Sep 24 11:38:36 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Mon, 24 Sep 2018 20:38:36 +0900
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
Message-ID: <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>

FWIW, I think the factory method pattern should be reconsidered 
entirely. In C/C++, when we want to call say getpid(), we don't start 
loading stuff up before calling getpid(), we call getpid()! Why not do 
the same from Java? From a usability point of view, not loading stuff 
manually works fine for JavaCPP...

Now, I know you're going to start taking about interfaces and what not. 
You said that you had plans to introduce an entirely new array type just 
to make it more friendly with vector instructions and native libraries. 
Why not start thinking about an "interface" that would be friendly to 
native libraries as well? Why stop at arrays?

Samuel

On 09/24/2018 08:10 PM, Maurizio Cimadamore wrote:
> Hi Jorn,
> thanks for the patch. As mentioned last time we have two options here: 
> one is to follow the approach forward in your patch; another would be to 
> ditch Library.getDefault() entirely and adopt a more explicit approach - 
> that is to always require 'implicit' libraries to be mentioned - either 
> by jextract artifacts or by API points.
> 
> A note on the latter - when you do:
> 
> Libraries.bindRaw(lookup, Foo.class)
> 
> the code lookup the @NativeHeader annotation on Foo.class and tries to 
> extract required library names from there. Currently, we do not add 
> library names for standard libraries such as "c" or "m" (math), which is 
> what led us down the (slippery?) slope of having a Library.getDefault 
> somewhere.
> 
> Another option would be to have jextract to always generate the names of 
> the libraries an artifact depends on, and then the API should throw an 
> exception if a @NativeHeader is found with no libraries. More 
> specifically, jextract should always add "c" to the set of used 
> libraries in the @NativeHeader annotation (other libraries can be 
> explicitly supplied using the -l flag).
> 
> Note that I'm not saying "the binder should always add in 'clib'" for 
> you, because that's kind of a lame assumption: it works obviously well 
> for C but it doesn't make a lot of sense if you want to use Panama for 
> other purposes/languages. Which seems to suggest that, as far as the 
> binder is concerned, library dependencies should always be explicit.
> 
> Thoughts?
> 
> Maurizio
> 
> 
> 
> On 24/09/18 11:52, Jorn Vernee wrote:
>> Hello,
>>
>> I'd like to contribute a patch that adds support for the default 
>> library on windows.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>> Diff: https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>>
>> As mentioned before [1], this fixes 2 bugs:
>>
>> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
>> threw an NPE (at least on windows). That is fixed by returning
>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>>
>> 2.) The default library search was not working on windows. It was using
>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not on
>> windows (see https://stackoverflow.com/q/23437007). I have changed the
>> implementation from using a default handle to using a new native
>> function findEntryInProcess, which does the right thing for Windows
>> (iterate over all loaded modules), and does the same thing it used to
>> for unix. findEntry is now a Java method, and the original findEntry is
>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>> forwards to findEntry0, and the anonymous class of the default library
>> overrides to forward to findEntryInProcess.
>>
>> Please test this patch on unix as well, since I don't have the ability 
>> to do so.
>>
>> Jorn
>>
>> [1] : 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html 
>>
> 


From maurizio.cimadamore at oracle.com  Mon Sep 24 12:00:15 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 24 Sep 2018 13:00:15 +0100
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
Message-ID: <0760f00f-51dd-05f5-10b4-5a03d88f636e@oracle.com>

Can you please expand a bit on this?

But also, warning: this is a RFR thread and I don't like it being side 
tracked to re-discuss higher level goals such as modeling headers with 
interfaces. If you want to raise this concern (seems so, but not 100% 
sure), as otherwise stated, please raise that in the appropriate forum 
(panama-spec-experts). Let's keep this list for discussion on the 
prototype, feedback on real usage, bugs, etc.

Maurizio

On 24/09/18 12:38, Samuel Audet wrote:
> FWIW, I think the factory method pattern should be reconsidered 
> entirely. In C/C++, when we want to call say getpid(), we don't start 
> loading stuff up before calling getpid(), we call getpid()! Why not do 
> the same from Java? From a usability point of view, not loading stuff 
> manually works fine for JavaCPP...
>
> Now, I know you're going to start taking about interfaces and what 
> not. You said that you had plans to introduce an entirely new array 
> type just to make it more friendly with vector instructions and native 
> libraries. Why not start thinking about an "interface" that would be 
> friendly to native libraries as well? Why stop at arrays?
>
> Samuel
>
> On 09/24/2018 08:10 PM, Maurizio Cimadamore wrote:
>> Hi Jorn,
>> thanks for the patch. As mentioned last time we have two options 
>> here: one is to follow the approach forward in your patch; another 
>> would be to ditch Library.getDefault() entirely and adopt a more 
>> explicit approach - that is to always require 'implicit' libraries to 
>> be mentioned - either by jextract artifacts or by API points.
>>
>> A note on the latter - when you do:
>>
>> Libraries.bindRaw(lookup, Foo.class)
>>
>> the code lookup the @NativeHeader annotation on Foo.class and tries 
>> to extract required library names from there. Currently, we do not 
>> add library names for standard libraries such as "c" or "m" (math), 
>> which is what led us down the (slippery?) slope of having a 
>> Library.getDefault somewhere.
>>
>> Another option would be to have jextract to always generate the names 
>> of the libraries an artifact depends on, and then the API should 
>> throw an exception if a @NativeHeader is found with no libraries. 
>> More specifically, jextract should always add "c" to the set of used 
>> libraries in the @NativeHeader annotation (other libraries can be 
>> explicitly supplied using the -l flag).
>>
>> Note that I'm not saying "the binder should always add in 'clib'" for 
>> you, because that's kind of a lame assumption: it works obviously 
>> well for C but it doesn't make a lot of sense if you want to use 
>> Panama for other purposes/languages. Which seems to suggest that, as 
>> far as the binder is concerned, library dependencies should always be 
>> explicit.
>>
>> Thoughts?
>>
>> Maurizio
>>
>>
>>
>> On 24/09/18 11:52, Jorn Vernee wrote:
>>> Hello,
>>>
>>> I'd like to contribute a patch that adds support for the default 
>>> library on windows.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>>> Diff: 
>>> https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>>>
>>> As mentioned before [1], this fixes 2 bugs:
>>>
>>> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
>>> threw an NPE (at least on windows). That is fixed by returning
>>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>>>
>>> 2.) The default library search was not working on windows. It was using
>>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not on
>>> windows (see https://stackoverflow.com/q/23437007). I have changed the
>>> implementation from using a default handle to using a new native
>>> function findEntryInProcess, which does the right thing for Windows
>>> (iterate over all loaded modules), and does the same thing it used to
>>> for unix. findEntry is now a Java method, and the original findEntry is
>>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>>> forwards to findEntry0, and the anonymous class of the default library
>>> overrides to forward to findEntryInProcess.
>>>
>>> Please test this patch on unix as well, since I don't have the 
>>> ability to do so.
>>>
>>> Jorn
>>>
>>> [1] : 
>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html 
>>>
>>
>


From sundararajan.athijegannathan at oracle.com  Mon Sep 24 12:22:02 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Mon, 24 Sep 2018 17:52:02 +0530
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <e9daf33dc915c4916ee9156ad87a2d12@xs4all.nl>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <e9daf33dc915c4916ee9156ad87a2d12@xs4all.nl>
Message-ID: <5BA8D6EA.1000702@oracle.com>

Also it seems EnumProcessModules is meant for debuggers [1]. I agree 
that we should probably not have the notion of the default library.

[1] 
https://docs.microsoft.com/en-us/windows/desktop/api/psapi/nf-psapi-enumprocessmodules

-Sundar

On 24/09/18, 5:02 PM, Jorn Vernee wrote:
> I also think the approach of removing the default library is more 
> robust, since a symbol can be loaded from several libraries at once, 
> but when using the default library you don't know which one you're 
> gonna get. I can imagine a big project using several decencies which 
> try to load the same symbol from different libraries through the 
> default library and things breaking when the wrong function address is 
> loaded.
>
> On the other hand, library names for the standard library are platform 
> specific. e.g. the standard C library dll is named msvcrt on windows, 
> so writing platform agnostic code that just depends on the portable 
> standard library, like the tests we have, would be more difficult.
>
> Maybe that problem could be solved by shipping pre-computed library 
> interfaces for the standard library with the JDK, where the value of 
> NativeHeader#libraries would be platform specific. That would also 
> decrease the time-to-helloworld for foreign, and I think that at least 
> the standard C library is small enough that that won't be problematic.
>
> Jorn
>
> Maurizio Cimadamore schreef op 2018-09-24 13:10:
>> Hi Jorn,
>> thanks for the patch. As mentioned last time we have two options here:
>> one is to follow the approach forward in your patch; another would be
>> to ditch Library.getDefault() entirely and adopt a more explicit
>> approach - that is to always require 'implicit' libraries to be
>> mentioned - either by jextract artifacts or by API points.
>>
>> A note on the latter - when you do:
>>
>> Libraries.bindRaw(lookup, Foo.class)
>>
>> the code lookup the @NativeHeader annotation on Foo.class and tries to
>> extract required library names from there. Currently, we do not add
>> library names for standard libraries such as "c" or "m" (math), which
>> is what led us down the (slippery?) slope of having a
>> Library.getDefault somewhere.
>>
>> Another option would be to have jextract to always generate the names
>> of the libraries an artifact depends on, and then the API should throw
>> an exception if a @NativeHeader is found with no libraries. More
>> specifically, jextract should always add "c" to the set of used
>> libraries in the @NativeHeader annotation (other libraries can be
>> explicitly supplied using the -l flag).
>>
>> Note that I'm not saying "the binder should always add in 'clib'" for
>> you, because that's kind of a lame assumption: it works obviously well
>> for C but it doesn't make a lot of sense if you want to use Panama for
>> other purposes/languages. Which seems to suggest that, as far as the
>> binder is concerned, library dependencies should always be explicit.
>>
>> Thoughts?
>>
>> Maurizio
>>
>>
>>
>> On 24/09/18 11:52, Jorn Vernee wrote:
>>> Hello,
>>>
>>> I'd like to contribute a patch that adds support for the default 
>>> library on windows.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>>> Diff: 
>>> https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>>>
>>> As mentioned before [1], this fixes 2 bugs:
>>>
>>> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
>>> threw an NPE (at least on windows). That is fixed by returning
>>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>>>
>>> 2.) The default library search was not working on windows. It was using
>>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not on
>>> windows (see https://stackoverflow.com/q/23437007). I have changed the
>>> implementation from using a default handle to using a new native
>>> function findEntryInProcess, which does the right thing for Windows
>>> (iterate over all loaded modules), and does the same thing it used to
>>> for unix. findEntry is now a Java method, and the original findEntry is
>>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>>> forwards to findEntry0, and the anonymous class of the default library
>>> overrides to forward to findEntryInProcess.
>>>
>>> Please test this patch on unix as well, since I don't have the 
>>> ability to do so.
>>>
>>> Jorn
>>>
>>> [1] : 
>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html

From jbvernee at xs4all.nl  Mon Sep 24 12:34:35 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 24 Sep 2018 14:34:35 +0200
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
Message-ID: <c1e938f3967fb53b0831fea30a4e710c@xs4all.nl>

I agree with the usability point. In C++ it's as simple to call puts as 
doing:

     #include <stdio.h>

     int main() {
         puts("Hello World!");
     }

And I think the optimal Java equivalent would be something like:

     import static org.openjdk.stdio.*;

     public class Main {

         public static void main(String[] args) {
             puts("Hello World!");
         }

     }

This can be facilitated by creating a 'singleton facade' for the library 
interface like so:

     public class stdio {

         private static final stdioImpl lib = Libraries.bind(lookup(), 
stdioImpl.class);

         public static int puts (String message) {
             try(Scope scope = Scope.newNativeScope()) {
                 Pointer<Byte> msg = scope.toCString(message);
                 return lib.puts(msg);
             }
         }

         ...
     }

Such a facade class could be shipped with the JDK or perhaps as an 
artifact on maven central, or maybe it could be an additional output of 
jextract.

But there is only so much you can do automagically from the Java side. 
When working from C/C++ you have the compiler filling in the blanks. For 
instance, it automatically allocates storage for string literals. Java 
does that as well for Java string literals `String s = "Hello";`, but it 
can not do that for native strings, and you have to use the Scope API to 
do that manually. In some cases, like the above, you can write glue-code 
to make that automatic, but I think at some point things become too 
complex for that, and there will always be some usability barrier to 
interop.

Jorn

Samuel Audet schreef op 2018-09-24 13:38:
> FWIW, I think the factory method pattern should be reconsidered
> entirely. In C/C++, when we want to call say getpid(), we don't start
> loading stuff up before calling getpid(), we call getpid()! Why not do
> the same from Java? From a usability point of view, not loading stuff
> manually works fine for JavaCPP...
> 
> Now, I know you're going to start taking about interfaces and what
> not. You said that you had plans to introduce an entirely new array
> type just to make it more friendly with vector instructions and native
> libraries. Why not start thinking about an "interface" that would be
> friendly to native libraries as well? Why stop at arrays?
> 
> Samuel
> 
> On 09/24/2018 08:10 PM, Maurizio Cimadamore wrote:
>> Hi Jorn,
>> thanks for the patch. As mentioned last time we have two options here: 
>> one is to follow the approach forward in your patch; another would be 
>> to ditch Library.getDefault() entirely and adopt a more explicit 
>> approach - that is to always require 'implicit' libraries to be 
>> mentioned - either by jextract artifacts or by API points.
>> 
>> A note on the latter - when you do:
>> 
>> Libraries.bindRaw(lookup, Foo.class)
>> 
>> the code lookup the @NativeHeader annotation on Foo.class and tries to 
>> extract required library names from there. Currently, we do not add 
>> library names for standard libraries such as "c" or "m" (math), which 
>> is what led us down the (slippery?) slope of having a 
>> Library.getDefault somewhere.
>> 
>> Another option would be to have jextract to always generate the names 
>> of the libraries an artifact depends on, and then the API should throw 
>> an exception if a @NativeHeader is found with no libraries. More 
>> specifically, jextract should always add "c" to the set of used 
>> libraries in the @NativeHeader annotation (other libraries can be 
>> explicitly supplied using the -l flag).
>> 
>> Note that I'm not saying "the binder should always add in 'clib'" for 
>> you, because that's kind of a lame assumption: it works obviously well 
>> for C but it doesn't make a lot of sense if you want to use Panama for 
>> other purposes/languages. Which seems to suggest that, as far as the 
>> binder is concerned, library dependencies should always be explicit.
>> 
>> Thoughts?
>> 
>> Maurizio
>> 
>> 
>> 
>> On 24/09/18 11:52, Jorn Vernee wrote:
>>> Hello,
>>> 
>>> I'd like to contribute a patch that adds support for the default 
>>> library on windows.
>>> 
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>>> Diff: 
>>> https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>>> 
>>> As mentioned before [1], this fixes 2 bugs:
>>> 
>>> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
>>> threw an NPE (at least on windows). That is fixed by returning
>>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>>> 
>>> 2.) The default library search was not working on windows. It was 
>>> using
>>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not 
>>> on
>>> windows (see https://stackoverflow.com/q/23437007). I have changed 
>>> the
>>> implementation from using a default handle to using a new native
>>> function findEntryInProcess, which does the right thing for Windows
>>> (iterate over all loaded modules), and does the same thing it used to
>>> for unix. findEntry is now a Java method, and the original findEntry 
>>> is
>>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>>> forwards to findEntry0, and the anonymous class of the default 
>>> library
>>> overrides to forward to findEntryInProcess.
>>> 
>>> Please test this patch on unix as well, since I don't have the 
>>> ability to do so.
>>> 
>>> Jorn
>>> 
>>> [1] : 
>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html
>> 

From jbvernee at xs4all.nl  Mon Sep 24 12:55:19 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 24 Sep 2018 14:55:19 +0200
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <5BA8D6EA.1000702@oracle.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <e9daf33dc915c4916ee9156ad87a2d12@xs4all.nl> <5BA8D6EA.1000702@oracle.com>
Message-ID: <e847ea5946487caf263f0b116b3a48d9@xs4all.nl>

Ok, then I think we agree on dropping the default library, and the 
question becomes how to handle the current tests now having to load 
different libraries on different platforms.

For the most part the Java side is completely the same, only the value 
being passed to `NativeHeader#libraries` would have to change depending 
on the platform.

Any suggestions about how to go about that? My only idea of just 
copy-pasting every test, adding a @requires tag and then changing just 
that string seems like overkill.

Jorn

Sundararajan Athijegannathan schreef op 2018-09-24 14:22:
> Also it seems EnumProcessModules is meant for debuggers [1]. I agree
> that we should probably not have the notion of the default library.
> 
> [1]
> https://docs.microsoft.com/en-us/windows/desktop/api/psapi/nf-psapi-enumprocessmodules
> 
> -Sundar
> 
> On 24/09/18, 5:02 PM, Jorn Vernee wrote:
>> I also think the approach of removing the default library is more 
>> robust, since a symbol can be loaded from several libraries at once, 
>> but when using the default library you don't know which one you're 
>> gonna get. I can imagine a big project using several decencies which 
>> try to load the same symbol from different libraries through the 
>> default library and things breaking when the wrong function address is 
>> loaded.
>> 
>> On the other hand, library names for the standard library are platform 
>> specific. e.g. the standard C library dll is named msvcrt on windows, 
>> so writing platform agnostic code that just depends on the portable 
>> standard library, like the tests we have, would be more difficult.
>> 
>> Maybe that problem could be solved by shipping pre-computed library 
>> interfaces for the standard library with the JDK, where the value of 
>> NativeHeader#libraries would be platform specific. That would also 
>> decrease the time-to-helloworld for foreign, and I think that at least 
>> the standard C library is small enough that that won't be problematic.
>> 
>> Jorn
>> 
>> Maurizio Cimadamore schreef op 2018-09-24 13:10:
>>> Hi Jorn,
>>> thanks for the patch. As mentioned last time we have two options 
>>> here:
>>> one is to follow the approach forward in your patch; another would be
>>> to ditch Library.getDefault() entirely and adopt a more explicit
>>> approach - that is to always require 'implicit' libraries to be
>>> mentioned - either by jextract artifacts or by API points.
>>> 
>>> A note on the latter - when you do:
>>> 
>>> Libraries.bindRaw(lookup, Foo.class)
>>> 
>>> the code lookup the @NativeHeader annotation on Foo.class and tries 
>>> to
>>> extract required library names from there. Currently, we do not add
>>> library names for standard libraries such as "c" or "m" (math), which
>>> is what led us down the (slippery?) slope of having a
>>> Library.getDefault somewhere.
>>> 
>>> Another option would be to have jextract to always generate the names
>>> of the libraries an artifact depends on, and then the API should 
>>> throw
>>> an exception if a @NativeHeader is found with no libraries. More
>>> specifically, jextract should always add "c" to the set of used
>>> libraries in the @NativeHeader annotation (other libraries can be
>>> explicitly supplied using the -l flag).
>>> 
>>> Note that I'm not saying "the binder should always add in 'clib'" for
>>> you, because that's kind of a lame assumption: it works obviously 
>>> well
>>> for C but it doesn't make a lot of sense if you want to use Panama 
>>> for
>>> other purposes/languages. Which seems to suggest that, as far as the
>>> binder is concerned, library dependencies should always be explicit.
>>> 
>>> Thoughts?
>>> 
>>> Maurizio
>>> 
>>> 
>>> 
>>> On 24/09/18 11:52, Jorn Vernee wrote:
>>>> Hello,
>>>> 
>>>> I'd like to contribute a patch that adds support for the default 
>>>> library on windows.
>>>> 
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>>>> Diff: 
>>>> https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>>>> 
>>>> As mentioned before [1], this fixes 2 bugs:
>>>> 
>>>> 1.) When no library was loaded 
>>>> ClassLoader#NativeLibrary#getFromClass
>>>> threw an NPE (at least on windows). That is fixed by returning
>>>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>>>> 
>>>> 2.) The default library search was not working on windows. It was 
>>>> using
>>>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not 
>>>> on
>>>> windows (see https://stackoverflow.com/q/23437007). I have changed 
>>>> the
>>>> implementation from using a default handle to using a new native
>>>> function findEntryInProcess, which does the right thing for Windows
>>>> (iterate over all loaded modules), and does the same thing it used 
>>>> to
>>>> for unix. findEntry is now a Java method, and the original findEntry 
>>>> is
>>>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>>>> forwards to findEntry0, and the anonymous class of the default 
>>>> library
>>>> overrides to forward to findEntryInProcess.
>>>> 
>>>> Please test this patch on unix as well, since I don't have the 
>>>> ability to do so.
>>>> 
>>>> Jorn
>>>> 
>>>> [1] : 
>>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html

From jbvernee at xs4all.nl  Mon Sep 24 13:03:10 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 24 Sep 2018 15:03:10 +0200
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <e847ea5946487caf263f0b116b3a48d9@xs4all.nl>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <e9daf33dc915c4916ee9156ad87a2d12@xs4all.nl> <5BA8D6EA.1000702@oracle.com>
 <e847ea5946487caf263f0b116b3a48d9@xs4all.nl>
Message-ID: <6887055f9e47f5c1f840bfc292efbf15@xs4all.nl>

Jorn Vernee schreef op 2018-09-24 14:55:
> Ok, then I think we agree on dropping the default library, and the
> question becomes how to handle the current tests now having to load
> different libraries on different platforms.
> 
> For the most part the Java side is completely the same, only the value
> being passed to `NativeHeader#libraries` would have to change
> depending on the platform.
> 
> Any suggestions about how to go about that? My only idea of just
> copy-pasting every test, adding a @requires tag and then changing just
> that string seems like overkill.
> 
> Jorn

Thinking more about that; if I can get jextract to build and run on 
windows, and it is changed to also supply a value for the `libraries` 
field (as suggested), a shell script could be ran to generate the 
interfaces on demand when the tests run. Then the tests would also 
double as tests for jextract.

Jorn

From maurizio.cimadamore at oracle.com  Mon Sep 24 13:37:58 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 24 Sep 2018 14:37:58 +0100
Subject: on usability (was Re: [foreign] RFR 8211060: Library.getDefault()
 does not work on Windows)
In-Reply-To: <c1e938f3967fb53b0831fea30a4e710c@xs4all.nl>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
 <c1e938f3967fb53b0831fea30a4e710c@xs4all.nl>
Message-ID: <1dde768f-2067-de80-2655-a881ae78c771@oracle.com>

Having a pre-extracted stdlib bundle is something we have considered - 
quoting from [1]:

"Now, since most (all?) of the libraries out there are going to assume
the availability of some 'standard library', let's also assume that some
extracted artifact for such library is available and that jextract
always knows how to find it - this is the equivalent of java.base for
the module system, or java.lang for the Java import system. This
addresses the bootstrapping issue."

In time we'll get there, I don't see any real technical obstacles to get 
to your 'optimal' snippet.

I think there are two aspects that I'd like to draw attention upon:

1) Magic does not come for free. E.g. it might "seem" that JNI has a 
more direct approach to calling native methods (ease of use issues 
aside). In reality it's just that the complexity of calling that native 
method, marshalling arguments, unmarshalling returns, dealing with 
native thread transitions and what's not has just been pushed under the 
JVM rug. So, yes, you can "just" call getpid - but the burden is on the 
VM. Now, the JNI support is already quite complex - I can't honestly 
imagine a sane way for the VM to support any given invocation scheme 
that a user might wish to see supported. This is why Panama is betting 
on Java code + layouts to do the lifting: that way the VM interface can 
be kept simple (which has significant payoffs - as the resulting code 
can be optimized much more - see the linkToNative experimental results 
in [2]).

2) As your example points out, while calling 'getpid' is something that 
seems 'easy' enough - 'puts' is already some other beast. It takes a 
pointer to some memory location where the string is stored. The JNI 
approach is to pass the Java string as is, and then do the wiring in 
native code. That is, there's no free lunch here - either you do the 
adaptation in Java, or you do it in native code (**). Panama gives you a 
rich enough API to do all such adaptations in Java, so that all native 
calls are... just native calls (again this means more regularity which 
means more performances). Having opaque native code snippets which do 
argument adaptation is not very optimal (and optimizable) for the JVM. 
With Panama you can create a good-looking API which internally uses 
pointers/scopes and delegates to the right native method - all done in 
Java. On top of that, our plans cover a so called 'civilization' layer 
(see [3]), by which users will be able to customize what comes out of 
jextract in order e.g. to tell that for 'puts' they really want a Java 
String argument and not a Pointer<Byte>; again this will be done in a 
more general way, so that the binder will be pointed at a pair of 
functions which can be used to map the user provided data to and from 
native code.

(**) for an example of how interfacing with standard libraries needs 
some kind of wrapping, even in JNI - look at [4]; this file is 
essentially a collection of system calls which are wrapped by some logic 
(e.g. to check errno, ...). I claim that there is something 
_fundamentally_ wrong with code like this, in that the native code is 
mixing two concerns: (i) performing the required native call and (ii) 
adjusting input/output/errors of the call in a way that is suitable to 
the corresponding Java API. Why shouldn't the Java API itself be in 
charge of doing (ii) ?

Maurizio

[1] - 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-August/002560.html
[2] - 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
[3] - 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-April/001537.html
[4] - 
http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c#l314




On 24/09/18 13:34, Jorn Vernee wrote:
> I agree with the usability point. In C++ it's as simple to call puts 
> as doing:
>
> ??? #include <stdio.h>
>
> ??? int main() {
> ??????? puts("Hello World!");
> ??? }
>
> And I think the optimal Java equivalent would be something like:
>
> ??? import static org.openjdk.stdio.*;
>
> ??? public class Main {
>
> ??????? public static void main(String[] args) {
> ??????????? puts("Hello World!");
> ??????? }
>
> ??? }
>
> This can be facilitated by creating a 'singleton facade' for the 
> library interface like so:
>
> ??? public class stdio {
>
> ??????? private static final stdioImpl lib = Libraries.bind(lookup(), 
> stdioImpl.class);
>
> ??????? public static int puts (String message) {
> ??????????? try(Scope scope = Scope.newNativeScope()) {
> ??????????????? Pointer<Byte> msg = scope.toCString(message);
> ??????????????? return lib.puts(msg);
> ??????????? }
> ??????? }
>
> ??????? ...
> ??? }
>
> Such a facade class could be shipped with the JDK or perhaps as an 
> artifact on maven central, or maybe it could be an additional output 
> of jextract.
>
> But there is only so much you can do automagically from the Java side. 
> When working from C/C++ you have the compiler filling in the blanks. 
> For instance, it automatically allocates storage for string literals. 
> Java does that as well for Java string literals `String s = "Hello";`, 
> but it can not do that for native strings, and you have to use the 
> Scope API to do that manually. In some cases, like the above, you can 
> write glue-code to make that automatic, but I think at some point 
> things become too complex for that, and there will always be some 
> usability barrier to interop.
>
> Jorn
>
> Samuel Audet schreef op 2018-09-24 13:38:
>> FWIW, I think the factory method pattern should be reconsidered
>> entirely. In C/C++, when we want to call say getpid(), we don't start
>> loading stuff up before calling getpid(), we call getpid()! Why not do
>> the same from Java? From a usability point of view, not loading stuff
>> manually works fine for JavaCPP...
>>
>> Now, I know you're going to start taking about interfaces and what
>> not. You said that you had plans to introduce an entirely new array
>> type just to make it more friendly with vector instructions and native
>> libraries. Why not start thinking about an "interface" that would be
>> friendly to native libraries as well? Why stop at arrays?
>>
>> Samuel
>>
>> On 09/24/2018 08:10 PM, Maurizio Cimadamore wrote:
>>> Hi Jorn,
>>> thanks for the patch. As mentioned last time we have two options 
>>> here: one is to follow the approach forward in your patch; another 
>>> would be to ditch Library.getDefault() entirely and adopt a more 
>>> explicit approach - that is to always require 'implicit' libraries 
>>> to be mentioned - either by jextract artifacts or by API points.
>>>
>>> A note on the latter - when you do:
>>>
>>> Libraries.bindRaw(lookup, Foo.class)
>>>
>>> the code lookup the @NativeHeader annotation on Foo.class and tries 
>>> to extract required library names from there. Currently, we do not 
>>> add library names for standard libraries such as "c" or "m" (math), 
>>> which is what led us down the (slippery?) slope of having a 
>>> Library.getDefault somewhere.
>>>
>>> Another option would be to have jextract to always generate the 
>>> names of the libraries an artifact depends on, and then the API 
>>> should throw an exception if a @NativeHeader is found with no 
>>> libraries. More specifically, jextract should always add "c" to the 
>>> set of used libraries in the @NativeHeader annotation (other 
>>> libraries can be explicitly supplied using the -l flag).
>>>
>>> Note that I'm not saying "the binder should always add in 'clib'" 
>>> for you, because that's kind of a lame assumption: it works 
>>> obviously well for C but it doesn't make a lot of sense if you want 
>>> to use Panama for other purposes/languages. Which seems to suggest 
>>> that, as far as the binder is concerned, library dependencies should 
>>> always be explicit.
>>>
>>> Thoughts?
>>>
>>> Maurizio
>>>
>>>
>>>
>>> On 24/09/18 11:52, Jorn Vernee wrote:
>>>> Hello,
>>>>
>>>> I'd like to contribute a patch that adds support for the default 
>>>> library on windows.
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>>>> Diff: 
>>>> https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>>>>
>>>> As mentioned before [1], this fixes 2 bugs:
>>>>
>>>> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
>>>> threw an NPE (at least on windows). That is fixed by returning
>>>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>>>>
>>>> 2.) The default library search was not working on windows. It was 
>>>> using
>>>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but 
>>>> not on
>>>> windows (see https://stackoverflow.com/q/23437007). I have changed the
>>>> implementation from using a default handle to using a new native
>>>> function findEntryInProcess, which does the right thing for Windows
>>>> (iterate over all loaded modules), and does the same thing it used to
>>>> for unix. findEntry is now a Java method, and the original 
>>>> findEntry is
>>>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>>>> forwards to findEntry0, and the anonymous class of the default library
>>>> overrides to forward to findEntryInProcess.
>>>>
>>>> Please test this patch on unix as well, since I don't have the 
>>>> ability to do so.
>>>>
>>>> Jorn
>>>>
>>>> [1] : 
>>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html
>>>


From sundararajan.athijegannathan at oracle.com  Mon Sep 24 13:50:20 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Mon, 24 Sep 2018 19:20:20 +0530
Subject: [foreign] RFR 8211006: Add API points to support by-value
 semantics for structs and arrays
In-Reply-To: <45aafb81-8fb9-2b89-c194-4cea63274837@oracle.com>
References: <45aafb81-8fb9-2b89-c194-4cea63274837@oracle.com>
Message-ID: <5BA8EB9C.8060009@oracle.com>

Looks good

-Sundar

On 21/09/18, 8:47 PM, Maurizio Cimadamore wrote:
> As written in the issue summary (see for more details):
>
> https://bugs.openjdk.java.net/browse/JDK-8211006
>
> We need API points to perform deep copy of structs/arrays, which would 
> allow developers to easily mimic what happens when operating on 
> struct/array struct fields.
>
> I also did a minor tweak on References.ofArray, as the setter was 
> redundantly creating an Array object instead of just copying.
>
> Webrev:
>
> http://cr.openjdk.java.net/~mcimadamore/panama/8211006/
>
> Cheers
> Maurizio
>
>

From maurizio.cimadamore at oracle.com  Mon Sep 24 14:35:24 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Mon, 24 Sep 2018 14:35:24 +0000
Subject: hg: panama/dev: 8211006: Add API points to support by-value semantics
 for structs and arrays
Message-ID: <201809241435.w8OEZO7E019473@aojmv0008.oracle.com>

Changeset: 7313653f7bc5
Author:    mcimadamore
Date:      2018-09-24 15:34 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/7313653f7bc5

8211006: Add API points to support by-value semantics for structs and arrays
Reviewed-by: sundar

! src/java.base/share/classes/java/foreign/memory/Array.java
! src/java.base/share/classes/java/foreign/memory/Struct.java
! src/java.base/share/classes/jdk/internal/foreign/memory/References.java
+ test/jdk/java/foreign/types/DeepAssignTest.java


From maurizio.cimadamore at oracle.com  Mon Sep 24 14:36:16 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 24 Sep 2018 15:36:16 +0100
Subject: [foreign] RFR 8211006: Add API points to support by-value
 semantics for structs and arrays
In-Reply-To: <5BA8EB9C.8060009@oracle.com>
References: <45aafb81-8fb9-2b89-c194-4cea63274837@oracle.com>
 <5BA8EB9C.8060009@oracle.com>
Message-ID: <789abe37-4063-3eda-edee-9c0d36c04161@oracle.com>

Thanks - pushed (as discussed privately, I also updated the signatures 
of the two new methods to be polymorphic in the array/struct type, to 
enhance static safety).

Maurizio


On 24/09/18 14:50, Sundararajan Athijegannathan wrote:
> Looks good
>
> -Sundar
>
> On 21/09/18, 8:47 PM, Maurizio Cimadamore wrote:
>> As written in the issue summary (see for more details):
>>
>> https://bugs.openjdk.java.net/browse/JDK-8211006
>>
>> We need API points to perform deep copy of structs/arrays, which 
>> would allow developers to easily mimic what happens when operating on 
>> struct/array struct fields.
>>
>> I also did a minor tweak on References.ofArray, as the setter was 
>> redundantly creating an Array object instead of just copying.
>>
>> Webrev:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/8211006/
>>
>> Cheers
>> Maurizio
>>
>>


From maurizio.cimadamore at oracle.com  Mon Sep 24 14:36:46 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Mon, 24 Sep 2018 14:36:46 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809241436.w8OEalSn020295@aojmv0008.oracle.com>

Changeset: 93be18efd5b9
Author:    mcimadamore
Date:      2018-09-24 16:39 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/93be18efd5b9

Automatic merge with foreign



From jbvernee at xs4all.nl  Mon Sep 24 15:58:07 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Mon, 24 Sep 2018 17:58:07 +0200
Subject: [foreign] RFR : Fixes to StdLibTest
Message-ID: <e1c2b91fef4fbfc3a4426398ab9c89ef@xs4all.nl>

Hello,

Please review this patch which does a few fixes to StdLibTest:

Diff: 
https://gist.github.com/JornVernee/312ecb03d93882efac1f86ccc8ad1f25

This patch fixes 3 things:

1.) 'puts' returns a non-negative value on success [1], but the test was 
testing > 0. I changed it to >= 0, since the MSVC implementation returns 
exactly 0.

2.) Switched the test_time test from testing localtime(), which is 
locale dependent to testing gmtime() which always uses the GMT timezone, 
since the locale dependency was giving trouble in cygwin [2].

3.) Also for the test_time test, I changed the way the test parameters 
are being generated, since it was apparently generating invalid values 
for windows, so the native function was returning null, and the test was 
failing.

Since I'm not a committer someone else will have to create a commit and 
push this.

Thanks,
Jorn

[1] : https://en.cppreference.com/w/c/io/puts
[2] : 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002753.html

From sundararajan.athijegannathan at oracle.com  Mon Sep 24 16:43:23 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Mon, 24 Sep 2018 22:13:23 +0530
Subject: [foreign] RFR : Fixes to StdLibTest
In-Reply-To: <e1c2b91fef4fbfc3a4426398ab9c89ef@xs4all.nl>
References: <e1c2b91fef4fbfc3a4426398ab9c89ef@xs4all.nl>
Message-ID: <5BA9142B.4020306@oracle.com>

Thanks for this patch Jorn!

I filed a bug to track this:  
https://bugs.openjdk.java.net/browse/JDK-8211063

PS. I'm testing your patch on Mac.

Thanks,
-Sundar

On 24/09/18, 9:28 PM, Jorn Vernee wrote:
> Hello,
>
> Please review this patch which does a few fixes to StdLibTest:
>
> Diff: https://gist.github.com/JornVernee/312ecb03d93882efac1f86ccc8ad1f25
>
> This patch fixes 3 things:
>
> 1.) 'puts' returns a non-negative value on success [1], but the test 
> was testing > 0. I changed it to >= 0, since the MSVC implementation 
> returns exactly 0.
>
> 2.) Switched the test_time test from testing localtime(), which is 
> locale dependent to testing gmtime() which always uses the GMT 
> timezone, since the locale dependency was giving trouble in cygwin [2].
>
> 3.) Also for the test_time test, I changed the way the test parameters 
> are being generated, since it was apparently generating invalid values 
> for windows, so the native function was returning null, and the test 
> was failing.
>
> Since I'm not a committer someone else will have to create a commit 
> and push this.
>
> Thanks,
> Jorn
>
> [1] : https://en.cppreference.com/w/c/io/puts
> [2] : 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002753.html

From maurizio.cimadamore at oracle.com  Mon Sep 24 16:59:30 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 24 Sep 2018 17:59:30 +0100
Subject: [foreign] RFR : Fixes to StdLibTest
In-Reply-To: <5BA9142B.4020306@oracle.com>
References: <e1c2b91fef4fbfc3a4426398ab9c89ef@xs4all.nl>
 <5BA9142B.4020306@oracle.com>
Message-ID: <f8e8b6a1-4f52-bb86-e001-e61b6b5abfec@oracle.com>



On 24/09/18 17:43, Sundararajan Athijegannathan wrote:
> Thanks for this patch Jorn!
>
> I filed a bug to track this: 
> https://bugs.openjdk.java.net/browse/JDK-8211063
>
> PS. I'm testing your patch on Mac.
Works on Linux

Maurizio
>
> Thanks,
> -Sundar
>
> On 24/09/18, 9:28 PM, Jorn Vernee wrote:
>> Hello,
>>
>> Please review this patch which does a few fixes to StdLibTest:
>>
>> Diff: 
>> https://gist.github.com/JornVernee/312ecb03d93882efac1f86ccc8ad1f25
>>
>> This patch fixes 3 things:
>>
>> 1.) 'puts' returns a non-negative value on success [1], but the test 
>> was testing > 0. I changed it to >= 0, since the MSVC implementation 
>> returns exactly 0.
>>
>> 2.) Switched the test_time test from testing localtime(), which is 
>> locale dependent to testing gmtime() which always uses the GMT 
>> timezone, since the locale dependency was giving trouble in cygwin [2].
>>
>> 3.) Also for the test_time test, I changed the way the test 
>> parameters are being generated, since it was apparently generating 
>> invalid values for windows, so the native function was returning 
>> null, and the test was failing.
>>
>> Since I'm not a committer someone else will have to create a commit 
>> and push this.
>>
>> Thanks,
>> Jorn
>>
>> [1] : https://en.cppreference.com/w/c/io/puts
>> [2] : 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002753.html


From jonathan.gibbons at oracle.com  Fri Sep 21 21:29:28 2018
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Fri, 21 Sep 2018 14:29:28 -0700
Subject: [foreign] test_time timezone trouble (TZ)
In-Reply-To: <9FDB2BFC-2A7C-4485-9EAE-401669F86BF0@oracle.com>
References: <548afe90ca8727551817bd4a3b215345@xs4all.nl>
 <9FDB2BFC-2A7C-4485-9EAE-401669F86BF0@oracle.com>
Message-ID: <5BA562B8.50807@oracle.com>

Jorn,

There is only limited amount of support for setting env variables in 
jtreg tests, because at least in the
early releases of JDK, the use of environment variables was discouraged.

Using `@run shell unsettz.sh` will not have the desired effect because 
it will only affect the shell
that is created to run your shell script, and will not have a persistent 
environment on any other
process.

If you don't mind always setting a command-line option, you might try 
using `-eTZ=` on the jtreg
command-line, which will unset TZ for all tests, although it would be 
more accurate to say that
it will not set it for any tests.

The only way to change the setting of TZ for any specific test will be 
to do one of the
following:

1. Write the test as a shell script that unsets TZ and then runs any 
Java code, using various
     shell environment variables that will be available; these generally 
begin with "TEST".

2. Write the test as a Java program that execs a child process to run 
Java, with the exact set
     of environment variables you want to pass in. There are system 
properties that identify
     the test JDK, test class path and so on; these generally begin with 
"test.".

These days, writing shell tests is generally discouraged, because it is 
notoriously difficult to get it
right on all relevant platforms, and because these days, it is 
reasonably easy to use either
the Java SE Process[Builder] API or test library API to invoke java.

Hope that helps; if you have follow-up questions, please cc: me directly 
or use one of the jtreg-* aliases,
since I am not on the panama-dev list.

-- Jon

On 09/21/2018 02:12 PM, Michel Trudeau wrote:
> [adding jtreg mailing list to seek out answer about jtreg and TZ on Windows]
>
> On Sep 21, 2018, at 12:44 PM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>
> Hello guys,
>
> I was running into a problem with the test_time test in StdLibTest. The generator for test values was apparently generating invalid values. After fixing that, all the test iterations were failing because the hours were off by one (the output of LocalDateTime.getHours() is being compared to the output of localtime().hour()). Good thing somebody else on the internet seems to have had the same problem [1]. TL;DR when using the MSVC version of localtime, and when running in cygwin, the function tries to interpret the TZ environment variable, but since that has a unix format (courtesy of cygwin), the interpretation fails and defaults to GMT. When TZ is not set, it defaults to the system default timezone, which is also what's being tested against.
>
> I can get the tests to pass by using `unset TZ` in the cygwin terminal before running them, but I'd have to do that every time I reboot it. I was trying to unset TZ automatically by using jtreq `@run shell unsettz.sh` where unsettz.sh is a file containing just the command `unset TZ`. It seems to be running successfully according to the test output:
>
> ACTION: shell -- Passed. Execution successful
> REASON: User specified action: run shell unsettz.sh
> TIME:   0.126 seconds
> messages:
> command: shell unsettz.sh
> reason: User specified action: run shell unsettz.sh
> elapsed time (seconds): 0.126
> STDOUT:
> STDERR:
>
> But it doesn't seem to affect the tests itself, and they still fail (still off by 1 hour).
>
> I was wondering if there is a way to let jtreg control environment variables? Or maybe you can suggest a different solution?
>
> The jtreg guide [2] mentions that TZ will be propagated from Windows 'if set', but I need it to be not set, or automatically set to the system's default time zone (by default it's blank).
>
> (other than that, tests are looking good: `passed: 24; failed: 4; error: 1`. I just need to fix structs by value, which on Windows cheats and just passes a pointer. 2 failing tests are from jextract missing)
>
> Thanks,
> Jorn
>
> [1] : https://stackoverflow.com/q/11655003
> [2] : http://openjdk.java.net/jtreg/tag-spec.html
>


From shravya.rukmannagari at intel.com  Mon Sep 24 22:04:45 2018
From: shravya.rukmannagari at intel.com (shravya.rukmannagari at intel.com)
Date: Mon, 24 Sep 2018 22:04:45 +0000
Subject: hg: panama/dev: Linux SVML Support for VectorAPI
Message-ID: <201809242204.w8OM4jN7002844@aojmv0008.oracle.com>

Changeset: 8ac77d1b0891
Author:    srukmannagar
Date:      2018-09-24 15:03 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/8ac77d1b0891

Linux SVML Support for VectorAPI

+ src/hotspot/os_cpu/linux_x86/globals_vectorApiSupport_linux.hpp
! src/hotspot/os_cpu/linux_x86/svml_d_acos_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_asin_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_atan2_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_atan_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_cbrt_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_cos_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_cosh_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_exp_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_expm1_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_hypot_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_log10_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_log1p_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_log_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_pow_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_sin_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_sinh_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_tan_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_d_tanh_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_acos_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_asin_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_atan2_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_atan_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_cbrt_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_cos_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_cosh_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_exp_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_expm1_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_hypot_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_log10_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_log1p_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_log_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_pow_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_sin_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_sinh_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_tan_linux_x86.s
! src/hotspot/os_cpu/linux_x86/svml_s_tanh_linux_x86.s
! src/hotspot/share/utilities/globalDefinitions_vecApi.hpp


From maurizio.cimadamore at oracle.com  Mon Sep 24 22:06:47 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Mon, 24 Sep 2018 22:06:47 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809242206.w8OM6m0B003974@aojmv0008.oracle.com>

Changeset: 1761ca646d70
Author:    mcimadamore
Date:      2018-09-25 00:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/1761ca646d70

Automatic merge with vectorIntrinsics



From maurizio.cimadamore at oracle.com  Mon Sep 24 23:38:22 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 25 Sep 2018 00:38:22 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
Message-ID: <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>



On 20/09/18 10:29, Magnus Ihse Bursie wrote:
> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>> Try this:
>>
>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>> @@ -60,6 +60,9 @@
>> ?????? VER=`ls $with_libclang/lib/clang/`
>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>> ?????? CLANG_LIB_PATH="$with_libclang/lib"
>> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>> +??? ? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>> ???? fi
>>
>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>
>>
>> Now, if you use only the --with-libclang option with a Unix-style 
>> path (the thing you were trying at first), I believe it should do the 
>> right thing.
>
> I have not followed the entire conversation, but this part looks sane. 
> As Maurizio says, we should use BASIC_FIXUP_PATH on all paths from 
> configure argument. (And these should, yes indeed, be in unix style). 
> However, this might not be all fixes that are needed. I can help take 
> a look at it, if someone points me to where the problematic code resides.
>
Magnus, I followed up a bit on this, and managed to reproduce the issue.

The underlying issue is that our lib-clang.m4 conf file checks for 
headers in the usual way with AC_CHECK_HEADER. This check relies on 
setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above 
fixups, what you end up with is this:

CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include

This is the correct _cygwin_ path, but it's not the correct Windows 
path. In fact, if you try to compile a dummy header that contains this 
(this was also observed by Jorn)

#include <clang-c/Index.h>

with a -I option that includes the above path using the cl.exe compiler 
within cygwin, you get an error because the include file cannot be 
found. The only solution is to use a Windows path, which in cygwin can 
only be done using double backslashes (ugh).

Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix path, 
while here we need the opposite operation. I was under the impression 
that the build configure framework needed to solve this particular issue 
anyway, to check for other headers, but I realized that all calls to 
AC_CHECK_HEADER seems to be Unix-style only. So, is AC_CHECK_HEADER even 
supposed to work under Windows? If not, what should we use in order to 
make our configure script portable?

P.S.
I note that there's also a FIXPATH script which does exactly what we 
need here - but this script seems just to be set by the configure step 
and is probably only used by the proper build, so I don't think we can 
refer to it from here?

Thanks
Maurizio
> /Magnus
>
>
>>
>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming paths 
>> which can possibly contain forward slashes - e.g.
>>
>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>
>>
>> Maurizio
>>
>>
>>
>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>> FTR, as I'm looking at other configure files, I've seen
>>>
>>> ?"xwindows" used not "xmicrosoft"
>>>
>>> As for the need for double backslash, I don't see any special 
>>> processing done in other configure files prior to the 
>>> AC_CHECK_HEADER call.
>>>
>>> The build guide [1] is very explicit that forward slashes should be 
>>> used in options (unlike what you did) and mentions some 'fixpath' 
>>> tool which is used by configure to convert Unix paths into Windows 
>>> ones. I wonder if this could be a bug in that tool?
>>>
>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 - 
>>> it could be worth chasing down as to where the compiled 'fixpath' 
>>> executable is put (should be somewhere in 
>>> build/<conf>/configure-support/) call it with the clang include 
>>> folder with Unix-style forward slash and see whether it spits the 
>>> correct path.
>>>
>>> The source code for this tool can be found here:
>>>
>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>>
>>>
>>> Maurizio
>>>
>>> [1] - 
>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>>>
>>>
>>> On 20/09/18 00:41, Henry Jen wrote:
>>>> Actually, -I works, and link option need to be passed with /link, 
>>>> but you got the idea?
>>>>
>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not 
>>>> sure it would work. Need to find a better way to check the lib.
>>>>
>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 -0700
>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018 -0700
>>>> @@ -68,7 +68,11 @@
>>>> ????????? LIBCLANG_CPPFLAGS=""
>>>> ????? fi
>>>> ????? if test "x$CLANG_LIB_PATH" != "x"; then
>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>> ????????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>> +????? else
>>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>> +????? fi
>>>> ????? else
>>>> ????????? LIBCLANG_LDFLAGS=""
>>>> ????? fi
>>>>
>>>> Cheers,
>>>> Henry
>>>>
>>>>
>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>>
>>>>> Haven?t test it, but try this,
>>>>>
>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 
>>>>> -0700
>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018 
>>>>> -0700
>>>>> @@ -63,12 +63,20 @@
>>>>> ???? fi
>>>>>
>>>>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> ???????? LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>> +????? fi
>>>>> ???? else
>>>>> ???????? LIBCLANG_CPPFLAGS=""
>>>>> ???? fi
>>>>> ???? if test "x$CLANG_LIB_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> ???????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>> +????? fi
>>>>> ???? else
>>>>> ???????? LIBCLANG_LDFLAGS=""
>>>>> ???? fi
>>>>>
>>>>> We still need to fix the copy of DLL.
>>>>>
>>>>> Cheers,
>>>>> Henry
>>>>>
>>>>>
>>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore 
>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>
>>>>>> Looks like good progress; tomorrow I'll take a look at some of 
>>>>>> our build files and see if something special is done for mangling 
>>>>>> windows include paths.
>>>>>>
>>>>>> Cheers
>>>>>> Maurizio
>>>>>>
>>>>>>
>>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>>> If I use the following flags:
>>>>>>>
>>>>>>> $ bash configure --disable-warnings-as-errors 
>>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' 
>>>>>>> --with-libclang-include='J:\\LLVM\\include' 
>>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib' 
>>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>>>
>>>>>>> (Notice that I'm having to use different path styles)
>>>>>>>
>>>>>>> It's detecting the header files, but it's failing on being 
>>>>>>> passed the wrong flags. from config.log (see at the bottom):
>>>>>>>
>>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>>> configure:60273: 
>>>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib 
>>>>>>> conftest.cpp -lclang?? >&5
>>>>>>> conftest.cpp
>>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>>
>>>>>>> /out:conftest.exe
>>>>>>> /out:conftest.exe
>>>>>>> conftest.obj
>>>>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>>>>> clang_getClangVersion referenced in function main
>>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>>
>>>>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>>>>> and will be removed in a future release
>>>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>>>> '-LJ:\\LLVM\\lib'
>>>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>>>>
>>>>>>> That seems to be a simple problem of casing-off windows and 
>>>>>>> passing the right flags, but I call it a night here :)
>>>>>>>
>>>>>>> Jorn
>>>>>>>
>>>
>>
>


From Ningsheng.Jian at arm.com  Tue Sep 25 08:52:11 2018
From: Ningsheng.Jian at arm.com (Ningsheng Jian (Arm Technology China))
Date: Tue, 25 Sep 2018 08:52:11 +0000
Subject: [vector] RFC: Add scalable shapes for Arm Scalable Vector
 Extension
In-Reply-To: <5a9d8c0d-dfd9-efb0-0198-f5d84e8ea925@oracle.com>
References: <AM0PR08MB3523229FCD23E33A488724E790000@AM0PR08MB3523.eurprd08.prod.outlook.com>
 <5a9d8c0d-dfd9-efb0-0198-f5d84e8ea925@oracle.com>
Message-ID: <AM0PR08MB3523F26BA08F247CB8BA02B090160@AM0PR08MB3523.eurprd08.prod.outlook.com>

Hi Vladimir,

Thanks a lot for your comments! 

>
> First of all, it's great to see Arm looking into Vector API! Welcome!
> 
> > As you may know that Arm SVE architecturally supports scalable vector length,
> from 128 to 2048bits, multiple of 128bits. Currently Vector API only supports 64,
> 128, 256 and 512 bit vector shapes. It's inappropriate to pre-generate all SVE
> supported shapes, hence, we propose a new type of vector for scalable length
> (or length agnostic) shapes.
> >
> > Webrev:
> > http://cr.openjdk.java.net/~njian/sve/vectorapi.webrev00/
> >
> > Could you please help to take a look at this? The *ScalableVector are the new
> scalable length vectors and to be compatible with existing architectures, the
> preferredSpecies() will return those known species first.
> 
> Overall, I like how it shapes out. There are some aspects I'd like to clarify though.
> 
> The name (*ScalableVector) is slightly misleading to me, but I'm fine with it.  I
> interpret new shapes as representing vectors of maximally supported size at
> runtime without specifying the actual size.
> 

I am happy to change the name if you have some suggestion. The new shapes are representing current (SVE) hardware supported (max) vector register size, and the actual size is known at runtime. 

> New shapes don't interoperate well with existing ones, so unless you change
> how vector shapes are checked (==), at runtime scalable and fixed shape variants
> shouldn't meet. That leads to the next question: how do you expect vector
> shape changing operation working (resize(), rebracket(), cast()) with new variants?
> 

Those vector changing operations are in my TODO list. The scalable shapes are actually fixed length shape during runtime, so I think it is possible to implement those operations. But I am still don't quite understand the motivation and use cases of  those vector shape changing APIs. Could you please help to explain a bit?

> > We are working on SVE support on OpenJDK, and current Vector API's intrinsics
> can really help us on our SVE backend work. We have an initial prototype which
> adds SVE backend and updates a bit for current register allocator to support
> scalable length vector types. Our WIP SVE backend shows that this scalable
> vector proposal works fine with current Vector API intrinsics. We would like to
> collect your comments about this scalable vector patch so that we can move
> forward for the SVE backend support. I notice that Vladimir is also thinking about
> alternative loop shapes, that's really helpful for SVE! We will also take a look at
> that further.
> 
> It's very encouraging to hear existing intrinsics generalize well to SVE. Looking
> forward to seeing the results of SVE backend work. Feel free to share your pain
> points while implementing it.

Thanks! We will also evaluate the Arm NEON implementation and performance for current Vector API.

Thanks,
Ningsheng

From sundararajan.athijegannathan at oracle.com  Tue Sep 25 10:17:59 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Tue, 25 Sep 2018 15:47:59 +0530
Subject: [foreign] RFR 8211063: StdLibTest improvements
Message-ID: <5BAA0B57.2070900@oracle.com>

Please review.

Bug: https://bugs.openjdk.java.net/browse/JDK-8211063
Webrev: http://cr.openjdk.java.net/~sundar/8211063/webrev.00/

This patch is a contribution from Jorn Vernee [1]. It has been tested on 
Mac and Linux (by me & Maurizio)

[1] 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002773.html

Thanks,
-Sundar

From maurizio.cimadamore at oracle.com  Tue Sep 25 10:13:55 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 25 Sep 2018 11:13:55 +0100
Subject: [foreign] RFR 8211063: StdLibTest improvements
In-Reply-To: <5BAA0B57.2070900@oracle.com>
References: <5BAA0B57.2070900@oracle.com>
Message-ID: <8e517923-0522-056a-e772-df7b05594611@oracle.com>

Changes look good to me

Maurizio


On 25/09/18 11:17, Sundararajan Athijegannathan wrote:
> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8211063
> Webrev: http://cr.openjdk.java.net/~sundar/8211063/webrev.00/
>
> This patch is a contribution from Jorn Vernee [1]. It has been tested 
> on Mac and Linux (by me & Maurizio)
>
> [1] 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002773.html
>
> Thanks,
> -Sundar


From sundararajan.athijegannathan at oracle.com  Tue Sep 25 10:36:37 2018
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Tue, 25 Sep 2018 10:36:37 +0000
Subject: hg: panama/dev: 8211063: StdLibTest improvements
Message-ID: <201809251036.w8PAac90008877@aojmv0008.oracle.com>

Changeset: 966f750e424d
Author:    sundar
Date:      2018-09-25 16:15 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/966f750e424d

8211063: StdLibTest improvements
Reviewed-by: mcimadamore
Contributed-by: jbvernee at xs4all.nl

! test/jdk/java/foreign/StdLibTest.java


From sundararajan.athijegannathan at oracle.com  Tue Sep 25 10:49:44 2018
From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan)
Date: Tue, 25 Sep 2018 16:19:44 +0530
Subject: [foreign] RFR 8211063: StdLibTest improvements
In-Reply-To: <8e517923-0522-056a-e772-df7b05594611@oracle.com>
References: <5BAA0B57.2070900@oracle.com>
 <8e517923-0522-056a-e772-df7b05594611@oracle.com>
Message-ID: <5BAA12C8.5000009@oracle.com>

Thanks for the review Maurizio. Thanks for the contribution Jorn Vernee!

PS. Pushed the changes

-Sundar

On 25/09/18, 3:43 PM, Maurizio Cimadamore wrote:
> Changes look good to me
>
> Maurizio
>
>
> On 25/09/18 11:17, Sundararajan Athijegannathan wrote:
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211063
>> Webrev: http://cr.openjdk.java.net/~sundar/8211063/webrev.00/
>>
>> This patch is a contribution from Jorn Vernee [1]. It has been tested 
>> on Mac and Linux (by me & Maurizio)
>>
>> [1] 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002773.html
>>
>> Thanks,
>> -Sundar
>

From maurizio.cimadamore at oracle.com  Tue Sep 25 10:41:41 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Tue, 25 Sep 2018 10:41:41 +0000
Subject: hg: panama/dev: Automatic merge with foreign
Message-ID: <201809251041.w8PAffST012245@aojmv0008.oracle.com>

Changeset: c762f015a181
Author:    mcimadamore
Date:      2018-09-25 12:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/c762f015a181

Automatic merge with foreign



From jbvernee at xs4all.nl  Tue Sep 25 13:27:41 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Tue, 25 Sep 2018 15:27:41 +0200
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
Message-ID: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>

Hello,

Thanks for accepting my last contribution!

During testing I was running into an exception like this:

     Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 
Index 0 out of bounds for length 0
	at 
java.base/jdk.internal.foreign.NativeInvoker.invokeNormal(NativeInvoker.java:216)
	at 
java.base/jdk.internal.foreign.NativeInvoker.invoke(NativeInvoker.java:200)
	at com.github.jdnl.Main$MyLib$Impl/0x000000010009c440.exit(Unknown 
Source)
   	at com.github.jdnl.Main.main(Main.java:17)

The cause turned out to be a mismatch between the Java method signature 
and the native function layout:

     @NativeHeader(declarations =
         "exit=()v" // missing param
     )
     interface MyLib {
         void exit(int code);
     }

Since this exception is pretty cryptic, I'd like to contribute a patch 
that adds an explicit check for this to NativeInvoker, which does some 
basic verification that the two signatures match. With the patch I get 
this error instead (note the last 'Caused by'):

     Exception in thread "main" java.lang.RuntimeException: Failed to 
generate implementation for class interface com.github.jdnl.Main$MyLib
	at 
java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:65)
	at 
java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:132)
	at 
java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:124)
	at java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:226)
	at java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:208)
	at java.base/java.lang.ClassValue.get(ClassValue.java:114)
	at 
java.base/jdk.internal.foreign.LibrariesHelper.getHeaderImplClass(LibrariesHelper.java:155)
	at 
java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:241)
	at 
java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:260)
	at java.base/java.foreign.Libraries.bind(Libraries.java:61)
	at com.github.jdnl.Main.main(Main.java:17)
     Caused by: java.lang.RuntimeException: Failed to generate method 
public abstract void com.github.jdnl.Main$MyLib.exit(int)
	at 
java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:153)
	at 
java.base/jdk.internal.foreign.HeaderImplGenerator.generateMembers(HeaderImplGenerator.java:103)
	at 
java.base/jdk.internal.foreign.BinderClassGenerator.generate(BinderClassGenerator.java:103)
	at 
java.base/jdk.internal.foreign.LibrariesHelper.lambda$generateImpl$0(LibrariesHelper.java:62)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at 
java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:61)
	... 10 more
     Caused by: java.lang.IllegalArgumentException: Java method signature 
and native layout not compatible: public abstract void 
com.github.jdnl.Main$MyLib.exit(int) : ()v
	at 
java.base/jdk.internal.foreign.NativeInvoker.of(NativeInvoker.java:127)
	at 
java.base/jdk.internal.foreign.HeaderImplGenerator.generateFunctionMethod(HeaderImplGenerator.java:130)
	at 
java.base/jdk.internal.foreign.HeaderImplGenerator.generateMethodImplementation(HeaderImplGenerator.java:121)
	at 
java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:150)
	... 15 more

And it is thrown when binding the interface, while the original 
exception was thrown when invoking the method.

I have kept the implementation fairly permissive, so things like binding 
`void m(Pointer<?> p)` to `(u64)v` (converting a pointer to an integer) 
should still be allowed, but maybe a stricter approach is preferable? I 
have added a test to verify that it catches some basic mistakes like 
missing parameters or return types.

Diff: 
https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27

I saw an RFR and JBS bug was made for my last contribution (thanks 
Sundar!), so I've not added 'RFR' to the subject line this time.

Regards,
Jorn

From maurizio.cimadamore at oracle.com  Tue Sep 25 16:43:46 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 25 Sep 2018 17:43:46 +0100
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
In-Reply-To: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
References: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
Message-ID: <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>

Hi Jorn,
your patch looks good, but I think I would like it to just limit things 
at a basic arity check (and void vs. non void check) in the function. 
Given how we treat native types (layout + carrier), how carriers are 
assigned to layouts is up to the method signature - if somebody really 
wants to see a pointer as a float, so be it, although I agree it looks 
silly.

More generally, I think that if we go down the path of enforcing more 
checks, we probably want to enforce carrier vs. layout compatibility in 
a different place - that is LayoutType. That is the natural place where 
to rule out ill-formed combinations.

So there are two checks here: one is the one you needed, which checks 
that a method signature is compatible with its function layout (same 
arity, same varargness, same kind of return). The other is a more in 
depth check performed on LayoutType, which I think is rather orthogonal 
w.r.t. former. I'd suggest to go ahead with the first.

Makes sense?

Maurizio



On 25/09/18 14:27, Jorn Vernee wrote:
> Hello,
>
> Thanks for accepting my last contribution!
>
> During testing I was running into an exception like this:
>
> ??? Exception in thread "main" 
> java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for 
> length 0
> ????at 
> java.base/jdk.internal.foreign.NativeInvoker.invokeNormal(NativeInvoker.java:216)
> ????at 
> java.base/jdk.internal.foreign.NativeInvoker.invoke(NativeInvoker.java:200)
> ????at com.github.jdnl.Main$MyLib$Impl/0x000000010009c440.exit(Unknown 
> Source)
> ????? at com.github.jdnl.Main.main(Main.java:17)
>
> The cause turned out to be a mismatch between the Java method 
> signature and the native function layout:
>
> ??? @NativeHeader(declarations =
> ??????? "exit=()v" // missing param
> ??? )
> ??? interface MyLib {
> ??????? void exit(int code);
> ??? }
>
> Since this exception is pretty cryptic, I'd like to contribute a patch 
> that adds an explicit check for this to NativeInvoker, which does some 
> basic verification that the two signatures match. With the patch I get 
> this error instead (note the last 'Caused by'):
>
> ??? Exception in thread "main" java.lang.RuntimeException: Failed to 
> generate implementation for class interface com.github.jdnl.Main$MyLib
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:65)
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:132)
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:124)
> ????at java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:226)
> ????at java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:208)
> ????at java.base/java.lang.ClassValue.get(ClassValue.java:114)
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper.getHeaderImplClass(LibrariesHelper.java:155)
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:241)
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:260)
> ????at java.base/java.foreign.Libraries.bind(Libraries.java:61)
> ????at com.github.jdnl.Main.main(Main.java:17)
> ??? Caused by: java.lang.RuntimeException: Failed to generate method 
> public abstract void com.github.jdnl.Main$MyLib.exit(int)
> ????at 
> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:153)
> ????at 
> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMembers(HeaderImplGenerator.java:103)
> ????at 
> java.base/jdk.internal.foreign.BinderClassGenerator.generate(BinderClassGenerator.java:103)
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper.lambda$generateImpl$0(LibrariesHelper.java:62)
> ????at java.base/java.security.AccessController.doPrivileged(Native 
> Method)
> ????at 
> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:61)
> ????... 10 more
> ??? Caused by: java.lang.IllegalArgumentException: Java method 
> signature and native layout not compatible: public abstract void 
> com.github.jdnl.Main$MyLib.exit(int) : ()v
> ????at 
> java.base/jdk.internal.foreign.NativeInvoker.of(NativeInvoker.java:127)
> ????at 
> java.base/jdk.internal.foreign.HeaderImplGenerator.generateFunctionMethod(HeaderImplGenerator.java:130)
> ????at 
> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMethodImplementation(HeaderImplGenerator.java:121)
> ????at 
> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:150)
> ????... 15 more
>
> And it is thrown when binding the interface, while the original 
> exception was thrown when invoking the method.
>
> I have kept the implementation fairly permissive, so things like 
> binding `void m(Pointer<?> p)` to `(u64)v` (converting a pointer to an 
> integer) should still be allowed, but maybe a stricter approach is 
> preferable? I have added a test to verify that it catches some basic 
> mistakes like missing parameters or return types.
>
> Diff: https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>
> I saw an RFR and JBS bug was made for my last contribution (thanks 
> Sundar!), so I've not added 'RFR' to the subject line this time.
>
> Regards,
> Jorn


From jbvernee at xs4all.nl  Tue Sep 25 18:33:15 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Tue, 25 Sep 2018 20:33:15 +0200
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
In-Reply-To: <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>
References: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
 <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>
Message-ID: <75153dbf40951b2594e092a3267d3834@xs4all.nl>

Hi Maurizio,
Thanks for looking at the patch.

I have made the check simpler as you suggested [1]. Doing carrier <-> 
layout checks in LayoutType makes sense to me.

If you appreciate this kind of contribution I had 2 more things I ran 
into;

1.) When a native function returns `null`, it seems that this value is 
always boxed using `References#ofVoid`, and when you de-reference it you 
get an UOE without a message. I'm not sure if boxing as Void is 
intentional, but either way it would be nice if it gave a reason like 
'can not de-reference null pointer'.

2.) The way NativeHeader#declarations is parsed makes it so that a 
faulty layout string like `exit(i32)v` (should be `exit=(i32)v`) is 
silently ignored, and you get an AbstractMethodError when invoking the 
method. Part of the problem I think, is that the parser splits the 
declarations string by `=`, and then goes from there. I'm thinking a 
more robust approach would be to make NativeHeader#declarations a 
String[] instead of a String, and then try to parse exactly 1 
declaration for each element. What do you think?

I could try and make patches for those later this week.

Jorn

[1] : 
https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27

Maurizio Cimadamore schreef op 2018-09-25 18:43:
> Hi Jorn,
> your patch looks good, but I think I would like it to just limit
> things at a basic arity check (and void vs. non void check) in the
> function. Given how we treat native types (layout + carrier), how
> carriers are assigned to layouts is up to the method signature - if
> somebody really wants to see a pointer as a float, so be it, although
> I agree it looks silly.
> 
> More generally, I think that if we go down the path of enforcing more
> checks, we probably want to enforce carrier vs. layout compatibility
> in a different place - that is LayoutType. That is the natural place
> where to rule out ill-formed combinations.
> 
> So there are two checks here: one is the one you needed, which checks
> that a method signature is compatible with its function layout (same
> arity, same varargness, same kind of return). The other is a more in
> depth check performed on LayoutType, which I think is rather
> orthogonal w.r.t. former. I'd suggest to go ahead with the first.
> 
> Makes sense?
> 
> Maurizio
> 
> 
> 
> On 25/09/18 14:27, Jorn Vernee wrote:
>> Hello,
>> 
>> Thanks for accepting my last contribution!
>> 
>> During testing I was running into an exception like this:
>> 
>> ??? Exception in thread "main" 
>> java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for 
>> length 0
>> ????at 
>> java.base/jdk.internal.foreign.NativeInvoker.invokeNormal(NativeInvoker.java:216)
>> ????at 
>> java.base/jdk.internal.foreign.NativeInvoker.invoke(NativeInvoker.java:200)
>> ????at com.github.jdnl.Main$MyLib$Impl/0x000000010009c440.exit(Unknown 
>> Source)
>> ????? at com.github.jdnl.Main.main(Main.java:17)
>> 
>> The cause turned out to be a mismatch between the Java method 
>> signature and the native function layout:
>> 
>> ??? @NativeHeader(declarations =
>> ??????? "exit=()v" // missing param
>> ??? )
>> ??? interface MyLib {
>> ??????? void exit(int code);
>> ??? }
>> 
>> Since this exception is pretty cryptic, I'd like to contribute a patch 
>> that adds an explicit check for this to NativeInvoker, which does some 
>> basic verification that the two signatures match. With the patch I get 
>> this error instead (note the last 'Caused by'):
>> 
>> ??? Exception in thread "main" java.lang.RuntimeException: Failed to 
>> generate implementation for class interface com.github.jdnl.Main$MyLib
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:65)
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:132)
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:124)
>> ????at 
>> java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:226)
>> ????at 
>> java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:208)
>> ????at java.base/java.lang.ClassValue.get(ClassValue.java:114)
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper.getHeaderImplClass(LibrariesHelper.java:155)
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:241)
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:260)
>> ????at java.base/java.foreign.Libraries.bind(Libraries.java:61)
>> ????at com.github.jdnl.Main.main(Main.java:17)
>> ??? Caused by: java.lang.RuntimeException: Failed to generate method 
>> public abstract void com.github.jdnl.Main$MyLib.exit(int)
>> ????at 
>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:153)
>> ????at 
>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMembers(HeaderImplGenerator.java:103)
>> ????at 
>> java.base/jdk.internal.foreign.BinderClassGenerator.generate(BinderClassGenerator.java:103)
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper.lambda$generateImpl$0(LibrariesHelper.java:62)
>> ????at java.base/java.security.AccessController.doPrivileged(Native 
>> Method)
>> ????at 
>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:61)
>> ????... 10 more
>> ??? Caused by: java.lang.IllegalArgumentException: Java method 
>> signature and native layout not compatible: public abstract void 
>> com.github.jdnl.Main$MyLib.exit(int) : ()v
>> ????at 
>> java.base/jdk.internal.foreign.NativeInvoker.of(NativeInvoker.java:127)
>> ????at 
>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateFunctionMethod(HeaderImplGenerator.java:130)
>> ????at 
>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMethodImplementation(HeaderImplGenerator.java:121)
>> ????at 
>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:150)
>> ????... 15 more
>> 
>> And it is thrown when binding the interface, while the original 
>> exception was thrown when invoking the method.
>> 
>> I have kept the implementation fairly permissive, so things like 
>> binding `void m(Pointer<?> p)` to `(u64)v` (converting a pointer to an 
>> integer) should still be allowed, but maybe a stricter approach is 
>> preferable? I have added a test to verify that it catches some basic 
>> mistakes like missing parameters or return types.
>> 
>> Diff: 
>> https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>> 
>> I saw an RFR and JBS bug was made for my last contribution (thanks 
>> Sundar!), so I've not added 'RFR' to the subject line this time.
>> 
>> Regards,
>> Jorn

From shravya.rukmannagari at intel.com  Tue Sep 25 19:29:13 2018
From: shravya.rukmannagari at intel.com (shravya.rukmannagari at intel.com)
Date: Tue, 25 Sep 2018 19:29:13 +0000
Subject: hg: panama/dev: Linux SVML Support Fix for file extension
Message-ID: <201809251929.w8PJTD8o020292@aojmv0008.oracle.com>

Changeset: e8f06cc3fe82
Author:    srukmannagar
Date:      2018-09-25 04:07 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e8f06cc3fe82

Linux SVML Support Fix for file extension

+ src/hotspot/os_cpu/linux_x86/svml_d_acos_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_acos_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_asin_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_asin_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_atan2_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_atan2_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_atan_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_atan_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_cbrt_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_cbrt_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_cos_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_cos_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_cosh_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_cosh_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_exp_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_exp_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_expm1_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_expm1_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_hypot_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_hypot_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_log10_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_log10_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_log1p_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_log1p_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_log_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_log_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_pow_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_pow_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_sin_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_sin_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_sinh_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_sinh_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_tan_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_tan_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_d_tanh_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_d_tanh_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_acos_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_acos_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_asin_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_asin_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_atan2_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_atan2_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_atan_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_atan_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_cbrt_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_cbrt_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_cos_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_cos_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_cosh_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_cosh_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_exp_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_exp_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_expm1_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_expm1_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_hypot_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_hypot_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_log10_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_log10_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_log1p_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_log1p_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_log_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_log_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_pow_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_pow_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_sin_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_sin_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_sinh_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_sinh_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_tan_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_tan_linux_x86.s
+ src/hotspot/os_cpu/linux_x86/svml_s_tanh_linux_x86.S
- src/hotspot/os_cpu/linux_x86/svml_s_tanh_linux_x86.s


From maurizio.cimadamore at oracle.com  Tue Sep 25 19:31:53 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Tue, 25 Sep 2018 19:31:53 +0000
Subject: hg: panama/dev: Automatic merge with vectorIntrinsics
Message-ID: <201809251931.w8PJVrCd021581@aojmv0008.oracle.com>

Changeset: bae89ae7aa41
Author:    mcimadamore
Date:      2018-09-25 21:34 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/bae89ae7aa41

Automatic merge with vectorIntrinsics

- src/hotspot/os_cpu/linux_x86/svml_d_acos_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_asin_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_atan2_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_atan_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_cbrt_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_cos_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_cosh_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_exp_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_expm1_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_hypot_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_log10_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_log1p_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_log_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_pow_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_sin_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_sinh_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_tan_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_d_tanh_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_acos_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_asin_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_atan2_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_atan_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_cbrt_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_cos_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_cosh_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_exp_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_expm1_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_hypot_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_log10_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_log1p_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_log_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_pow_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_sin_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_sinh_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_tan_linux_x86.s
- src/hotspot/os_cpu/linux_x86/svml_s_tanh_linux_x86.s


From samuel.audet at gmail.com  Wed Sep 26 00:47:00 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Wed, 26 Sep 2018 09:47:00 +0900
Subject: [foreign] RFR 8211060: Library.getDefault() does not work on
 Windows
In-Reply-To: <0760f00f-51dd-05f5-10b4-5a03d88f636e@oracle.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
 <0760f00f-51dd-05f5-10b4-5a03d88f636e@oracle.com>
Message-ID: <bb202e4d-279f-b587-3237-ad2bcd537992@gmail.com>

I'll reply to the renamed thread, sorry about that.

"re-discuss", well, AFAIK, it's never been discussed publicly once 
anywhere. One day, someone came up with the idea that interfaces should 
be used for everything, and that was the end of the "discussion". 
Although I know we're not supposed to have discussions here, there are 
no other places to have them, so I'm trying here anyway until you guys 
come up with with forums for that. :) The "spec" list is quite low 
volume and moderated, not exactly an inviting place for discussion...

Samuel

On 09/24/2018 09:00 PM, Maurizio Cimadamore wrote:
> Can you please expand a bit on this?
> 
> But also, warning: this is a RFR thread and I don't like it being side 
> tracked to re-discuss higher level goals such as modeling headers with 
> interfaces. If you want to raise this concern (seems so, but not 100% 
> sure), as otherwise stated, please raise that in the appropriate forum 
> (panama-spec-experts). Let's keep this list for discussion on the 
> prototype, feedback on real usage, bugs, etc.
> 
> Maurizio
> 
> On 24/09/18 12:38, Samuel Audet wrote:
>> FWIW, I think the factory method pattern should be reconsidered 
>> entirely. In C/C++, when we want to call say getpid(), we don't start 
>> loading stuff up before calling getpid(), we call getpid()! Why not do 
>> the same from Java? From a usability point of view, not loading stuff 
>> manually works fine for JavaCPP...
>>
>> Now, I know you're going to start taking about interfaces and what 
>> not. You said that you had plans to introduce an entirely new array 
>> type just to make it more friendly with vector instructions and native 
>> libraries. Why not start thinking about an "interface" that would be 
>> friendly to native libraries as well? Why stop at arrays?
>>
>> Samuel
>>
>> On 09/24/2018 08:10 PM, Maurizio Cimadamore wrote:
>>> Hi Jorn,
>>> thanks for the patch. As mentioned last time we have two options 
>>> here: one is to follow the approach forward in your patch; another 
>>> would be to ditch Library.getDefault() entirely and adopt a more 
>>> explicit approach - that is to always require 'implicit' libraries to 
>>> be mentioned - either by jextract artifacts or by API points.
>>>
>>> A note on the latter - when you do:
>>>
>>> Libraries.bindRaw(lookup, Foo.class)
>>>
>>> the code lookup the @NativeHeader annotation on Foo.class and tries 
>>> to extract required library names from there. Currently, we do not 
>>> add library names for standard libraries such as "c" or "m" (math), 
>>> which is what led us down the (slippery?) slope of having a 
>>> Library.getDefault somewhere.
>>>
>>> Another option would be to have jextract to always generate the names 
>>> of the libraries an artifact depends on, and then the API should 
>>> throw an exception if a @NativeHeader is found with no libraries. 
>>> More specifically, jextract should always add "c" to the set of used 
>>> libraries in the @NativeHeader annotation (other libraries can be 
>>> explicitly supplied using the -l flag).
>>>
>>> Note that I'm not saying "the binder should always add in 'clib'" for 
>>> you, because that's kind of a lame assumption: it works obviously 
>>> well for C but it doesn't make a lot of sense if you want to use 
>>> Panama for other purposes/languages. Which seems to suggest that, as 
>>> far as the binder is concerned, library dependencies should always be 
>>> explicit.
>>>
>>> Thoughts?
>>>
>>> Maurizio
>>>
>>>
>>>
>>> On 24/09/18 11:52, Jorn Vernee wrote:
>>>> Hello,
>>>>
>>>> I'd like to contribute a patch that adds support for the default 
>>>> library on windows.
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8211060
>>>> Diff: 
>>>> https://gist.github.com/JornVernee/7d45780df082cbfb27417b437c7b13a8
>>>>
>>>> As mentioned before [1], this fixes 2 bugs:
>>>>
>>>> 1.) When no library was loaded ClassLoader#NativeLibrary#getFromClass
>>>> threw an NPE (at least on windows). That is fixed by returning
>>>> defaultLibrary.fromClass when the nativeLibraryContext is empty.
>>>>
>>>> 2.) The default library search was not working on windows. It was using
>>>> a default handle, which works on unix (dlsym(RTLD_DEFAULT)), but not on
>>>> windows (see https://stackoverflow.com/q/23437007). I have changed the
>>>> implementation from using a default handle to using a new native
>>>> function findEntryInProcess, which does the right thing for Windows
>>>> (iterate over all loaded modules), and does the same thing it used to
>>>> for unix. findEntry is now a Java method, and the original findEntry is
>>>> renamed to findEntry0. The NativeLibrary implementation of findEntry
>>>> forwards to findEntry0, and the anonymous class of the default library
>>>> overrides to forward to findEntryInProcess.
>>>>
>>>> Please test this patch on unix as well, since I don't have the 
>>>> ability to do so.
>>>>
>>>> Jorn
>>>>
>>>> [1] : 
>>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002644.html 
>>>>
>>>
>>
> 


From samuel.audet at gmail.com  Wed Sep 26 01:14:49 2018
From: samuel.audet at gmail.com (Samuel Audet)
Date: Wed, 26 Sep 2018 10:14:49 +0900
Subject: on usability (was Re: [foreign] RFR 8211060: Library.getDefault()
 does not work on Windows)
In-Reply-To: <1dde768f-2067-de80-2655-a881ae78c771@oracle.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
 <c1e938f3967fb53b0831fea30a4e710c@xs4all.nl>
 <1dde768f-2067-de80-2655-a881ae78c771@oracle.com>
Message-ID: <39381ab4-f888-f7e9-3764-59b5dba1eca9@gmail.com>

We can do a lot of wrapper magic in either Java or C++. JavaCPP already 
does for JNI what Jorn is describing we could do for Panama:
https://github.com/bytedeco/javacpp-presets/tree/master/systems#the-srcmainjavatestavxjava-source-file

If we consider JNI to be "legacy", it makes a lot of sense to try and do 
some acrobatics like that to support legacy systems, but why not make 
the new hotness actually _usable_? Take a look at how Swift does it:
https://developer.apple.com/documentation/swift/imported_c_and_objective-c_apis/using_imported_c_structs_and_unions_in_swift

Now that's what I call *usable*. Panama is very far from that level of 
usability. And it's not because the Java language is somehow 
handicapped. Check what the guys over at GraalVM are doing:
https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.tutorial/src/com/oracle/svm/tutorial/CInterfaceTutorial.java

It also features performance that's already apparently higher than JNI:
https://cornerwings.github.io/2018/07/graal-native-methods/

Why not do something user friendly like that in Panama's case as well? 
What's the rationale to make it all in the end as complicated to use as 
JNI? Maybe there's something I'm missing, so please point it out.

Samuel


On 09/24/2018 10:37 PM, Maurizio Cimadamore wrote:
> Having a pre-extracted stdlib bundle is something we have considered - 
> quoting from [1]:
> 
> "Now, since most (all?) of the libraries out there are going to assume
> the availability of some 'standard library', let's also assume that some
> extracted artifact for such library is available and that jextract
> always knows how to find it - this is the equivalent of java.base for
> the module system, or java.lang for the Java import system. This
> addresses the bootstrapping issue."
> 
> In time we'll get there, I don't see any real technical obstacles to get 
> to your 'optimal' snippet.
> 
> I think there are two aspects that I'd like to draw attention upon:
> 
> 1) Magic does not come for free. E.g. it might "seem" that JNI has a 
> more direct approach to calling native methods (ease of use issues 
> aside). In reality it's just that the complexity of calling that native 
> method, marshalling arguments, unmarshalling returns, dealing with 
> native thread transitions and what's not has just been pushed under the 
> JVM rug. So, yes, you can "just" call getpid - but the burden is on the 
> VM. Now, the JNI support is already quite complex - I can't honestly 
> imagine a sane way for the VM to support any given invocation scheme 
> that a user might wish to see supported. This is why Panama is betting 
> on Java code + layouts to do the lifting: that way the VM interface can 
> be kept simple (which has significant payoffs - as the resulting code 
> can be optimized much more - see the linkToNative experimental results 
> in [2]).
> 
> 2) As your example points out, while calling 'getpid' is something that 
> seems 'easy' enough - 'puts' is already some other beast. It takes a 
> pointer to some memory location where the string is stored. The JNI 
> approach is to pass the Java string as is, and then do the wiring in 
> native code. That is, there's no free lunch here - either you do the 
> adaptation in Java, or you do it in native code (**). Panama gives you a 
> rich enough API to do all such adaptations in Java, so that all native 
> calls are... just native calls (again this means more regularity which 
> means more performances). Having opaque native code snippets which do 
> argument adaptation is not very optimal (and optimizable) for the JVM. 
> With Panama you can create a good-looking API which internally uses 
> pointers/scopes and delegates to the right native method - all done in 
> Java. On top of that, our plans cover a so called 'civilization' layer 
> (see [3]), by which users will be able to customize what comes out of 
> jextract in order e.g. to tell that for 'puts' they really want a Java 
> String argument and not a Pointer<Byte>; again this will be done in a 
> more general way, so that the binder will be pointed at a pair of 
> functions which can be used to map the user provided data to and from 
> native code.
> 
> (**) for an example of how interfacing with standard libraries needs 
> some kind of wrapping, even in JNI - look at [4]; this file is 
> essentially a collection of system calls which are wrapped by some logic 
> (e.g. to check errno, ...). I claim that there is something 
> _fundamentally_ wrong with code like this, in that the native code is 
> mixing two concerns: (i) performing the required native call and (ii) 
> adjusting input/output/errors of the call in a way that is suitable to 
> the corresponding Java API. Why shouldn't the Java API itself be in 
> charge of doing (ii) ?
> 
> Maurizio
> 
> [1] - 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-August/002560.html
> [2] - 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html 
> 
> [3] - 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-April/001537.html
> [4] - 
> http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c#l314 
> 
> 
> 
> 
> 
> On 24/09/18 13:34, Jorn Vernee wrote:
>> I agree with the usability point. In C++ it's as simple to call puts 
>> as doing:
>>
>> ??? #include <stdio.h>
>>
>> ??? int main() {
>> ??????? puts("Hello World!");
>> ??? }
>>
>> And I think the optimal Java equivalent would be something like:
>>
>> ??? import static org.openjdk.stdio.*;
>>
>> ??? public class Main {
>>
>> ??????? public static void main(String[] args) {
>> ??????????? puts("Hello World!");
>> ??????? }
>>
>> ??? }
>>
>> This can be facilitated by creating a 'singleton facade' for the 
>> library interface like so:
>>
>> ??? public class stdio {
>>
>> ??????? private static final stdioImpl lib = Libraries.bind(lookup(), 
>> stdioImpl.class);
>>
>> ??????? public static int puts (String message) {
>> ??????????? try(Scope scope = Scope.newNativeScope()) {
>> ??????????????? Pointer<Byte> msg = scope.toCString(message);
>> ??????????????? return lib.puts(msg);
>> ??????????? }
>> ??????? }
>>
>> ??????? ...
>> ??? }
>>
>> Such a facade class could be shipped with the JDK or perhaps as an 
>> artifact on maven central, or maybe it could be an additional output 
>> of jextract.
>>
>> But there is only so much you can do automagically from the Java side. 
>> When working from C/C++ you have the compiler filling in the blanks. 
>> For instance, it automatically allocates storage for string literals. 
>> Java does that as well for Java string literals `String s = "Hello";`, 
>> but it can not do that for native strings, and you have to use the 
>> Scope API to do that manually. In some cases, like the above, you can 
>> write glue-code to make that automatic, but I think at some point 
>> things become too complex for that, and there will always be some 
>> usability barrier to interop.
>>
>> Jorn
>>
>> Samuel Audet schreef op 2018-09-24 13:38:
>>> FWIW, I think the factory method pattern should be reconsidered
>>> entirely. In C/C++, when we want to call say getpid(), we don't start
>>> loading stuff up before calling getpid(), we call getpid()! Why not do
>>> the same from Java? From a usability point of view, not loading stuff
>>> manually works fine for JavaCPP...
>>>
>>> Now, I know you're going to start taking about interfaces and what
>>> not. You said that you had plans to introduce an entirely new array
>>> type just to make it more friendly with vector instructions and native
>>> libraries. Why not start thinking about an "interface" that would be
>>> friendly to native libraries as well? Why stop at arrays?
>>>
>>> Samuel
>>>

From jbvernee at xs4all.nl  Wed Sep 26 09:39:00 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 26 Sep 2018 11:39:00 +0200
Subject: on usability
In-Reply-To: <39381ab4-f888-f7e9-3764-59b5dba1eca9@gmail.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
 <c1e938f3967fb53b0831fea30a4e710c@xs4all.nl>
 <1dde768f-2067-de80-2655-a881ae78c771@oracle.com>
 <39381ab4-f888-f7e9-3764-59b5dba1eca9@gmail.com>
Message-ID: <daedc829c7f052f3f034313499f2e984@xs4all.nl>

The Swift example looks cool and I can say 2 things about that:

1.) Swift seems to have properties (i.e. syntactic sugar for getters and 
setter), so It's much easier to inject some code that accesses an 
underlying C struct instead of a backing field. I think at least for the 
time being, the Java equivalent would be getters and setters. But tbh I 
don't see that much usability problems with that, since it's more or 
less the same amount of characters to write: `myStruct.x = 10` vs. 
`myStruct.x$set(10)` (especially considering auto-completion). It just 
doesn't look as fancy.

2.) I guess jextract could generate an equivalent of the 2 `Init()` 
functions as well for generated structs, and that would be part of the 
'civilisation layer' Maurizio mentioned. With current tech, it would 
probably either create and leak a scope internally, or you'd have to 
pass a scope. Maybe a long term solution could be something like using a 
default scope that is managed by the garbage collector. The generated 
struct would not have a tight life-cycle (GC lazily collects objects), 
but it would be easier to use.

The Graal native access stuff uses truffle, i.e. it is baked into the 
interpreter. But the truffle interpreter is built to be very 
customizeable, so I think doing the same with the Hotspot interpreter 
would be far more difficult. But either way, the way Graal maps a C 
struct to an interface in that example looks pretty similar to what 
panama is doing to me.

I think panama is making sane choices, and focusing on capability before 
usability. panama is just not as far along as the other projects you 
mention, and I think more usability (jextract civilization layer) and 
better performance (linkToNative backend) are yet to come.

Jorn

Samuel Audet schreef op 2018-09-26 03:14:
> We can do a lot of wrapper magic in either Java or C++. JavaCPP
> already does for JNI what Jorn is describing we could do for Panama:
> https://github.com/bytedeco/javacpp-presets/tree/master/systems#the-srcmainjavatestavxjava-source-file
> 
> If we consider JNI to be "legacy", it makes a lot of sense to try and
> do some acrobatics like that to support legacy systems, but why not
> make the new hotness actually _usable_? Take a look at how Swift does
> it:
> https://developer.apple.com/documentation/swift/imported_c_and_objective-c_apis/using_imported_c_structs_and_unions_in_swift
> 
> Now that's what I call *usable*. Panama is very far from that level of
> usability. And it's not because the Java language is somehow
> handicapped. Check what the guys over at GraalVM are doing:
> https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.tutorial/src/com/oracle/svm/tutorial/CInterfaceTutorial.java
> 
> It also features performance that's already apparently higher than JNI:
> https://cornerwings.github.io/2018/07/graal-native-methods/
> 
> Why not do something user friendly like that in Panama's case as well?
> What's the rationale to make it all in the end as complicated to use
> as JNI? Maybe there's something I'm missing, so please point it out.
> 
> Samuel
> 
> 
> On 09/24/2018 10:37 PM, Maurizio Cimadamore wrote:
>> Having a pre-extracted stdlib bundle is something we have considered - 
>> quoting from [1]:
>> 
>> "Now, since most (all?) of the libraries out there are going to assume
>> the availability of some 'standard library', let's also assume that 
>> some
>> extracted artifact for such library is available and that jextract
>> always knows how to find it - this is the equivalent of java.base for
>> the module system, or java.lang for the Java import system. This
>> addresses the bootstrapping issue."
>> 
>> In time we'll get there, I don't see any real technical obstacles to 
>> get to your 'optimal' snippet.
>> 
>> I think there are two aspects that I'd like to draw attention upon:
>> 
>> 1) Magic does not come for free. E.g. it might "seem" that JNI has a 
>> more direct approach to calling native methods (ease of use issues 
>> aside). In reality it's just that the complexity of calling that 
>> native method, marshalling arguments, unmarshalling returns, dealing 
>> with native thread transitions and what's not has just been pushed 
>> under the JVM rug. So, yes, you can "just" call getpid - but the 
>> burden is on the VM. Now, the JNI support is already quite complex - I 
>> can't honestly imagine a sane way for the VM to support any given 
>> invocation scheme that a user might wish to see supported. This is why 
>> Panama is betting on Java code + layouts to do the lifting: that way 
>> the VM interface can be kept simple (which has significant payoffs - 
>> as the resulting code can be optimized much more - see the 
>> linkToNative experimental results in [2]).
>> 
>> 2) As your example points out, while calling 'getpid' is something 
>> that seems 'easy' enough - 'puts' is already some other beast. It 
>> takes a pointer to some memory location where the string is stored. 
>> The JNI approach is to pass the Java string as is, and then do the 
>> wiring in native code. That is, there's no free lunch here - either 
>> you do the adaptation in Java, or you do it in native code (**). 
>> Panama gives you a rich enough API to do all such adaptations in Java, 
>> so that all native calls are... just native calls (again this means 
>> more regularity which means more performances). Having opaque native 
>> code snippets which do argument adaptation is not very optimal (and 
>> optimizable) for the JVM. With Panama you can create a good-looking 
>> API which internally uses pointers/scopes and delegates to the right 
>> native method - all done in Java. On top of that, our plans cover a so 
>> called 'civilization' layer (see [3]), by which users will be able to 
>> customize what comes out of jextract in order e.g. to tell that for 
>> 'puts' they really want a Java String argument and not a 
>> Pointer<Byte>; again this will be done in a more general way, so that 
>> the binder will be pointed at a pair of functions which can be used to 
>> map the user provided data to and from native code.
>> 
>> (**) for an example of how interfacing with standard libraries needs 
>> some kind of wrapping, even in JNI - look at [4]; this file is 
>> essentially a collection of system calls which are wrapped by some 
>> logic (e.g. to check errno, ...). I claim that there is something 
>> _fundamentally_ wrong with code like this, in that the native code is 
>> mixing two concerns: (i) performing the required native call and (ii) 
>> adjusting input/output/errors of the call in a way that is suitable to 
>> the corresponding Java API. Why shouldn't the Java API itself be in 
>> charge of doing (ii) ?
>> 
>> Maurizio
>> 
>> [1] - 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-August/002560.html
>> [2] - 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html 
>> [3] - 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-April/001537.html
>> [4] - 
>> http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c#l314 
>> On 24/09/18 13:34, Jorn Vernee wrote:
>>> I agree with the usability point. In C++ it's as simple to call puts 
>>> as doing:
>>> 
>>> ??? #include <stdio.h>
>>> 
>>> ??? int main() {
>>> ??????? puts("Hello World!");
>>> ??? }
>>> 
>>> And I think the optimal Java equivalent would be something like:
>>> 
>>> ??? import static org.openjdk.stdio.*;
>>> 
>>> ??? public class Main {
>>> 
>>> ??????? public static void main(String[] args) {
>>> ??????????? puts("Hello World!");
>>> ??????? }
>>> 
>>> ??? }
>>> 
>>> This can be facilitated by creating a 'singleton facade' for the 
>>> library interface like so:
>>> 
>>> ??? public class stdio {
>>> 
>>> ??????? private static final stdioImpl lib = Libraries.bind(lookup(), 
>>> stdioImpl.class);
>>> 
>>> ??????? public static int puts (String message) {
>>> ??????????? try(Scope scope = Scope.newNativeScope()) {
>>> ??????????????? Pointer<Byte> msg = scope.toCString(message);
>>> ??????????????? return lib.puts(msg);
>>> ??????????? }
>>> ??????? }
>>> 
>>> ??????? ...
>>> ??? }
>>> 
>>> Such a facade class could be shipped with the JDK or perhaps as an 
>>> artifact on maven central, or maybe it could be an additional output 
>>> of jextract.
>>> 
>>> But there is only so much you can do automagically from the Java 
>>> side. When working from C/C++ you have the compiler filling in the 
>>> blanks. For instance, it automatically allocates storage for string 
>>> literals. Java does that as well for Java string literals `String s = 
>>> "Hello";`, but it can not do that for native strings, and you have to 
>>> use the Scope API to do that manually. In some cases, like the above, 
>>> you can write glue-code to make that automatic, but I think at some 
>>> point things become too complex for that, and there will always be 
>>> some usability barrier to interop.
>>> 
>>> Jorn
>>> 
>>> Samuel Audet schreef op 2018-09-24 13:38:
>>>> FWIW, I think the factory method pattern should be reconsidered
>>>> entirely. In C/C++, when we want to call say getpid(), we don't 
>>>> start
>>>> loading stuff up before calling getpid(), we call getpid()! Why not 
>>>> do
>>>> the same from Java? From a usability point of view, not loading 
>>>> stuff
>>>> manually works fine for JavaCPP...
>>>> 
>>>> Now, I know you're going to start taking about interfaces and what
>>>> not. You said that you had plans to introduce an entirely new array
>>>> type just to make it more friendly with vector instructions and 
>>>> native
>>>> libraries. Why not start thinking about an "interface" that would be
>>>> friendly to native libraries as well? Why stop at arrays?
>>>> 
>>>> Samuel
>>>> 

From maurizio.cimadamore at oracle.com  Wed Sep 26 11:05:31 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 26 Sep 2018 12:05:31 +0100
Subject: on usability (was Re: [foreign] RFR 8211060: Library.getDefault()
 does not work on Windows)
In-Reply-To: <39381ab4-f888-f7e9-3764-59b5dba1eca9@gmail.com>
References: <138140e4ab654e59fa538b5438fdee9d@xs4all.nl>
 <71d58dfa-4bf2-382a-9449-e726f291b1e9@oracle.com>
 <3e25bf4a-1a75-357f-be72-fd36cbd3cf15@gmail.com>
 <c1e938f3967fb53b0831fea30a4e710c@xs4all.nl>
 <1dde768f-2067-de80-2655-a881ae78c771@oracle.com>
 <39381ab4-f888-f7e9-3764-59b5dba1eca9@gmail.com>
Message-ID: <8d27e877-2105-a033-45da-08e825d349fd@oracle.com>

Jorn actually made most of my points, but I will try to add something in 
here



On 26/09/18 02:14, Samuel Audet wrote:
> We can do a lot of wrapper magic in either Java or C++. JavaCPP 
> already does for JNI what Jorn is describing we could do for Panama:
> https://github.com/bytedeco/javacpp-presets/tree/master/systems#the-srcmainjavatestavxjava-source-file 
>
>
> If we consider JNI to be "legacy",
It's not that JNI is legacy - but JNI performances (and safety, and 
usability) cannot be improved much. If you want to get performances that 
are similar to those you'd get with a native compiler, using JNI is just 
not the tool for the job. Existing solutions such as JavaCPP or JNR can 
take the usability pain out of JNI and offer a smoother experience. But, 
being based on JNI, they can't improve over what JNI has to offer, at 
least not performance-wise. For that you need to reach out for some 
trick under the hood (either in Hotspot or in Graal).

> it makes a lot of sense to try and do some acrobatics like that to 
> support legacy systems, but why not make the new hotness actually 
> _usable_? Take a look at how Swift does it:
> https://developer.apple.com/documentation/swift/imported_c_and_objective-c_apis/using_imported_c_structs_and_unions_in_swift 
>
Now, I believe in Swift, when you interop with native code you really 
"import" that native code - meaning that some definition will be 
generated from the corresponding C/Objective-C header. If you squint, 
this is not too different from what we do with jextract, is it? After 
the importing process is done, you can use the struct as if it were a 
Swift struct; in Panama you can use it as an interface. There are some 
syntactic differences of course - the fact that Swift has properties 
allows it to model struct access (get/set) as field access, in Panama we 
have to do with interface calls.

Btw, on this topic of accessors, native structs really need 3 accessors: 
get/set/addressof - I don't think even Swift is powerful enough to do 
the 3rd, which means for that you are back to a regular method call?

Anyway, I don't see a huge usability gap here other than the one imposed 
by different language restrictions.
>
> Now that's what I call *usable*. Panama is very far from that level of 
> usability. And it's not because the Java language is somehow 
> handicapped. Check what the guys over at GraalVM are doing:
> https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.tutorial/src/com/oracle/svm/tutorial/CInterfaceTutorial.java 
>
Well, if you read most of the code you would actually have trouble 
telling whether it's Panama or Graal. The main concepts are there - 
there is a way to speak about pointers, structs are modeled as 
interfaces, ...

The only real difference is native methods (and even that is a very 
small one) - Graal allows you to model native calls as calls to 'Java 
native' methods which have some annotations; if you AOT everything (with 
substrateVM/native image) the AOT compiler can compile the call away 
using a direct native call. You get better performances (no JNI), you 
lose flexibility (dynamic loading). I'd say the Graal solution is in 
spirit very similar to the one we provide here, with the difference that 
Panama does the magic at runtime (JIT) as opposed to AOT.

So, to summarize, the biggest difference I see (apart from naming 
differences for e.g. pointer types etc.) is that Graal/Native Image 
support uses 'annotated' 'native' methods. Panama uses 'annotated' 
'abstract' (interface) methods. Oh - there's also another difference - 
Panama does not attempt to model enums as Java enums as that's not 
complete, but that's a different story. Again, not much difference to 
see here?

Maybe you refer to the fact that the annotations in the Graal example 
are easier to read than Panama's (which you didn't say, but I could 
agree); in that case please do keep in mind that Panama metadata is 
designed to be language neutral so as to maximize applicability and not 
to bake any language assumptions (e.g. pointers are 64bit) into the VM. 
If we're only talking C/C++ this might sound odd, but it pays off in 
that you can also naturally model things that do not come from C (e.g. 
protobuf/flatbuffer schemas).
>
> It also features performance that's already apparently higher than JNI:
> https://cornerwings.github.io/2018/07/graal-native-methods/
>
> Why not do something user friendly like that in Panama's case as well? 
> What's the rationale to make it all in the end as complicated to use 
> as JNI? Maybe there's something I'm missing, so please point it out.
I believed I showed in the JMH thread how linkToNative is similarly 
capable of sailing well past of JNI. And I have reasons to believe a 
similar optimization as the one done in C2 for linkToNative can be done 
with the Graal compiler.

So, all things consider I (still) fail to see how the design choices we 
have made are precluding us certain usability goals, or certain 
performance targets? - which is what you seem to suggest.

Maurizio
>
> Samuel
>
>
> On 09/24/2018 10:37 PM, Maurizio Cimadamore wrote:
>> Having a pre-extracted stdlib bundle is something we have considered 
>> - quoting from [1]:
>>
>> "Now, since most (all?) of the libraries out there are going to assume
>> the availability of some 'standard library', let's also assume that some
>> extracted artifact for such library is available and that jextract
>> always knows how to find it - this is the equivalent of java.base for
>> the module system, or java.lang for the Java import system. This
>> addresses the bootstrapping issue."
>>
>> In time we'll get there, I don't see any real technical obstacles to 
>> get to your 'optimal' snippet.
>>
>> I think there are two aspects that I'd like to draw attention upon:
>>
>> 1) Magic does not come for free. E.g. it might "seem" that JNI has a 
>> more direct approach to calling native methods (ease of use issues 
>> aside). In reality it's just that the complexity of calling that 
>> native method, marshalling arguments, unmarshalling returns, dealing 
>> with native thread transitions and what's not has just been pushed 
>> under the JVM rug. So, yes, you can "just" call getpid - but the 
>> burden is on the VM. Now, the JNI support is already quite complex - 
>> I can't honestly imagine a sane way for the VM to support any given 
>> invocation scheme that a user might wish to see supported. This is 
>> why Panama is betting on Java code + layouts to do the lifting: that 
>> way the VM interface can be kept simple (which has significant 
>> payoffs - as the resulting code can be optimized much more - see the 
>> linkToNative experimental results in [2]).
>>
>> 2) As your example points out, while calling 'getpid' is something 
>> that seems 'easy' enough - 'puts' is already some other beast. It 
>> takes a pointer to some memory location where the string is stored. 
>> The JNI approach is to pass the Java string as is, and then do the 
>> wiring in native code. That is, there's no free lunch here - either 
>> you do the adaptation in Java, or you do it in native code (**). 
>> Panama gives you a rich enough API to do all such adaptations in 
>> Java, so that all native calls are... just native calls (again this 
>> means more regularity which means more performances). Having opaque 
>> native code snippets which do argument adaptation is not very optimal 
>> (and optimizable) for the JVM. With Panama you can create a 
>> good-looking API which internally uses pointers/scopes and delegates 
>> to the right native method - all done in Java. On top of that, our 
>> plans cover a so called 'civilization' layer (see [3]), by which 
>> users will be able to customize what comes out of jextract in order 
>> e.g. to tell that for 'puts' they really want a Java String argument 
>> and not a Pointer<Byte>; again this will be done in a more general 
>> way, so that the binder will be pointed at a pair of functions which 
>> can be used to map the user provided data to and from native code.
>>
>> (**) for an example of how interfacing with standard libraries needs 
>> some kind of wrapping, even in JNI - look at [4]; this file is 
>> essentially a collection of system calls which are wrapped by some 
>> logic (e.g. to check errno, ...). I claim that there is something 
>> _fundamentally_ wrong with code like this, in that the native code is 
>> mixing two concerns: (i) performing the required native call and (ii) 
>> adjusting input/output/errors of the call in a way that is suitable 
>> to the corresponding Java API. Why shouldn't the Java API itself be 
>> in charge of doing (ii) ?
>>
>> Maurizio
>>
>> [1] - 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-August/002560.html
>> [2] - 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html 
>>
>> [3] - 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-April/001537.html
>> [4] - 
>> http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c#l314 
>>
>>
>>
>>
>>
>> On 24/09/18 13:34, Jorn Vernee wrote:
>>> I agree with the usability point. In C++ it's as simple to call puts 
>>> as doing:
>>>
>>> ??? #include <stdio.h>
>>>
>>> ??? int main() {
>>> ??????? puts("Hello World!");
>>> ??? }
>>>
>>> And I think the optimal Java equivalent would be something like:
>>>
>>> ??? import static org.openjdk.stdio.*;
>>>
>>> ??? public class Main {
>>>
>>> ??????? public static void main(String[] args) {
>>> ??????????? puts("Hello World!");
>>> ??????? }
>>>
>>> ??? }
>>>
>>> This can be facilitated by creating a 'singleton facade' for the 
>>> library interface like so:
>>>
>>> ??? public class stdio {
>>>
>>> ??????? private static final stdioImpl lib = 
>>> Libraries.bind(lookup(), stdioImpl.class);
>>>
>>> ??????? public static int puts (String message) {
>>> ??????????? try(Scope scope = Scope.newNativeScope()) {
>>> ??????????????? Pointer<Byte> msg = scope.toCString(message);
>>> ??????????????? return lib.puts(msg);
>>> ??????????? }
>>> ??????? }
>>>
>>> ??????? ...
>>> ??? }
>>>
>>> Such a facade class could be shipped with the JDK or perhaps as an 
>>> artifact on maven central, or maybe it could be an additional output 
>>> of jextract.
>>>
>>> But there is only so much you can do automagically from the Java 
>>> side. When working from C/C++ you have the compiler filling in the 
>>> blanks. For instance, it automatically allocates storage for string 
>>> literals. Java does that as well for Java string literals `String s 
>>> = "Hello";`, but it can not do that for native strings, and you have 
>>> to use the Scope API to do that manually. In some cases, like the 
>>> above, you can write glue-code to make that automatic, but I think 
>>> at some point things become too complex for that, and there will 
>>> always be some usability barrier to interop.
>>>
>>> Jorn
>>>
>>> Samuel Audet schreef op 2018-09-24 13:38:
>>>> FWIW, I think the factory method pattern should be reconsidered
>>>> entirely. In C/C++, when we want to call say getpid(), we don't start
>>>> loading stuff up before calling getpid(), we call getpid()! Why not do
>>>> the same from Java? From a usability point of view, not loading stuff
>>>> manually works fine for JavaCPP...
>>>>
>>>> Now, I know you're going to start taking about interfaces and what
>>>> not. You said that you had plans to introduce an entirely new array
>>>> type just to make it more friendly with vector instructions and native
>>>> libraries. Why not start thinking about an "interface" that would be
>>>> friendly to native libraries as well? Why stop at arrays?
>>>>
>>>> Samuel
>>>>


From maurizio.cimadamore at oracle.com  Wed Sep 26 14:42:44 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 26 Sep 2018 15:42:44 +0100
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
In-Reply-To: <75153dbf40951b2594e092a3267d3834@xs4all.nl>
References: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
 <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>
 <75153dbf40951b2594e092a3267d3834@xs4all.nl>
Message-ID: <c45c2b1d-e0e0-82b7-e8db-139273dc9122@oracle.com>

I'll start the paperwork for [1]. I'll probably change it slightly to 
still use a constructor, so that there are less clashes with the ongoing 
direct invoker patch which also touches the same area. But what you did 
was sensible of course.

Regarding the other suggestions:

1) that looks sensible; right now we return a nullPointer() which as a 
References.ofVoid, so that's why you get the UOE. We could create a 
References.ofNull to be used specifically for nullPointer()

2) Yes, I'm aware of this issue; I think at this stage we don't want to 
make metadata changes - for when we will ready to rediscuss metadata 
again (soon!) I think we need to go for a more radical change here, as I 
don't think that pushing all symbol declarations on a toplevel 
annotation is helpful here. So stay tuned for more. Also, I have the 
feeling that now that the parser has improved, we could tackle this 
usability issue in another way - that is, by parsing the following grammar:

declarations ::= declaration [declarations]

declaration ::= symbolName '=' layout |
 ??????????????????????? symbolName '=' function

symbolName ::= Ident

In the past, when I tried to do this I faced issues with the scanner 
sometimes consuming token I didn't want, but then I completely revamped 
the code for parsing annotations (and other productions too), and this 
might improve things. It might be worth another try.

Thanks
Maurizio



On 25/09/18 19:33, Jorn Vernee wrote:
> Hi Maurizio,
> Thanks for looking at the patch.
>
> I have made the check simpler as you suggested [1]. Doing carrier <-> 
> layout checks in LayoutType makes sense to me.
>
> If you appreciate this kind of contribution I had 2 more things I ran 
> into;
>
> 1.) When a native function returns `null`, it seems that this value is 
> always boxed using `References#ofVoid`, and when you de-reference it 
> you get an UOE without a message. I'm not sure if boxing as Void is 
> intentional, but either way it would be nice if it gave a reason like 
> 'can not de-reference null pointer'.
>
> 2.) The way NativeHeader#declarations is parsed makes it so that a 
> faulty layout string like `exit(i32)v` (should be `exit=(i32)v`) is 
> silently ignored, and you get an AbstractMethodError when invoking the 
> method. Part of the problem I think, is that the parser splits the 
> declarations string by `=`, and then goes from there. I'm thinking a 
> more robust approach would be to make NativeHeader#declarations a 
> String[] instead of a String, and then try to parse exactly 1 
> declaration for each element. What do you think?
>
> I could try and make patches for those later this week.
>
> Jorn
>
> [1] : https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>
> Maurizio Cimadamore schreef op 2018-09-25 18:43:
>> Hi Jorn,
>> your patch looks good, but I think I would like it to just limit
>> things at a basic arity check (and void vs. non void check) in the
>> function. Given how we treat native types (layout + carrier), how
>> carriers are assigned to layouts is up to the method signature - if
>> somebody really wants to see a pointer as a float, so be it, although
>> I agree it looks silly.
>>
>> More generally, I think that if we go down the path of enforcing more
>> checks, we probably want to enforce carrier vs. layout compatibility
>> in a different place - that is LayoutType. That is the natural place
>> where to rule out ill-formed combinations.
>>
>> So there are two checks here: one is the one you needed, which checks
>> that a method signature is compatible with its function layout (same
>> arity, same varargness, same kind of return). The other is a more in
>> depth check performed on LayoutType, which I think is rather
>> orthogonal w.r.t. former. I'd suggest to go ahead with the first.
>>
>> Makes sense?
>>
>> Maurizio
>>
>>
>>
>> On 25/09/18 14:27, Jorn Vernee wrote:
>>> Hello,
>>>
>>> Thanks for accepting my last contribution!
>>>
>>> During testing I was running into an exception like this:
>>>
>>> ??? Exception in thread "main" 
>>> java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for 
>>> length 0
>>> ????at 
>>> java.base/jdk.internal.foreign.NativeInvoker.invokeNormal(NativeInvoker.java:216)
>>> ????at 
>>> java.base/jdk.internal.foreign.NativeInvoker.invoke(NativeInvoker.java:200)
>>> ????at 
>>> com.github.jdnl.Main$MyLib$Impl/0x000000010009c440.exit(Unknown Source)
>>> ????? at com.github.jdnl.Main.main(Main.java:17)
>>>
>>> The cause turned out to be a mismatch between the Java method 
>>> signature and the native function layout:
>>>
>>> ??? @NativeHeader(declarations =
>>> ??????? "exit=()v" // missing param
>>> ??? )
>>> ??? interface MyLib {
>>> ??????? void exit(int code);
>>> ??? }
>>>
>>> Since this exception is pretty cryptic, I'd like to contribute a 
>>> patch that adds an explicit check for this to NativeInvoker, which 
>>> does some basic verification that the two signatures match. With the 
>>> patch I get this error instead (note the last 'Caused by'):
>>>
>>> ??? Exception in thread "main" java.lang.RuntimeException: Failed to 
>>> generate implementation for class interface com.github.jdnl.Main$MyLib
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:65)
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:132)
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:124)
>>> ????at 
>>> java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:226)
>>> ????at 
>>> java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:208)
>>> ????at java.base/java.lang.ClassValue.get(ClassValue.java:114)
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper.getHeaderImplClass(LibrariesHelper.java:155)
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:241)
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:260)
>>> ????at java.base/java.foreign.Libraries.bind(Libraries.java:61)
>>> ????at com.github.jdnl.Main.main(Main.java:17)
>>> ??? Caused by: java.lang.RuntimeException: Failed to generate method 
>>> public abstract void com.github.jdnl.Main$MyLib.exit(int)
>>> ????at 
>>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:153)
>>> ????at 
>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMembers(HeaderImplGenerator.java:103)
>>> ????at 
>>> java.base/jdk.internal.foreign.BinderClassGenerator.generate(BinderClassGenerator.java:103)
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper.lambda$generateImpl$0(LibrariesHelper.java:62)
>>> ????at java.base/java.security.AccessController.doPrivileged(Native 
>>> Method)
>>> ????at 
>>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:61)
>>> ????... 10 more
>>> ??? Caused by: java.lang.IllegalArgumentException: Java method 
>>> signature and native layout not compatible: public abstract void 
>>> com.github.jdnl.Main$MyLib.exit(int) : ()v
>>> ????at 
>>> java.base/jdk.internal.foreign.NativeInvoker.of(NativeInvoker.java:127)
>>> ????at 
>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateFunctionMethod(HeaderImplGenerator.java:130)
>>> ????at 
>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMethodImplementation(HeaderImplGenerator.java:121)
>>> ????at 
>>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:150)
>>> ????... 15 more
>>>
>>> And it is thrown when binding the interface, while the original 
>>> exception was thrown when invoking the method.
>>>
>>> I have kept the implementation fairly permissive, so things like 
>>> binding `void m(Pointer<?> p)` to `(u64)v` (converting a pointer to 
>>> an integer) should still be allowed, but maybe a stricter approach 
>>> is preferable? I have added a test to verify that it catches some 
>>> basic mistakes like missing parameters or return types.
>>>
>>> Diff: 
>>> https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>>>
>>> I saw an RFR and JBS bug was made for my last contribution (thanks 
>>> Sundar!), so I've not added 'RFR' to the subject line this time.
>>>
>>> Regards,
>>> Jorn


From jbvernee at xs4all.nl  Wed Sep 26 15:34:22 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 26 Sep 2018 17:34:22 +0200
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
In-Reply-To: <c45c2b1d-e0e0-82b7-e8db-139273dc9122@oracle.com>
References: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
 <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>
 <75153dbf40951b2594e092a3267d3834@xs4all.nl>
 <c45c2b1d-e0e0-82b7-e8db-139273dc9122@oracle.com>
Message-ID: <cad02720b46a9ff90c9a186272baf318@xs4all.nl>

Maurizio Cimadamore schreef op 2018-09-26 16:42:
> I'll start the paperwork for [1]. I'll probably change it slightly to
> still use a constructor, so that there are less clashes with the
> ongoing direct invoker patch which also touches the same area. But
> what you did was sensible of course.
> 
> Regarding the other suggestions:
> 
> 1) that looks sensible; right now we return a nullPointer() which as a
> References.ofVoid, so that's why you get the UOE. We could create a
> References.ofNull to be used specifically for nullPointer()

Heh, I was just working on a patch which does exactly that :)

I also made it so that `nullPointer()` takes a LayoutType, so that a 
method returning e.g. `Pointer<Integer>` doesn't suddenly have a 
LayoutType with a Void carrier. Seeing the References.ofVoid anonymous 
class in the stack trace was also a source of confusion for me.

Also, a lot of places do casts to get at the carrier type of 
LayoutTypeImpl, and I've had to do that as well. I'm wondering if 
`Class<X> carrier()` should be a public method on LayoutType? Since a 
LayoutType seems to capture a mapping between a Java carrier type and a 
Layout, only being able to access the Layout through the public API 
feels a-symmetrical to me.

> 2) Yes, I'm aware of this issue; I think at this stage we don't want
> to make metadata changes - for when we will ready to rediscuss
> metadata again (soon!) I think we need to go for a more radical change
> here, as I don't think that pushing all symbol declarations on a
> toplevel annotation is helpful here. So stay tuned for more. Also, I
> have the feeling that now that the parser has improved, we could
> tackle this usability issue in another way - that is, by parsing the
> following grammar:
> 
> declarations ::= declaration [declarations]
> 
> declaration ::= symbolName '=' layout |
> ??????????????????????? symbolName '=' function
> 
> symbolName ::= Ident
> 
> In the past, when I tried to do this I faced issues with the scanner
> sometimes consuming token I didn't want, but then I completely
> revamped the code for parsing annotations (and other productions too),
> and this might improve things. It might be worth another try.

Cool! I'll stay tuned.

Jorn

> Thanks
> Maurizio
> 
> 
> 
> On 25/09/18 19:33, Jorn Vernee wrote:
>> Hi Maurizio,
>> Thanks for looking at the patch.
>> 
>> I have made the check simpler as you suggested [1]. Doing carrier <-> 
>> layout checks in LayoutType makes sense to me.
>> 
>> If you appreciate this kind of contribution I had 2 more things I ran 
>> into;
>> 
>> 1.) When a native function returns `null`, it seems that this value is 
>> always boxed using `References#ofVoid`, and when you de-reference it 
>> you get an UOE without a message. I'm not sure if boxing as Void is 
>> intentional, but either way it would be nice if it gave a reason like 
>> 'can not de-reference null pointer'.
>> 
>> 2.) The way NativeHeader#declarations is parsed makes it so that a 
>> faulty layout string like `exit(i32)v` (should be `exit=(i32)v`) is 
>> silently ignored, and you get an AbstractMethodError when invoking the 
>> method. Part of the problem I think, is that the parser splits the 
>> declarations string by `=`, and then goes from there. I'm thinking a 
>> more robust approach would be to make NativeHeader#declarations a 
>> String[] instead of a String, and then try to parse exactly 1 
>> declaration for each element. What do you think?
>> 
>> I could try and make patches for those later this week.
>> 
>> Jorn
>> 
>> [1] : 
>> https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>> 
>> Maurizio Cimadamore schreef op 2018-09-25 18:43:
>>> Hi Jorn,
>>> your patch looks good, but I think I would like it to just limit
>>> things at a basic arity check (and void vs. non void check) in the
>>> function. Given how we treat native types (layout + carrier), how
>>> carriers are assigned to layouts is up to the method signature - if
>>> somebody really wants to see a pointer as a float, so be it, although
>>> I agree it looks silly.
>>> 
>>> More generally, I think that if we go down the path of enforcing more
>>> checks, we probably want to enforce carrier vs. layout compatibility
>>> in a different place - that is LayoutType. That is the natural place
>>> where to rule out ill-formed combinations.
>>> 
>>> So there are two checks here: one is the one you needed, which checks
>>> that a method signature is compatible with its function layout (same
>>> arity, same varargness, same kind of return). The other is a more in
>>> depth check performed on LayoutType, which I think is rather
>>> orthogonal w.r.t. former. I'd suggest to go ahead with the first.
>>> 
>>> Makes sense?
>>> 
>>> Maurizio
>>> 
>>> 
>>> 
>>> On 25/09/18 14:27, Jorn Vernee wrote:
>>>> Hello,
>>>> 
>>>> Thanks for accepting my last contribution!
>>>> 
>>>> During testing I was running into an exception like this:
>>>> 
>>>> ??? Exception in thread "main" 
>>>> java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for 
>>>> length 0
>>>> ????at 
>>>> java.base/jdk.internal.foreign.NativeInvoker.invokeNormal(NativeInvoker.java:216)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.NativeInvoker.invoke(NativeInvoker.java:200)
>>>> ????at 
>>>> com.github.jdnl.Main$MyLib$Impl/0x000000010009c440.exit(Unknown 
>>>> Source)
>>>> ????? at com.github.jdnl.Main.main(Main.java:17)
>>>> 
>>>> The cause turned out to be a mismatch between the Java method 
>>>> signature and the native function layout:
>>>> 
>>>> ??? @NativeHeader(declarations =
>>>> ??????? "exit=()v" // missing param
>>>> ??? )
>>>> ??? interface MyLib {
>>>> ??????? void exit(int code);
>>>> ??? }
>>>> 
>>>> Since this exception is pretty cryptic, I'd like to contribute a 
>>>> patch that adds an explicit check for this to NativeInvoker, which 
>>>> does some basic verification that the two signatures match. With the 
>>>> patch I get this error instead (note the last 'Caused by'):
>>>> 
>>>> ??? Exception in thread "main" java.lang.RuntimeException: Failed to 
>>>> generate implementation for class interface 
>>>> com.github.jdnl.Main$MyLib
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:65)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:132)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:124)
>>>> ????at 
>>>> java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:226)
>>>> ????at 
>>>> java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:208)
>>>> ????at java.base/java.lang.ClassValue.get(ClassValue.java:114)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper.getHeaderImplClass(LibrariesHelper.java:155)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:241)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:260)
>>>> ????at java.base/java.foreign.Libraries.bind(Libraries.java:61)
>>>> ????at com.github.jdnl.Main.main(Main.java:17)
>>>> ??? Caused by: java.lang.RuntimeException: Failed to generate method 
>>>> public abstract void com.github.jdnl.Main$MyLib.exit(int)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:153)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMembers(HeaderImplGenerator.java:103)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.BinderClassGenerator.generate(BinderClassGenerator.java:103)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper.lambda$generateImpl$0(LibrariesHelper.java:62)
>>>> ????at java.base/java.security.AccessController.doPrivileged(Native 
>>>> Method)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:61)
>>>> ????... 10 more
>>>> ??? Caused by: java.lang.IllegalArgumentException: Java method 
>>>> signature and native layout not compatible: public abstract void 
>>>> com.github.jdnl.Main$MyLib.exit(int) : ()v
>>>> ????at 
>>>> java.base/jdk.internal.foreign.NativeInvoker.of(NativeInvoker.java:127)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateFunctionMethod(HeaderImplGenerator.java:130)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMethodImplementation(HeaderImplGenerator.java:121)
>>>> ????at 
>>>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:150)
>>>> ????... 15 more
>>>> 
>>>> And it is thrown when binding the interface, while the original 
>>>> exception was thrown when invoking the method.
>>>> 
>>>> I have kept the implementation fairly permissive, so things like 
>>>> binding `void m(Pointer<?> p)` to `(u64)v` (converting a pointer to 
>>>> an integer) should still be allowed, but maybe a stricter approach 
>>>> is preferable? I have added a test to verify that it catches some 
>>>> basic mistakes like missing parameters or return types.
>>>> 
>>>> Diff: 
>>>> https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>>>> 
>>>> I saw an RFR and JBS bug was made for my last contribution (thanks 
>>>> Sundar!), so I've not added 'RFR' to the subject line this time.
>>>> 
>>>> Regards,
>>>> Jorn

From maurizio.cimadamore at oracle.com  Wed Sep 26 15:49:43 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 26 Sep 2018 16:49:43 +0100
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
In-Reply-To: <cad02720b46a9ff90c9a186272baf318@xs4all.nl>
References: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
 <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>
 <75153dbf40951b2594e092a3267d3834@xs4all.nl>
 <c45c2b1d-e0e0-82b7-e8db-139273dc9122@oracle.com>
 <cad02720b46a9ff90c9a186272baf318@xs4all.nl>
Message-ID: <e0ab47fe-adeb-0c51-8bad-e095d05acfb6@oracle.com>



On 26/09/18 16:34, Jorn Vernee wrote:
> Maurizio Cimadamore schreef op 2018-09-26 16:42:
>> I'll start the paperwork for [1]. I'll probably change it slightly to
>> still use a constructor, so that there are less clashes with the
>> ongoing direct invoker patch which also touches the same area. But
>> what you did was sensible of course.
>>
>> Regarding the other suggestions:
>>
>> 1) that looks sensible; right now we return a nullPointer() which as a
>> References.ofVoid, so that's why you get the UOE. We could create a
>> References.ofNull to be used specifically for nullPointer()
>
> Heh, I was just working on a patch which does exactly that :)
>
> I also made it so that `nullPointer()` takes a LayoutType, so that a 
> method returning e.g. `Pointer<Integer>` doesn't suddenly have a 
> LayoutType with a Void carrier. Seeing the References.ofVoid anonymous 
> class in the stack trace was also a source of confusion for me.
Not sure you need to add a parameter to nullPointer for that. I think 
there should be only one null pointer, and we should use unchecked 
conversion internally to cast it to the type the user wants - 
dynamically the carrier will not be used - since the Reference attached 
to that will prevent dereference ops.
>
> Also, a lot of places do casts to get at the carrier type of 
> LayoutTypeImpl, and I've had to do that as well. I'm wondering if 
> `Class<X> carrier()` should be a public method on LayoutType? Since a 
> LayoutType seems to capture a mapping between a Java carrier type and 
> a Layout, only being able to access the Layout through the public API 
> feels a-symmetrical to me.
The reason why carrier is hidden is that it's only part of the story - 
e.g. all pointers of pointers will have Pointer.class a carrier in the 
layout type, but some are really Pointer<Integer> some are Pointer<Byte> 
- that is a j.l.Class is not enough. And I wanted to avoid committing to 
java.lang.reflect.Type too much in this API. I get what you mean about 
asymmetry, but my feeling is that client code doesn't probably care much 
about the carrier component - the part that clients mainly care about is 
in having the carrier reflected in the generic type e.g. Pointer<X> - so 
the carrier is there, but only as a source of static safety.

Of course if you write generic code that works on any pointer Pointer<?> 
(as the binder does) then you might need to look at the carrier, but I 
wanted to wait and see what these use cases looked like first. Also, 
this is an area where pattern matching (when available) will help a lot:

Pointer<?> pointer = ...
switch (pointer.type) {
 ??? case Pointer(Pointer(int.class)): ...
}

(probably similar improvements will help with layout API as well).

Thanks
Maurizio
>
>> 2) Yes, I'm aware of this issue; I think at this stage we don't want
>> to make metadata changes - for when we will ready to rediscuss
>> metadata again (soon!) I think we need to go for a more radical change
>> here, as I don't think that pushing all symbol declarations on a
>> toplevel annotation is helpful here. So stay tuned for more. Also, I
>> have the feeling that now that the parser has improved, we could
>> tackle this usability issue in another way - that is, by parsing the
>> following grammar:
>>
>> declarations ::= declaration [declarations]
>>
>> declaration ::= symbolName '=' layout |
>> ??????????????????????? symbolName '=' function
>>
>> symbolName ::= Ident
>>
>> In the past, when I tried to do this I faced issues with the scanner
>> sometimes consuming token I didn't want, but then I completely
>> revamped the code for parsing annotations (and other productions too),
>> and this might improve things. It might be worth another try.
>
> Cool! I'll stay tuned.
>
> Jorn
>
>> Thanks
>> Maurizio
>>
>>
>>
>> On 25/09/18 19:33, Jorn Vernee wrote:
>>> Hi Maurizio,
>>> Thanks for looking at the patch.
>>>
>>> I have made the check simpler as you suggested [1]. Doing carrier 
>>> <-> layout checks in LayoutType makes sense to me.
>>>
>>> If you appreciate this kind of contribution I had 2 more things I 
>>> ran into;
>>>
>>> 1.) When a native function returns `null`, it seems that this value 
>>> is always boxed using `References#ofVoid`, and when you de-reference 
>>> it you get an UOE without a message. I'm not sure if boxing as Void 
>>> is intentional, but either way it would be nice if it gave a reason 
>>> like 'can not de-reference null pointer'.
>>>
>>> 2.) The way NativeHeader#declarations is parsed makes it so that a 
>>> faulty layout string like `exit(i32)v` (should be `exit=(i32)v`) is 
>>> silently ignored, and you get an AbstractMethodError when invoking 
>>> the method. Part of the problem I think, is that the parser splits 
>>> the declarations string by `=`, and then goes from there. I'm 
>>> thinking a more robust approach would be to make 
>>> NativeHeader#declarations a String[] instead of a String, and then 
>>> try to parse exactly 1 declaration for each element. What do you think?
>>>
>>> I could try and make patches for those later this week.
>>>
>>> Jorn
>>>
>>> [1] : 
>>> https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>>>
>>> Maurizio Cimadamore schreef op 2018-09-25 18:43:
>>>> Hi Jorn,
>>>> your patch looks good, but I think I would like it to just limit
>>>> things at a basic arity check (and void vs. non void check) in the
>>>> function. Given how we treat native types (layout + carrier), how
>>>> carriers are assigned to layouts is up to the method signature - if
>>>> somebody really wants to see a pointer as a float, so be it, although
>>>> I agree it looks silly.
>>>>
>>>> More generally, I think that if we go down the path of enforcing more
>>>> checks, we probably want to enforce carrier vs. layout compatibility
>>>> in a different place - that is LayoutType. That is the natural place
>>>> where to rule out ill-formed combinations.
>>>>
>>>> So there are two checks here: one is the one you needed, which checks
>>>> that a method signature is compatible with its function layout (same
>>>> arity, same varargness, same kind of return). The other is a more in
>>>> depth check performed on LayoutType, which I think is rather
>>>> orthogonal w.r.t. former. I'd suggest to go ahead with the first.
>>>>
>>>> Makes sense?
>>>>
>>>> Maurizio
>>>>
>>>>
>>>>
>>>> On 25/09/18 14:27, Jorn Vernee wrote:
>>>>> Hello,
>>>>>
>>>>> Thanks for accepting my last contribution!
>>>>>
>>>>> During testing I was running into an exception like this:
>>>>>
>>>>> ??? Exception in thread "main" 
>>>>> java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds 
>>>>> for length 0
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.NativeInvoker.invokeNormal(NativeInvoker.java:216)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.NativeInvoker.invoke(NativeInvoker.java:200)
>>>>> ????at 
>>>>> com.github.jdnl.Main$MyLib$Impl/0x000000010009c440.exit(Unknown 
>>>>> Source)
>>>>> ????? at com.github.jdnl.Main.main(Main.java:17)
>>>>>
>>>>> The cause turned out to be a mismatch between the Java method 
>>>>> signature and the native function layout:
>>>>>
>>>>> ??? @NativeHeader(declarations =
>>>>> ??????? "exit=()v" // missing param
>>>>> ??? )
>>>>> ??? interface MyLib {
>>>>> ??????? void exit(int code);
>>>>> ??? }
>>>>>
>>>>> Since this exception is pretty cryptic, I'd like to contribute a 
>>>>> patch that adds an explicit check for this to NativeInvoker, which 
>>>>> does some basic verification that the two signatures match. With 
>>>>> the patch I get this error instead (note the last 'Caused by'):
>>>>>
>>>>> ??? Exception in thread "main" java.lang.RuntimeException: Failed 
>>>>> to generate implementation for class interface 
>>>>> com.github.jdnl.Main$MyLib
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:65)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:132)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper$3.computeValue(LibrariesHelper.java:124)
>>>>> ????at 
>>>>> java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:226)
>>>>> ????at 
>>>>> java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:208)
>>>>> ????at java.base/java.lang.ClassValue.get(ClassValue.java:114)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper.getHeaderImplClass(LibrariesHelper.java:155)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:241)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper.bind(LibrariesHelper.java:260)
>>>>> ????at java.base/java.foreign.Libraries.bind(Libraries.java:61)
>>>>> ????at com.github.jdnl.Main.main(Main.java:17)
>>>>> ??? Caused by: java.lang.RuntimeException: Failed to generate 
>>>>> method public abstract void com.github.jdnl.Main$MyLib.exit(int)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:153)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMembers(HeaderImplGenerator.java:103)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.BinderClassGenerator.generate(BinderClassGenerator.java:103)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper.lambda$generateImpl$0(LibrariesHelper.java:62)
>>>>> ????at 
>>>>> java.base/java.security.AccessController.doPrivileged(Native Method)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.LibrariesHelper.generateImpl(LibrariesHelper.java:61)
>>>>> ????... 10 more
>>>>> ??? Caused by: java.lang.IllegalArgumentException: Java method 
>>>>> signature and native layout not compatible: public abstract void 
>>>>> com.github.jdnl.Main$MyLib.exit(int) : ()v
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.NativeInvoker.of(NativeInvoker.java:127) 
>>>>>
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateFunctionMethod(HeaderImplGenerator.java:130)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.HeaderImplGenerator.generateMethodImplementation(HeaderImplGenerator.java:121)
>>>>> ????at 
>>>>> java.base/jdk.internal.foreign.BinderClassGenerator.generateMembers(BinderClassGenerator.java:150)
>>>>> ????... 15 more
>>>>>
>>>>> And it is thrown when binding the interface, while the original 
>>>>> exception was thrown when invoking the method.
>>>>>
>>>>> I have kept the implementation fairly permissive, so things like 
>>>>> binding `void m(Pointer<?> p)` to `(u64)v` (converting a pointer 
>>>>> to an integer) should still be allowed, but maybe a stricter 
>>>>> approach is preferable? I have added a test to verify that it 
>>>>> catches some basic mistakes like missing parameters or return types.
>>>>>
>>>>> Diff: 
>>>>> https://gist.github.com/JornVernee/2fb68d70bad1aa807b50b77840117b27
>>>>>
>>>>> I saw an RFR and JBS bug was made for my last contribution (thanks 
>>>>> Sundar!), so I've not added 'RFR' to the subject line this time.
>>>>>
>>>>> Regards,
>>>>> Jorn


From jbvernee at xs4all.nl  Wed Sep 26 17:31:38 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Wed, 26 Sep 2018 19:31:38 +0200
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
In-Reply-To: <e0ab47fe-adeb-0c51-8bad-e095d05acfb6@oracle.com>
References: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
 <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>
 <75153dbf40951b2594e092a3267d3834@xs4all.nl>
 <c45c2b1d-e0e0-82b7-e8db-139273dc9122@oracle.com>
 <cad02720b46a9ff90c9a186272baf318@xs4all.nl>
 <e0ab47fe-adeb-0c51-8bad-e095d05acfb6@oracle.com>
Message-ID: <0bfa7d55d1ab32c5c2afbd343cc9b7d8@xs4all.nl>

Maurizio Cimadamore schreef op 2018-09-26 17:49:
>> I also made it so that `nullPointer()` takes a LayoutType, so that a 
>> method returning e.g. `Pointer<Integer>` doesn't suddenly have a 
>> LayoutType with a Void carrier. Seeing the References.ofVoid anonymous 
>> class in the stack trace was also a source of confusion for me.
> Not sure you need to add a parameter to nullPointer for that. I think
> there should be only one null pointer, and we should use unchecked
> conversion internally to cast it to the type the user wants -
> dynamically the carrier will not be used - since the Reference
> attached to that will prevent dereference ops.

Yeah, you're right, it wasn't needed in the end [1]. I did create a 
BoundedPointer.nullPointer() as well so I wouldn't have to make the 
LayoutTypeImpl constructor public. That also allowed me to simplify the 
type variable declaration, since the one place that needed a null 
BoundedPointer can now just call BoundedPointer.nullPointer().

I also removed the type parameter from the carrier field in 
LayoutTypeImpl, since it didn't really seem to be used, as 
LayoutTypeImpl.carrier() just returned a Class<?>. That allowed me to 
avoid doing an unchecked cast in BoundedPointer.nullPointer().

Jorn

[1] : 
https://gist.github.com/JornVernee/7727db9b8ebdbb5c88205e3bbe97227c


From maurizio.cimadamore at oracle.com  Wed Sep 26 19:54:46 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 26 Sep 2018 20:54:46 +0100
Subject: [foreign] Improve error reporting when Java signature and native
 function layout don't match.
In-Reply-To: <0bfa7d55d1ab32c5c2afbd343cc9b7d8@xs4all.nl>
References: <d2ffdee9ea116abb888ade5c6c36e363@xs4all.nl>
 <c68031ed-591c-e52e-2fc6-ad7ee550b157@oracle.com>
 <75153dbf40951b2594e092a3267d3834@xs4all.nl>
 <c45c2b1d-e0e0-82b7-e8db-139273dc9122@oracle.com>
 <cad02720b46a9ff90c9a186272baf318@xs4all.nl>
 <e0ab47fe-adeb-0c51-8bad-e095d05acfb6@oracle.com>
 <0bfa7d55d1ab32c5c2afbd343cc9b7d8@xs4all.nl>
Message-ID: <b649d68e-0352-ffa6-b945-d7428378c6dc@oracle.com>

Looks very good, thanks for fixing the javadoc too.

I will prepare an RFR for this too (probably tomorrow).

Maurizio


On 26/09/18 18:31, Jorn Vernee wrote:
> Maurizio Cimadamore schreef op 2018-09-26 17:49:
>>> I also made it so that `nullPointer()` takes a LayoutType, so that a 
>>> method returning e.g. `Pointer<Integer>` doesn't suddenly have a 
>>> LayoutType with a Void carrier. Seeing the References.ofVoid 
>>> anonymous class in the stack trace was also a source of confusion 
>>> for me.
>> Not sure you need to add a parameter to nullPointer for that. I think
>> there should be only one null pointer, and we should use unchecked
>> conversion internally to cast it to the type the user wants -
>> dynamically the carrier will not be used - since the Reference
>> attached to that will prevent dereference ops.
>
> Yeah, you're right, it wasn't needed in the end [1]. I did create a 
> BoundedPointer.nullPointer() as well so I wouldn't have to make the 
> LayoutTypeImpl constructor public. That also allowed me to simplify 
> the type variable declaration, since the one place that needed a null 
> BoundedPointer can now just call BoundedPointer.nullPointer().
>
> I also removed the type parameter from the carrier field in 
> LayoutTypeImpl, since it didn't really seem to be used, as 
> LayoutTypeImpl.carrier() just returned a Class<?>. That allowed me to 
> avoid doing an unchecked cast in BoundedPointer.nullPointer().
>
> Jorn
>
> [1] : https://gist.github.com/JornVernee/7727db9b8ebdbb5c88205e3bbe97227c
>


From maurizio.cimadamore at oracle.com  Wed Sep 26 20:03:22 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 26 Sep 2018 20:03:22 +0000
Subject: hg: panama/dev: 81 new changesets
Message-ID: <201809262003.w8QK3SKX013413@aojmv0008.oracle.com>

Changeset: 3f32076a43a6
Author:    ihse
Date:      2018-09-19 22:42 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/3f32076a43a6

8210919: Remove statically linked libjli on Windows
Reviewed-by: erikj

! make/ExplodedImageOptimize.gmk
! make/launcher/Launcher-java.base.gmk
! make/launcher/LauncherCommon.gmk
! make/lib/CoreLibraries.gmk
! make/lib/Lib-java.instrument.gmk

Changeset: 61f62fb4d4bf
Author:    zgu
Date:      2018-09-19 16:51 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/61f62fb4d4bf

8210879: ClassLoaderStatsClosure does raw oop comparison
Summary: Uses oopDesc::equals() for comparing oop equality
Reviewed-by: shade

! src/hotspot/share/classfile/classLoaderStats.hpp

Changeset: 414bd559dba8
Author:    jwilhelm
Date:      2018-09-20 00:16 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/414bd559dba8

Added tag jdk-12+12 for changeset 15094d12a632

! .hgtags

Changeset: 4be08a5102b0
Author:    ihse
Date:      2018-09-20 00:19 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/4be08a5102b0

8210924: Remove PACKAGE_PATH
Reviewed-by: tbell

! make/autoconf/spec.gmk.in
! make/autoconf/toolchain.m4
! make/launcher/LauncherCommon.gmk
! make/lib/Awt2dLibraries.gmk
! make/lib/CoreLibraries.gmk
! src/java.base/macosx/native/libjli/java_md_macosx.h
! src/java.base/unix/native/libjli/java_md_solinux.h

Changeset: 30e6079a9a12
Author:    jnimeh
Date:      2018-09-19 16:07 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/30e6079a9a12

8210846: TLSv.1.3 interop problems with OpenSSL 1.1.1 when used on the client side with mutual auth
Reviewed-by: wetmore

! src/java.base/share/classes/sun/security/ssl/SignatureAlgorithmsExtension.java

Changeset: cfa50d6a6fba
Author:    kbarrett
Date:      2018-09-19 20:07 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/cfa50d6a6fba

8210889: Some service thread cleanups can be starved
Summary: Do all available work on each iteration.
Reviewed-by: pliden, tschatzl, coleenp

! src/hotspot/share/runtime/serviceThread.cpp

Changeset: 43668e3cae4d
Author:    ihse
Date:      2018-09-20 08:59 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/43668e3cae4d

8210920: Native C++ tests are not using CXXFLAGS
Reviewed-by: tbell, erikj

! make/autoconf/flags-cflags.m4
! make/autoconf/flags-ldflags.m4
! make/autoconf/spec.gmk.in
! make/common/TestFilesCompilation.gmk

Changeset: d23d7389142f
Author:    phedlin
Date:      2018-09-12 14:08 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/d23d7389142f

8210284: "assert((av & 0x00000001) == 0) failed: unsupported V8" on Solaris 11.4
Summary: Sanity checks on V8 legacy properties removed.
Reviewed-by: neliasso, eosterlund, kvn

! src/hotspot/os_cpu/solaris_sparc/vm_version_solaris_sparc.cpp

Changeset: 1fd0f300d4b7
Author:    coleenp
Date:      2018-09-20 08:11 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/1fd0f300d4b7

8202201: All oop stores in the x64 interpreter are treated as volatile when using G1
Summary: ran out of registers, generated volatile and non-volatile branches.
Reviewed-by: eosterlund, dholmes

! src/hotspot/cpu/x86/templateTable_x86.cpp
! src/hotspot/cpu/x86/templateTable_x86.hpp

Changeset: 46eac084082d
Author:    pliden
Date:      2018-09-20 14:04 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/46eac084082d

8210857: Allow retiring TLABs and collecting statistics in parallel
Reviewed-by: sjohanss, eosterlund

! src/hotspot/share/gc/shared/collectedHeap.cpp
! src/hotspot/share/gc/shared/memAllocator.cpp
! src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp
! src/hotspot/share/gc/shared/threadLocalAllocBuffer.hpp
! src/hotspot/share/runtime/thread.cpp

Changeset: 75e4ce0fa1ba
Author:    pliden
Date:      2018-09-20 14:04 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/75e4ce0fa1ba

8210883: ZGC: Parallel retire/resize/remap of TLABs
Reviewed-by: eosterlund

! src/hotspot/share/gc/z/zCollectedHeap.cpp
! src/hotspot/share/gc/z/zCollectedHeap.hpp
! src/hotspot/share/gc/z/zHeap.cpp
! src/hotspot/share/gc/z/zInitialize.cpp
! src/hotspot/share/gc/z/zMark.cpp
! src/hotspot/share/gc/z/zObjectAllocator.cpp
! src/hotspot/share/gc/z/zObjectAllocator.hpp
! src/hotspot/share/gc/z/zRelocate.cpp
+ src/hotspot/share/gc/z/zStatTLAB.cpp
+ src/hotspot/share/gc/z/zStatTLAB.hpp
! src/hotspot/share/gc/z/zValue.hpp

Changeset: f12165de3cc0
Author:    ihse
Date:      2018-09-20 18:33 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f12165de3cc0

8210949: Stop filtering out -xc99=%none for liblcms
Reviewed-by: erikj

! make/lib/Awt2dLibraries.gmk

Changeset: 74c67f87fe80
Author:    ihse
Date:      2018-09-20 18:37 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/74c67f87fe80

8210944: Stop replacing -MD with -MT in libwindowsaccessbridge
Reviewed-by: erikj

! make/lib/Lib-jdk.accessibility.gmk

Changeset: 844cd8887372
Author:    ihse
Date:      2018-09-20 18:38 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/844cd8887372

8210941: Stop filtering out -xregs=no%appl for libsunec
Reviewed-by: erikj

! make/lib/Lib-jdk.crypto.ec.gmk

Changeset: f3c1945fa8aa
Author:    ihse
Date:      2018-09-20 18:39 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/f3c1945fa8aa

8210960: Allow --with-boot-jdk-jvmargs to work during configure
Reviewed-by: erikj

! make/autoconf/basics.m4
! make/autoconf/boot-jdk.m4

Changeset: 2a51125b2794
Author:    iklam
Date:      2018-09-18 21:47 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/2a51125b2794

8210875: Refactor CompactHashtable
Reviewed-by: ccheung, jiangli

! src/hotspot/share/classfile/compactHashtable.cpp
! src/hotspot/share/classfile/compactHashtable.hpp
- src/hotspot/share/classfile/compactHashtable.inline.hpp
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/classfile/stringTable.cpp
! src/hotspot/share/classfile/stringTable.hpp
! src/hotspot/share/classfile/symbolTable.cpp
! src/hotspot/share/classfile/symbolTable.hpp
! src/hotspot/share/classfile/systemDictionaryShared.cpp
! src/hotspot/share/memory/filemap.cpp
! src/hotspot/share/services/diagnosticCommand.cpp
! src/hotspot/share/utilities/utf8.cpp
! src/hotspot/share/utilities/utf8.hpp

Changeset: 9777d724ace8
Author:    ihse
Date:      2018-09-20 20:54 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/9777d724ace8

8210962: Deprecate jdk-variant
Reviewed-by: shade, erikj

! make/autoconf/basics.m4
! make/autoconf/configure.ac
! make/autoconf/jdk-options.m4
! make/autoconf/spec.gmk.in

Changeset: e3632b4706c4
Author:    ihse
Date:      2018-09-20 21:05 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/e3632b4706c4

8210931: JLI and launchers normalization and cleanup
Reviewed-by: alanb, erikj

! make/MacBundles.gmk
! make/launcher/Launcher-java.base.gmk
! make/launcher/Launcher-jdk.jconsole.gmk
! make/launcher/LauncherCommon.gmk
! make/lib/CoreLibraries.gmk
! make/lib/Lib-java.base.gmk
! make/lib/Lib-java.instrument.gmk
- src/java.base/macosx/native/libjli/java_md_macosx.c
+ src/java.base/macosx/native/libjli/java_md_macosx.m

Changeset: e777e997e7c1
Author:    shade
Date:      2018-09-20 21:14 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/e777e997e7c1

8210963: Build failures after "8210829: Modularize allocations in C2"
Reviewed-by: rkennke, thartmann

! src/hotspot/share/gc/shared/c2/barrierSetC2.cpp
! src/hotspot/share/opto/macro.cpp

Changeset: 07179f7db03d
Author:    vromero
Date:      2018-09-20 12:49 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/07179f7db03d

8209022: Missing checkcast when casting to type parameter bounded by intersection type
Reviewed-by: mcimadamore

! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java
+ test/langtools/tools/javac/typeVariableCast/TypeVariableCastTest.java

Changeset: a7448f4f89a0
Author:    roland
Date:      2018-09-12 16:38 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/a7448f4f89a0

8210390: C2 still crashes with "assert(mode == ControlAroundStripMined && use == sfpt) failed: missed a node"
Reviewed-by: thartmann, kvn

! src/hotspot/share/opto/loopopts.cpp
+ test/hotspot/jtreg/compiler/loopstripmining/StripMinedLoopReorgOffsets.java

Changeset: c26fbf1434c4
Author:    xuelei
Date:      2018-09-20 14:19 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/c26fbf1434c4

8210974: No extensions debug log for ClientHello
Reviewed-by: jnimeh, wetmore

! src/java.base/share/classes/sun/security/ssl/SSLExtensions.java

Changeset: d2c72de3cf83
Author:    xyin
Date:      2018-09-21 15:49 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/d2c72de3cf83

8199931: java/net/MulticastSocket/UnreferencedMulticastSockets.java fails with "incorrect data received"
Reviewed-by: chegar

! test/jdk/java/net/MulticastSocket/UnreferencedMulticastSockets.java

Changeset: ec03768578c2
Author:    xyin
Date:      2018-09-21 16:13 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/ec03768578c2

8169246: java/net/DatagramSocket/ReportSocketClosed.java fails intermittently with BindException
Reviewed-by: chegar

! test/jdk/java/net/DatagramSocket/ReportSocketClosed.java

Changeset: bf1d479fe7eb
Author:    jlahoda
Date:      2018-09-21 12:29 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/bf1d479fe7eb

8209865: Incorrect 'multiple elements' notes with Elements#getTypeElement and --release
Summary: Changing ct.sym to be module-path oriented, rather than class-path oriented.
Reviewed-by: jjg

! make/langtools/src/classes/build/tools/symbolgenerator/CreateSymbols.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/platform/JDKPlatformProvider.java
+ test/langtools/tools/javac/platform/ReleaseModulesAndTypeElement.java

Changeset: 91fd24cf57d5
Author:    jlahoda
Date:      2018-09-21 12:29 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/91fd24cf57d5

8209058: Cannot find annotation method 'value()' in type 'Profile+Annotation'
Summary: Correct detection of the Profile+Annotation synthetic annotation for classes that are not from the java.base module.
Reviewed-by: jjg

! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java
+ test/langtools/tools/javac/platform/NoProfileAnnotationWarning.java

Changeset: 5dd9f3ac52a4
Author:    mbaesken
Date:      2018-09-20 13:59 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/5dd9f3ac52a4

8210961: [aix] enhance list of environment variables reported in error log file on AIX
Reviewed-by: clanger, simonis

! src/hotspot/share/utilities/vmError.cpp

Changeset: b177af763b82
Author:    tschatzl
Date:      2018-09-21 15:11 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b177af763b82

8210557: G1 next bitmap verification at the end of concurrent mark sometimes fails
Summary: Removed unnecessary verification that can cause spurious false alarm.
Reviewed-by: sjohanss, kbarrett

! src/hotspot/share/gc/g1/g1CollectedHeap.cpp
! src/hotspot/share/gc/g1/g1ConcurrentMark.cpp
! src/hotspot/share/gc/g1/g1ConcurrentMark.hpp
! src/hotspot/share/gc/g1/g1ConcurrentMarkThread.cpp

Changeset: d62ebdfd8f18
Author:    amenkov
Date:      2018-09-21 10:18 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/d62ebdfd8f18

8210725: com/sun/jdi/RedefineClearBreakpoint.java fails with waitForPrompt timed out after 60 seconds
8210748: [TESTBUG] lib.jdb.Jdb.waitForPrompt() should clarify which output is the pending reply after a timeout
Reviewed-by: jcbeyler, gadams, sspitsyn

! test/jdk/com/sun/jdi/lib/jdb/Jdb.java
! test/jdk/com/sun/jdi/lib/jdb/JdbTest.java

Changeset: 46ca82c15f6c
Author:    amenkov
Date:      2018-09-21 11:28 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/46ca82c15f6c

8210760: [TEST] rewrite com/sun/jdi shell tests to java version - step4
Reviewed-by: jcbeyler, sspitsyn, cjplummer

! test/jdk/ProblemList.txt
+ test/jdk/com/sun/jdi/RedefineException.java
- test/jdk/com/sun/jdi/RedefineException.sh
+ test/jdk/com/sun/jdi/RedefineFinal.java
- test/jdk/com/sun/jdi/RedefineFinal.sh
+ test/jdk/com/sun/jdi/RedefineIntConstantToLong.java
- test/jdk/com/sun/jdi/RedefineIntConstantToLong.sh
+ test/jdk/com/sun/jdi/RedefineMulti.java
- test/jdk/com/sun/jdi/RedefineMulti.sh
+ test/jdk/com/sun/jdi/RedefinePop.java
- test/jdk/com/sun/jdi/RedefinePop.sh
+ test/jdk/com/sun/jdi/RedefineStep.java
- test/jdk/com/sun/jdi/RedefineStep.sh
+ test/jdk/com/sun/jdi/RedefineTTYLineNumber.java
- test/jdk/com/sun/jdi/RedefineTTYLineNumber.sh
+ test/jdk/com/sun/jdi/StringConvertTest.java
- test/jdk/com/sun/jdi/StringConvertTest.sh
+ test/jdk/com/sun/jdi/WatchFramePop.java
- test/jdk/com/sun/jdi/WatchFramePop.sh

Changeset: f8af1f1f3518
Author:    tbell
Date:      2018-09-21 12:08 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/f8af1f1f3518

8190985: .jcheck/conf files contain 'project=jdk10'
Reviewed-by: mr, iris, erikj

! .jcheck/conf

Changeset: ab54a4d61d7f
Author:    ihse
Date:      2018-09-21 21:35 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/ab54a4d61d7f

8210988: Improved handling of compiler warnings in the build
Reviewed-by: erikj

! make/autoconf/flags-cflags.m4
! make/autoconf/flags-ldflags.m4
! make/autoconf/spec.gmk.in
! make/common/NativeCompilation.gmk
! make/hotspot/lib/CompileGtest.gmk
! make/hotspot/lib/CompileJvm.gmk

Changeset: dc15e45122b2
Author:    iignatyev
Date:      2018-09-21 14:50 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/dc15e45122b2

8210894: remove jdk/testlibrary/Asserts
Reviewed-by: serb

! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/AddReads.java
! test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/ExportModule.java
! test/jdk/com/sun/crypto/provider/Mac/HmacSHA512.java
! test/jdk/com/sun/jdi/NoLaunchOptionTest.java
! test/jdk/com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogAppModal1Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogAppModal2Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogAppModal3Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogAppModal4Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogAppModal5Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogAppModal6Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogDocModal1Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogDocModal2Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogDocModal3Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogDocModal4Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogDocModal5Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogDocModal6Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogDocModal7Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogModal1Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogModal2Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogModal3Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogModal4Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogModal5Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogModal6Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogNonModal1Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogNonModal2Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogNonModal3Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogNonModal4Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogNonModal5Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogNonModal6Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogNonModal7Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogTKModal1Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogTKModal2Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogTKModal3Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogTKModal4Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogTKModal5Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogTKModal6Test.java
! test/jdk/java/awt/Modal/FileDialog/FileDialogTKModal7Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDDAppModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDDDocModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDDModelessTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDDNonModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDDSetModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDDTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDDToolkitModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFAppModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFSetModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFToolkitModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFWModeless1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFWModeless2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFWNonModal1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFWNonModal2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDFWTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingDocModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDAppModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDDocModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDModelessTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDNonModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDSetModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDToolkitModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWDocModal1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWDocModal2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWDocModal3Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWDocModal4Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWModeless1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWModeless2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWModeless3Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWModeless4Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWNonModal1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWNonModal2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWNonModal3Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWNonModal4Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingFDWTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsAppModal1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsAppModal2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsAppModal3Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsAppModal4Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsAppModal5Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsAppModal6Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsDocModal1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsDocModal2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsDocModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsSetModal1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsSetModal2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsSetModal3Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsSetModal4Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsSetModal5Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsSetModal6Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsToolkitModal1Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsToolkitModal2Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsToolkitModal3Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsToolkitModal4Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsToolkitModal5Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/BlockingWindowsToolkitModal6Test.java
! test/jdk/java/awt/Modal/ModalBlockingTests/UnblockedDialogAppModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/UnblockedDialogDocModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/UnblockedDialogModelessTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/UnblockedDialogNonModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/UnblockedDialogSetModalTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/UnblockedDialogTest.java
! test/jdk/java/awt/Modal/ModalBlockingTests/UnblockedDialogToolkitModalTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ApplicationExcludeDialogFileTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ApplicationExcludeDialogPageSetupTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ApplicationExcludeDialogPrintSetupTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ApplicationExcludeFrameFileTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ApplicationExcludeFramePageSetupTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ApplicationExcludeFramePrintSetupTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ExcludeDialogTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ExcludeFrameTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ToolkitExcludeDialogFileTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ToolkitExcludeDialogPageSetupTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ToolkitExcludeDialogPrintSetupTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ToolkitExcludeFrameFileTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ToolkitExcludeFramePageSetupTest.java
! test/jdk/java/awt/Modal/ModalExclusionTests/ToolkitExcludeFramePrintSetupTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDWFAppModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDWFDocModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDWFModelessTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDWFNonModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDWFTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDialogsAppModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDialogsDocModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDialogsModelessTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferDialogsNonModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFDWAppModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFDWDocModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFDWModelessTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFDWNonModalTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDAppModal1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDAppModal2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDAppModal3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDAppModal4Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDDocModal1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDDocModal2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDDocModal3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDDocModal4Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDModeless1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDModeless2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDModeless3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDModeless4Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDNonModal1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDNonModal2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDNonModal3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDNonModal4Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferFWDTest.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFAppModal1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFAppModal2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFAppModal3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFDocModal1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFDocModal2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFDocModal3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFModeless1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFModeless2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFModeless3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFNonModal1Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFNonModal2Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFNonModal3Test.java
! test/jdk/java/awt/Modal/ModalFocusTransferTests/FocusTransferWDFTest.java
! test/jdk/java/awt/Modal/ModalitySettingsTest/ModalitySettingsTest.java
! test/jdk/java/awt/Modal/MultipleDialogs/MultipleDialogs1Test.java
! test/jdk/java/awt/Modal/MultipleDialogs/MultipleDialogs2Test.java
! test/jdk/java/awt/Modal/MultipleDialogs/MultipleDialogs3Test.java
! test/jdk/java/awt/Modal/MultipleDialogs/MultipleDialogs4Test.java
! test/jdk/java/awt/Modal/MultipleDialogs/MultipleDialogs5Test.java
! test/jdk/java/awt/Modal/NullModalityDialogTest/NullModalityDialogTest.java
! test/jdk/java/awt/Modal/OnTop/OnTopAppModal1Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopAppModal2Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopAppModal3Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopAppModal4Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopAppModal5Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopAppModal6Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopDDFTest.java
! test/jdk/java/awt/Modal/OnTop/OnTopDocModal1Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopDocModal2Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopDocModal3Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopDocModal4Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopDocModal5Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopDocModal6Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopFDFTest.java
! test/jdk/java/awt/Modal/OnTop/OnTopModal1Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModal2Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModal3Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModal4Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModal5Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModal6Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModeless1Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModeless2Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModeless3Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModeless4Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModeless5Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopModeless6Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopTKModal1Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopTKModal2Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopTKModal3Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopTKModal4Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopTKModal5Test.java
! test/jdk/java/awt/Modal/OnTop/OnTopTKModal6Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackAppModal1Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackAppModal2Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackAppModal3Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackAppModal4Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackAppModal5Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackAppModal6Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackDDFTest.java
! test/jdk/java/awt/Modal/ToBack/ToBackDocModal1Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackDocModal2Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackDocModal3Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackDocModal4Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackDocModal5Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackDocModal6Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackFDFTest.java
! test/jdk/java/awt/Modal/ToBack/ToBackModal1Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModal2Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModal3Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModal4Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModal5Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModal6Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModeless1Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModeless2Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModeless3Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModeless4Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModeless5Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackModeless6Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackNonModal1Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackNonModal2Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackNonModal3Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackNonModal4Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackNonModal5Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackNonModal6Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackTKModal1Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackTKModal2Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackTKModal3Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackTKModal4Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackTKModal5Test.java
! test/jdk/java/awt/Modal/ToBack/ToBackTKModal6Test.java
! test/jdk/java/awt/Modal/ToFront/DialogToFrontAppModalTest.java
! test/jdk/java/awt/Modal/ToFront/DialogToFrontDocModalTest.java
! test/jdk/java/awt/Modal/ToFront/DialogToFrontModalTest.java
! test/jdk/java/awt/Modal/ToFront/DialogToFrontModeless1Test.java
! test/jdk/java/awt/Modal/ToFront/DialogToFrontNonModalTest.java
! test/jdk/java/awt/Modal/ToFront/DialogToFrontTKModalTest.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontAppModal1Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontAppModal2Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontAppModal3Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontAppModal4Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontAppModal5Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontDocModal1Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontDocModal2Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontModal1Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontModal2Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontModal3Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontModal4Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontModal5Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontModeless1Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontNonModalTest.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontTKModal1Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontTKModal2Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontTKModal3Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontTKModal4Test.java
! test/jdk/java/awt/Modal/ToFront/FrameToFrontTKModal5Test.java
! test/jdk/java/awt/Modal/helpers/TestDialog.java
! test/jdk/java/awt/Modal/helpers/TestFrame.java
! test/jdk/java/awt/Modal/helpers/TestWindow.java
! test/jdk/java/awt/Robot/ModifierRobotKey/ModifierRobotEnhancedKeyTest.java
! test/jdk/java/awt/Robot/ModifierRobotKey/ModifierRobotKeyTest.java
! test/jdk/java/awt/dnd/DragSourceListenerSerializationTest/DragSourceListenerSerializationTest.java
! test/jdk/java/awt/event/KeyEvent/ExtendedModifiersTest/ExtendedModifiersTest.java
! test/jdk/java/awt/event/KeyEvent/KeyMaskTest/KeyMaskTest.java
! test/jdk/java/awt/event/MouseEvent/MouseButtonsAndKeyMasksTest/MouseButtonsAndKeyMasksTest.java
! test/jdk/java/awt/event/MouseEvent/MouseButtonsTest/MouseButtonsTest.java
! test/jdk/java/awt/event/MouseEvent/MultipleMouseButtonsTest/MultipleMouseButtonsTest.java
! test/jdk/java/lang/annotation/repeatingAnnotations/CustomRepeatingWithSecurityManager.java
! test/jdk/java/lang/annotation/repeatingAnnotations/RepeatingWithSecurityManager.java
! test/jdk/java/lang/annotation/typeAnnotations/GetAnnotatedOwnerType.java
! test/jdk/java/lang/invoke/ExplicitCastArgumentsTest.java
! test/jdk/java/lang/invoke/MethodHandles/CatchExceptionTest.java
! test/jdk/java/lang/invoke/common/test/java/lang/invoke/lib/Helper.java
! test/jdk/java/security/SecureRandom/GetInstanceTest.java
! test/jdk/java/security/SecureRandom/SerializedSeedTest.java
! test/jdk/java/util/logging/TestLoggerWeakRefLeak.java
! test/jdk/jdk/internal/reflect/constantPool/ConstantPoolTest.java
! test/jdk/lib/testlibrary/AssertsTest.java
- test/jdk/lib/testlibrary/jdk/testlibrary/Asserts.java
! test/jdk/sun/jvmstat/monitor/MonitoredVm/TestPollingInterval.java
! test/jdk/sun/management/jdp/JdpDefaultsTest.java
! test/jdk/sun/management/jdp/JdpJmxRemoteDynamicPortTest.java
! test/jdk/sun/management/jdp/JdpOffTest.java
! test/jdk/sun/management/jdp/JdpOnTestCase.java
! test/jdk/sun/management/jdp/JdpSpecificAddressTest.java
! test/jdk/sun/management/jdp/JdpTestUtilTest.java
! test/jdk/sun/security/provider/MessageDigest/SHA512.java
! test/jdk/sun/tools/jcmd/TestJcmdDefaults.java
! test/jdk/sun/tools/jcmd/TestJcmdSanity.java
! test/jdk/sun/tools/jhsdb/HeapDumpTest.java
! test/jdk/sun/tools/jmap/BasicJMapTest.java
! test/jdk/sun/tools/jps/JpsHelper.java
! test/jdk/sun/tools/jps/TestJps.java
! test/jdk/sun/tools/jps/TestJpsSanity.java
! test/jdk/sun/tools/jstatd/JstatGCUtilParser.java
! test/jdk/sun/tools/jstatd/JstatdTest.java
! test/jdk/sun/tools/jstatd/TestJstatdDefaults.java
! test/jdk/sun/tools/jstatd/TestJstatdExternalRegistry.java
! test/jdk/sun/tools/jstatd/TestJstatdPort.java
! test/jdk/sun/tools/jstatd/TestJstatdPortAndServer.java
! test/jdk/sun/tools/jstatd/TestJstatdServer.java
! test/lib/jdk/test/lib/Asserts.java

Changeset: f191aca8f96d
Author:    jjg
Date:      2018-09-21 15:38 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/f191aca8f96d

8210275: Source Launcher should fail if --source is used without a source file
Reviewed-by: mchung, alanb, mcimadamore

! src/java.base/share/native/libjli/java.c
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/launcher.properties
! test/jdk/tools/launcher/SourceMode.java
! test/langtools/tools/javac/launcher/SourceLauncherTest.java

Changeset: b7153eff0558
Author:    cjplummer
Date:      2018-09-22 14:12 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/b7153eff0558

8210987: Extra newlines on Windows when running nsk jdb tests
Summary: fix skipping over newlines
Reviewed-by: dholmes, jcbeyler, amenkov

! test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java

Changeset: 16f0deae8fa6
Author:    gadams
Date:      2018-09-21 08:13 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/16f0deae8fa6

8208471: nsk/jdb/unwatch/unwatch002/unwatch002.java fails with "Prompt is not received during 300200 milliseconds"
Reviewed-by: cjplummer, amenkov

! test/hotspot/jtreg/vmTestbase/nsk/jdb/unwatch/unwatch002/unwatch002.java

Changeset: 354fb27fd38a
Author:    jnimeh
Date:      2018-09-24 00:13 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/354fb27fd38a

8210918: Add test to exercise server-side client hello processing
Reviewed-by: xuelei

+ test/jdk/javax/net/ssl/compatibility/ClientHelloProcessing.java

Changeset: f5daffd7ec7a
Author:    avoitylov
Date:      2018-09-24 16:39 +0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/f5daffd7ec7a

8210465: ARM: Object equals abstraction for BarrierSetAssembler
Reviewed-by: rkennke, dsamersoff

! src/hotspot/cpu/arm/c1_LIRAssembler_arm.cpp
! src/hotspot/cpu/arm/gc/shared/barrierSetAssembler_arm.cpp
! src/hotspot/cpu/arm/gc/shared/barrierSetAssembler_arm.hpp
! src/hotspot/cpu/arm/macroAssembler_arm.cpp
! src/hotspot/cpu/arm/macroAssembler_arm.hpp
! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/arm/templateTable_arm.cpp

Changeset: cc1a4a267798
Author:    avoitylov
Date:      2018-09-24 16:44 +0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/cc1a4a267798

8210466: Modularize allocations in assembler
Reviewed-by: rkennke, dsamersoff

! src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp
! src/hotspot/cpu/arm/c1_Runtime1_arm.cpp
! src/hotspot/cpu/arm/gc/shared/barrierSetAssembler_arm.cpp
! src/hotspot/cpu/arm/gc/shared/barrierSetAssembler_arm.hpp
! src/hotspot/cpu/arm/macroAssembler_arm.cpp
! src/hotspot/cpu/arm/macroAssembler_arm.hpp
! src/hotspot/cpu/arm/templateTable_arm.cpp

Changeset: 34e2180a6d51
Author:    avoitylov
Date:      2018-09-24 16:52 +0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/34e2180a6d51

8209695: ARM: Explicit barriers for interpreter
Reviewed-by: dsamersoff

! src/hotspot/cpu/arm/gc/shared/barrierSetAssembler_arm.hpp
! src/hotspot/cpu/arm/macroAssembler_arm.cpp
! src/hotspot/cpu/arm/macroAssembler_arm.hpp
! src/hotspot/cpu/arm/sharedRuntime_arm.cpp
! src/hotspot/cpu/arm/templateInterpreterGenerator_arm.cpp
! src/hotspot/cpu/arm/templateTable_arm.cpp

Changeset: 54afe70c50b6
Author:    avoitylov
Date:      2018-09-24 16:54 +0300
URL:       http://hg.openjdk.java.net/panama/dev/rev/54afe70c50b6

8209697: ARM: Explicit barriers for C1/assembler
Reviewed-by: dsamersoff

! src/hotspot/cpu/arm/c1_LIRAssembler_arm.cpp

Changeset: 4010c90156d1
Author:    sgehwolf
Date:      2018-09-21 16:58 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/4010c90156d1

8210761: libjsig is being compiled without optimization
Reviewed-by: erikj, ihse

! make/lib/Lib-java.base.gmk

Changeset: 9b8f2ef4663a
Author:    rkennke
Date:      2018-09-14 14:43 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/9b8f2ef4663a

8210752: Remaining explicit barriers for C2
Reviewed-by: roland, shade

! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/opto/parse2.cpp

Changeset: 6c17cf410d7c
Author:    dholmes
Date:      2018-09-24 12:18 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/6c17cf410d7c

8211045: [Testbug] Fix for 8144279 didn't define a test case!
Summary: add missing file from original commit
Reviewed-by: shade, sgehwolf

+ test/hotspot/jtreg/runtime/jsig/Testjsig.java

Changeset: e240625311ad
Author:    dcubed
Date:      2018-09-24 13:05 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/e240625311ad

8209019: Remove tests affected by JDK-8208690 from the ProblemList
Reviewed-by: dfuchs

! test/jdk/ProblemList.txt

Changeset: ec62d6cab037
Author:    ihse
Date:      2018-09-24 19:26 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/ec62d6cab037

8211029: Have a common set of enabled warnings for all native libraries
Reviewed-by: erikj

! make/autoconf/flags-cflags.m4
! make/common/TestFilesCompilation.gmk
! make/hotspot/gensrc/GensrcDtrace.gmk
! make/hotspot/lib/CompileJvm.gmk
! make/launcher/Launcher-jdk.pack.gmk
! make/launcher/LauncherCommon.gmk
! make/lib/Awt2dLibraries.gmk
! make/lib/CoreLibraries.gmk
! make/lib/Lib-java.base.gmk
! make/lib/Lib-java.desktop.gmk
! make/lib/Lib-java.instrument.gmk
! make/lib/Lib-java.security.jgss.gmk
! make/lib/Lib-jdk.crypto.cryptoki.gmk
! make/lib/Lib-jdk.crypto.ec.gmk
! make/lib/Lib-jdk.hotspot.agent.gmk
! make/lib/Lib-jdk.jdwp.agent.gmk
! make/lib/Lib-jdk.management.gmk
! make/lib/Lib-jdk.pack.gmk
! make/lib/Lib-jdk.sctp.gmk
! src/java.desktop/unix/native/common/awt/awt_Font.c

Changeset: 3c6d285c8168
Author:    bchristi
Date:      2018-09-24 10:41 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/3c6d285c8168

8072130: java/lang/instrument/BootClassPath/BootClassPathTest.sh fails on Mac OSX
Reviewed-by: sherman

! src/java.base/unix/native/libjava/java_props_md.c
! src/java.instrument/unix/native/libinstrument/EncodingSupport_md.c
! test/jdk/ProblemList.txt
! test/jdk/java/lang/instrument/BootClassPath/BootClassPathTest.sh
! test/jdk/java/lang/instrument/BootClassPath/Setup.java

Changeset: 8bbb5cbac92c
Author:    ihse
Date:      2018-09-24 20:45 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/8bbb5cbac92c

8210705: Stop exporting all symbols on macosx
Reviewed-by: erikj

! make/launcher/LauncherCommon.gmk
! make/lib/LibCommon.gmk
! src/java.desktop/macosx/native/libosxapp/AWT_debug.h
! src/java.desktop/macosx/native/libosxapp/NSApplicationAWT.h
! src/java.desktop/macosx/native/libosxapp/PropertiesUtilities.h
! src/java.desktop/macosx/native/libosxapp/ThreadUtilities.h

Changeset: 11b9d3a6f31c
Author:    erikj
Date:      2018-09-24 13:51 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/11b9d3a6f31c

8211037: Load jib jars dynamically from JibArtifactManager
Reviewed-by: ihse

! make/RunTests.gmk
! make/autoconf/spec.gmk.in
! make/autoconf/toolchain.m4
! make/conf/jib-profiles.js
! test/TestCommon.gmk
! test/lib/jdk/test/lib/artifacts/ArtifactResolver.java
! test/lib/jdk/test/lib/artifacts/JibArtifactManager.java

Changeset: 9978fea8a371
Author:    kvn
Date:      2018-09-24 16:37 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/9978fea8a371

8210764: Update avx512 implementation
Reviewed-by: kvn
Contributed-by: sandhya.viswanathan at intel.com

! src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.cpp
! src/hotspot/cpu/arm/c1_LIRAssembler_arm.cpp
! src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
! src/hotspot/cpu/s390/c1_LIRAssembler_s390.cpp
! src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp
! src/hotspot/cpu/x86/c1_LIRGenerator_x86.cpp
! src/hotspot/cpu/x86/globals_x86.hpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/cpu/x86/vm_version_x86.cpp
! src/hotspot/cpu/x86/vm_version_x86.hpp
! src/hotspot/cpu/x86/x86.ad
! src/hotspot/cpu/x86/x86_32.ad
! src/hotspot/cpu/x86/x86_64.ad
! src/hotspot/share/c1/c1_LIR.cpp
! src/hotspot/share/c1/c1_LIR.hpp
! src/hotspot/share/c1/c1_LIRAssembler.cpp
! src/hotspot/share/c1/c1_LIRAssembler.hpp

Changeset: cef2c1ea2f60
Author:    mli
Date:      2018-09-25 11:18 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/cef2c1ea2f60

8210443: Migrate Locale matching tests to JDK Repo.
Reviewed-by: naoto
Contributed-by: dan.z.zhou at oracle.com

! test/jdk/java/util/Locale/Bug7069824.java
+ test/jdk/java/util/Locale/FilteringModeTest.java

Changeset: 1a35c474e4a6
Author:    fyuan
Date:      2018-09-25 11:24 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/1a35c474e4a6

8210934: Move sun/net/www/protocol/http/GetErrorStream.java to OpenJDK
Summary: repalce internet website dependency with a built-in http server
Reviewed-by: chegar

+ test/jdk/sun/net/www/protocol/http/GetErrorStream.java

Changeset: 54aafb3ba9ab
Author:    mikael
Date:      2018-09-24 22:12 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/54aafb3ba9ab

8210848: Obsolete SyncKnobs
Reviewed-by: redestad, coleenp, dholmes, dcubed

! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/runtime/arguments.cpp
! src/hotspot/share/runtime/globals.hpp
! src/hotspot/share/runtime/objectMonitor.cpp
! src/hotspot/share/runtime/objectMonitor.hpp
! src/hotspot/share/runtime/synchronizer.cpp
! src/hotspot/share/runtime/synchronizer.hpp
! src/hotspot/share/runtime/thread.cpp
! src/hotspot/share/runtime/vframe.cpp

Changeset: f7d40158eb2f
Author:    pmuthuswamy
Date:      2018-09-25 12:36 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/f7d40158eb2f

8205593: Javadoc -link makes broken links if module name matches package name
Reviewed-by: jjg

! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlConfiguration.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDocletWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/LinkFactoryImpl.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/Navigation.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/BaseConfiguration.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets.properties
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Extern.java
! test/langtools/jdk/javadoc/doclet/JavascriptWinTitle/JavascriptWinTitle.java
! test/langtools/jdk/javadoc/doclet/testClassCrossReferences/C.java
! test/langtools/jdk/javadoc/doclet/testClassCrossReferences/TestClassCrossReferences.java
! test/langtools/jdk/javadoc/doclet/testDocRootInlineTag/TestDocRootInlineTag.java
! test/langtools/jdk/javadoc/doclet/testExternalOverridenMethod/TestExternalOverridenMethod.java
! test/langtools/jdk/javadoc/doclet/testHref/TestHref.java
! test/langtools/jdk/javadoc/doclet/testLinkOption/TestLinkOption.java
+ test/langtools/jdk/javadoc/doclet/testLinkOption/TestLinkOptionWithModule.java
! test/langtools/jdk/javadoc/doclet/testLinkOption/TestNewLineInLink.java
! test/langtools/jdk/javadoc/doclet/testModules/TestModules.java
! test/langtools/jdk/javadoc/doclet/testTitleInHref/TestTitleInHref.java

Changeset: c319db69099c
Author:    pmuthuswamy
Date:      2018-09-25 13:58 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/c319db69099c

8202462: {@index} may cause duplicate labels
Reviewed-by: jjg

! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDocletWriter.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModuleWriterImpl.java
! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/TagletWriterImpl.java
+ test/langtools/jdk/javadoc/doclet/testIndexTaglet/TestIndexTaglet.java
! test/langtools/jdk/javadoc/doclet/testModules/TestModules.java

Changeset: bc38c75eed57
Author:    thartmann
Date:      2018-09-25 14:16 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/bc38c75eed57

8210152: Optimize integer divisible by power-of-2 check
Summary: Integer conditional negation operation before zero check is eliminated
Reviewed-by: kvn, thartmann
Contributed-by: pengfei.li at arm.com

! src/hotspot/share/opto/subnode.cpp

Changeset: 490d9001eba9
Author:    plevart
Date:      2018-09-25 14:23 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/490d9001eba9

8205461: Create Collector which merges results of two other collectors
Reviewed-by: briangoetz, smarks, plevart
Contributed-by: amaembo at gmail.com

! src/java.base/share/classes/java/util/stream/Collectors.java
! test/jdk/java/util/stream/test/org/openjdk/tests/java/util/stream/CollectorsTest.java

Changeset: eb954a4b6083
Author:    rkennke
Date:      2018-09-24 18:44 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/eb954a4b6083

8132849: Increased stop time in cleanup phase because of single-threaded walk of thread stacks in NMethodSweeper::mark_active_nmethods()
Reviewed-by: eosterlund, zgu, thartmann

! src/hotspot/share/runtime/safepoint.cpp
! src/hotspot/share/runtime/sweeper.cpp
! src/hotspot/share/runtime/sweeper.hpp

Changeset: 703813b05838
Author:    aph
Date:      2018-09-24 18:19 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/703813b05838

8211064: [AArch64] Interpreter and c1 don't correctly handle jboolean results in native calls
Reviewed-by: aph
Contributed-by: andrey.petushkov at gmail.com

! src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
! src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp
! src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp
! src/hotspot/cpu/aarch64/templateInterpreterGenerator_aarch64.cpp

Changeset: faafa910a9a5
Author:    chegar
Date:      2018-09-25 15:35 +0100
URL:       http://hg.openjdk.java.net/panama/dev/rev/faafa910a9a5

8211099: ProblemList two networking tests until jtreg b14 is promoted
Reviewed-by: alanb

! test/jdk/ProblemList.txt

Changeset: 92960b0e6191
Author:    rkennke
Date:      2018-09-25 16:41 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/92960b0e6191

8211061: Tests fail with assert(VM_Version::supports_sse4_1()) on ThreadRipper CPU
Reviewed-by: thartmann, roland

! src/hotspot/cpu/x86/vm_version_x86.cpp

Changeset: f8f2f7ee52cb
Author:    dcubed
Date:      2018-09-25 11:31 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/f8f2f7ee52cb

8211103: ProblemList runtime/XCheckJniJsig/XCheckJSig.java on MacOS X
Reviewed-by: mikael, dholmes

! test/hotspot/jtreg/ProblemList.txt

Changeset: cdfabab3413f
Author:    jcbeyler
Date:      2018-09-25 09:34 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/cdfabab3413f

8210689: Remove the multi-line old C style for string literals
Summary: Remove the multi-line old C style and prefer C++ multi-line
Reviewed-by: amenkov, cjplummer

! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Breakpoint/breakpoint001/breakpoint001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ClassLoad/classload001/classload001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GarbageCollectionStart/gcstart001/gcstart001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassSignature/getclsig006/getclsig006.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetFieldName/getfldnm005/getfldnm005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetJNIFunctionTable/getjniftab002/getjniftab002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariableTable/localtab004/localtab004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetLocalVariableTable/localtab005/localtab005.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind001/nativemethbind001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind002/nativemethbind002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind003/nativemethbind003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/NativeMethodBind/nativemethbind004/nativemethbind004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ObjectFree/objfree001/objfree001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ObjectFree/objfree002/objfree002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass003/redefclass003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass004/redefclass004.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass008/redefclass008.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass009/redefclass009.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass010/redefclass010.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetJNIFunctionTable/setjniftab001/setjniftab001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetJNIFunctionTable/setjniftab002/setjniftab002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SingleStep/singlestep001/singlestep001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SingleStep/singlestep002/singlestep002.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SingleStep/singlestep003/singlestep003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP01/ap01t001/ap01t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP02/ap02t001/ap02t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP03/ap03t001/ap03t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP10/ap10t001/ap10t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP12/ap12t001/ap12t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS201/hs201t003/hs201t003.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/ji05t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/jni_interception/JI06/ji06t001/ji06t001.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/JVMTIagent.cpp
! test/hotspot/jtreg/vmTestbase/nsk/share/aod/aod.cpp

Changeset: 8f66a57054b7
Author:    jjg
Date:      2018-09-25 10:30 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/8f66a57054b7

8210839: Improve interaction between source launcher and classpath
Reviewed-by: alanb, mchung

! src/jdk.compiler/share/classes/com/sun/tools/javac/launcher/Main.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/launcher.properties
! test/langtools/tools/javac/launcher/SourceLauncherTest.java
! test/langtools/tools/javac/launcher/src/CLTest.java

Changeset: a6bdb6d5f167
Author:    darcy
Date:      2018-09-25 11:31 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/a6bdb6d5f167

8173730: Stop including enhanced for-loop tip for enum values() method
Reviewed-by: jjg

! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets.properties

Changeset: e0153fc0a843
Author:    darcy
Date:      2018-09-25 13:31 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/e0153fc0a843

8211127: TestNewLanguageFeatures.java fails after JDK-8173730
Reviewed-by: jjg

! test/langtools/jdk/javadoc/doclet/testNewLanguageFeatures/TestNewLanguageFeatures.java

Changeset: a0426bc28519
Author:    naoto
Date:      2018-09-25 13:57 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/a0426bc28519

8210633: Cannot parse JapaneseDate string with DateTimeFormatterBuilder Mapped-values
Reviewed-by: scolebourne, rriggs

! src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java
! test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java

Changeset: e1368526699d
Author:    fyang
Date:      2018-09-26 06:26 +0800
URL:       http://hg.openjdk.java.net/panama/dev/rev/e1368526699d

8210413: AArch64: Optimize div/rem by constant in C1
Summary: Remove div-by-zero check for non-zero divisor and generate cheap instructions if divisor is power-of-2
Reviewed-by: aph
Contributed-by: pengfei.li at arm.com

! src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.cpp
! src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.hpp
! src/hotspot/cpu/aarch64/c1_LIRGenerator_aarch64.cpp

Changeset: d2b381ea8477
Author:    iignatyev
Date:      2018-09-25 18:26 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/d2b381ea8477

8211134: problem list compiler/whitebox/ForceNMethodSweepTest.java
Reviewed-by: kvn

! test/hotspot/jtreg/ProblemList.txt

Changeset: 5f931e3e7a63
Author:    lkorinth
Date:      2018-09-21 18:57 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/5f931e3e7a63

8196341: Add JFR events for parallel phases of G1
Reviewed-by: tschatzl, sangheki

! src/hotspot/share/gc/g1/g1CollectedHeap.cpp
! src/hotspot/share/gc/g1/g1GCPhaseTimes.cpp
! src/hotspot/share/gc/g1/g1GCPhaseTimes.hpp
! src/hotspot/share/gc/g1/g1RemSet.cpp
! src/hotspot/share/jfr/metadata/metadata.xml
! src/jdk.jfr/share/conf/jfr/default.jfc
! src/jdk.jfr/share/conf/jfr/profile.jfc
! test/jdk/TEST.groups
+ test/jdk/jdk/jfr/event/gc/collection/TestG1ParallelPhases.java
! test/lib/jdk/test/lib/jfr/EventNames.java

Changeset: 511a9946f83e
Author:    mbaesken
Date:      2018-09-26 14:28 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/511a9946f83e

8211146: fix problematic elif-tests after recent gcc warning changes Werror=undef
Reviewed-by: stuefe, clanger, dholmes, chegar, alanb

! src/java.base/unix/native/libnet/net_util_md.h
! src/java.base/unix/native/libnio/ch/NativeThread.c
! src/java.base/unix/native/libnio/ch/nio_util.h
! src/jdk.jdwp.agent/unix/native/libjdwp/util_md.h

Changeset: 6ffa38b8da65
Author:    mbaesken
Date:      2018-09-12 11:13 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/6ffa38b8da65

8207768: Improve exception messages during manifest parsing of jar archives
Reviewed-by: clanger, mullan, weijun

! src/java.base/share/classes/java/util/jar/Attributes.java
! src/java.base/share/classes/java/util/jar/JarFile.java
! src/java.base/share/classes/java/util/jar/Manifest.java
! src/java.base/share/classes/sun/net/util/SocketExceptions.java
+ src/java.base/share/classes/sun/security/util/SecurityProperties.java
! src/java.base/share/conf/security/java.security

Changeset: ec4c3c287ca7
Author:    roland
Date:      2018-09-18 20:49 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/ec4c3c287ca7

8210885: Convert left over loads/stores to access api
Reviewed-by: thartmann, rkennke

! src/hotspot/share/gc/shared/c2/barrierSetC2.cpp
! src/hotspot/share/gc/shared/c2/barrierSetC2.hpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/opto/parse2.cpp
! src/hotspot/share/opto/parse3.cpp
! src/hotspot/share/opto/parseHelper.cpp
! src/hotspot/share/opto/stringopts.cpp

Changeset: 69faed47bf35
Author:    mikael
Date:      2018-09-26 10:37 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/69faed47bf35

8210892: Deprecate TLABStats
Reviewed-by: pliden, tschatzl

! src/hotspot/share/runtime/arguments.cpp
! test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java

Changeset: f6e15aa9c16e
Author:    lancea
Date:      2018-09-26 13:56 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/f6e15aa9c16e

8211121: Remove sun.reflect.ReflectionFactory::newInstanceForSerialization
Reviewed-by: mchung, alanb, darcy, dfuchs

! src/jdk.unsupported/share/classes/sun/reflect/ReflectionFactory.java

Changeset: 7e78be444e68
Author:    cushon
Date:      2018-09-25 21:33 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/7e78be444e68

8211138: Missing Flag enum constants
Reviewed-by: mcimadamore, vromero

! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java
+ test/langtools/tools/javac/flags/FlagsTest.java

Changeset: 2ee7e1b7ba66
Author:    smarks
Date:      2018-08-25 20:16 +0530
URL:       http://hg.openjdk.java.net/panama/dev/rev/2ee7e1b7ba66

7033681: Arrays.asList methods needs better documentation
Reviewed-by: smarks
Contributed-by: Jaikiran Pai <jaikiran.pai at gmail.com>

! src/java.base/share/classes/java/util/Arrays.java

Changeset: 789cc1561621
Author:    jjg
Date:      2018-09-26 11:41 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/789cc1561621

8210274: Source Launcher should work with a security manager
Reviewed-by: mchung, alanb

! src/jdk.compiler/share/classes/com/sun/tools/javac/launcher/Main.java
! test/langtools/tools/javac/launcher/SourceLauncherTest.java

Changeset: bdf62f266de4
Author:    akolarkunnu
Date:      2018-09-24 03:25 -0700
URL:       http://hg.openjdk.java.net/panama/dev/rev/bdf62f266de4

8210994: Create test for SwingSet3 FrameDemo
Reviewed-by: serb
Contributed-by: abdul.kolarkunnu at oracle.com

+ test/jdk/sanity/client/SwingSet/src/FrameDemoTest.java
! test/jdk/sanity/client/SwingSet/src/InternalFrameDemoTest.java
! test/jdk/sanity/client/SwingSet/src/TestHelpers.java
+ test/jdk/sanity/client/lib/SwingSet3/src/com/sun/swingset3/demos/frame/BusyGlass.java
+ test/jdk/sanity/client/lib/SwingSet3/src/com/sun/swingset3/demos/frame/FrameDemo.java
+ test/jdk/sanity/client/lib/SwingSet3/src/com/sun/swingset3/demos/frame/resources/images/FrameDemo.gif

Changeset: 32161fbea3fe
Author:    coleenp
Date:      2018-09-26 14:01 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/32161fbea3fe

8210856: Move InstanceKlass DependencyContext cleaning to SystemDictionary::do_unloading()
Summary: Already walk classes in ClassLoaderData::unload so generalize to also clean nmethod dependencies.
Reviewed-by: eosterlund, dlong, vlivanov

! src/hotspot/share/classfile/classLoaderData.cpp
! src/hotspot/share/code/dependencyContext.cpp
! src/hotspot/share/code/dependencyContext.hpp
! src/hotspot/share/oops/instanceKlass.cpp
! src/hotspot/share/oops/instanceKlass.hpp
! test/hotspot/gtest/code/test_dependencyContext.cpp

Changeset: 8b02303915bc
Author:    coleenp
Date:      2018-09-26 14:56 -0400
URL:       http://hg.openjdk.java.net/panama/dev/rev/8b02303915bc

Merge



From maurizio.cimadamore at oracle.com  Wed Sep 26 20:06:29 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 26 Sep 2018 20:06:29 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809262006.w8QK6UIA014504@aojmv0008.oracle.com>

Changeset: b86bc6673265
Author:    mcimadamore
Date:      2018-09-26 22:08 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/b86bc6673265

Automatic merge with default

! make/RunTests.gmk
! make/autoconf/basics.m4
! make/autoconf/spec.gmk.in
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
- src/hotspot/share/classfile/compactHashtable.inline.hpp
- src/java.base/macosx/native/libjli/java_md_macosx.c
! test/jdk/TEST.groups
- test/jdk/com/sun/jdi/RedefineException.sh
- test/jdk/com/sun/jdi/RedefineFinal.sh
- test/jdk/com/sun/jdi/RedefineIntConstantToLong.sh
- test/jdk/com/sun/jdi/RedefineMulti.sh
- test/jdk/com/sun/jdi/RedefinePop.sh
- test/jdk/com/sun/jdi/RedefineStep.sh
- test/jdk/com/sun/jdi/RedefineTTYLineNumber.sh
- test/jdk/com/sun/jdi/StringConvertTest.sh
- test/jdk/com/sun/jdi/WatchFramePop.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Asserts.java


From maurizio.cimadamore at oracle.com  Wed Sep 26 20:06:52 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 26 Sep 2018 20:06:52 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809262006.w8QK6qYr014780@aojmv0008.oracle.com>

Changeset: 04dbedb07947
Author:    mcimadamore
Date:      2018-09-26 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/04dbedb07947

Automatic merge with default

- src/hotspot/share/classfile/compactHashtable.inline.hpp
- src/java.base/macosx/native/libjli/java_md_macosx.c
- test/jdk/com/sun/jdi/RedefineException.sh
- test/jdk/com/sun/jdi/RedefineFinal.sh
- test/jdk/com/sun/jdi/RedefineIntConstantToLong.sh
- test/jdk/com/sun/jdi/RedefineMulti.sh
- test/jdk/com/sun/jdi/RedefinePop.sh
- test/jdk/com/sun/jdi/RedefineStep.sh
- test/jdk/com/sun/jdi/RedefineTTYLineNumber.sh
- test/jdk/com/sun/jdi/StringConvertTest.sh
- test/jdk/com/sun/jdi/WatchFramePop.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Asserts.java


From maurizio.cimadamore at oracle.com  Wed Sep 26 20:07:11 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 26 Sep 2018 20:07:11 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809262007.w8QK7BPD015079@aojmv0008.oracle.com>

Changeset: fc838b7d2c3f
Author:    mcimadamore
Date:      2018-09-26 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/fc838b7d2c3f

Automatic merge with default

! src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp
! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/arm/sharedRuntime_arm.cpp
! src/hotspot/cpu/x86/assembler_x86.cpp
! src/hotspot/cpu/x86/assembler_x86.hpp
! src/hotspot/cpu/x86/macroAssembler_x86.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
! src/hotspot/cpu/x86/x86.ad
! src/hotspot/cpu/x86/x86_64.ad
- src/hotspot/share/classfile/compactHashtable.inline.hpp
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
! src/hotspot/share/opto/library_call.cpp
! src/hotspot/share/opto/macro.cpp
! src/hotspot/share/runtime/globals.hpp
- src/java.base/macosx/native/libjli/java_md_macosx.c
- test/jdk/com/sun/jdi/RedefineException.sh
- test/jdk/com/sun/jdi/RedefineFinal.sh
- test/jdk/com/sun/jdi/RedefineIntConstantToLong.sh
- test/jdk/com/sun/jdi/RedefineMulti.sh
- test/jdk/com/sun/jdi/RedefinePop.sh
- test/jdk/com/sun/jdi/RedefineStep.sh
- test/jdk/com/sun/jdi/RedefineTTYLineNumber.sh
- test/jdk/com/sun/jdi/StringConvertTest.sh
- test/jdk/com/sun/jdi/WatchFramePop.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Asserts.java


From maurizio.cimadamore at oracle.com  Wed Sep 26 20:07:29 2018
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 26 Sep 2018 20:07:29 +0000
Subject: hg: panama/dev: Automatic merge with default
Message-ID: <201809262007.w8QK7Uvv015428@aojmv0008.oracle.com>

Changeset: 5f51479f239f
Author:    mcimadamore
Date:      2018-09-26 22:09 +0200
URL:       http://hg.openjdk.java.net/panama/dev/rev/5f51479f239f

Automatic merge with default

! src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp
! src/hotspot/cpu/arm/methodHandles_arm.cpp
! src/hotspot/cpu/x86/macroAssembler_x86.hpp
- src/hotspot/share/classfile/compactHashtable.inline.hpp
! src/hotspot/share/classfile/javaClasses.cpp
! src/hotspot/share/classfile/javaClasses.hpp
! src/hotspot/share/opto/graphKit.cpp
! src/hotspot/share/opto/graphKit.hpp
- src/java.base/macosx/native/libjli/java_md_macosx.c
- test/jdk/com/sun/jdi/RedefineException.sh
- test/jdk/com/sun/jdi/RedefineFinal.sh
- test/jdk/com/sun/jdi/RedefineIntConstantToLong.sh
- test/jdk/com/sun/jdi/RedefineMulti.sh
- test/jdk/com/sun/jdi/RedefinePop.sh
- test/jdk/com/sun/jdi/RedefineStep.sh
- test/jdk/com/sun/jdi/RedefineTTYLineNumber.sh
- test/jdk/com/sun/jdi/StringConvertTest.sh
- test/jdk/com/sun/jdi/WatchFramePop.sh
- test/jdk/lib/testlibrary/jdk/testlibrary/Asserts.java


From magnus.ihse.bursie at oracle.com  Thu Sep 27 13:35:54 2018
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 27 Sep 2018 15:35:54 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
Message-ID: <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>



On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>
>
>
> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>> Try this:
>>>
>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>> @@ -60,6 +60,9 @@
>>> ?????? VER=`ls $with_libclang/lib/clang/`
>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>> ?????? CLANG_LIB_PATH="$with_libclang/lib"
>>> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>> +??? ? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>> ???? fi
>>>
>>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>
>>>
>>> Now, if you use only the --with-libclang option with a Unix-style 
>>> path (the thing you were trying at first), I believe it should do 
>>> the right thing.
>>
>> I have not followed the entire conversation, but this part looks 
>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths 
>> from configure argument. (And these should, yes indeed, be in unix 
>> style). However, this might not be all fixes that are needed. I can 
>> help take a look at it, if someone points me to where the problematic 
>> code resides.
>>
> Magnus, I followed up a bit on this, and managed to reproduce the issue.
>
> The underlying issue is that our lib-clang.m4 conf file checks for 
> headers in the usual way with AC_CHECK_HEADER. This check relies on 
> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above 
> fixups, what you end up with is this:
>
> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>
> This is the correct _cygwin_ path, but it's not the correct Windows 
> path. In fact, if you try to compile a dummy header that contains this 
> (this was also observed by Jorn)
>
> #include <clang-c/Index.h>
>
> with a -I option that includes the above path using the cl.exe 
> compiler within cygwin, you get an error because the include file 
> cannot be found. The only solution is to use a Windows path, which in 
> cygwin can only be done using double backslashes (ugh).
>
> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix 
> path, while here we need the opposite operation. I was under the 
> impression that the build configure framework needed to solve this 
> particular issue anyway, to check for other headers, but I realized 
> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is 
> AC_CHECK_HEADER even supposed to work under Windows? If not, what 
> should we use in order to make our configure script portable?
>
> P.S.
> I note that there's also a FIXPATH script which does exactly what we 
> need here - but this script seems just to be set by the configure step 
> and is probably only used by the proper build, so I don't think we can 
> refer to it from here?

Hi,

Sorry for not having had time to look at this yet.

The solution should be to use FIXPATH at all times when compiling on 
Windows if using local paths, even in the configure script. It might be 
the case that we have not had to do this as of yet, so it might not work 
out of the box. :-( But I thought we set up CC to "$FIXPATH $CC" on 
Windows, so I thought it should work by itself.

To get around the mess of windows and unix style paths, the decision was 
made to use only unix paths *all the time*. The duct tape needed to get 
this to work is the fixpath launcher, which takes a command-line with 
unix paths and converts it to windows paths at the very last moment 
before launching the executable.

Does things work better if you to:
CC="$FIXPATH $CC"
before your AC_CHECK_HEADER?

/Magnus


>
> Thanks
> Maurizio
>> /Magnus
>>
>>
>>>
>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming 
>>> paths which can possibly contain forward slashes - e.g.
>>>
>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>>
>>>
>>> Maurizio
>>>
>>>
>>>
>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>> FTR, as I'm looking at other configure files, I've seen
>>>>
>>>> ?"xwindows" used not "xmicrosoft"
>>>>
>>>> As for the need for double backslash, I don't see any special 
>>>> processing done in other configure files prior to the 
>>>> AC_CHECK_HEADER call.
>>>>
>>>> The build guide [1] is very explicit that forward slashes should be 
>>>> used in options (unlike what you did) and mentions some 'fixpath' 
>>>> tool which is used by configure to convert Unix paths into Windows 
>>>> ones. I wonder if this could be a bug in that tool?
>>>>
>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 
>>>> - it could be worth chasing down as to where the compiled 'fixpath' 
>>>> executable is put (should be somewhere in 
>>>> build/<conf>/configure-support/) call it with the clang include 
>>>> folder with Unix-style forward slash and see whether it spits the 
>>>> correct path.
>>>>
>>>> The source code for this tool can be found here:
>>>>
>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>>>
>>>>
>>>> Maurizio
>>>>
>>>> [1] - 
>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>>>>
>>>>
>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>> Actually, -I works, and link option need to be passed with /link, 
>>>>> but you got the idea?
>>>>>
>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not 
>>>>> sure it would work. Need to find a better way to check the lib.
>>>>>
>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 
>>>>> -0700
>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018 
>>>>> -0700
>>>>> @@ -68,7 +68,11 @@
>>>>> ????????? LIBCLANG_CPPFLAGS=""
>>>>> ????? fi
>>>>> ????? if test "x$CLANG_LIB_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> ????????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>> +????? fi
>>>>> ????? else
>>>>> ????????? LIBCLANG_LDFLAGS=""
>>>>> ????? fi
>>>>>
>>>>> Cheers,
>>>>> Henry
>>>>>
>>>>>
>>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>>>
>>>>>> Haven?t test it, but try this,
>>>>>>
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018 
>>>>>> -0700
>>>>>> @@ -63,12 +63,20 @@
>>>>>> ???? fi
>>>>>>
>>>>>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> ???????? LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>>> +????? else
>>>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>>> +????? fi
>>>>>> ???? else
>>>>>> ???????? LIBCLANG_CPPFLAGS=""
>>>>>> ???? fi
>>>>>> ???? if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> ???????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +????? else
>>>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>>> +????? fi
>>>>>> ???? else
>>>>>> ???????? LIBCLANG_LDFLAGS=""
>>>>>> ???? fi
>>>>>>
>>>>>> We still need to fix the copy of DLL.
>>>>>>
>>>>>> Cheers,
>>>>>> Henry
>>>>>>
>>>>>>
>>>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore 
>>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>>
>>>>>>> Looks like good progress; tomorrow I'll take a look at some of 
>>>>>>> our build files and see if something special is done for 
>>>>>>> mangling windows include paths.
>>>>>>>
>>>>>>> Cheers
>>>>>>> Maurizio
>>>>>>>
>>>>>>>
>>>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>>>> If I use the following flags:
>>>>>>>>
>>>>>>>> $ bash configure --disable-warnings-as-errors 
>>>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' 
>>>>>>>> --with-libclang-include='J:\\LLVM\\include' 
>>>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib' 
>>>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>>>>
>>>>>>>> (Notice that I'm having to use different path styles)
>>>>>>>>
>>>>>>>> It's detecting the header files, but it's failing on being 
>>>>>>>> passed the wrong flags. from config.log (see at the bottom):
>>>>>>>>
>>>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>>>> configure:60273: 
>>>>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>>>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib 
>>>>>>>> conftest.cpp -lclang?? >&5
>>>>>>>> conftest.cpp
>>>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>>>
>>>>>>>> /out:conftest.exe
>>>>>>>> /out:conftest.exe
>>>>>>>> conftest.obj
>>>>>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>>>>>> clang_getClangVersion referenced in function main
>>>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for 
>>>>>>>> x64
>>>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>>>
>>>>>>>> cl : Command line warning D9035 : option 'o' has been 
>>>>>>>> deprecated and will be removed in a future release
>>>>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>>>>> '-LJ:\\LLVM\\lib'
>>>>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>>>>> '-lclang'
>>>>>>>>
>>>>>>>> That seems to be a simple problem of casing-off windows and 
>>>>>>>> passing the right flags, but I call it a night here :)
>>>>>>>>
>>>>>>>> Jorn
>>>>>>>>
>>>>
>>>
>>
>


From maurizio.cimadamore at oracle.com  Thu Sep 27 13:41:40 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 27 Sep 2018 14:41:40 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
Message-ID: <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>



On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>
>
> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>
>>
>>
>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>> Try this:
>>>>
>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>>> @@ -60,6 +60,9 @@
>>>> ?????? VER=`ls $with_libclang/lib/clang/`
>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>> ?????? CLANG_LIB_PATH="$with_libclang/lib"
>>>> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>> +??? ? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>> +??? ? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>> ???? fi
>>>>
>>>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>
>>>>
>>>> Now, if you use only the --with-libclang option with a Unix-style 
>>>> path (the thing you were trying at first), I believe it should do 
>>>> the right thing.
>>>
>>> I have not followed the entire conversation, but this part looks 
>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths 
>>> from configure argument. (And these should, yes indeed, be in unix 
>>> style). However, this might not be all fixes that are needed. I can 
>>> help take a look at it, if someone points me to where the 
>>> problematic code resides.
>>>
>> Magnus, I followed up a bit on this, and managed to reproduce the issue.
>>
>> The underlying issue is that our lib-clang.m4 conf file checks for 
>> headers in the usual way with AC_CHECK_HEADER. This check relies on 
>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above 
>> fixups, what you end up with is this:
>>
>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>
>> This is the correct _cygwin_ path, but it's not the correct Windows 
>> path. In fact, if you try to compile a dummy header that contains 
>> this (this was also observed by Jorn)
>>
>> #include <clang-c/Index.h>
>>
>> with a -I option that includes the above path using the cl.exe 
>> compiler within cygwin, you get an error because the include file 
>> cannot be found. The only solution is to use a Windows path, which in 
>> cygwin can only be done using double backslashes (ugh).
>>
>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix 
>> path, while here we need the opposite operation. I was under the 
>> impression that the build configure framework needed to solve this 
>> particular issue anyway, to check for other headers, but I realized 
>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is 
>> AC_CHECK_HEADER even supposed to work under Windows? If not, what 
>> should we use in order to make our configure script portable?
>>
>> P.S.
>> I note that there's also a FIXPATH script which does exactly what we 
>> need here - but this script seems just to be set by the configure 
>> step and is probably only used by the proper build, so I don't think 
>> we can refer to it from here?
>
> Hi,
>
> Sorry for not having had time to look at this yet.
>
> The solution should be to use FIXPATH at all times when compiling on 
> Windows if using local paths, even in the configure script. It might 
> be the case that we have not had to do this as of yet, so it might not 
> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC" 
> on Windows, so I thought it should work by itself.
>
> To get around the mess of windows and unix style paths, the decision 
> was made to use only unix paths *all the time*. The duct tape needed 
> to get this to work is the fixpath launcher, which takes a 
> command-line with unix paths and converts it to windows paths at the 
> very last moment before launching the executable.
>
> Does things work better if you to:
> CC="$FIXPATH $CC"
> before your AC_CHECK_HEADER?

I'll give this a try - thanks

Maurizio
>
> /Magnus
>
>
>>
>> Thanks
>> Maurizio
>>> /Magnus
>>>
>>>
>>>>
>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming 
>>>> paths which can possibly contain forward slashes - e.g.
>>>>
>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>>>
>>>>
>>>> Maurizio
>>>>
>>>>
>>>>
>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>>> FTR, as I'm looking at other configure files, I've seen
>>>>>
>>>>> ?"xwindows" used not "xmicrosoft"
>>>>>
>>>>> As for the need for double backslash, I don't see any special 
>>>>> processing done in other configure files prior to the 
>>>>> AC_CHECK_HEADER call.
>>>>>
>>>>> The build guide [1] is very explicit that forward slashes should 
>>>>> be used in options (unlike what you did) and mentions some 
>>>>> 'fixpath' tool which is used by configure to convert Unix paths 
>>>>> into Windows ones. I wonder if this could be a bug in that tool?
>>>>>
>>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 
>>>>> - it could be worth chasing down as to where the compiled 
>>>>> 'fixpath' executable is put (should be somewhere in 
>>>>> build/<conf>/configure-support/) call it with the clang include 
>>>>> folder with Unix-style forward slash and see whether it spits the 
>>>>> correct path.
>>>>>
>>>>> The source code for this tool can be found here:
>>>>>
>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>>>>
>>>>>
>>>>> Maurizio
>>>>>
>>>>> [1] - 
>>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>>>>>
>>>>>
>>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>>> Actually, -I works, and link option need to be passed with /link, 
>>>>>> but you got the idea?
>>>>>>
>>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not 
>>>>>> sure it would work. Need to find a better way to check the lib.
>>>>>>
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018 
>>>>>> -0700
>>>>>> @@ -68,7 +68,11 @@
>>>>>> ????????? LIBCLANG_CPPFLAGS=""
>>>>>> ????? fi
>>>>>> ????? if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> ????????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +????? else
>>>>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>>> +????? fi
>>>>>> ????? else
>>>>>> ????????? LIBCLANG_LDFLAGS=""
>>>>>> ????? fi
>>>>>>
>>>>>> Cheers,
>>>>>> Henry
>>>>>>
>>>>>>
>>>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> 
>>>>>>> wrote:
>>>>>>>
>>>>>>> Haven?t test it, but try this,
>>>>>>>
>>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018 
>>>>>>> -0700
>>>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018 
>>>>>>> -0700
>>>>>>> @@ -63,12 +63,20 @@
>>>>>>> ???? fi
>>>>>>>
>>>>>>> ???? if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>>> ???????? LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>>>> +????? else
>>>>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>>>> +????? fi
>>>>>>> ???? else
>>>>>>> ???????? LIBCLANG_CPPFLAGS=""
>>>>>>> ???? fi
>>>>>>> ???? if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>>> ???????? LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>>> +????? else
>>>>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>>>> +????? fi
>>>>>>> ???? else
>>>>>>> ???????? LIBCLANG_LDFLAGS=""
>>>>>>> ???? fi
>>>>>>>
>>>>>>> We still need to fix the copy of DLL.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Henry
>>>>>>>
>>>>>>>
>>>>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore 
>>>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>>>
>>>>>>>> Looks like good progress; tomorrow I'll take a look at some of 
>>>>>>>> our build files and see if something special is done for 
>>>>>>>> mangling windows include paths.
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>>
>>>>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>>>>> If I use the following flags:
>>>>>>>>>
>>>>>>>>> $ bash configure --disable-warnings-as-errors 
>>>>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13' 
>>>>>>>>> --with-libclang-include='J:\\LLVM\\include' 
>>>>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib' 
>>>>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>>>>>
>>>>>>>>> (Notice that I'm having to use different path styles)
>>>>>>>>>
>>>>>>>>> It's detecting the header files, but it's failing on being 
>>>>>>>>> passed the wrong flags. from config.log (see at the bottom):
>>>>>>>>>
>>>>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>>>>> configure:60273: 
>>>>>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>>>>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib 
>>>>>>>>> conftest.cpp -lclang?? >&5
>>>>>>>>> conftest.cpp
>>>>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>>>>
>>>>>>>>> /out:conftest.exe
>>>>>>>>> /out:conftest.exe
>>>>>>>>> conftest.obj
>>>>>>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>>>>>>> clang_getClangVersion referenced in function main
>>>>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 
>>>>>>>>> for x64
>>>>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>>>>
>>>>>>>>> cl : Command line warning D9035 : option 'o' has been 
>>>>>>>>> deprecated and will be removed in a future release
>>>>>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>>>>>> '-LJ:\\LLVM\\lib'
>>>>>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>>>>>> '-lclang'
>>>>>>>>>
>>>>>>>>> That seems to be a simple problem of casing-off windows and 
>>>>>>>>> passing the right flags, but I call it a night here :)
>>>>>>>>>
>>>>>>>>> Jorn
>>>>>>>>>
>>>>>
>>>>
>>>
>>
>


From jbvernee at xs4all.nl  Thu Sep 27 14:37:03 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 27 Sep 2018 16:37:03 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
Message-ID: <069e341fd48b736d55160cca9447b857@xs4all.nl>

AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it is 
there when using `CXX="$FIXPATH $CXX"".

Checking the header now works, but it's breaking when using 
`AC_CHECK_LIB`:

configure:60545: checking for clang_getClangVersion in -lclang
configure:60570: 
/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
-c 
/cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
-o conftest.exe    -I/cygdrive/j/LLVM/include -L/cygdrive/j/LLVM/lib 
conftest.cpp -lclang   >&5
conftest.cpp
Microsoft (R) Incremental Linker Version 14.14.26430.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:conftest.exe
/out:conftest.exe
conftest.obj
conftest.obj : error LNK2019: unresolved external symbol 
clang_getClangVersion referenced in function main
conftest.exe : fatal error LNK1120: 1 unresolved externals
Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and 
will be removed in a future release
cl : Command line warning D9002 : ignoring unknown option 
'-Lj:/LLVM/lib'
cl : Command line warning D9002 : ignoring unknown option '-lclang'
configure:60570: $? = 2

I'm also seeing the following warnings:

The following warnings were produced. Repeated here for convenience:
WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
preprocessor!
WARNING: clang-c/Index.h: proceeding with the compiler's result

FWIW Not sure if the lib check can work with AC_CHECK_LIB, which seems 
to add the `-l` on it's own (note the unrecognized option warnings in 
the log). For the MS linker AFAIK you'd need `/DEFAULTLIB:clang`, or to 
pass a .lib or .pdb file on the command line instead.

Jorn

[1] : 
https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017

Maurizio Cimadamore schreef op 2018-09-27 15:41:
> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
> 
>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>> 
>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>> 
>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>> Try this:
>> 
>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4    Tue Sep 11 18:39:11 2018 +0100
>> +++ b/make/autoconf/lib-clang.m4    Thu Sep 20 09:59:08 2018 +0100
>> @@ -60,6 +60,9 @@
>> VER=`ls $with_libclang/lib/clang/`
>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>> CLANG_LIB_PATH="$with_libclang/lib"
>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>> +      BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>> fi
>> 
>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>> 
>> Now, if you use only the --with-libclang option with a Unix-style
>> path (the thing you were trying at first), I believe it should do
>> the right thing.
>> 
>> I have not followed the entire conversation, but this part looks
>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>> from configure argument. (And these should, yes indeed, be in unix
>> style). However, this might not be all fixes that are needed. I can
>> help take a look at it, if someone points me to where the
>> problematic code resides.
>  Magnus, I followed up a bit on this, and managed to reproduce the
> issue.
> 
> The underlying issue is that our lib-clang.m4 conf file checks for
> headers in the usual way with AC_CHECK_HEADER. This check relies on
> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
> fixups, what you end up with is this:
> 
> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
> 
> This is the correct _cygwin_ path, but it's not the correct Windows
> path. In fact, if you try to compile a dummy header that contains this
> (this was also observed by Jorn)
> 
> #include <clang-c/Index.h>
> 
> with a -I option that includes the above path using the cl.exe
> compiler within cygwin, you get an error because the include file
> cannot be found. The only solution is to use a Windows path, which in
> cygwin can only be done using double backslashes (ugh).
> 
> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
> path, while here we need the opposite operation. I was under the
> impression that the build configure framework needed to solve this
> particular issue anyway, to check for other headers, but I realized
> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is
> AC_CHECK_HEADER even supposed to work under Windows? If not, what
> should we use in order to make our configure script portable?
> 
> P.S.
> I note that there's also a FIXPATH script which does exactly what we
> need here - but this script seems just to be set by the configure step
> and is probably only used by the proper build, so I don't think we can
> refer to it from here?
> 
> Hi,
> 
> Sorry for not having had time to look at this yet.
> 
> The solution should be to use FIXPATH at all times when compiling on
> Windows if using local paths, even in the configure script. It might
> be the case that we have not had to do this as of yet, so it might not
> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC"
> on Windows, so I thought it should work by itself.
> 
> To get around the mess of windows and unix style paths, the decision
> was made to use only unix paths *all the time*. The duct tape needed
> to get this to work is the fixpath launcher, which takes a
> command-line with unix paths and converts it to windows paths at the
> very last moment before launching the executable.
> 
> Does things work better if you to:
> CC="$FIXPATH $CC"
> before your AC_CHECK_HEADER?
> 
> I'll give this a try - thanks
> 
> Maurizio
> 
>> /Magnus
>> 
>> Thanks
>> Maurizio
>> /Magnus
>> 
>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>> paths which can possibly contain forward slashes - e.g.
>> 
>> 
> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49
>> 
>> 
>> Maurizio
>> 
>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>> FTR, as I'm looking at other configure files, I've seen
>> 
>> "xwindows" used not "xmicrosoft"
>> 
>> As for the need for double backslash, I don't see any special
>> processing done in other configure files prior to the
>> AC_CHECK_HEADER call.
>> 
>> The build guide [1] is very explicit that forward slashes should be
>> used in options (unlike what you did) and mentions some 'fixpath'
>> tool which is used by configure to convert Unix paths into Windows
>> ones. I wonder if this could be a bug in that tool?
>> 
>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>> it could be worth chasing down as to where the compiled 'fixpath'
>> executable is put (should be somewhere in
>> build/<conf>/configure-support/) call it with the clang include
>> folder with Unix-style forward slash and see whether it spits the
>> correct path.
>> 
>> The source code for this tool can be found here:
>> 
>> 
> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c
>> 
>> 
>> Maurizio
>> 
>> [1] -
>> 
> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>> 
>> On 20/09/18 00:41, Henry Jen wrote:
>> Actually, -I works, and link option need to be passed with /link,
>> but you got the idea?
>> 
>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>> sure it would work. Need to find a better way to check the lib.
>> 
>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>> -0700
>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018
>> -0700
>> @@ -68,7 +68,11 @@
>> LIBCLANG_CPPFLAGS=""
>> fi
>> if test "x$CLANG_LIB_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>> +      else
>> +        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>> +      fi
>> else
>> LIBCLANG_LDFLAGS=""
>> fi
>> 
>> Cheers,
>> Henry
>> 
>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>> 
>> 
>> Haven?t test it, but try this,
>> 
>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>> -0700
>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018
>> -0700
>> @@ -63,12 +63,20 @@
>> fi
>> 
>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>> +      else
>> +        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>> +      fi
>> else
>> LIBCLANG_CPPFLAGS=""
>> fi
>> if test "x$CLANG_LIB_PATH" != "x"; then
>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>> +      else
>> +        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>> +      fi
>> else
>> LIBCLANG_LDFLAGS=""
>> fi
>> 
>> We still need to fix the copy of DLL.
>> 
>> Cheers,
>> Henry
>> 
>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com> wrote:
>> 
>> Looks like good progress; tomorrow I'll take a look at some of our
>> build files and see if something special is done for mangling
>> windows include paths.
>> 
>> Cheers
>> Maurizio
>> 
>> On 19/09/18 23:12, Jorn Vernee wrote:
>> If I use the following flags:
>> 
>> $ bash configure --disable-warnings-as-errors
>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>> --with-libclang-include='J:\\LLVM\\include'
>> --with-libclang-include-aux='J:\\LLVM\\lib'
>> --with-libclang-lib='J:\\LLVM\\lib'
>> 
>> (Notice that I'm having to use different path styles)
>> 
>> It's detecting the header files, but it's failing on being passed
>> the wrong flags. from config.log (see at the bottom):
>> 
>> configure:60248: checking for clang_getClangVersion in -lclang
>> configure:60273:
>> 
> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl
>> -o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>> -lclang   >&5
>> conftest.cpp
>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>> 
>> /out:conftest.exe
>> /out:conftest.exe
>> conftest.obj
>> conftest.obj : error LNK2019: unresolved external symbol
>> clang_getClangVersion referenced in function main
>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>> 
>> cl : Command line warning D9035 : option 'o' has been deprecated and
>> will be removed in a future release
>> cl : Command line warning D9002 : ignoring unknown option
>> '-LJ:\\LLVM\\lib'
>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>> 
>> That seems to be a simple problem of casing-off windows and passing
>> the right flags, but I call it a night here :)
>> 
>> Jorn

From maurizio.cimadamore at oracle.com  Thu Sep 27 14:58:19 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 27 Sep 2018 15:58:19 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <069e341fd48b736d55160cca9447b857@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <F99F0C9F-B3AC-4307-AB4A-71D8111C41CC@oracle.com>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
 <069e341fd48b736d55160cca9447b857@xs4all.nl>
Message-ID: <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>

Got the same on my windows box. I also replaced Magnus's CC suggestion 
with CXX which got me a bit further.

Maurizio



On 27/09/18 15:37, Jorn Vernee wrote:
> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
> fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it is 
> there when using `CXX="$FIXPATH $CXX"".
>
> Checking the header now works, but it's breaking when using 
> `AC_CHECK_LIB`:
>
> configure:60545: checking for clang_getClangVersion in -lclang
> configure:60570: 
> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
> -c 
> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
> -o conftest.exe??? -I/cygdrive/j/LLVM/include -L/cygdrive/j/LLVM/lib 
> conftest.cpp -lclang?? >&5
> conftest.cpp
> Microsoft (R) Incremental Linker Version 14.14.26430.0
> Copyright (C) Microsoft Corporation.? All rights reserved.
>
> /out:conftest.exe
> /out:conftest.exe
> conftest.obj
> conftest.obj : error LNK2019: unresolved external symbol 
> clang_getClangVersion referenced in function main
> conftest.exe : fatal error LNK1120: 1 unresolved externals
> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
> Copyright (C) Microsoft Corporation.? All rights reserved.
>
> cl : Command line warning D9035 : option 'o' has been deprecated and 
> will be removed in a future release
> cl : Command line warning D9002 : ignoring unknown option '-Lj:/LLVM/lib'
> cl : Command line warning D9002 : ignoring unknown option '-lclang'
> configure:60570: $? = 2
>
> I'm also seeing the following warnings:
>
> The following warnings were produced. Repeated here for convenience:
> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
> preprocessor!
> WARNING: clang-c/Index.h: proceeding with the compiler's result
>
> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which seems 
> to add the `-l` on it's own (note the unrecognized option warnings in 
> the log). For the MS linker AFAIK you'd need `/DEFAULTLIB:clang`, or 
> to pass a .lib or .pdb file on the command line instead.
>
> Jorn
>
> [1] : 
> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>
> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>
>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>
>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>
>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>> Try this:
>>>
>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>> @@ -60,6 +60,9 @@
>>> VER=`ls $with_libclang/lib/clang/`
>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>> CLANG_LIB_PATH="$with_libclang/lib"
>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>> +????? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>> fi
>>>
>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>
>>> Now, if you use only the --with-libclang option with a Unix-style
>>> path (the thing you were trying at first), I believe it should do
>>> the right thing.
>>>
>>> I have not followed the entire conversation, but this part looks
>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>>> from configure argument. (And these should, yes indeed, be in unix
>>> style). However, this might not be all fixes that are needed. I can
>>> help take a look at it, if someone points me to where the
>>> problematic code resides.
>> ?Magnus, I followed up a bit on this, and managed to reproduce the
>> issue.
>>
>> The underlying issue is that our lib-clang.m4 conf file checks for
>> headers in the usual way with AC_CHECK_HEADER. This check relies on
>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>> fixups, what you end up with is this:
>>
>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>
>> This is the correct _cygwin_ path, but it's not the correct Windows
>> path. In fact, if you try to compile a dummy header that contains this
>> (this was also observed by Jorn)
>>
>> #include <clang-c/Index.h>
>>
>> with a -I option that includes the above path using the cl.exe
>> compiler within cygwin, you get an error because the include file
>> cannot be found. The only solution is to use a Windows path, which in
>> cygwin can only be done using double backslashes (ugh).
>>
>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>> path, while here we need the opposite operation. I was under the
>> impression that the build configure framework needed to solve this
>> particular issue anyway, to check for other headers, but I realized
>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is
>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>> should we use in order to make our configure script portable?
>>
>> P.S.
>> I note that there's also a FIXPATH script which does exactly what we
>> need here - but this script seems just to be set by the configure step
>> and is probably only used by the proper build, so I don't think we can
>> refer to it from here?
>>
>> Hi,
>>
>> Sorry for not having had time to look at this yet.
>>
>> The solution should be to use FIXPATH at all times when compiling on
>> Windows if using local paths, even in the configure script. It might
>> be the case that we have not had to do this as of yet, so it might not
>> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC"
>> on Windows, so I thought it should work by itself.
>>
>> To get around the mess of windows and unix style paths, the decision
>> was made to use only unix paths *all the time*. The duct tape needed
>> to get this to work is the fixpath launcher, which takes a
>> command-line with unix paths and converts it to windows paths at the
>> very last moment before launching the executable.
>>
>> Does things work better if you to:
>> CC="$FIXPATH $CC"
>> before your AC_CHECK_HEADER?
>>
>> I'll give this a try - thanks
>>
>> Maurizio
>>
>>> /Magnus
>>>
>>> Thanks
>>> Maurizio
>>> /Magnus
>>>
>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>> paths which can possibly contain forward slashes - e.g.
>>>
>>>
>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>
>>>
>>>
>>> Maurizio
>>>
>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>> FTR, as I'm looking at other configure files, I've seen
>>>
>>> "xwindows" used not "xmicrosoft"
>>>
>>> As for the need for double backslash, I don't see any special
>>> processing done in other configure files prior to the
>>> AC_CHECK_HEADER call.
>>>
>>> The build guide [1] is very explicit that forward slashes should be
>>> used in options (unlike what you did) and mentions some 'fixpath'
>>> tool which is used by configure to convert Unix paths into Windows
>>> ones. I wonder if this could be a bug in that tool?
>>>
>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>>> it could be worth chasing down as to where the compiled 'fixpath'
>>> executable is put (should be somewhere in
>>> build/<conf>/configure-support/) call it with the clang include
>>> folder with Unix-style forward slash and see whether it spits the
>>> correct path.
>>>
>>> The source code for this tool can be found here:
>>>
>>>
>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>
>>>
>>>
>>> Maurizio
>>>
>>> [1] -
>>>
>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows 
>>
>>>
>>> On 20/09/18 00:41, Henry Jen wrote:
>>> Actually, -I works, and link option need to be passed with /link,
>>> but you got the idea?
>>>
>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>> sure it would work. Need to find a better way to check the lib.
>>>
>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>> -0700
>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018
>>> -0700
>>> @@ -68,7 +68,11 @@
>>> LIBCLANG_CPPFLAGS=""
>>> fi
>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>> +????? else
>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>> +????? fi
>>> else
>>> LIBCLANG_LDFLAGS=""
>>> fi
>>>
>>> Cheers,
>>> Henry
>>>
>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>
>>>
>>> Haven?t test it, but try this,
>>>
>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>> -0700
>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018
>>> -0700
>>> @@ -63,12 +63,20 @@
>>> fi
>>>
>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>> +????? else
>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>> +????? fi
>>> else
>>> LIBCLANG_CPPFLAGS=""
>>> fi
>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>> +????? else
>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>> +????? fi
>>> else
>>> LIBCLANG_LDFLAGS=""
>>> fi
>>>
>>> We still need to fix the copy of DLL.
>>>
>>> Cheers,
>>> Henry
>>>
>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>> <maurizio.cimadamore at oracle.com> wrote:
>>>
>>> Looks like good progress; tomorrow I'll take a look at some of our
>>> build files and see if something special is done for mangling
>>> windows include paths.
>>>
>>> Cheers
>>> Maurizio
>>>
>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>> If I use the following flags:
>>>
>>> $ bash configure --disable-warnings-as-errors
>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>> --with-libclang-include='J:\\LLVM\\include'
>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>
>>> (Notice that I'm having to use different path styles)
>>>
>>> It's detecting the header files, but it's failing on being passed
>>> the wrong flags. from config.log (see at the bottom):
>>>
>>> configure:60248: checking for clang_getClangVersion in -lclang
>>> configure:60273:
>>>
>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>
>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>> -lclang?? >&5
>>> conftest.cpp
>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>
>>> /out:conftest.exe
>>> /out:conftest.exe
>>> conftest.obj
>>> conftest.obj : error LNK2019: unresolved external symbol
>>> clang_getClangVersion referenced in function main
>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>
>>> cl : Command line warning D9035 : option 'o' has been deprecated and
>>> will be removed in a future release
>>> cl : Command line warning D9002 : ignoring unknown option
>>> '-LJ:\\LLVM\\lib'
>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>
>>> That seems to be a simple problem of casing-off windows and passing
>>> the right flags, but I call it a night here :)
>>>
>>> Jorn


From maurizio.cimadamore at oracle.com  Thu Sep 27 15:11:03 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 27 Sep 2018 16:11:03 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
 <069e341fd48b736d55160cca9447b857@xs4all.nl>
 <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>
Message-ID: <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>

Btw, the issue in my case seems to be cause by the fact that cl.exe cant 
locate <time.h> (which is included by clang-c/Index.h header)

Maurizio


On 27/09/18 15:58, Maurizio Cimadamore wrote:
> Got the same on my windows box. I also replaced Magnus's CC suggestion 
> with CXX which got me a bit further.
>
> Maurizio
>
>
>
> On 27/09/18 15:37, Jorn Vernee wrote:
>> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
>> fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it is 
>> there when using `CXX="$FIXPATH $CXX"".
>>
>> Checking the header now works, but it's breaking when using 
>> `AC_CHECK_LIB`:
>>
>> configure:60545: checking for clang_getClangVersion in -lclang
>> configure:60570: 
>> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
>> -c 
>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>> -o conftest.exe??? -I/cygdrive/j/LLVM/include -L/cygdrive/j/LLVM/lib 
>> conftest.cpp -lclang?? >&5
>> conftest.cpp
>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>
>> /out:conftest.exe
>> /out:conftest.exe
>> conftest.obj
>> conftest.obj : error LNK2019: unresolved external symbol 
>> clang_getClangVersion referenced in function main
>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>
>> cl : Command line warning D9035 : option 'o' has been deprecated and 
>> will be removed in a future release
>> cl : Command line warning D9002 : ignoring unknown option 
>> '-Lj:/LLVM/lib'
>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>> configure:60570: $? = 2
>>
>> I'm also seeing the following warnings:
>>
>> The following warnings were produced. Repeated here for convenience:
>> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
>> preprocessor!
>> WARNING: clang-c/Index.h: proceeding with the compiler's result
>>
>> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which 
>> seems to add the `-l` on it's own (note the unrecognized option 
>> warnings in the log). For the MS linker AFAIK you'd need 
>> `/DEFAULTLIB:clang`, or to pass a .lib or .pdb file on the command 
>> line instead.
>>
>> Jorn
>>
>> [1] : 
>> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>>
>> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>>
>>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>>
>>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>>
>>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>> Try this:
>>>>
>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>>> @@ -60,6 +60,9 @@
>>>> VER=`ls $with_libclang/lib/clang/`
>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>> CLANG_LIB_PATH="$with_libclang/lib"
>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>> +????? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>> fi
>>>>
>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>
>>>> Now, if you use only the --with-libclang option with a Unix-style
>>>> path (the thing you were trying at first), I believe it should do
>>>> the right thing.
>>>>
>>>> I have not followed the entire conversation, but this part looks
>>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>>>> from configure argument. (And these should, yes indeed, be in unix
>>>> style). However, this might not be all fixes that are needed. I can
>>>> help take a look at it, if someone points me to where the
>>>> problematic code resides.
>>> ?Magnus, I followed up a bit on this, and managed to reproduce the
>>> issue.
>>>
>>> The underlying issue is that our lib-clang.m4 conf file checks for
>>> headers in the usual way with AC_CHECK_HEADER. This check relies on
>>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>>> fixups, what you end up with is this:
>>>
>>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>>
>>> This is the correct _cygwin_ path, but it's not the correct Windows
>>> path. In fact, if you try to compile a dummy header that contains this
>>> (this was also observed by Jorn)
>>>
>>> #include <clang-c/Index.h>
>>>
>>> with a -I option that includes the above path using the cl.exe
>>> compiler within cygwin, you get an error because the include file
>>> cannot be found. The only solution is to use a Windows path, which in
>>> cygwin can only be done using double backslashes (ugh).
>>>
>>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>>> path, while here we need the opposite operation. I was under the
>>> impression that the build configure framework needed to solve this
>>> particular issue anyway, to check for other headers, but I realized
>>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is
>>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>>> should we use in order to make our configure script portable?
>>>
>>> P.S.
>>> I note that there's also a FIXPATH script which does exactly what we
>>> need here - but this script seems just to be set by the configure step
>>> and is probably only used by the proper build, so I don't think we can
>>> refer to it from here?
>>>
>>> Hi,
>>>
>>> Sorry for not having had time to look at this yet.
>>>
>>> The solution should be to use FIXPATH at all times when compiling on
>>> Windows if using local paths, even in the configure script. It might
>>> be the case that we have not had to do this as of yet, so it might not
>>> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC"
>>> on Windows, so I thought it should work by itself.
>>>
>>> To get around the mess of windows and unix style paths, the decision
>>> was made to use only unix paths *all the time*. The duct tape needed
>>> to get this to work is the fixpath launcher, which takes a
>>> command-line with unix paths and converts it to windows paths at the
>>> very last moment before launching the executable.
>>>
>>> Does things work better if you to:
>>> CC="$FIXPATH $CC"
>>> before your AC_CHECK_HEADER?
>>>
>>> I'll give this a try - thanks
>>>
>>> Maurizio
>>>
>>>> /Magnus
>>>>
>>>> Thanks
>>>> Maurizio
>>>> /Magnus
>>>>
>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>>> paths which can possibly contain forward slashes - e.g.
>>>>
>>>>
>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>>
>>>>
>>>>
>>>> Maurizio
>>>>
>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>> FTR, as I'm looking at other configure files, I've seen
>>>>
>>>> "xwindows" used not "xmicrosoft"
>>>>
>>>> As for the need for double backslash, I don't see any special
>>>> processing done in other configure files prior to the
>>>> AC_CHECK_HEADER call.
>>>>
>>>> The build guide [1] is very explicit that forward slashes should be
>>>> used in options (unlike what you did) and mentions some 'fixpath'
>>>> tool which is used by configure to convert Unix paths into Windows
>>>> ones. I wonder if this could be a bug in that tool?
>>>>
>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>>>> it could be worth chasing down as to where the compiled 'fixpath'
>>>> executable is put (should be somewhere in
>>>> build/<conf>/configure-support/) call it with the clang include
>>>> folder with Unix-style forward slash and see whether it spits the
>>>> correct path.
>>>>
>>>> The source code for this tool can be found here:
>>>>
>>>>
>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>>
>>>>
>>>>
>>>> Maurizio
>>>>
>>>> [1] -
>>>>
>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows 
>>>
>>>>
>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>> Actually, -I works, and link option need to be passed with /link,
>>>> but you got the idea?
>>>>
>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>>> sure it would work. Need to find a better way to check the lib.
>>>>
>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>> -0700
>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018
>>>> -0700
>>>> @@ -68,7 +68,11 @@
>>>> LIBCLANG_CPPFLAGS=""
>>>> fi
>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>> +????? else
>>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>> +????? fi
>>>> else
>>>> LIBCLANG_LDFLAGS=""
>>>> fi
>>>>
>>>> Cheers,
>>>> Henry
>>>>
>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>
>>>>
>>>> Haven?t test it, but try this,
>>>>
>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>> -0700
>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018
>>>> -0700
>>>> @@ -63,12 +63,20 @@
>>>> fi
>>>>
>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>> +????? else
>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>> +????? fi
>>>> else
>>>> LIBCLANG_CPPFLAGS=""
>>>> fi
>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>> +????? else
>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>> +????? fi
>>>> else
>>>> LIBCLANG_LDFLAGS=""
>>>> fi
>>>>
>>>> We still need to fix the copy of DLL.
>>>>
>>>> Cheers,
>>>> Henry
>>>>
>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>
>>>> Looks like good progress; tomorrow I'll take a look at some of our
>>>> build files and see if something special is done for mangling
>>>> windows include paths.
>>>>
>>>> Cheers
>>>> Maurizio
>>>>
>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>> If I use the following flags:
>>>>
>>>> $ bash configure --disable-warnings-as-errors
>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>>> --with-libclang-include='J:\\LLVM\\include'
>>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>
>>>> (Notice that I'm having to use different path styles)
>>>>
>>>> It's detecting the header files, but it's failing on being passed
>>>> the wrong flags. from config.log (see at the bottom):
>>>>
>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>> configure:60273:
>>>>
>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>
>>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>>> -lclang?? >&5
>>>> conftest.cpp
>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>
>>>> /out:conftest.exe
>>>> /out:conftest.exe
>>>> conftest.obj
>>>> conftest.obj : error LNK2019: unresolved external symbol
>>>> clang_getClangVersion referenced in function main
>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>
>>>> cl : Command line warning D9035 : option 'o' has been deprecated and
>>>> will be removed in a future release
>>>> cl : Command line warning D9002 : ignoring unknown option
>>>> '-LJ:\\LLVM\\lib'
>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>
>>>> That seems to be a simple problem of casing-off windows and passing
>>>> the right flags, but I call it a night here :)
>>>>
>>>> Jorn
>


From maurizio.cimadamore at oracle.com  Thu Sep 27 15:55:09 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Thu, 27 Sep 2018 16:55:09 +0100
Subject: [foreign] Running tests on Windows
In-Reply-To: <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
 <069e341fd48b736d55160cca9447b857@xs4all.nl>
 <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>
 <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>
Message-ID: <8f22bf90-3c69-d5ed-a84e-943585b676f6@oracle.com>

NVM,
I think the real issue is how AC_CHECK_LIB invokes the compiler - e.g. 
with -lclang

In MSVC this should be /link libclang.lib

Not sure how you tell AC_CHECK_LIB to use a different command, but 
frankly it seems like AC_XYZ checks are GNU-centric and they don't play 
well with Windows - it might be a lost cause to try and make them work 
under cygwin.

Maurizio


On 27/09/18 16:11, Maurizio Cimadamore wrote:
> Btw, the issue in my case seems to be cause by the fact that cl.exe 
> cant locate <time.h> (which is included by clang-c/Index.h header)
>
> Maurizio
>
>
> On 27/09/18 15:58, Maurizio Cimadamore wrote:
>> Got the same on my windows box. I also replaced Magnus's CC 
>> suggestion with CXX which got me a bit further.
>>
>> Maurizio
>>
>>
>>
>> On 27/09/18 15:37, Jorn Vernee wrote:
>>> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
>>> fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it is 
>>> there when using `CXX="$FIXPATH $CXX"".
>>>
>>> Checking the header now works, but it's breaking when using 
>>> `AC_CHECK_LIB`:
>>>
>>> configure:60545: checking for clang_getClangVersion in -lclang
>>> configure:60570: 
>>> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
>>> -c 
>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>> -o conftest.exe??? -I/cygdrive/j/LLVM/include -L/cygdrive/j/LLVM/lib 
>>> conftest.cpp -lclang?? >&5
>>> conftest.cpp
>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>
>>> /out:conftest.exe
>>> /out:conftest.exe
>>> conftest.obj
>>> conftest.obj : error LNK2019: unresolved external symbol 
>>> clang_getClangVersion referenced in function main
>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>
>>> cl : Command line warning D9035 : option 'o' has been deprecated and 
>>> will be removed in a future release
>>> cl : Command line warning D9002 : ignoring unknown option 
>>> '-Lj:/LLVM/lib'
>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>> configure:60570: $? = 2
>>>
>>> I'm also seeing the following warnings:
>>>
>>> The following warnings were produced. Repeated here for convenience:
>>> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
>>> preprocessor!
>>> WARNING: clang-c/Index.h: proceeding with the compiler's result
>>>
>>> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which 
>>> seems to add the `-l` on it's own (note the unrecognized option 
>>> warnings in the log). For the MS linker AFAIK you'd need 
>>> `/DEFAULTLIB:clang`, or to pass a .lib or .pdb file on the command 
>>> line instead.
>>>
>>> Jorn
>>>
>>> [1] : 
>>> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>>>
>>> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>>>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>>>
>>>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>>>
>>>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>>>
>>>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>>> Try this:
>>>>>
>>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>>>> @@ -60,6 +60,9 @@
>>>>> VER=`ls $with_libclang/lib/clang/`
>>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>>> CLANG_LIB_PATH="$with_libclang/lib"
>>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>>> +????? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>>> fi
>>>>>
>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>
>>>>> Now, if you use only the --with-libclang option with a Unix-style
>>>>> path (the thing you were trying at first), I believe it should do
>>>>> the right thing.
>>>>>
>>>>> I have not followed the entire conversation, but this part looks
>>>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>>>>> from configure argument. (And these should, yes indeed, be in unix
>>>>> style). However, this might not be all fixes that are needed. I can
>>>>> help take a look at it, if someone points me to where the
>>>>> problematic code resides.
>>>> ?Magnus, I followed up a bit on this, and managed to reproduce the
>>>> issue.
>>>>
>>>> The underlying issue is that our lib-clang.m4 conf file checks for
>>>> headers in the usual way with AC_CHECK_HEADER. This check relies on
>>>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>>>> fixups, what you end up with is this:
>>>>
>>>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>>>
>>>> This is the correct _cygwin_ path, but it's not the correct Windows
>>>> path. In fact, if you try to compile a dummy header that contains this
>>>> (this was also observed by Jorn)
>>>>
>>>> #include <clang-c/Index.h>
>>>>
>>>> with a -I option that includes the above path using the cl.exe
>>>> compiler within cygwin, you get an error because the include file
>>>> cannot be found. The only solution is to use a Windows path, which in
>>>> cygwin can only be done using double backslashes (ugh).
>>>>
>>>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>>>> path, while here we need the opposite operation. I was under the
>>>> impression that the build configure framework needed to solve this
>>>> particular issue anyway, to check for other headers, but I realized
>>>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is
>>>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>>>> should we use in order to make our configure script portable?
>>>>
>>>> P.S.
>>>> I note that there's also a FIXPATH script which does exactly what we
>>>> need here - but this script seems just to be set by the configure step
>>>> and is probably only used by the proper build, so I don't think we can
>>>> refer to it from here?
>>>>
>>>> Hi,
>>>>
>>>> Sorry for not having had time to look at this yet.
>>>>
>>>> The solution should be to use FIXPATH at all times when compiling on
>>>> Windows if using local paths, even in the configure script. It might
>>>> be the case that we have not had to do this as of yet, so it might not
>>>> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC"
>>>> on Windows, so I thought it should work by itself.
>>>>
>>>> To get around the mess of windows and unix style paths, the decision
>>>> was made to use only unix paths *all the time*. The duct tape needed
>>>> to get this to work is the fixpath launcher, which takes a
>>>> command-line with unix paths and converts it to windows paths at the
>>>> very last moment before launching the executable.
>>>>
>>>> Does things work better if you to:
>>>> CC="$FIXPATH $CC"
>>>> before your AC_CHECK_HEADER?
>>>>
>>>> I'll give this a try - thanks
>>>>
>>>> Maurizio
>>>>
>>>>> /Magnus
>>>>>
>>>>> Thanks
>>>>> Maurizio
>>>>> /Magnus
>>>>>
>>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>>>> paths which can possibly contain forward slashes - e.g.
>>>>>
>>>>>
>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>>>
>>>>>
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>>> FTR, as I'm looking at other configure files, I've seen
>>>>>
>>>>> "xwindows" used not "xmicrosoft"
>>>>>
>>>>> As for the need for double backslash, I don't see any special
>>>>> processing done in other configure files prior to the
>>>>> AC_CHECK_HEADER call.
>>>>>
>>>>> The build guide [1] is very explicit that forward slashes should be
>>>>> used in options (unlike what you did) and mentions some 'fixpath'
>>>>> tool which is used by configure to convert Unix paths into Windows
>>>>> ones. I wonder if this could be a bug in that tool?
>>>>>
>>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>>>>> it could be worth chasing down as to where the compiled 'fixpath'
>>>>> executable is put (should be somewhere in
>>>>> build/<conf>/configure-support/) call it with the clang include
>>>>> folder with Unix-style forward slash and see whether it spits the
>>>>> correct path.
>>>>>
>>>>> The source code for this tool can be found here:
>>>>>
>>>>>
>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>>>
>>>>>
>>>>>
>>>>> Maurizio
>>>>>
>>>>> [1] -
>>>>>
>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows 
>>>>
>>>>>
>>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>> Actually, -I works, and link option need to be passed with /link,
>>>>> but you got the idea?
>>>>>
>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>>>> sure it would work. Need to find a better way to check the lib.
>>>>>
>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>>> -0700
>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018
>>>>> -0700
>>>>> @@ -68,7 +68,11 @@
>>>>> LIBCLANG_CPPFLAGS=""
>>>>> fi
>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>> +????? fi
>>>>> else
>>>>> LIBCLANG_LDFLAGS=""
>>>>> fi
>>>>>
>>>>> Cheers,
>>>>> Henry
>>>>>
>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>>
>>>>>
>>>>> Haven?t test it, but try this,
>>>>>
>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>>> -0700
>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018
>>>>> -0700
>>>>> @@ -63,12 +63,20 @@
>>>>> fi
>>>>>
>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>> +????? fi
>>>>> else
>>>>> LIBCLANG_CPPFLAGS=""
>>>>> fi
>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>> +????? fi
>>>>> else
>>>>> LIBCLANG_LDFLAGS=""
>>>>> fi
>>>>>
>>>>> We still need to fix the copy of DLL.
>>>>>
>>>>> Cheers,
>>>>> Henry
>>>>>
>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>
>>>>> Looks like good progress; tomorrow I'll take a look at some of our
>>>>> build files and see if something special is done for mangling
>>>>> windows include paths.
>>>>>
>>>>> Cheers
>>>>> Maurizio
>>>>>
>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>> If I use the following flags:
>>>>>
>>>>> $ bash configure --disable-warnings-as-errors
>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>>>> --with-libclang-include='J:\\LLVM\\include'
>>>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>
>>>>> (Notice that I'm having to use different path styles)
>>>>>
>>>>> It's detecting the header files, but it's failing on being passed
>>>>> the wrong flags. from config.log (see at the bottom):
>>>>>
>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>> configure:60273:
>>>>>
>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>
>>>>> -o conftest.exe -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>>>> -lclang?? >&5
>>>>> conftest.cpp
>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>
>>>>> /out:conftest.exe
>>>>> /out:conftest.exe
>>>>> conftest.obj
>>>>> conftest.obj : error LNK2019: unresolved external symbol
>>>>> clang_getClangVersion referenced in function main
>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>
>>>>> cl : Command line warning D9035 : option 'o' has been deprecated and
>>>>> will be removed in a future release
>>>>> cl : Command line warning D9002 : ignoring unknown option
>>>>> '-LJ:\\LLVM\\lib'
>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>>
>>>>> That seems to be a simple problem of casing-off windows and passing
>>>>> the right flags, but I call it a night here :)
>>>>>
>>>>> Jorn
>>
>


From jbvernee at xs4all.nl  Thu Sep 27 16:00:45 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 27 Sep 2018 18:00:45 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
 <069e341fd48b736d55160cca9447b857@xs4all.nl>
 <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>
 <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>
Message-ID: <d24349bd2327b4980b5c1264ecd1bf54@xs4all.nl>

I have commented out the AC_CHECK_LIB for now, since I know that the 
library is there any way, and now getting stuck in 
/make/lib/Lib-jdk.internal.clang.gmk with the following error during 
`make images`:

LINK : warning LNK4044: unrecognized option '/ljsig'; ignored
LINK : fatal error LNK1181: cannot open input file 'j:\LLVM\lib.obj'
make[3]: *** [Lib-jdk.internal.clang.gmk:36: 
/cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/modules_libs/jdk.internal.clang/jclang.dll] 
Error 1
make[2]: *** [make/Main.gmk:215: jdk.internal.clang-libs] Error 2
make[2]: *** Waiting for unfinished jobs....
Creating support/modules_libs/jdk.management/management_ext.dll from 7 
file(s)

ERROR: Build failed for target 'images' in configuration 
'windows-x86_64-normal-server-release' (exit code 2)

I can get rid of the warning by changing `-ljsig` to `/DEFAULTLIB:jsig`, 
but the error stays.

The linking is done using the `link-file-relative` macro from 
/make/common/MakeBase.gmk It might be another pathing issue?

Not sure how to debug that further, there doesn't seem to be a `make 
images` equivalent of config.log where I can look at the commands being 
run?

Jorn

Maurizio Cimadamore schreef op 2018-09-27 17:11:
> Btw, the issue in my case seems to be cause by the fact that cl.exe
> cant locate <time.h> (which is included by clang-c/Index.h header)
> 
> Maurizio
> 
> 
> On 27/09/18 15:58, Maurizio Cimadamore wrote:
>> Got the same on my windows box. I also replaced Magnus's CC suggestion 
>> with CXX which got me a bit further.
>> 
>> Maurizio
>> 
>> 
>> 
>> On 27/09/18 15:37, Jorn Vernee wrote:
>>> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
>>> fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it is 
>>> there when using `CXX="$FIXPATH $CXX"".
>>> 
>>> Checking the header now works, but it's breaking when using 
>>> `AC_CHECK_LIB`:
>>> 
>>> configure:60545: checking for clang_getClangVersion in -lclang
>>> configure:60570: 
>>> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
>>> -c 
>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>> -o conftest.exe??? -I/cygdrive/j/LLVM/include -L/cygdrive/j/LLVM/lib 
>>> conftest.cpp -lclang?? >&5
>>> conftest.cpp
>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>> 
>>> /out:conftest.exe
>>> /out:conftest.exe
>>> conftest.obj
>>> conftest.obj : error LNK2019: unresolved external symbol 
>>> clang_getClangVersion referenced in function main
>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>> 
>>> cl : Command line warning D9035 : option 'o' has been deprecated and 
>>> will be removed in a future release
>>> cl : Command line warning D9002 : ignoring unknown option 
>>> '-Lj:/LLVM/lib'
>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>> configure:60570: $? = 2
>>> 
>>> I'm also seeing the following warnings:
>>> 
>>> The following warnings were produced. Repeated here for convenience:
>>> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
>>> preprocessor!
>>> WARNING: clang-c/Index.h: proceeding with the compiler's result
>>> 
>>> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which 
>>> seems to add the `-l` on it's own (note the unrecognized option 
>>> warnings in the log). For the MS linker AFAIK you'd need 
>>> `/DEFAULTLIB:clang`, or to pass a .lib or .pdb file on the command 
>>> line instead.
>>> 
>>> Jorn
>>> 
>>> [1] : 
>>> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>>> 
>>> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>>>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>>> 
>>>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>>> 
>>>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>>> 
>>>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>>> Try this:
>>>>> 
>>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>>>> @@ -60,6 +60,9 @@
>>>>> VER=`ls $with_libclang/lib/clang/`
>>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>>> CLANG_LIB_PATH="$with_libclang/lib"
>>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>>> +????? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>>> fi
>>>>> 
>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>> 
>>>>> Now, if you use only the --with-libclang option with a Unix-style
>>>>> path (the thing you were trying at first), I believe it should do
>>>>> the right thing.
>>>>> 
>>>>> I have not followed the entire conversation, but this part looks
>>>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>>>>> from configure argument. (And these should, yes indeed, be in unix
>>>>> style). However, this might not be all fixes that are needed. I can
>>>>> help take a look at it, if someone points me to where the
>>>>> problematic code resides.
>>>> ?Magnus, I followed up a bit on this, and managed to reproduce the
>>>> issue.
>>>> 
>>>> The underlying issue is that our lib-clang.m4 conf file checks for
>>>> headers in the usual way with AC_CHECK_HEADER. This check relies on
>>>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>>>> fixups, what you end up with is this:
>>>> 
>>>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>>> 
>>>> This is the correct _cygwin_ path, but it's not the correct Windows
>>>> path. In fact, if you try to compile a dummy header that contains 
>>>> this
>>>> (this was also observed by Jorn)
>>>> 
>>>> #include <clang-c/Index.h>
>>>> 
>>>> with a -I option that includes the above path using the cl.exe
>>>> compiler within cygwin, you get an error because the include file
>>>> cannot be found. The only solution is to use a Windows path, which 
>>>> in
>>>> cygwin can only be done using double backslashes (ugh).
>>>> 
>>>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>>>> path, while here we need the opposite operation. I was under the
>>>> impression that the build configure framework needed to solve this
>>>> particular issue anyway, to check for other headers, but I realized
>>>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, 
>>>> is
>>>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>>>> should we use in order to make our configure script portable?
>>>> 
>>>> P.S.
>>>> I note that there's also a FIXPATH script which does exactly what we
>>>> need here - but this script seems just to be set by the configure 
>>>> step
>>>> and is probably only used by the proper build, so I don't think we 
>>>> can
>>>> refer to it from here?
>>>> 
>>>> Hi,
>>>> 
>>>> Sorry for not having had time to look at this yet.
>>>> 
>>>> The solution should be to use FIXPATH at all times when compiling on
>>>> Windows if using local paths, even in the configure script. It might
>>>> be the case that we have not had to do this as of yet, so it might 
>>>> not
>>>> work out of the box. :-( But I thought we set up CC to "$FIXPATH 
>>>> $CC"
>>>> on Windows, so I thought it should work by itself.
>>>> 
>>>> To get around the mess of windows and unix style paths, the decision
>>>> was made to use only unix paths *all the time*. The duct tape needed
>>>> to get this to work is the fixpath launcher, which takes a
>>>> command-line with unix paths and converts it to windows paths at the
>>>> very last moment before launching the executable.
>>>> 
>>>> Does things work better if you to:
>>>> CC="$FIXPATH $CC"
>>>> before your AC_CHECK_HEADER?
>>>> 
>>>> I'll give this a try - thanks
>>>> 
>>>> Maurizio
>>>> 
>>>>> /Magnus
>>>>> 
>>>>> Thanks
>>>>> Maurizio
>>>>> /Magnus
>>>>> 
>>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>>>> paths which can possibly contain forward slashes - e.g.
>>>>> 
>>>>> 
>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49
>>>>> 
>>>>> 
>>>>> Maurizio
>>>>> 
>>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>>> FTR, as I'm looking at other configure files, I've seen
>>>>> 
>>>>> "xwindows" used not "xmicrosoft"
>>>>> 
>>>>> As for the need for double backslash, I don't see any special
>>>>> processing done in other configure files prior to the
>>>>> AC_CHECK_HEADER call.
>>>>> 
>>>>> The build guide [1] is very explicit that forward slashes should be
>>>>> used in options (unlike what you did) and mentions some 'fixpath'
>>>>> tool which is used by configure to convert Unix paths into Windows
>>>>> ones. I wonder if this could be a bug in that tool?
>>>>> 
>>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 
>>>>> -
>>>>> it could be worth chasing down as to where the compiled 'fixpath'
>>>>> executable is put (should be somewhere in
>>>>> build/<conf>/configure-support/) call it with the clang include
>>>>> folder with Unix-style forward slash and see whether it spits the
>>>>> correct path.
>>>>> 
>>>>> The source code for this tool can be found here:
>>>>> 
>>>>> 
>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c
>>>>> 
>>>>> 
>>>>> Maurizio
>>>>> 
>>>>> [1] -
>>>>> 
>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>>>>> 
>>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>> Actually, -I works, and link option need to be passed with /link,
>>>>> but you got the idea?
>>>>> 
>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>>>> sure it would work. Need to find a better way to check the lib.
>>>>> 
>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>>> -0700
>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018
>>>>> -0700
>>>>> @@ -68,7 +68,11 @@
>>>>> LIBCLANG_CPPFLAGS=""
>>>>> fi
>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>> +????? fi
>>>>> else
>>>>> LIBCLANG_LDFLAGS=""
>>>>> fi
>>>>> 
>>>>> Cheers,
>>>>> Henry
>>>>> 
>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> 
>>>>> wrote:
>>>>> 
>>>>> 
>>>>> Haven?t test it, but try this,
>>>>> 
>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>>> -0700
>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018
>>>>> -0700
>>>>> @@ -63,12 +63,20 @@
>>>>> fi
>>>>> 
>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>> +????? fi
>>>>> else
>>>>> LIBCLANG_CPPFLAGS=""
>>>>> fi
>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>> +????? else
>>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>> +????? fi
>>>>> else
>>>>> LIBCLANG_LDFLAGS=""
>>>>> fi
>>>>> 
>>>>> We still need to fix the copy of DLL.
>>>>> 
>>>>> Cheers,
>>>>> Henry
>>>>> 
>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>> 
>>>>> Looks like good progress; tomorrow I'll take a look at some of our
>>>>> build files and see if something special is done for mangling
>>>>> windows include paths.
>>>>> 
>>>>> Cheers
>>>>> Maurizio
>>>>> 
>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>> If I use the following flags:
>>>>> 
>>>>> $ bash configure --disable-warnings-as-errors
>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>>>> --with-libclang-include='J:\\LLVM\\include'
>>>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>> 
>>>>> (Notice that I'm having to use different path styles)
>>>>> 
>>>>> It's detecting the header files, but it's failing on being passed
>>>>> the wrong flags. from config.log (see at the bottom):
>>>>> 
>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>> configure:60273:
>>>>> 
>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl
>>>>> -o conftest.exe??? -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>>>> -lclang?? >&5
>>>>> conftest.cpp
>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>> 
>>>>> /out:conftest.exe
>>>>> /out:conftest.exe
>>>>> conftest.obj
>>>>> conftest.obj : error LNK2019: unresolved external symbol
>>>>> clang_getClangVersion referenced in function main
>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>> 
>>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>>> and
>>>>> will be removed in a future release
>>>>> cl : Command line warning D9002 : ignoring unknown option
>>>>> '-LJ:\\LLVM\\lib'
>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>> 
>>>>> That seems to be a simple problem of casing-off windows and passing
>>>>> the right flags, but I call it a night here :)
>>>>> 
>>>>> Jorn
>> 

From henry.jen at oracle.com  Thu Sep 27 17:00:59 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Thu, 27 Sep 2018 10:00:59 -0700
Subject: [foreign] Running tests on Windows
In-Reply-To: <d24349bd2327b4980b5c1264ecd1bf54@xs4all.nl>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
 <069e341fd48b736d55160cca9447b857@xs4all.nl>
 <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>
 <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>
 <d24349bd2327b4980b5c1264ecd1bf54@xs4all.nl>
Message-ID: <25E15003-3B8D-4EF4-AB9C-1A27C9B9A322@oracle.com>

If you cannot get AC_CHECK_LIB to work, that means the LIBCLANG_LDFLAGS is not setup correctly. Based on the information so far, simply add /DEFAULTLIB ourselves should work assuming MSVC will ignore the -l.

Something like this, and perhaps also use FIXPATH so we can use unix style path in configure.


diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
--- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018 -0700
+++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018 -0700
@@ -68,7 +68,11 @@
        LIBCLANG_CPPFLAGS=""
    fi
    if test "x$CLANG_LIB_PATH" != "x"; then
+      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
        LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
+      else
+        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH /DEFAULTLIB:clang"
+      fi
    else
        LIBCLANG_LDFLAGS=""
    fi


Cheers,
Henry


> On Sep 27, 2018, at 9:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
> 
> I have commented out the AC_CHECK_LIB for now, since I know that the library is there any way, and now getting stuck in /make/lib/Lib-jdk.internal.clang.gmk with the following error during `make images`:
> 
> LINK : warning LNK4044: unrecognized option '/ljsig'; ignored
> LINK : fatal error LNK1181: cannot open input file 'j:\LLVM\lib.obj'
> make[3]: *** [Lib-jdk.internal.clang.gmk:36: /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/modules_libs/jdk.internal.clang/jclang.dll] Error 1
> make[2]: *** [make/Main.gmk:215: jdk.internal.clang-libs] Error 2
> make[2]: *** Waiting for unfinished jobs....
> Creating support/modules_libs/jdk.management/management_ext.dll from 7 file(s)
> 
> ERROR: Build failed for target 'images' in configuration 'windows-x86_64-normal-server-release' (exit code 2)
> 
> I can get rid of the warning by changing `-ljsig` to `/DEFAULTLIB:jsig`, but the error stays.
> 
> The linking is done using the `link-file-relative` macro from /make/common/MakeBase.gmk It might be another pathing issue?
> 
> Not sure how to debug that further, there doesn't seem to be a `make images` equivalent of config.log where I can look at the commands being run?
> 
> Jorn
> 
> Maurizio Cimadamore schreef op 2018-09-27 17:11:
>> Btw, the issue in my case seems to be cause by the fact that cl.exe
>> cant locate <time.h> (which is included by clang-c/Index.h header)
>> Maurizio
>> On 27/09/18 15:58, Maurizio Cimadamore wrote:
>>> Got the same on my windows box. I also replaced Magnus's CC suggestion with CXX which got me a bit further.
>>> Maurizio
>>> On 27/09/18 15:37, Jorn Vernee wrote:
>>>> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it is there when using `CXX="$FIXPATH $CXX"".
>>>> Checking the header now works, but it's breaking when using `AC_CHECK_LIB`:
>>>> configure:60545: checking for clang_getClangVersion in -lclang
>>>> configure:60570: /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -c /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl -o conftest.exe    -I/cygdrive/j/LLVM/include -L/cygdrive/j/LLVM/lib conftest.cpp -lclang   >&5
>>>> conftest.cpp
>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>> /out:conftest.exe
>>>> /out:conftest.exe
>>>> conftest.obj
>>>> conftest.obj : error LNK2019: unresolved external symbol clang_getClangVersion referenced in function main
>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>> cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release
>>>> cl : Command line warning D9002 : ignoring unknown option '-Lj:/LLVM/lib'
>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>> configure:60570: $? = 2
>>>> I'm also seeing the following warnings:
>>>> The following warnings were produced. Repeated here for convenience:
>>>> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the preprocessor!
>>>> WARNING: clang-c/Index.h: proceeding with the compiler's result
>>>> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which seems to add the `-l` on it's own (note the unrecognized option warnings in the log). For the MS linker AFAIK you'd need `/DEFAULTLIB:clang`, or to pass a .lib or .pdb file on the command line instead.
>>>> Jorn
>>>> [1] : https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>>>> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>>>>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>>>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>>>> Try this:
>>>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4    Tue Sep 11 18:39:11 2018 +0100
>>>>>> +++ b/make/autoconf/lib-clang.m4    Thu Sep 20 09:59:08 2018 +0100
>>>>>> @@ -60,6 +60,9 @@
>>>>>> VER=`ls $with_libclang/lib/clang/`
>>>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>>>> CLANG_LIB_PATH="$with_libclang/lib"
>>>>>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>>>> +      BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>>>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>>>> fi
>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>> Now, if you use only the --with-libclang option with a Unix-style
>>>>>> path (the thing you were trying at first), I believe it should do
>>>>>> the right thing.
>>>>>> I have not followed the entire conversation, but this part looks
>>>>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>>>>>> from configure argument. (And these should, yes indeed, be in unix
>>>>>> style). However, this might not be all fixes that are needed. I can
>>>>>> help take a look at it, if someone points me to where the
>>>>>> problematic code resides.
>>>>>  Magnus, I followed up a bit on this, and managed to reproduce the
>>>>> issue.
>>>>> The underlying issue is that our lib-clang.m4 conf file checks for
>>>>> headers in the usual way with AC_CHECK_HEADER. This check relies on
>>>>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>>>>> fixups, what you end up with is this:
>>>>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>>>> This is the correct _cygwin_ path, but it's not the correct Windows
>>>>> path. In fact, if you try to compile a dummy header that contains this
>>>>> (this was also observed by Jorn)
>>>>> #include <clang-c/Index.h>
>>>>> with a -I option that includes the above path using the cl.exe
>>>>> compiler within cygwin, you get an error because the include file
>>>>> cannot be found. The only solution is to use a Windows path, which in
>>>>> cygwin can only be done using double backslashes (ugh).
>>>>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>>>>> path, while here we need the opposite operation. I was under the
>>>>> impression that the build configure framework needed to solve this
>>>>> particular issue anyway, to check for other headers, but I realized
>>>>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is
>>>>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>>>>> should we use in order to make our configure script portable?
>>>>> P.S.
>>>>> I note that there's also a FIXPATH script which does exactly what we
>>>>> need here - but this script seems just to be set by the configure step
>>>>> and is probably only used by the proper build, so I don't think we can
>>>>> refer to it from here?
>>>>> Hi,
>>>>> Sorry for not having had time to look at this yet.
>>>>> The solution should be to use FIXPATH at all times when compiling on
>>>>> Windows if using local paths, even in the configure script. It might
>>>>> be the case that we have not had to do this as of yet, so it might not
>>>>> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC"
>>>>> on Windows, so I thought it should work by itself.
>>>>> To get around the mess of windows and unix style paths, the decision
>>>>> was made to use only unix paths *all the time*. The duct tape needed
>>>>> to get this to work is the fixpath launcher, which takes a
>>>>> command-line with unix paths and converts it to windows paths at the
>>>>> very last moment before launching the executable.
>>>>> Does things work better if you to:
>>>>> CC="$FIXPATH $CC"
>>>>> before your AC_CHECK_HEADER?
>>>>> I'll give this a try - thanks
>>>>> Maurizio
>>>>>> /Magnus
>>>>>> Thanks
>>>>>> Maurizio
>>>>>> /Magnus
>>>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>>>>> paths which can possibly contain forward slashes - e.g.
>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49
>>>>>> Maurizio
>>>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>>>> FTR, as I'm looking at other configure files, I've seen
>>>>>> "xwindows" used not "xmicrosoft"
>>>>>> As for the need for double backslash, I don't see any special
>>>>>> processing done in other configure files prior to the
>>>>>> AC_CHECK_HEADER call.
>>>>>> The build guide [1] is very explicit that forward slashes should be
>>>>>> used in options (unlike what you did) and mentions some 'fixpath'
>>>>>> tool which is used by configure to convert Unix paths into Windows
>>>>>> ones. I wonder if this could be a bug in that tool?
>>>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>>>>>> it could be worth chasing down as to where the compiled 'fixpath'
>>>>>> executable is put (should be somewhere in
>>>>>> build/<conf>/configure-support/) call it with the clang include
>>>>>> folder with Unix-style forward slash and see whether it spits the
>>>>>> correct path.
>>>>>> The source code for this tool can be found here:
>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c
>>>>>> Maurizio
>>>>>> [1] -
>>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>>>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>>> Actually, -I works, and link option need to be passed with /link,
>>>>>> but you got the idea?
>>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>>>>> sure it would work. Need to find a better way to check the lib.
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018
>>>>>> -0700
>>>>>> @@ -68,7 +68,11 @@
>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>> fi
>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +      else
>>>>>> +        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>>> +      fi
>>>>>> else
>>>>>> LIBCLANG_LDFLAGS=""
>>>>>> fi
>>>>>> Cheers,
>>>>>> Henry
>>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>>> Haven?t test it, but try this,
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018
>>>>>> -0700
>>>>>> @@ -63,12 +63,20 @@
>>>>>> fi
>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>>> +      else
>>>>>> +        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>>> +      fi
>>>>>> else
>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>> fi
>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +      else
>>>>>> +        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>>> +      fi
>>>>>> else
>>>>>> LIBCLANG_LDFLAGS=""
>>>>>> fi
>>>>>> We still need to fix the copy of DLL.
>>>>>> Cheers,
>>>>>> Henry
>>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>> Looks like good progress; tomorrow I'll take a look at some of our
>>>>>> build files and see if something special is done for mangling
>>>>>> windows include paths.
>>>>>> Cheers
>>>>>> Maurizio
>>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>> If I use the following flags:
>>>>>> $ bash configure --disable-warnings-as-errors
>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>>>>> --with-libclang-include='J:\\LLVM\\include'
>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>> (Notice that I'm having to use different path styles)
>>>>>> It's detecting the header files, but it's failing on being passed
>>>>>> the wrong flags. from config.log (see at the bottom):
>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>> configure:60273:
>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl
>>>>>> -o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>>>>> -lclang   >&5
>>>>>> conftest.cpp
>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>>> /out:conftest.exe
>>>>>> /out:conftest.exe
>>>>>> conftest.obj
>>>>>> conftest.obj : error LNK2019: unresolved external symbol
>>>>>> clang_getClangVersion referenced in function main
>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>>> cl : Command line warning D9035 : option 'o' has been deprecated and
>>>>>> will be removed in a future release
>>>>>> cl : Command line warning D9002 : ignoring unknown option
>>>>>> '-LJ:\\LLVM\\lib'
>>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>>> That seems to be a simple problem of casing-off windows and passing
>>>>>> the right flags, but I call it a night here :)
>>>>>> Jorn


From jbvernee at xs4all.nl  Thu Sep 27 17:21:34 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Thu, 27 Sep 2018 19:21:34 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <25E15003-3B8D-4EF4-AB9C-1A27C9B9A322@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <8729429f9b13c9f5cbf29cac3f5e42e2@xs4all.nl>
 <719E0847-D5D8-4E9E-9BCE-C1FDB7003E15@oracle.com>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
 <069e341fd48b736d55160cca9447b857@xs4all.nl>
 <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>
 <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>
 <d24349bd2327b4980b5c1264ecd1bf54@xs4all.nl>
 <25E15003-3B8D-4EF4-AB9C-1A27C9B9A322@oracle.com>
Message-ID: <bc9a0a439d7a6d3954ad3951eee9a051@xs4all.nl>

I had tried that as well (looks like I forgot to send an email about 
this), but the /link flag has to be passed after all the compiler 
arguments [1], since it swallows all the remaining inputs. I can't 
control the order of arguments being passed, and AC_CHECK_LIB is passing 
the .cpp file as the last argument, so I get an error about missing 
source files from the compiler.

Later on those link flags will be passed to the linker directly as well, 
which does not recognize the /link flag, so I have been using:

       if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
         LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
       else
         LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH /DEFAULTLIB:libclang"
       fi

Without the /link flag.

Cheers,
Jorn

[1] : 
https://docs.microsoft.com/en-us/cpp/build/reference/link-pass-options-to-linker?view=vs-2017

Henry Jen schreef op 2018-09-27 19:00:
> If you cannot get AC_CHECK_LIB to work, that means the
> LIBCLANG_LDFLAGS is not setup correctly. Based on the information so
> far, simply add /DEFAULTLIB ourselves should work assuming MSVC will
> ignore the -l.
> 
> Something like this, and perhaps also use FIXPATH so we can use unix
> style path in configure.
> 
> 
> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018 -0700
> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018 -0700
> @@ -68,7 +68,11 @@
>         LIBCLANG_CPPFLAGS=""
>     fi
>     if test "x$CLANG_LIB_PATH" != "x"; then
> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>         LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
> +      else
> +        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH 
> /DEFAULTLIB:clang"
> +      fi
>     else
>         LIBCLANG_LDFLAGS=""
>     fi
> 
> 
> Cheers,
> Henry
> 
> 
>> On Sep 27, 2018, at 9:00 AM, Jorn Vernee <jbvernee at xs4all.nl> wrote:
>> 
>> I have commented out the AC_CHECK_LIB for now, since I know that the 
>> library is there any way, and now getting stuck in 
>> /make/lib/Lib-jdk.internal.clang.gmk with the following error during 
>> `make images`:
>> 
>> LINK : warning LNK4044: unrecognized option '/ljsig'; ignored
>> LINK : fatal error LNK1181: cannot open input file 'j:\LLVM\lib.obj'
>> make[3]: *** [Lib-jdk.internal.clang.gmk:36: 
>> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/support/modules_libs/jdk.internal.clang/jclang.dll] 
>> Error 1
>> make[2]: *** [make/Main.gmk:215: jdk.internal.clang-libs] Error 2
>> make[2]: *** Waiting for unfinished jobs....
>> Creating support/modules_libs/jdk.management/management_ext.dll from 7 
>> file(s)
>> 
>> ERROR: Build failed for target 'images' in configuration 
>> 'windows-x86_64-normal-server-release' (exit code 2)
>> 
>> I can get rid of the warning by changing `-ljsig` to 
>> `/DEFAULTLIB:jsig`, but the error stays.
>> 
>> The linking is done using the `link-file-relative` macro from 
>> /make/common/MakeBase.gmk It might be another pathing issue?
>> 
>> Not sure how to debug that further, there doesn't seem to be a `make 
>> images` equivalent of config.log where I can look at the commands 
>> being run?
>> 
>> Jorn
>> 
>> Maurizio Cimadamore schreef op 2018-09-27 17:11:
>>> Btw, the issue in my case seems to be cause by the fact that cl.exe
>>> cant locate <time.h> (which is included by clang-c/Index.h header)
>>> Maurizio
>>> On 27/09/18 15:58, Maurizio Cimadamore wrote:
>>>> Got the same on my windows box. I also replaced Magnus's CC 
>>>> suggestion with CXX which got me a bit further.
>>>> Maurizio
>>>> On 27/09/18 15:37, Jorn Vernee wrote:
>>>>> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
>>>>> fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it 
>>>>> is there when using `CXX="$FIXPATH $CXX"".
>>>>> Checking the header now works, but it's breaking when using 
>>>>> `AC_CHECK_LIB`:
>>>>> configure:60545: checking for clang_getClangVersion in -lclang
>>>>> configure:60570: 
>>>>> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
>>>>> -c 
>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>> -o conftest.exe    -I/cygdrive/j/LLVM/include 
>>>>> -L/cygdrive/j/LLVM/lib conftest.cpp -lclang   >&5
>>>>> conftest.cpp
>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>> /out:conftest.exe
>>>>> /out:conftest.exe
>>>>> conftest.obj
>>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>>> clang_getClangVersion referenced in function main
>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>>> and will be removed in a future release
>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>> '-Lj:/LLVM/lib'
>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>> configure:60570: $? = 2
>>>>> I'm also seeing the following warnings:
>>>>> The following warnings were produced. Repeated here for 
>>>>> convenience:
>>>>> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
>>>>> preprocessor!
>>>>> WARNING: clang-c/Index.h: proceeding with the compiler's result
>>>>> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which 
>>>>> seems to add the `-l` on it's own (note the unrecognized option 
>>>>> warnings in the log). For the MS linker AFAIK you'd need 
>>>>> `/DEFAULTLIB:clang`, or to pass a .lib or .pdb file on the command 
>>>>> line instead.
>>>>> Jorn
>>>>> [1] : 
>>>>> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>>>>> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>>>>>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>>>>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>>>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>>>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>>>>> Try this:
>>>>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>>>>> --- a/make/autoconf/lib-clang.m4    Tue Sep 11 18:39:11 2018 
>>>>>>> +0100
>>>>>>> +++ b/make/autoconf/lib-clang.m4    Thu Sep 20 09:59:08 2018 
>>>>>>> +0100
>>>>>>> @@ -60,6 +60,9 @@
>>>>>>> VER=`ls $with_libclang/lib/clang/`
>>>>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>>>>> CLANG_LIB_PATH="$with_libclang/lib"
>>>>>>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>>>>> +      BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>>>>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>>>>> fi
>>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>>> Now, if you use only the --with-libclang option with a Unix-style
>>>>>>> path (the thing you were trying at first), I believe it should do
>>>>>>> the right thing.
>>>>>>> I have not followed the entire conversation, but this part looks
>>>>>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all 
>>>>>>> paths
>>>>>>> from configure argument. (And these should, yes indeed, be in 
>>>>>>> unix
>>>>>>> style). However, this might not be all fixes that are needed. I 
>>>>>>> can
>>>>>>> help take a look at it, if someone points me to where the
>>>>>>> problematic code resides.
>>>>>>  Magnus, I followed up a bit on this, and managed to reproduce the
>>>>>> issue.
>>>>>> The underlying issue is that our lib-clang.m4 conf file checks for
>>>>>> headers in the usual way with AC_CHECK_HEADER. This check relies 
>>>>>> on
>>>>>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>>>>>> fixups, what you end up with is this:
>>>>>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>>>>> This is the correct _cygwin_ path, but it's not the correct 
>>>>>> Windows
>>>>>> path. In fact, if you try to compile a dummy header that contains 
>>>>>> this
>>>>>> (this was also observed by Jorn)
>>>>>> #include <clang-c/Index.h>
>>>>>> with a -I option that includes the above path using the cl.exe
>>>>>> compiler within cygwin, you get an error because the include file
>>>>>> cannot be found. The only solution is to use a Windows path, which 
>>>>>> in
>>>>>> cygwin can only be done using double backslashes (ugh).
>>>>>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>>>>>> path, while here we need the opposite operation. I was under the
>>>>>> impression that the build configure framework needed to solve this
>>>>>> particular issue anyway, to check for other headers, but I 
>>>>>> realized
>>>>>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, 
>>>>>> is
>>>>>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>>>>>> should we use in order to make our configure script portable?
>>>>>> P.S.
>>>>>> I note that there's also a FIXPATH script which does exactly what 
>>>>>> we
>>>>>> need here - but this script seems just to be set by the configure 
>>>>>> step
>>>>>> and is probably only used by the proper build, so I don't think we 
>>>>>> can
>>>>>> refer to it from here?
>>>>>> Hi,
>>>>>> Sorry for not having had time to look at this yet.
>>>>>> The solution should be to use FIXPATH at all times when compiling 
>>>>>> on
>>>>>> Windows if using local paths, even in the configure script. It 
>>>>>> might
>>>>>> be the case that we have not had to do this as of yet, so it might 
>>>>>> not
>>>>>> work out of the box. :-( But I thought we set up CC to "$FIXPATH 
>>>>>> $CC"
>>>>>> on Windows, so I thought it should work by itself.
>>>>>> To get around the mess of windows and unix style paths, the 
>>>>>> decision
>>>>>> was made to use only unix paths *all the time*. The duct tape 
>>>>>> needed
>>>>>> to get this to work is the fixpath launcher, which takes a
>>>>>> command-line with unix paths and converts it to windows paths at 
>>>>>> the
>>>>>> very last moment before launching the executable.
>>>>>> Does things work better if you to:
>>>>>> CC="$FIXPATH $CC"
>>>>>> before your AC_CHECK_HEADER?
>>>>>> I'll give this a try - thanks
>>>>>> Maurizio
>>>>>>> /Magnus
>>>>>>> Thanks
>>>>>>> Maurizio
>>>>>>> /Magnus
>>>>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>>>>>> paths which can possibly contain forward slashes - e.g.
>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49
>>>>>>> Maurizio
>>>>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>>>>> FTR, as I'm looking at other configure files, I've seen
>>>>>>> "xwindows" used not "xmicrosoft"
>>>>>>> As for the need for double backslash, I don't see any special
>>>>>>> processing done in other configure files prior to the
>>>>>>> AC_CHECK_HEADER call.
>>>>>>> The build guide [1] is very explicit that forward slashes should 
>>>>>>> be
>>>>>>> used in options (unlike what you did) and mentions some 'fixpath'
>>>>>>> tool which is used by configure to convert Unix paths into 
>>>>>>> Windows
>>>>>>> ones. I wonder if this could be a bug in that tool?
>>>>>>> Seems like this tool is compiled in 
>>>>>>> make/autoconf/basic_windows.m4 -
>>>>>>> it could be worth chasing down as to where the compiled 'fixpath'
>>>>>>> executable is put (should be somewhere in
>>>>>>> build/<conf>/configure-support/) call it with the clang include
>>>>>>> folder with Unix-style forward slash and see whether it spits the
>>>>>>> correct path.
>>>>>>> The source code for this tool can be found here:
>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c
>>>>>>> Maurizio
>>>>>>> [1] -
>>>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows
>>>>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>>>> Actually, -I works, and link option need to be passed with /link,
>>>>>>> but you got the idea?
>>>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>>>>>> sure it would work. Need to find a better way to check the lib.
>>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>>>>>>> -0700
>>>>>>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018
>>>>>>> -0700
>>>>>>> @@ -68,7 +68,11 @@
>>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>>> fi
>>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>>> +      else
>>>>>>> +        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>>>> +      fi
>>>>>>> else
>>>>>>> LIBCLANG_LDFLAGS=""
>>>>>>> fi
>>>>>>> Cheers,
>>>>>>> Henry
>>>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> 
>>>>>>> wrote:
>>>>>>> Haven?t test it, but try this,
>>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>>>>>>> -0700
>>>>>>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018
>>>>>>> -0700
>>>>>>> @@ -63,12 +63,20 @@
>>>>>>> fi
>>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>>>> +      else
>>>>>>> +        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>>>> +      fi
>>>>>>> else
>>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>>> fi
>>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>>> +      else
>>>>>>> +        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>>>> +      fi
>>>>>>> else
>>>>>>> LIBCLANG_LDFLAGS=""
>>>>>>> fi
>>>>>>> We still need to fix the copy of DLL.
>>>>>>> Cheers,
>>>>>>> Henry
>>>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>> Looks like good progress; tomorrow I'll take a look at some of 
>>>>>>> our
>>>>>>> build files and see if something special is done for mangling
>>>>>>> windows include paths.
>>>>>>> Cheers
>>>>>>> Maurizio
>>>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>>> If I use the following flags:
>>>>>>> $ bash configure --disable-warnings-as-errors
>>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>>>>>> --with-libclang-include='J:\\LLVM\\include'
>>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>>> (Notice that I'm having to use different path styles)
>>>>>>> It's detecting the header files, but it's failing on being passed
>>>>>>> the wrong flags. from config.log (see at the bottom):
>>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>>> configure:60273:
>>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl
>>>>>>> -o conftest.exe    -IJ:\\LLVM\\include -LJ:\\LLVM\\lib 
>>>>>>> conftest.cpp
>>>>>>> -lclang   >&5
>>>>>>> conftest.cpp
>>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>>>> /out:conftest.exe
>>>>>>> /out:conftest.exe
>>>>>>> conftest.obj
>>>>>>> conftest.obj : error LNK2019: unresolved external symbol
>>>>>>> clang_getClangVersion referenced in function main
>>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for 
>>>>>>> x64
>>>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>>>>> and
>>>>>>> will be removed in a future release
>>>>>>> cl : Command line warning D9002 : ignoring unknown option
>>>>>>> '-LJ:\\LLVM\\lib'
>>>>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>>>>> '-lclang'
>>>>>>> That seems to be a simple problem of casing-off windows and 
>>>>>>> passing
>>>>>>> the right flags, but I call it a night here :)
>>>>>>> Jorn

From magnus.ihse.bursie at oracle.com  Thu Sep 27 18:15:29 2018
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 27 Sep 2018 20:15:29 +0200
Subject: [foreign] Running tests on Windows
In-Reply-To: <8f22bf90-3c69-d5ed-a84e-943585b676f6@oracle.com>
References: <1a5d831165850f79b88b0a7c0e24f121@xs4all.nl>
 <19d34989f673a8399bfe96f4bea29834@xs4all.nl>
 <69e894bd-83e3-a49c-0736-f47bd2f20070@oracle.com>
 <d71266d2ef2469661f8819d19132678e@xs4all.nl>
 <7e795613-8b0f-dcfe-734e-157d6f82cc28@oracle.com>
 <f1d8d05a8ecf9e678f7e7f7f4c5cf42a@xs4all.nl>
 <fc0c69cc5994ee4abf32c2b3e1452c7c@xs4all.nl>
 <812e8c4c-e77b-10dc-0ba0-ecd39554bf9b@oracle.com>
 <D2F78712-5DBC-4777-B47A-1FF9B66DBF29@oracle.com>
 <B7E9B126-FB20-4E95-8EDA-87027243B920@oracle.com>
 <90b3a234-5fae-17ff-5ff0-e06128d4b33e@oracle.com>
 <df6cc707-b72b-917c-2051-788e1adb1734@oracle.com>
 <8571b1c0-89d0-e015-b2c4-2161cf73aad5@oracle.com>
 <4df08fa5-6293-4c15-030a-fa268cef83c0@oracle.com>
 <595d3568-f6ee-beae-0f2e-2e8a2fc5e51f@oracle.com>
 <57ce427a-d5f0-3840-e5b3-8c577b974de8@oracle.com>
 <069e341fd48b736d55160cca9447b857@xs4all.nl>
 <be959816-36c0-9a98-38dc-a3c7d593fb60@oracle.com>
 <8687214b-4c1e-7b72-d658-aa48edc93d04@oracle.com>
 <8f22bf90-3c69-d5ed-a84e-943585b676f6@oracle.com>
Message-ID: <e0509dd4-1f5e-559d-c9bd-31392ed10649@oracle.com>



On 2018-09-27 17:55, Maurizio Cimadamore wrote:
> NVM,
> I think the real issue is how AC_CHECK_LIB invokes the compiler - e.g. 
> with -lclang
>
> In MSVC this should be /link libclang.lib
>
> Not sure how you tell AC_CHECK_LIB to use a different command, but 
> frankly it seems like AC_XYZ checks are GNU-centric and they don't 
> play well with Windows - it might be a lost cause to try and make them 
> work under cygwin.
That might very well be the case. We have not used AC_CHECK_LIB much (at 
all?), and I vaguely remember that we've tried to actually stay clear of 
AC_CHECK_LIB because of issues; that might be that they do not work on 
Microsoft.

For other external libraries on Windows, like freetype, if I remember 
correctly we have just provided the absolute path to the .LIB file as 
*_LIBS. So instead of the unix standard of using a LD_FLAGS -L<path> and 
LIBS -l<lib>, we just use <path>/<lib>.lib as LIBS.

/Magnus

>
> Maurizio
>
>
> On 27/09/18 16:11, Maurizio Cimadamore wrote:
>> Btw, the issue in my case seems to be cause by the fact that cl.exe 
>> cant locate <time.h> (which is included by clang-c/Index.h header)
>>
>> Maurizio
>>
>>
>> On 27/09/18 15:58, Maurizio Cimadamore wrote:
>>> Got the same on my windows box. I also replaced Magnus's CC 
>>> suggestion with CXX which got me a bit further.
>>>
>>> Maurizio
>>>
>>>
>>>
>>> On 27/09/18 15:37, Jorn Vernee wrote:
>>>> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
>>>> fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it 
>>>> is there when using `CXX="$FIXPATH $CXX"".
>>>>
>>>> Checking the header now works, but it's breaking when using 
>>>> `AC_CHECK_LIB`:
>>>>
>>>> configure:60545: checking for clang_getClangVersion in -lclang
>>>> configure:60570: 
>>>> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
>>>> -c 
>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>> -o conftest.exe??? -I/cygdrive/j/LLVM/include 
>>>> -L/cygdrive/j/LLVM/lib conftest.cpp -lclang?? >&5
>>>> conftest.cpp
>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>
>>>> /out:conftest.exe
>>>> /out:conftest.exe
>>>> conftest.obj
>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>> clang_getClangVersion referenced in function main
>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>
>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>> and will be removed in a future release
>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>> '-Lj:/LLVM/lib'
>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>> configure:60570: $? = 2
>>>>
>>>> I'm also seeing the following warnings:
>>>>
>>>> The following warnings were produced. Repeated here for convenience:
>>>> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
>>>> preprocessor!
>>>> WARNING: clang-c/Index.h: proceeding with the compiler's result
>>>>
>>>> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which 
>>>> seems to add the `-l` on it's own (note the unrecognized option 
>>>> warnings in the log). For the MS linker AFAIK you'd need 
>>>> `/DEFAULTLIB:clang`, or to pass a .lib or .pdb file on the command 
>>>> line instead.
>>>>
>>>> Jorn
>>>>
>>>> [1] : 
>>>> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>>>>
>>>> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>>>>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>>>>
>>>>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>>>>
>>>>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>>>>
>>>>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>>>> Try this:
>>>>>>
>>>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4??? Tue Sep 11 18:39:11 2018 +0100
>>>>>> +++ b/make/autoconf/lib-clang.m4??? Thu Sep 20 09:59:08 2018 +0100
>>>>>> @@ -60,6 +60,9 @@
>>>>>> VER=`ls $with_libclang/lib/clang/`
>>>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>>>> CLANG_LIB_PATH="$with_libclang/lib"
>>>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>>>> +????? BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>>>> +????? BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>>>> fi
>>>>>>
>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>>
>>>>>> Now, if you use only the --with-libclang option with a Unix-style
>>>>>> path (the thing you were trying at first), I believe it should do
>>>>>> the right thing.
>>>>>>
>>>>>> I have not followed the entire conversation, but this part looks
>>>>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>>>>>> from configure argument. (And these should, yes indeed, be in unix
>>>>>> style). However, this might not be all fixes that are needed. I can
>>>>>> help take a look at it, if someone points me to where the
>>>>>> problematic code resides.
>>>>> ?Magnus, I followed up a bit on this, and managed to reproduce the
>>>>> issue.
>>>>>
>>>>> The underlying issue is that our lib-clang.m4 conf file checks for
>>>>> headers in the usual way with AC_CHECK_HEADER. This check relies on
>>>>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>>>>> fixups, what you end up with is this:
>>>>>
>>>>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>>>>
>>>>> This is the correct _cygwin_ path, but it's not the correct Windows
>>>>> path. In fact, if you try to compile a dummy header that contains 
>>>>> this
>>>>> (this was also observed by Jorn)
>>>>>
>>>>> #include <clang-c/Index.h>
>>>>>
>>>>> with a -I option that includes the above path using the cl.exe
>>>>> compiler within cygwin, you get an error because the include file
>>>>> cannot be found. The only solution is to use a Windows path, which in
>>>>> cygwin can only be done using double backslashes (ugh).
>>>>>
>>>>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>>>>> path, while here we need the opposite operation. I was under the
>>>>> impression that the build configure framework needed to solve this
>>>>> particular issue anyway, to check for other headers, but I realized
>>>>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is
>>>>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>>>>> should we use in order to make our configure script portable?
>>>>>
>>>>> P.S.
>>>>> I note that there's also a FIXPATH script which does exactly what we
>>>>> need here - but this script seems just to be set by the configure 
>>>>> step
>>>>> and is probably only used by the proper build, so I don't think we 
>>>>> can
>>>>> refer to it from here?
>>>>>
>>>>> Hi,
>>>>>
>>>>> Sorry for not having had time to look at this yet.
>>>>>
>>>>> The solution should be to use FIXPATH at all times when compiling on
>>>>> Windows if using local paths, even in the configure script. It might
>>>>> be the case that we have not had to do this as of yet, so it might 
>>>>> not
>>>>> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC"
>>>>> on Windows, so I thought it should work by itself.
>>>>>
>>>>> To get around the mess of windows and unix style paths, the decision
>>>>> was made to use only unix paths *all the time*. The duct tape needed
>>>>> to get this to work is the fixpath launcher, which takes a
>>>>> command-line with unix paths and converts it to windows paths at the
>>>>> very last moment before launching the executable.
>>>>>
>>>>> Does things work better if you to:
>>>>> CC="$FIXPATH $CC"
>>>>> before your AC_CHECK_HEADER?
>>>>>
>>>>> I'll give this a try - thanks
>>>>>
>>>>> Maurizio
>>>>>
>>>>>> /Magnus
>>>>>>
>>>>>> Thanks
>>>>>> Maurizio
>>>>>> /Magnus
>>>>>>
>>>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>>>>> paths which can possibly contain forward slashes - e.g.
>>>>>>
>>>>>>
>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>>>>
>>>>>>
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>>>> FTR, as I'm looking at other configure files, I've seen
>>>>>>
>>>>>> "xwindows" used not "xmicrosoft"
>>>>>>
>>>>>> As for the need for double backslash, I don't see any special
>>>>>> processing done in other configure files prior to the
>>>>>> AC_CHECK_HEADER call.
>>>>>>
>>>>>> The build guide [1] is very explicit that forward slashes should be
>>>>>> used in options (unlike what you did) and mentions some 'fixpath'
>>>>>> tool which is used by configure to convert Unix paths into Windows
>>>>>> ones. I wonder if this could be a bug in that tool?
>>>>>>
>>>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>>>>>> it could be worth chasing down as to where the compiled 'fixpath'
>>>>>> executable is put (should be somewhere in
>>>>>> build/<conf>/configure-support/) call it with the clang include
>>>>>> folder with Unix-style forward slash and see whether it spits the
>>>>>> correct path.
>>>>>>
>>>>>> The source code for this tool can be found here:
>>>>>>
>>>>>>
>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>>>>
>>>>>>
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> [1] -
>>>>>>
>>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows 
>>>>>
>>>>>>
>>>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>>> Actually, -I works, and link option need to be passed with /link,
>>>>>> but you got the idea?
>>>>>>
>>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>>>>> sure it would work. Need to find a better way to check the lib.
>>>>>>
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:38:06 2018
>>>>>> -0700
>>>>>> @@ -68,7 +68,11 @@
>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>> fi
>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +????? else
>>>>>> +??????? LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>>> +????? fi
>>>>>> else
>>>>>> LIBCLANG_LDFLAGS=""
>>>>>> fi
>>>>>>
>>>>>> Cheers,
>>>>>> Henry
>>>>>>
>>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>>>
>>>>>>
>>>>>> Haven?t test it, but try this,
>>>>>>
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4??????? Mon Sep 17 22:17:08 2018
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4??????? Wed Sep 19 16:16:27 2018
>>>>>> -0700
>>>>>> @@ -63,12 +63,20 @@
>>>>>> fi
>>>>>>
>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>>> +????? else
>>>>>> +??????? LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>>> +????? fi
>>>>>> else
>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>> fi
>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +????? if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +????? else
>>>>>> +??????? LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>>> +????? fi
>>>>>> else
>>>>>> LIBCLANG_LDFLAGS=""
>>>>>> fi
>>>>>>
>>>>>> We still need to fix the copy of DLL.
>>>>>>
>>>>>> Cheers,
>>>>>> Henry
>>>>>>
>>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>
>>>>>> Looks like good progress; tomorrow I'll take a look at some of our
>>>>>> build files and see if something special is done for mangling
>>>>>> windows include paths.
>>>>>>
>>>>>> Cheers
>>>>>> Maurizio
>>>>>>
>>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>> If I use the following flags:
>>>>>>
>>>>>> $ bash configure --disable-warnings-as-errors
>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>>>>> --with-libclang-include='J:\\LLVM\\include'
>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>>
>>>>>> (Notice that I'm having to use different path styles)
>>>>>>
>>>>>> It's detecting the header files, but it's failing on being passed
>>>>>> the wrong flags. from config.log (see at the bottom):
>>>>>>
>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>> configure:60273:
>>>>>>
>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>>
>>>>>> -o conftest.exe -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>>>>> -lclang?? >&5
>>>>>> conftest.cpp
>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>
>>>>>> /out:conftest.exe
>>>>>> /out:conftest.exe
>>>>>> conftest.obj
>>>>>> conftest.obj : error LNK2019: unresolved external symbol
>>>>>> clang_getClangVersion referenced in function main
>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>>> Copyright (C) Microsoft Corporation.? All rights reserved.
>>>>>>
>>>>>> cl : Command line warning D9035 : option 'o' has been deprecated and
>>>>>> will be removed in a future release
>>>>>> cl : Command line warning D9002 : ignoring unknown option
>>>>>> '-LJ:\\LLVM\\lib'
>>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>>>
>>>>>> That seems to be a simple problem of casing-off windows and passing
>>>>>> the right flags, but I call it a night here :)
>>>>>>
>>>>>> Jorn
>>>
>>
>


From maurizio.cimadamore at oracle.com  Fri Sep 28 11:19:05 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 12:19:05 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
Message-ID: <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>

This is an updated version of the direct invocation scheme support. Very 
close to the last one, but there are some minor refactorings/improvements:

1) Added a @Stable annotation in DirectNativeInvoker's MH field
2) box/unbox routine used by the UniversalXYZ strategies have been moved 
from NativeInvoker to UniversalNativeInvoker
3) I revamped the logic which detects whether fastpath is applicable - 
now we create the calling sequence first, and we use that to check 
whether we can fast path it. Some internal benchmark have shown that 
with a large number of symbols, we were doing a lot of work because we 
were trying the fastpath always and then, in case of exception fallback 
to slow path; in such cases we would create calling sequence twice. This 
new technique might also be more friendly w.r.t. Windows and other ABIs.

I'd really like to move ahead with this (as this RFR has been out for 
quite a while now) - if there's no other comments I'll go ahead.

Maurizio


On 14/09/18 19:04, Maurizio Cimadamore wrote:
> Hi,
> as mentioned in [1], this patch adds binder support for the so called 
> 'direct' invocation scheme, which allows for greater native invocation 
> downcall/upcall performances by means of specialized adapters. The 
> core idea, also described in [1], is to define adapters of the kind:
>
> invokeNative_V_DDDDD
> invokeNative_V_JDDDD
> invokeNative_V_JJDDD
> invokeNative_V_JJJDD
> invokeNative_V_JJJJD
> invokeNative_V_JJJJJ
>
> Where long arguments come before double arguments (and do this for 
> each arity e.g. <=5).
>
> If all arguments are passed in register, then this reordering doesn't 
> affect behavior, and greatly limits the number of permutations to be 
> supported/generated.
>
> The downcall part (java to native) is relative straightforward: the 
> directNativeInvoker.cpp file defines a bunch of native entry points, 
> one per shape, which cast the input address to a function pointer of 
> the desired shape, and then call it:
>
> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, 
> jlong arg0, jdouble arg1) {
> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
> }
>
> The upcall business is a little trickier: first, if we are only to 
> optimize upcalls where argument passing happens in registers, then 
> it's crucial to note that by the time we get into the assembly stub, 
> all the registers will have been populated by the native code to 
> contain the right arguments in the right places. So we can avoid all 
> the shuffling in the assembly adapter and simply jump onto a C 
> function that looks like this:
>
> long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
> ????????????????????????????????????? double d0, double d1, double d2, 
> double d3,
> ?????????????????????????????????????? unsigned int mask, jobject rec) 
> { ... }
>
> Note here that the first 8 arguments are just longs and doubles, and 
> those will be expected to be in registers, according to the System V 
> ABI. (In windows, the situation will be a bit different as less 
> integer registers are available, so this will need some work there).
>
> So, to recap, the assembly upcall stub simply 'append' the receiver 
> object and a 'signature mask' in the last two available C registers 
> and then jump onto the helper function. The helper function will find 
> all the desired arguments in the right places - there will be, in the 
> general case, some unused arguments, but that's fine, after all it 
> didn't cost anything to us to load them in the first place!
>
> Note that we have three helper variants, one for each return type { 
> long, double, void }. This is required as we need the C helper to 
> return a value of the right type which will generate the right 
> assembly sequence to store the result in the right register (either 
> integer or MMX).
>
> So, with three helpers we can support all the shapes with up to 8 
> arguments. On the Java side we have, of course, to define a 
> specialized entry point for each shape.
>
> All the magic for adapting method handle to and from the specialized 
> adapters happen in the DirectSignatureShuffler class; this class is 
> responsible for adapting each argument e.g. from Java to native value, 
> and then reordering the adapted method handle to match the order in 
> which arguments are expected by the adapter (e.g. move all longs in 
> front). The challenge was in having DirectSignatureShuffle to be fully 
> symmetric - e.g. I did not want to have different code paths for 
> upcalls and downcalls, so the code tries quite hard to be parametric 
> in the shuffling direction (java->native or native->java) - which 
> means that adapters will be applied in one way or in the inverse way 
> depending on the shuffling direction (and as to whether we are 
> adapting an argument or a return). Since method handle filters are 
> composable, it all works out quite beautifully.
>
> Note that the resulting, adapted MH is stored in a @Stable field to 
> tell the JIT to optimize the heck out of it (as if it were a static 
> constant).
>
> This patch contains several other changes - which I discuss briefly 
> below:
>
> * we need to setup a framework in which new invocation strategies can 
> be plugged in - note that we now have essentially 4 cases:
>
> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>
> When the code wants e.g. a NativeInvoker, it asks for one to the 
> NativeInvoker::of factory (UpcallHandler work in a similar way); this 
> factory will attempt to go down the fast path - if an error occurs 
> when computing the fast path, the call will fallback to the universal 
> (slow) path.
>
> Most of the changes you see in the Java code are associated to this 
> refactoring - e.g. all clients of NativeInvoker/UpcallHandler should 
> now go through the factory
>
> * CallbackImplGenerator had a major issue since the new factory for 
> NativeInvoker wants to bind an address eagerly (this is required e.g. 
> to be forward compatible with linkToNative backend); which means that 
> at construction time we have to get the address of the callback, call 
> the NativeInvoker factory and then stash the target method handle into 
> a field of the anon callback class. Vlad tells me that fields of anon 
> classes are always 'trusted' by the JIT, which means they should be 
> treated as '@Stable' (note that I can't put a @Stable annotation 
> there, since this code will be spinned in user-land).
>
> * There are a bunch of properties that can be set to either force slow 
> path or force 'direct' path; in the latter case, if an error occurs 
> when instantiating the direct wrapper, an exception is thrown. This 
> mode is very useful for testing, and I indeed have tried to run all 
> our tests with this flag enabled, to see how many places could not be 
> optimized.
>
> * I've also reorganized all the native code in hotspot/prims so that 
> we have a separate file for each scheme (and so that native Java 
> methods could be added where they really belong). This should also 
> help in the long run as it should make adding/removing a given scheme 
> easier.
>
> * I've also added a small test which tries to pass structs of 
> different sizes, but I will also work on a more complex test which 
> will stress-test all invocation modes in a more complete fashion. With 
> respect to testing, I've also done a fastdebug build and ran all tests 
> with that (as fastdebug catches way many more hotspot assertion than 
> the product version); everything passed.
>
> Webrev:
>
> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>
> I'd like to thank Vladimir Ivanov for the prompt support whenever I 
> got stuck down the macro assembler rabbit hole :-)
>
> Cheers
> Maurizio
>
> [1] - 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>
>
>


From maurizio.cimadamore at oracle.com  Fri Sep 28 11:19:46 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 12:19:46 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
Message-ID: <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8210757_v2/

Maurizio


On 28/09/18 12:19, Maurizio Cimadamore wrote:
> This is an updated version of the direct invocation scheme support. 
> Very close to the last one, but there are some minor 
> refactorings/improvements:
>
> 1) Added a @Stable annotation in DirectNativeInvoker's MH field
> 2) box/unbox routine used by the UniversalXYZ strategies have been 
> moved from NativeInvoker to UniversalNativeInvoker
> 3) I revamped the logic which detects whether fastpath is applicable - 
> now we create the calling sequence first, and we use that to check 
> whether we can fast path it. Some internal benchmark have shown that 
> with a large number of symbols, we were doing a lot of work because we 
> were trying the fastpath always and then, in case of exception 
> fallback to slow path; in such cases we would create calling sequence 
> twice. This new technique might also be more friendly w.r.t. Windows 
> and other ABIs.
>
> I'd really like to move ahead with this (as this RFR has been out for 
> quite a while now) - if there's no other comments I'll go ahead.
>
> Maurizio
>
>
> On 14/09/18 19:04, Maurizio Cimadamore wrote:
>> Hi,
>> as mentioned in [1], this patch adds binder support for the so called 
>> 'direct' invocation scheme, which allows for greater native 
>> invocation downcall/upcall performances by means of specialized 
>> adapters. The core idea, also described in [1], is to define adapters 
>> of the kind:
>>
>> invokeNative_V_DDDDD
>> invokeNative_V_JDDDD
>> invokeNative_V_JJDDD
>> invokeNative_V_JJJDD
>> invokeNative_V_JJJJD
>> invokeNative_V_JJJJJ
>>
>> Where long arguments come before double arguments (and do this for 
>> each arity e.g. <=5).
>>
>> If all arguments are passed in register, then this reordering doesn't 
>> affect behavior, and greatly limits the number of permutations to be 
>> supported/generated.
>>
>> The downcall part (java to native) is relative straightforward: the 
>> directNativeInvoker.cpp file defines a bunch of native entry points, 
>> one per shape, which cast the input address to a function pointer of 
>> the desired shape, and then call it:
>>
>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, 
>> jlong arg0, jdouble arg1) {
>> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>> }
>>
>> The upcall business is a little trickier: first, if we are only to 
>> optimize upcalls where argument passing happens in registers, then 
>> it's crucial to note that by the time we get into the assembly stub, 
>> all the registers will have been populated by the native code to 
>> contain the right arguments in the right places. So we can avoid all 
>> the shuffling in the assembly adapter and simply jump onto a C 
>> function that looks like this:
>>
>> long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
>> ????????????????????????????????????? double d0, double d1, double 
>> d2, double d3,
>> ?????????????????????????????????????? unsigned int mask, jobject 
>> rec) { ... }
>>
>> Note here that the first 8 arguments are just longs and doubles, and 
>> those will be expected to be in registers, according to the System V 
>> ABI. (In windows, the situation will be a bit different as less 
>> integer registers are available, so this will need some work there).
>>
>> So, to recap, the assembly upcall stub simply 'append' the receiver 
>> object and a 'signature mask' in the last two available C registers 
>> and then jump onto the helper function. The helper function will find 
>> all the desired arguments in the right places - there will be, in the 
>> general case, some unused arguments, but that's fine, after all it 
>> didn't cost anything to us to load them in the first place!
>>
>> Note that we have three helper variants, one for each return type { 
>> long, double, void }. This is required as we need the C helper to 
>> return a value of the right type which will generate the right 
>> assembly sequence to store the result in the right register (either 
>> integer or MMX).
>>
>> So, with three helpers we can support all the shapes with up to 8 
>> arguments. On the Java side we have, of course, to define a 
>> specialized entry point for each shape.
>>
>> All the magic for adapting method handle to and from the specialized 
>> adapters happen in the DirectSignatureShuffler class; this class is 
>> responsible for adapting each argument e.g. from Java to native 
>> value, and then reordering the adapted method handle to match the 
>> order in which arguments are expected by the adapter (e.g. move all 
>> longs in front). The challenge was in having DirectSignatureShuffle 
>> to be fully symmetric - e.g. I did not want to have different code 
>> paths for upcalls and downcalls, so the code tries quite hard to be 
>> parametric in the shuffling direction (java->native or native->java) 
>> - which means that adapters will be applied in one way or in the 
>> inverse way depending on the shuffling direction (and as to whether 
>> we are adapting an argument or a return). Since method handle filters 
>> are composable, it all works out quite beautifully.
>>
>> Note that the resulting, adapted MH is stored in a @Stable field to 
>> tell the JIT to optimize the heck out of it (as if it were a static 
>> constant).
>>
>> This patch contains several other changes - which I discuss briefly 
>> below:
>>
>> * we need to setup a framework in which new invocation strategies can 
>> be plugged in - note that we now have essentially 4 cases:
>>
>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>
>> When the code wants e.g. a NativeInvoker, it asks for one to the 
>> NativeInvoker::of factory (UpcallHandler work in a similar way); this 
>> factory will attempt to go down the fast path - if an error occurs 
>> when computing the fast path, the call will fallback to the universal 
>> (slow) path.
>>
>> Most of the changes you see in the Java code are associated to this 
>> refactoring - e.g. all clients of NativeInvoker/UpcallHandler should 
>> now go through the factory
>>
>> * CallbackImplGenerator had a major issue since the new factory for 
>> NativeInvoker wants to bind an address eagerly (this is required e.g. 
>> to be forward compatible with linkToNative backend); which means that 
>> at construction time we have to get the address of the callback, call 
>> the NativeInvoker factory and then stash the target method handle 
>> into a field of the anon callback class. Vlad tells me that fields of 
>> anon classes are always 'trusted' by the JIT, which means they should 
>> be treated as '@Stable' (note that I can't put a @Stable annotation 
>> there, since this code will be spinned in user-land).
>>
>> * There are a bunch of properties that can be set to either force 
>> slow path or force 'direct' path; in the latter case, if an error 
>> occurs when instantiating the direct wrapper, an exception is thrown. 
>> This mode is very useful for testing, and I indeed have tried to run 
>> all our tests with this flag enabled, to see how many places could 
>> not be optimized.
>>
>> * I've also reorganized all the native code in hotspot/prims so that 
>> we have a separate file for each scheme (and so that native Java 
>> methods could be added where they really belong). This should also 
>> help in the long run as it should make adding/removing a given scheme 
>> easier.
>>
>> * I've also added a small test which tries to pass structs of 
>> different sizes, but I will also work on a more complex test which 
>> will stress-test all invocation modes in a more complete fashion. 
>> With respect to testing, I've also done a fastdebug build and ran all 
>> tests with that (as fastdebug catches way many more hotspot assertion 
>> than the product version); everything passed.
>>
>> Webrev:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>>
>> I'd like to thank Vladimir Ivanov for the prompt support whenever I 
>> got stuck down the macro assembler rabbit hole :-)
>>
>> Cheers
>> Maurizio
>>
>> [1] - 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>>
>>
>>
>


From jbvernee at xs4all.nl  Fri Sep 28 12:06:17 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Fri, 28 Sep 2018 14:06:17 +0200
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
 <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
Message-ID: <c9a6140fc69b24d6caa13bb1e0441d31@xs4all.nl>

Mostly out of curiosity; can you make the generated MethodHandle field 
in CallbackImplGenerator @Stable as well?

private void generateMethodHandleField(BinderClassWriter cw) {
     cw.visitField(ACC_PRIVATE | ACC_FINAL, MH_FIELD_NAME, 
Type.getDescriptor(MethodHandle.class), null, null)
         .visitAnnotation(Type.getDescriptor(Stable.class), true);
}

Jorn

Maurizio Cimadamore schreef op 2018-09-28 13:19:
> Webrev:
> 
> http://cr.openjdk.java.net/~mcimadamore/panama/8210757_v2/
> 
> Maurizio
> 
> 
> On 28/09/18 12:19, Maurizio Cimadamore wrote:
>> This is an updated version of the direct invocation scheme support. 
>> Very close to the last one, but there are some minor 
>> refactorings/improvements:
>> 
>> 1) Added a @Stable annotation in DirectNativeInvoker's MH field
>> 2) box/unbox routine used by the UniversalXYZ strategies have been 
>> moved from NativeInvoker to UniversalNativeInvoker
>> 3) I revamped the logic which detects whether fastpath is applicable - 
>> now we create the calling sequence first, and we use that to check 
>> whether we can fast path it. Some internal benchmark have shown that 
>> with a large number of symbols, we were doing a lot of work because we 
>> were trying the fastpath always and then, in case of exception 
>> fallback to slow path; in such cases we would create calling sequence 
>> twice. This new technique might also be more friendly w.r.t. Windows 
>> and other ABIs.
>> 
>> I'd really like to move ahead with this (as this RFR has been out for 
>> quite a while now) - if there's no other comments I'll go ahead.
>> 
>> Maurizio
>> 
>> 
>> On 14/09/18 19:04, Maurizio Cimadamore wrote:
>>> Hi,
>>> as mentioned in [1], this patch adds binder support for the so called 
>>> 'direct' invocation scheme, which allows for greater native 
>>> invocation downcall/upcall performances by means of specialized 
>>> adapters. The core idea, also described in [1], is to define adapters 
>>> of the kind:
>>> 
>>> invokeNative_V_DDDDD
>>> invokeNative_V_JDDDD
>>> invokeNative_V_JJDDD
>>> invokeNative_V_JJJDD
>>> invokeNative_V_JJJJD
>>> invokeNative_V_JJJJJ
>>> 
>>> Where long arguments come before double arguments (and do this for 
>>> each arity e.g. <=5).
>>> 
>>> If all arguments are passed in register, then this reordering doesn't 
>>> affect behavior, and greatly limits the number of permutations to be 
>>> supported/generated.
>>> 
>>> The downcall part (java to native) is relative straightforward: the 
>>> directNativeInvoker.cpp file defines a bunch of native entry points, 
>>> one per shape, which cast the input address to a function pointer of 
>>> the desired shape, and then call it:
>>> 
>>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, 
>>> jlong arg0, jdouble arg1) {
>>> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>>> }
>>> 
>>> The upcall business is a little trickier: first, if we are only to 
>>> optimize upcalls where argument passing happens in registers, then 
>>> it's crucial to note that by the time we get into the assembly stub, 
>>> all the registers will have been populated by the native code to 
>>> contain the right arguments in the right places. So we can avoid all 
>>> the shuffling in the assembly adapter and simply jump onto a C 
>>> function that looks like this:
>>> 
>>> long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
>>> ????????????????????????????????????? double d0, double d1, double 
>>> d2, double d3,
>>> ?????????????????????????????????????? unsigned int mask, jobject 
>>> rec) { ... }
>>> 
>>> Note here that the first 8 arguments are just longs and doubles, and 
>>> those will be expected to be in registers, according to the System V 
>>> ABI. (In windows, the situation will be a bit different as less 
>>> integer registers are available, so this will need some work there).
>>> 
>>> So, to recap, the assembly upcall stub simply 'append' the receiver 
>>> object and a 'signature mask' in the last two available C registers 
>>> and then jump onto the helper function. The helper function will find 
>>> all the desired arguments in the right places - there will be, in the 
>>> general case, some unused arguments, but that's fine, after all it 
>>> didn't cost anything to us to load them in the first place!
>>> 
>>> Note that we have three helper variants, one for each return type { 
>>> long, double, void }. This is required as we need the C helper to 
>>> return a value of the right type which will generate the right 
>>> assembly sequence to store the result in the right register (either 
>>> integer or MMX).
>>> 
>>> So, with three helpers we can support all the shapes with up to 8 
>>> arguments. On the Java side we have, of course, to define a 
>>> specialized entry point for each shape.
>>> 
>>> All the magic for adapting method handle to and from the specialized 
>>> adapters happen in the DirectSignatureShuffler class; this class is 
>>> responsible for adapting each argument e.g. from Java to native 
>>> value, and then reordering the adapted method handle to match the 
>>> order in which arguments are expected by the adapter (e.g. move all 
>>> longs in front). The challenge was in having DirectSignatureShuffle 
>>> to be fully symmetric - e.g. I did not want to have different code 
>>> paths for upcalls and downcalls, so the code tries quite hard to be 
>>> parametric in the shuffling direction (java->native or native->java) 
>>> - which means that adapters will be applied in one way or in the 
>>> inverse way depending on the shuffling direction (and as to whether 
>>> we are adapting an argument or a return). Since method handle filters 
>>> are composable, it all works out quite beautifully.
>>> 
>>> Note that the resulting, adapted MH is stored in a @Stable field to 
>>> tell the JIT to optimize the heck out of it (as if it were a static 
>>> constant).
>>> 
>>> This patch contains several other changes - which I discuss briefly 
>>> below:
>>> 
>>> * we need to setup a framework in which new invocation strategies can 
>>> be plugged in - note that we now have essentially 4 cases:
>>> 
>>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>> 
>>> When the code wants e.g. a NativeInvoker, it asks for one to the 
>>> NativeInvoker::of factory (UpcallHandler work in a similar way); this 
>>> factory will attempt to go down the fast path - if an error occurs 
>>> when computing the fast path, the call will fallback to the universal 
>>> (slow) path.
>>> 
>>> Most of the changes you see in the Java code are associated to this 
>>> refactoring - e.g. all clients of NativeInvoker/UpcallHandler should 
>>> now go through the factory
>>> 
>>> * CallbackImplGenerator had a major issue since the new factory for 
>>> NativeInvoker wants to bind an address eagerly (this is required e.g. 
>>> to be forward compatible with linkToNative backend); which means that 
>>> at construction time we have to get the address of the callback, call 
>>> the NativeInvoker factory and then stash the target method handle 
>>> into a field of the anon callback class. Vlad tells me that fields of 
>>> anon classes are always 'trusted' by the JIT, which means they should 
>>> be treated as '@Stable' (note that I can't put a @Stable annotation 
>>> there, since this code will be spinned in user-land).
>>> 
>>> * There are a bunch of properties that can be set to either force 
>>> slow path or force 'direct' path; in the latter case, if an error 
>>> occurs when instantiating the direct wrapper, an exception is thrown. 
>>> This mode is very useful for testing, and I indeed have tried to run 
>>> all our tests with this flag enabled, to see how many places could 
>>> not be optimized.
>>> 
>>> * I've also reorganized all the native code in hotspot/prims so that 
>>> we have a separate file for each scheme (and so that native Java 
>>> methods could be added where they really belong). This should also 
>>> help in the long run as it should make adding/removing a given scheme 
>>> easier.
>>> 
>>> * I've also added a small test which tries to pass structs of 
>>> different sizes, but I will also work on a more complex test which 
>>> will stress-test all invocation modes in a more complete fashion. 
>>> With respect to testing, I've also done a fastdebug build and ran all 
>>> tests with that (as fastdebug catches way many more hotspot assertion 
>>> than the product version); everything passed.
>>> 
>>> Webrev:
>>> 
>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>>> 
>>> I'd like to thank Vladimir Ivanov for the prompt support whenever I 
>>> got stuck down the macro assembler rabbit hole :-)
>>> 
>>> Cheers
>>> Maurizio
>>> 
>>> [1] - 
>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>>> 
>>> 
>>> 
>> 

From maurizio.cimadamore at oracle.com  Fri Sep 28 12:11:50 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 13:11:50 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <c9a6140fc69b24d6caa13bb1e0441d31@xs4all.nl>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
 <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
 <c9a6140fc69b24d6caa13bb1e0441d31@xs4all.nl>
Message-ID: <b888fb52-89cb-2778-c9ec-62ca3c456705@oracle.com>

No need for that, Vm treats all final fields on VM anon classes as 
@Stable (or so I've been told) :-)

Maurizio


On 28/09/18 13:06, Jorn Vernee wrote:
> Mostly out of curiosity; can you make the generated MethodHandle field 
> in CallbackImplGenerator @Stable as well?
>
> private void generateMethodHandleField(BinderClassWriter cw) {
> ??? cw.visitField(ACC_PRIVATE | ACC_FINAL, MH_FIELD_NAME, 
> Type.getDescriptor(MethodHandle.class), null, null)
> ??????? .visitAnnotation(Type.getDescriptor(Stable.class), true);
> }
>
> Jorn
>
> Maurizio Cimadamore schreef op 2018-09-28 13:19:
>> Webrev:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757_v2/
>>
>> Maurizio
>>
>>
>> On 28/09/18 12:19, Maurizio Cimadamore wrote:
>>> This is an updated version of the direct invocation scheme support. 
>>> Very close to the last one, but there are some minor 
>>> refactorings/improvements:
>>>
>>> 1) Added a @Stable annotation in DirectNativeInvoker's MH field
>>> 2) box/unbox routine used by the UniversalXYZ strategies have been 
>>> moved from NativeInvoker to UniversalNativeInvoker
>>> 3) I revamped the logic which detects whether fastpath is applicable 
>>> - now we create the calling sequence first, and we use that to check 
>>> whether we can fast path it. Some internal benchmark have shown that 
>>> with a large number of symbols, we were doing a lot of work because 
>>> we were trying the fastpath always and then, in case of exception 
>>> fallback to slow path; in such cases we would create calling 
>>> sequence twice. This new technique might also be more friendly 
>>> w.r.t. Windows and other ABIs.
>>>
>>> I'd really like to move ahead with this (as this RFR has been out 
>>> for quite a while now) - if there's no other comments I'll go ahead.
>>>
>>> Maurizio
>>>
>>>
>>> On 14/09/18 19:04, Maurizio Cimadamore wrote:
>>>> Hi,
>>>> as mentioned in [1], this patch adds binder support for the so 
>>>> called 'direct' invocation scheme, which allows for greater native 
>>>> invocation downcall/upcall performances by means of specialized 
>>>> adapters. The core idea, also described in [1], is to define 
>>>> adapters of the kind:
>>>>
>>>> invokeNative_V_DDDDD
>>>> invokeNative_V_JDDDD
>>>> invokeNative_V_JJDDD
>>>> invokeNative_V_JJJDD
>>>> invokeNative_V_JJJJD
>>>> invokeNative_V_JJJJJ
>>>>
>>>> Where long arguments come before double arguments (and do this for 
>>>> each arity e.g. <=5).
>>>>
>>>> If all arguments are passed in register, then this reordering 
>>>> doesn't affect behavior, and greatly limits the number of 
>>>> permutations to be supported/generated.
>>>>
>>>> The downcall part (java to native) is relative straightforward: the 
>>>> directNativeInvoker.cpp file defines a bunch of native entry 
>>>> points, one per shape, which cast the input address to a function 
>>>> pointer of the desired shape, and then call it:
>>>>
>>>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong 
>>>> addr, jlong arg0, jdouble arg1) {
>>>> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>>>> }
>>>>
>>>> The upcall business is a little trickier: first, if we are only to 
>>>> optimize upcalls where argument passing happens in registers, then 
>>>> it's crucial to note that by the time we get into the assembly 
>>>> stub, all the registers will have been populated by the native code 
>>>> to contain the right arguments in the right places. So we can avoid 
>>>> all the shuffling in the assembly adapter and simply jump onto a C 
>>>> function that looks like this:
>>>>
>>>> long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
>>>> ????????????????????????????????????? double d0, double d1, double 
>>>> d2, double d3,
>>>> ?????????????????????????????????????? unsigned int mask, jobject 
>>>> rec) { ... }
>>>>
>>>> Note here that the first 8 arguments are just longs and doubles, 
>>>> and those will be expected to be in registers, according to the 
>>>> System V ABI. (In windows, the situation will be a bit different as 
>>>> less integer registers are available, so this will need some work 
>>>> there).
>>>>
>>>> So, to recap, the assembly upcall stub simply 'append' the receiver 
>>>> object and a 'signature mask' in the last two available C registers 
>>>> and then jump onto the helper function. The helper function will 
>>>> find all the desired arguments in the right places - there will be, 
>>>> in the general case, some unused arguments, but that's fine, after 
>>>> all it didn't cost anything to us to load them in the first place!
>>>>
>>>> Note that we have three helper variants, one for each return type { 
>>>> long, double, void }. This is required as we need the C helper to 
>>>> return a value of the right type which will generate the right 
>>>> assembly sequence to store the result in the right register (either 
>>>> integer or MMX).
>>>>
>>>> So, with three helpers we can support all the shapes with up to 8 
>>>> arguments. On the Java side we have, of course, to define a 
>>>> specialized entry point for each shape.
>>>>
>>>> All the magic for adapting method handle to and from the 
>>>> specialized adapters happen in the DirectSignatureShuffler class; 
>>>> this class is responsible for adapting each argument e.g. from Java 
>>>> to native value, and then reordering the adapted method handle to 
>>>> match the order in which arguments are expected by the adapter 
>>>> (e.g. move all longs in front). The challenge was in having 
>>>> DirectSignatureShuffle to be fully symmetric - e.g. I did not want 
>>>> to have different code paths for upcalls and downcalls, so the code 
>>>> tries quite hard to be parametric in the shuffling direction 
>>>> (java->native or native->java) - which means that adapters will be 
>>>> applied in one way or in the inverse way depending on the shuffling 
>>>> direction (and as to whether we are adapting an argument or a 
>>>> return). Since method handle filters are composable, it all works 
>>>> out quite beautifully.
>>>>
>>>> Note that the resulting, adapted MH is stored in a @Stable field to 
>>>> tell the JIT to optimize the heck out of it (as if it were a static 
>>>> constant).
>>>>
>>>> This patch contains several other changes - which I discuss briefly 
>>>> below:
>>>>
>>>> * we need to setup a framework in which new invocation strategies 
>>>> can be plugged in - note that we now have essentially 4 cases:
>>>>
>>>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>>>
>>>> When the code wants e.g. a NativeInvoker, it asks for one to the 
>>>> NativeInvoker::of factory (UpcallHandler work in a similar way); 
>>>> this factory will attempt to go down the fast path - if an error 
>>>> occurs when computing the fast path, the call will fallback to the 
>>>> universal (slow) path.
>>>>
>>>> Most of the changes you see in the Java code are associated to this 
>>>> refactoring - e.g. all clients of NativeInvoker/UpcallHandler 
>>>> should now go through the factory
>>>>
>>>> * CallbackImplGenerator had a major issue since the new factory for 
>>>> NativeInvoker wants to bind an address eagerly (this is required 
>>>> e.g. to be forward compatible with linkToNative backend); which 
>>>> means that at construction time we have to get the address of the 
>>>> callback, call the NativeInvoker factory and then stash the target 
>>>> method handle into a field of the anon callback class. Vlad tells 
>>>> me that fields of anon classes are always 'trusted' by the JIT, 
>>>> which means they should be treated as '@Stable' (note that I can't 
>>>> put a @Stable annotation there, since this code will be spinned in 
>>>> user-land).
>>>>
>>>> * There are a bunch of properties that can be set to either force 
>>>> slow path or force 'direct' path; in the latter case, if an error 
>>>> occurs when instantiating the direct wrapper, an exception is 
>>>> thrown. This mode is very useful for testing, and I indeed have 
>>>> tried to run all our tests with this flag enabled, to see how many 
>>>> places could not be optimized.
>>>>
>>>> * I've also reorganized all the native code in hotspot/prims so 
>>>> that we have a separate file for each scheme (and so that native 
>>>> Java methods could be added where they really belong). This should 
>>>> also help in the long run as it should make adding/removing a given 
>>>> scheme easier.
>>>>
>>>> * I've also added a small test which tries to pass structs of 
>>>> different sizes, but I will also work on a more complex test which 
>>>> will stress-test all invocation modes in a more complete fashion. 
>>>> With respect to testing, I've also done a fastdebug build and ran 
>>>> all tests with that (as fastdebug catches way many more hotspot 
>>>> assertion than the product version); everything passed.
>>>>
>>>> Webrev:
>>>>
>>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>>>>
>>>> I'd like to thank Vladimir Ivanov for the prompt support whenever I 
>>>> got stuck down the macro assembler rabbit hole :-)
>>>>
>>>> Cheers
>>>> Maurizio
>>>>
>>>> [1] - 
>>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>>>>
>>>>
>>>>
>>>


From maurizio.cimadamore at oracle.com  Fri Sep 28 12:12:57 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 13:12:57 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <b888fb52-89cb-2778-c9ec-62ca3c456705@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
 <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
 <c9a6140fc69b24d6caa13bb1e0441d31@xs4all.nl>
 <b888fb52-89cb-2778-c9ec-62ca3c456705@oracle.com>
Message-ID: <a4e605a9-7f0a-3a85-39c1-c99538b07431@oracle.com>

And no, we can't do what you suggest (at least not in a straightforward 
fashion) because @Stable is in a non-exported package of java.base and 
the callback is spinned in the user-land. But it should be no issue.

Maurizio


On 28/09/18 13:11, Maurizio Cimadamore wrote:
> No need for that, Vm treats all final fields on VM anon classes as 
> @Stable (or so I've been told) :-)
>
> Maurizio
>
>
> On 28/09/18 13:06, Jorn Vernee wrote:
>> Mostly out of curiosity; can you make the generated MethodHandle 
>> field in CallbackImplGenerator @Stable as well?
>>
>> private void generateMethodHandleField(BinderClassWriter cw) {
>> ??? cw.visitField(ACC_PRIVATE | ACC_FINAL, MH_FIELD_NAME, 
>> Type.getDescriptor(MethodHandle.class), null, null)
>> ??????? .visitAnnotation(Type.getDescriptor(Stable.class), true);
>> }
>>
>> Jorn
>>
>> Maurizio Cimadamore schreef op 2018-09-28 13:19:
>>> Webrev:
>>>
>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757_v2/
>>>
>>> Maurizio
>>>
>>>
>>> On 28/09/18 12:19, Maurizio Cimadamore wrote:
>>>> This is an updated version of the direct invocation scheme support. 
>>>> Very close to the last one, but there are some minor 
>>>> refactorings/improvements:
>>>>
>>>> 1) Added a @Stable annotation in DirectNativeInvoker's MH field
>>>> 2) box/unbox routine used by the UniversalXYZ strategies have been 
>>>> moved from NativeInvoker to UniversalNativeInvoker
>>>> 3) I revamped the logic which detects whether fastpath is 
>>>> applicable - now we create the calling sequence first, and we use 
>>>> that to check whether we can fast path it. Some internal benchmark 
>>>> have shown that with a large number of symbols, we were doing a lot 
>>>> of work because we were trying the fastpath always and then, in 
>>>> case of exception fallback to slow path; in such cases we would 
>>>> create calling sequence twice. This new technique might also be 
>>>> more friendly w.r.t. Windows and other ABIs.
>>>>
>>>> I'd really like to move ahead with this (as this RFR has been out 
>>>> for quite a while now) - if there's no other comments I'll go ahead.
>>>>
>>>> Maurizio
>>>>
>>>>
>>>> On 14/09/18 19:04, Maurizio Cimadamore wrote:
>>>>> Hi,
>>>>> as mentioned in [1], this patch adds binder support for the so 
>>>>> called 'direct' invocation scheme, which allows for greater native 
>>>>> invocation downcall/upcall performances by means of specialized 
>>>>> adapters. The core idea, also described in [1], is to define 
>>>>> adapters of the kind:
>>>>>
>>>>> invokeNative_V_DDDDD
>>>>> invokeNative_V_JDDDD
>>>>> invokeNative_V_JJDDD
>>>>> invokeNative_V_JJJDD
>>>>> invokeNative_V_JJJJD
>>>>> invokeNative_V_JJJJJ
>>>>>
>>>>> Where long arguments come before double arguments (and do this for 
>>>>> each arity e.g. <=5).
>>>>>
>>>>> If all arguments are passed in register, then this reordering 
>>>>> doesn't affect behavior, and greatly limits the number of 
>>>>> permutations to be supported/generated.
>>>>>
>>>>> The downcall part (java to native) is relative straightforward: 
>>>>> the directNativeInvoker.cpp file defines a bunch of native entry 
>>>>> points, one per shape, which cast the input address to a function 
>>>>> pointer of the desired shape, and then call it:
>>>>>
>>>>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong 
>>>>> addr, jlong arg0, jdouble arg1) {
>>>>> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>>>>> }
>>>>>
>>>>> The upcall business is a little trickier: first, if we are only to 
>>>>> optimize upcalls where argument passing happens in registers, then 
>>>>> it's crucial to note that by the time we get into the assembly 
>>>>> stub, all the registers will have been populated by the native 
>>>>> code to contain the right arguments in the right places. So we can 
>>>>> avoid all the shuffling in the assembly adapter and simply jump 
>>>>> onto a C function that looks like this:
>>>>>
>>>>> long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
>>>>> ????????????????????????????????????? double d0, double d1, double 
>>>>> d2, double d3,
>>>>> ?????????????????????????????????????? unsigned int mask, jobject 
>>>>> rec) { ... }
>>>>>
>>>>> Note here that the first 8 arguments are just longs and doubles, 
>>>>> and those will be expected to be in registers, according to the 
>>>>> System V ABI. (In windows, the situation will be a bit different 
>>>>> as less integer registers are available, so this will need some 
>>>>> work there).
>>>>>
>>>>> So, to recap, the assembly upcall stub simply 'append' the 
>>>>> receiver object and a 'signature mask' in the last two available C 
>>>>> registers and then jump onto the helper function. The helper 
>>>>> function will find all the desired arguments in the right places - 
>>>>> there will be, in the general case, some unused arguments, but 
>>>>> that's fine, after all it didn't cost anything to us to load them 
>>>>> in the first place!
>>>>>
>>>>> Note that we have three helper variants, one for each return type 
>>>>> { long, double, void }. This is required as we need the C helper 
>>>>> to return a value of the right type which will generate the right 
>>>>> assembly sequence to store the result in the right register 
>>>>> (either integer or MMX).
>>>>>
>>>>> So, with three helpers we can support all the shapes with up to 8 
>>>>> arguments. On the Java side we have, of course, to define a 
>>>>> specialized entry point for each shape.
>>>>>
>>>>> All the magic for adapting method handle to and from the 
>>>>> specialized adapters happen in the DirectSignatureShuffler class; 
>>>>> this class is responsible for adapting each argument e.g. from 
>>>>> Java to native value, and then reordering the adapted method 
>>>>> handle to match the order in which arguments are expected by the 
>>>>> adapter (e.g. move all longs in front). The challenge was in 
>>>>> having DirectSignatureShuffle to be fully symmetric - e.g. I did 
>>>>> not want to have different code paths for upcalls and downcalls, 
>>>>> so the code tries quite hard to be parametric in the shuffling 
>>>>> direction (java->native or native->java) - which means that 
>>>>> adapters will be applied in one way or in the inverse way 
>>>>> depending on the shuffling direction (and as to whether we are 
>>>>> adapting an argument or a return). Since method handle filters are 
>>>>> composable, it all works out quite beautifully.
>>>>>
>>>>> Note that the resulting, adapted MH is stored in a @Stable field 
>>>>> to tell the JIT to optimize the heck out of it (as if it were a 
>>>>> static constant).
>>>>>
>>>>> This patch contains several other changes - which I discuss 
>>>>> briefly below:
>>>>>
>>>>> * we need to setup a framework in which new invocation strategies 
>>>>> can be plugged in - note that we now have essentially 4 cases:
>>>>>
>>>>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>>>>
>>>>> When the code wants e.g. a NativeInvoker, it asks for one to the 
>>>>> NativeInvoker::of factory (UpcallHandler work in a similar way); 
>>>>> this factory will attempt to go down the fast path - if an error 
>>>>> occurs when computing the fast path, the call will fallback to the 
>>>>> universal (slow) path.
>>>>>
>>>>> Most of the changes you see in the Java code are associated to 
>>>>> this refactoring - e.g. all clients of NativeInvoker/UpcallHandler 
>>>>> should now go through the factory
>>>>>
>>>>> * CallbackImplGenerator had a major issue since the new factory 
>>>>> for NativeInvoker wants to bind an address eagerly (this is 
>>>>> required e.g. to be forward compatible with linkToNative backend); 
>>>>> which means that at construction time we have to get the address 
>>>>> of the callback, call the NativeInvoker factory and then stash the 
>>>>> target method handle into a field of the anon callback class. Vlad 
>>>>> tells me that fields of anon classes are always 'trusted' by the 
>>>>> JIT, which means they should be treated as '@Stable' (note that I 
>>>>> can't put a @Stable annotation there, since this code will be 
>>>>> spinned in user-land).
>>>>>
>>>>> * There are a bunch of properties that can be set to either force 
>>>>> slow path or force 'direct' path; in the latter case, if an error 
>>>>> occurs when instantiating the direct wrapper, an exception is 
>>>>> thrown. This mode is very useful for testing, and I indeed have 
>>>>> tried to run all our tests with this flag enabled, to see how many 
>>>>> places could not be optimized.
>>>>>
>>>>> * I've also reorganized all the native code in hotspot/prims so 
>>>>> that we have a separate file for each scheme (and so that native 
>>>>> Java methods could be added where they really belong). This should 
>>>>> also help in the long run as it should make adding/removing a 
>>>>> given scheme easier.
>>>>>
>>>>> * I've also added a small test which tries to pass structs of 
>>>>> different sizes, but I will also work on a more complex test which 
>>>>> will stress-test all invocation modes in a more complete fashion. 
>>>>> With respect to testing, I've also done a fastdebug build and ran 
>>>>> all tests with that (as fastdebug catches way many more hotspot 
>>>>> assertion than the product version); everything passed.
>>>>>
>>>>> Webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>>>>>
>>>>> I'd like to thank Vladimir Ivanov for the prompt support whenever 
>>>>> I got stuck down the macro assembler rabbit hole :-)
>>>>>
>>>>> Cheers
>>>>> Maurizio
>>>>>
>>>>> [1] - 
>>>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>>>>>
>>>>>
>>>>>
>>>>
>


From jbvernee at xs4all.nl  Fri Sep 28 12:18:28 2018
From: jbvernee at xs4all.nl (Jorn Vernee)
Date: Fri, 28 Sep 2018 14:18:28 +0200
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <a4e605a9-7f0a-3a85-39c1-c99538b07431@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
 <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
 <c9a6140fc69b24d6caa13bb1e0441d31@xs4all.nl>
 <b888fb52-89cb-2778-c9ec-62ca3c456705@oracle.com>
 <a4e605a9-7f0a-3a85-39c1-c99538b07431@oracle.com>
Message-ID: <94b6ef1e29d53446e548a6d79a1cc4f1@xs4all.nl>

Oh sorry, I just remembered/noticed that you put that in the original 
email as well. Just got the idea by looking at the code.

Jorn

Maurizio Cimadamore schreef op 2018-09-28 14:12:
> And no, we can't do what you suggest (at least not in a
> straightforward fashion) because @Stable is in a non-exported package
> of java.base and the callback is spinned in the user-land. But it
> should be no issue.
> 
> Maurizio
> 
> 
> On 28/09/18 13:11, Maurizio Cimadamore wrote:
>> No need for that, Vm treats all final fields on VM anon classes as 
>> @Stable (or so I've been told) :-)
>> 
>> Maurizio
>> 
>> 
>> On 28/09/18 13:06, Jorn Vernee wrote:
>>> Mostly out of curiosity; can you make the generated MethodHandle 
>>> field in CallbackImplGenerator @Stable as well?
>>> 
>>> private void generateMethodHandleField(BinderClassWriter cw) {
>>> ??? cw.visitField(ACC_PRIVATE | ACC_FINAL, MH_FIELD_NAME, 
>>> Type.getDescriptor(MethodHandle.class), null, null)
>>> ??????? .visitAnnotation(Type.getDescriptor(Stable.class), true);
>>> }
>>> 
>>> Jorn
>>> 
>>> Maurizio Cimadamore schreef op 2018-09-28 13:19:
>>>> Webrev:
>>>> 
>>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757_v2/
>>>> 
>>>> Maurizio
>>>> 
>>>> 
>>>> On 28/09/18 12:19, Maurizio Cimadamore wrote:
>>>>> This is an updated version of the direct invocation scheme support. 
>>>>> Very close to the last one, but there are some minor 
>>>>> refactorings/improvements:
>>>>> 
>>>>> 1) Added a @Stable annotation in DirectNativeInvoker's MH field
>>>>> 2) box/unbox routine used by the UniversalXYZ strategies have been 
>>>>> moved from NativeInvoker to UniversalNativeInvoker
>>>>> 3) I revamped the logic which detects whether fastpath is 
>>>>> applicable - now we create the calling sequence first, and we use 
>>>>> that to check whether we can fast path it. Some internal benchmark 
>>>>> have shown that with a large number of symbols, we were doing a lot 
>>>>> of work because we were trying the fastpath always and then, in 
>>>>> case of exception fallback to slow path; in such cases we would 
>>>>> create calling sequence twice. This new technique might also be 
>>>>> more friendly w.r.t. Windows and other ABIs.
>>>>> 
>>>>> I'd really like to move ahead with this (as this RFR has been out 
>>>>> for quite a while now) - if there's no other comments I'll go 
>>>>> ahead.
>>>>> 
>>>>> Maurizio
>>>>> 
>>>>> 
>>>>> On 14/09/18 19:04, Maurizio Cimadamore wrote:
>>>>>> Hi,
>>>>>> as mentioned in [1], this patch adds binder support for the so 
>>>>>> called 'direct' invocation scheme, which allows for greater native 
>>>>>> invocation downcall/upcall performances by means of specialized 
>>>>>> adapters. The core idea, also described in [1], is to define 
>>>>>> adapters of the kind:
>>>>>> 
>>>>>> invokeNative_V_DDDDD
>>>>>> invokeNative_V_JDDDD
>>>>>> invokeNative_V_JJDDD
>>>>>> invokeNative_V_JJJDD
>>>>>> invokeNative_V_JJJJD
>>>>>> invokeNative_V_JJJJJ
>>>>>> 
>>>>>> Where long arguments come before double arguments (and do this for 
>>>>>> each arity e.g. <=5).
>>>>>> 
>>>>>> If all arguments are passed in register, then this reordering 
>>>>>> doesn't affect behavior, and greatly limits the number of 
>>>>>> permutations to be supported/generated.
>>>>>> 
>>>>>> The downcall part (java to native) is relative straightforward: 
>>>>>> the directNativeInvoker.cpp file defines a bunch of native entry 
>>>>>> points, one per shape, which cast the input address to a function 
>>>>>> pointer of the desired shape, and then call it:
>>>>>> 
>>>>>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong 
>>>>>> addr, jlong arg0, jdouble arg1) {
>>>>>> ??? return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>>>>>> }
>>>>>> 
>>>>>> The upcall business is a little trickier: first, if we are only to 
>>>>>> optimize upcalls where argument passing happens in registers, then 
>>>>>> it's crucial to note that by the time we get into the assembly 
>>>>>> stub, all the registers will have been populated by the native 
>>>>>> code to contain the right arguments in the right places. So we can 
>>>>>> avoid all the shuffling in the assembly adapter and simply jump 
>>>>>> onto a C function that looks like this:
>>>>>> 
>>>>>> long specialized_upcall_helper_J(long l0, long l1, long l2, long 
>>>>>> l3,
>>>>>> ????????????????????????????????????? double d0, double d1, double 
>>>>>> d2, double d3,
>>>>>> ?????????????????????????????????????? unsigned int mask, jobject 
>>>>>> rec) { ... }
>>>>>> 
>>>>>> Note here that the first 8 arguments are just longs and doubles, 
>>>>>> and those will be expected to be in registers, according to the 
>>>>>> System V ABI. (In windows, the situation will be a bit different 
>>>>>> as less integer registers are available, so this will need some 
>>>>>> work there).
>>>>>> 
>>>>>> So, to recap, the assembly upcall stub simply 'append' the 
>>>>>> receiver object and a 'signature mask' in the last two available C 
>>>>>> registers and then jump onto the helper function. The helper 
>>>>>> function will find all the desired arguments in the right places - 
>>>>>> there will be, in the general case, some unused arguments, but 
>>>>>> that's fine, after all it didn't cost anything to us to load them 
>>>>>> in the first place!
>>>>>> 
>>>>>> Note that we have three helper variants, one for each return type 
>>>>>> { long, double, void }. This is required as we need the C helper 
>>>>>> to return a value of the right type which will generate the right 
>>>>>> assembly sequence to store the result in the right register 
>>>>>> (either integer or MMX).
>>>>>> 
>>>>>> So, with three helpers we can support all the shapes with up to 8 
>>>>>> arguments. On the Java side we have, of course, to define a 
>>>>>> specialized entry point for each shape.
>>>>>> 
>>>>>> All the magic for adapting method handle to and from the 
>>>>>> specialized adapters happen in the DirectSignatureShuffler class; 
>>>>>> this class is responsible for adapting each argument e.g. from 
>>>>>> Java to native value, and then reordering the adapted method 
>>>>>> handle to match the order in which arguments are expected by the 
>>>>>> adapter (e.g. move all longs in front). The challenge was in 
>>>>>> having DirectSignatureShuffle to be fully symmetric - e.g. I did 
>>>>>> not want to have different code paths for upcalls and downcalls, 
>>>>>> so the code tries quite hard to be parametric in the shuffling 
>>>>>> direction (java->native or native->java) - which means that 
>>>>>> adapters will be applied in one way or in the inverse way 
>>>>>> depending on the shuffling direction (and as to whether we are 
>>>>>> adapting an argument or a return). Since method handle filters are 
>>>>>> composable, it all works out quite beautifully.
>>>>>> 
>>>>>> Note that the resulting, adapted MH is stored in a @Stable field 
>>>>>> to tell the JIT to optimize the heck out of it (as if it were a 
>>>>>> static constant).
>>>>>> 
>>>>>> This patch contains several other changes - which I discuss 
>>>>>> briefly below:
>>>>>> 
>>>>>> * we need to setup a framework in which new invocation strategies 
>>>>>> can be plugged in - note that we now have essentially 4 cases:
>>>>>> 
>>>>>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>>>>> 
>>>>>> When the code wants e.g. a NativeInvoker, it asks for one to the 
>>>>>> NativeInvoker::of factory (UpcallHandler work in a similar way); 
>>>>>> this factory will attempt to go down the fast path - if an error 
>>>>>> occurs when computing the fast path, the call will fallback to the 
>>>>>> universal (slow) path.
>>>>>> 
>>>>>> Most of the changes you see in the Java code are associated to 
>>>>>> this refactoring - e.g. all clients of NativeInvoker/UpcallHandler 
>>>>>> should now go through the factory
>>>>>> 
>>>>>> * CallbackImplGenerator had a major issue since the new factory 
>>>>>> for NativeInvoker wants to bind an address eagerly (this is 
>>>>>> required e.g. to be forward compatible with linkToNative backend); 
>>>>>> which means that at construction time we have to get the address 
>>>>>> of the callback, call the NativeInvoker factory and then stash the 
>>>>>> target method handle into a field of the anon callback class. Vlad 
>>>>>> tells me that fields of anon classes are always 'trusted' by the 
>>>>>> JIT, which means they should be treated as '@Stable' (note that I 
>>>>>> can't put a @Stable annotation there, since this code will be 
>>>>>> spinned in user-land).
>>>>>> 
>>>>>> * There are a bunch of properties that can be set to either force 
>>>>>> slow path or force 'direct' path; in the latter case, if an error 
>>>>>> occurs when instantiating the direct wrapper, an exception is 
>>>>>> thrown. This mode is very useful for testing, and I indeed have 
>>>>>> tried to run all our tests with this flag enabled, to see how many 
>>>>>> places could not be optimized.
>>>>>> 
>>>>>> * I've also reorganized all the native code in hotspot/prims so 
>>>>>> that we have a separate file for each scheme (and so that native 
>>>>>> Java methods could be added where they really belong). This should 
>>>>>> also help in the long run as it should make adding/removing a 
>>>>>> given scheme easier.
>>>>>> 
>>>>>> * I've also added a small test which tries to pass structs of 
>>>>>> different sizes, but I will also work on a more complex test which 
>>>>>> will stress-test all invocation modes in a more complete fashion. 
>>>>>> With respect to testing, I've also done a fastdebug build and ran 
>>>>>> all tests with that (as fastdebug catches way many more hotspot 
>>>>>> assertion than the product version); everything passed.
>>>>>> 
>>>>>> Webrev:
>>>>>> 
>>>>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>>>>>> 
>>>>>> I'd like to thank Vladimir Ivanov for the prompt support whenever 
>>>>>> I got stuck down the macro assembler rabbit hole :-)
>>>>>> 
>>>>>> Cheers
>>>>>> Maurizio
>>>>>> 
>>>>>> [1] - 
>>>>>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>> 

From maurizio.cimadamore at oracle.com  Fri Sep 28 12:40:23 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 13:40:23 +0100
Subject: [foreign] RFR 8211275: Binder should check that signature of bound
 function is compatible with layout descriptions
Message-ID: <12b07045-bb4e-c45e-0cd5-199c004177f1@oracle.com>

Hi,
this is a RFR for the changes discussed in [1] (thanks Jorn!). I've 
added few things:

* ensure that callbacks are checked too

* refactored the isCompatible method to be more usable from clients 
(e.g. reuse logic that throws assertions)

* added one more testcase to check a simple case of callback mismatch, 
just to make sure the logic is executed

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8211275/

Cheers
Maurizio

[1] - 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002786.html



From maurizio.cimadamore at oracle.com  Fri Sep 28 13:05:08 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 14:05:08 +0100
Subject: [foreign] RFR 8211277: Exception when dereferencing null pointer
 should be more specific
Message-ID: <3d502c5f-a56b-c856-1b15-a62ab4e8a760@oracle.com>

Hi,
this is a RFR for the changes discussed in [1] (again, thanks Jorn!). 
Some minor tweaks:

* consolidate the code paths for 'throwy' References - new 
Reference.OfGrumpy :-)

* tweaked test to check the exception cause - as otherwise the test 
passes even w/o the fix

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8211277/

Cheers
Maurizio

[1] - 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002796.html


From maurizio.cimadamore at oracle.com  Fri Sep 28 16:53:11 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 17:53:11 +0100
Subject: [foreign] RFR 8211281: Strenghten the logic for parsing header
 declarations
Message-ID: <8768816f-78cf-871d-49be-a04287077269@oracle.com>

Hi,
this patch revisits the logic for parsing the contents of header 
declaration annotations. As noted in [1] the old logic was too lax, and 
certain errors were not caught at parse time, which then generated weird 
issue at bind time.

The new logic works inside the parser (and not using parser as a black 
box, as before); this means we can now check for things like:

* the declaration name is not valid (e.g. "fo!*** = (i32)v")

* the declaration name has spaces (e.g. "fo o=(i32)v")

* there's no declaration name (e.g. "=(i32)v")

* there's no equal (e.g. "(i32)v")

I've added a test which checks many such conditions - the test is 
slightly more general than the examples above in that it tries to test a 
given configuration against different kinds of prefix and suffixes.

Webrev:

http://cr.openjdk.java.net/~mcimadamore/panama/8211281/

Cheers
Maurizio

[1] - 
http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002788.html


From maurizio.cimadamore at oracle.com  Fri Sep 28 17:01:38 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 18:01:38 +0100
Subject: [foreign] RFR 8211281: Strenghten the logic for parsing header
 declarations
In-Reply-To: <8768816f-78cf-871d-49be-a04287077269@oracle.com>
References: <8768816f-78cf-871d-49be-a04287077269@oracle.com>
Message-ID: <09be12fb-d8ff-f838-7b10-ae89a8770171@oracle.com>

The previous webrev link was pointing to an older version - please use 
the one below

http://cr.openjdk.java.net/~mcimadamore/panama/8211281-v3/

Apologies for the inconvenience.

Maurizio


On 28/09/18 17:53, Maurizio Cimadamore wrote:
> Hi,
> this patch revisits the logic for parsing the contents of header 
> declaration annotations. As noted in [1] the old logic was too lax, 
> and certain errors were not caught at parse time, which then generated 
> weird issue at bind time.
>
> The new logic works inside the parser (and not using parser as a black 
> box, as before); this means we can now check for things like:
>
> * the declaration name is not valid (e.g. "fo!*** = (i32)v")
>
> * the declaration name has spaces (e.g. "fo o=(i32)v")
>
> * there's no declaration name (e.g. "=(i32)v")
>
> * there's no equal (e.g. "(i32)v")
>
> I've added a test which checks many such conditions - the test is 
> slightly more general than the examples above in that it tries to test 
> a given configuration against different kinds of prefix and suffixes.
>
> Webrev:
>
> http://cr.openjdk.java.net/~mcimadamore/panama/8211281/
>
> Cheers
> Maurizio
>
> [1] - 
> http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002788.html
>


From henry.jen at oracle.com  Fri Sep 28 19:39:01 2018
From: henry.jen at oracle.com (Henry Jen)
Date: Fri, 28 Sep 2018 12:39:01 -0700
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
 <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
Message-ID: <16DB46B0-D81A-480C-A64E-817F6D31F298@oracle.com>

Looks good to me overall, although I only focus on java side and not dig into assembly. 

A really minor thing in the new RegisterStructTest.java, line 68 seems to be redundant?

 68             checkEquals(rs.l$get(), rs.l$get());

The test passes on Mac.

Cheers,
Henry


> On Sep 28, 2018, at 4:19 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> Webrev:
> 
> http://cr.openjdk.java.net/~mcimadamore/panama/8210757_v2/
> 
> Maurizio
> 
> 
> On 28/09/18 12:19, Maurizio Cimadamore wrote:
>> This is an updated version of the direct invocation scheme support. Very close to the last one, but there are some minor refactorings/improvements:
>> 
>> 1) Added a @Stable annotation in DirectNativeInvoker's MH field
>> 2) box/unbox routine used by the UniversalXYZ strategies have been moved from NativeInvoker to UniversalNativeInvoker
>> 3) I revamped the logic which detects whether fastpath is applicable - now we create the calling sequence first, and we use that to check whether we can fast path it. Some internal benchmark have shown that with a large number of symbols, we were doing a lot of work because we were trying the fastpath always and then, in case of exception fallback to slow path; in such cases we would create calling sequence twice. This new technique might also be more friendly w.r.t. Windows and other ABIs.
>> 
>> I'd really like to move ahead with this (as this RFR has been out for quite a while now) - if there's no other comments I'll go ahead.
>> 
>> Maurizio
>> 
>> 
>> On 14/09/18 19:04, Maurizio Cimadamore wrote:
>>> Hi,
>>> as mentioned in [1], this patch adds binder support for the so called 'direct' invocation scheme, which allows for greater native invocation downcall/upcall performances by means of specialized adapters. The core idea, also described in [1], is to define adapters of the kind:
>>> 
>>> invokeNative_V_DDDDD
>>> invokeNative_V_JDDDD
>>> invokeNative_V_JJDDD
>>> invokeNative_V_JJJDD
>>> invokeNative_V_JJJJD
>>> invokeNative_V_JJJJJ
>>> 
>>> Where long arguments come before double arguments (and do this for each arity e.g. <=5).
>>> 
>>> If all arguments are passed in register, then this reordering doesn't affect behavior, and greatly limits the number of permutations to be supported/generated.
>>> 
>>> The downcall part (java to native) is relative straightforward: the directNativeInvoker.cpp file defines a bunch of native entry points, one per shape, which cast the input address to a function pointer of the desired shape, and then call it:
>>> 
>>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) {
>>>     return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>>> }
>>> 
>>> The upcall business is a little trickier: first, if we are only to optimize upcalls where argument passing happens in registers, then it's crucial to note that by the time we get into the assembly stub, all the registers will have been populated by the native code to contain the right arguments in the right places. So we can avoid all the shuffling in the assembly adapter and simply jump onto a C function that looks like this:
>>> 
>>> long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
>>>                                       double d0, double d1, double d2, double d3,
>>>                                        unsigned int mask, jobject rec) { ... }
>>> 
>>> Note here that the first 8 arguments are just longs and doubles, and those will be expected to be in registers, according to the System V ABI. (In windows, the situation will be a bit different as less integer registers are available, so this will need some work there).
>>> 
>>> So, to recap, the assembly upcall stub simply 'append' the receiver object and a 'signature mask' in the last two available C registers and then jump onto the helper function. The helper function will find all the desired arguments in the right places - there will be, in the general case, some unused arguments, but that's fine, after all it didn't cost anything to us to load them in the first place!
>>> 
>>> Note that we have three helper variants, one for each return type { long, double, void }. This is required as we need the C helper to return a value of the right type which will generate the right assembly sequence to store the result in the right register (either integer or MMX).
>>> 
>>> So, with three helpers we can support all the shapes with up to 8 arguments. On the Java side we have, of course, to define a specialized entry point for each shape.
>>> 
>>> All the magic for adapting method handle to and from the specialized adapters happen in the DirectSignatureShuffler class; this class is responsible for adapting each argument e.g. from Java to native value, and then reordering the adapted method handle to match the order in which arguments are expected by the adapter (e.g. move all longs in front). The challenge was in having DirectSignatureShuffle to be fully symmetric - e.g. I did not want to have different code paths for upcalls and downcalls, so the code tries quite hard to be parametric in the shuffling direction (java->native or native->java) - which means that adapters will be applied in one way or in the inverse way depending on the shuffling direction (and as to whether we are adapting an argument or a return). Since method handle filters are composable, it all works out quite beautifully.
>>> 
>>> Note that the resulting, adapted MH is stored in a @Stable field to tell the JIT to optimize the heck out of it (as if it were a static constant).
>>> 
>>> This patch contains several other changes - which I discuss briefly below:
>>> 
>>> * we need to setup a framework in which new invocation strategies can be plugged in - note that we now have essentially 4 cases:
>>> 
>>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>> 
>>> When the code wants e.g. a NativeInvoker, it asks for one to the NativeInvoker::of factory (UpcallHandler work in a similar way); this factory will attempt to go down the fast path - if an error occurs when computing the fast path, the call will fallback to the universal (slow) path.
>>> 
>>> Most of the changes you see in the Java code are associated to this refactoring - e.g. all clients of NativeInvoker/UpcallHandler should now go through the factory
>>> 
>>> * CallbackImplGenerator had a major issue since the new factory for NativeInvoker wants to bind an address eagerly (this is required e.g. to be forward compatible with linkToNative backend); which means that at construction time we have to get the address of the callback, call the NativeInvoker factory and then stash the target method handle into a field of the anon callback class. Vlad tells me that fields of anon classes are always 'trusted' by the JIT, which means they should be treated as '@Stable' (note that I can't put a @Stable annotation there, since this code will be spinned in user-land).
>>> 
>>> * There are a bunch of properties that can be set to either force slow path or force 'direct' path; in the latter case, if an error occurs when instantiating the direct wrapper, an exception is thrown. This mode is very useful for testing, and I indeed have tried to run all our tests with this flag enabled, to see how many places could not be optimized.
>>> 
>>> * I've also reorganized all the native code in hotspot/prims so that we have a separate file for each scheme (and so that native Java methods could be added where they really belong). This should also help in the long run as it should make adding/removing a given scheme easier.
>>> 
>>> * I've also added a small test which tries to pass structs of different sizes, but I will also work on a more complex test which will stress-test all invocation modes in a more complete fashion. With respect to testing, I've also done a fastdebug build and ran all tests with that (as fastdebug catches way many more hotspot assertion than the product version); everything passed.
>>> 
>>> Webrev:
>>> 
>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>>> 
>>> I'd like to thank Vladimir Ivanov for the prompt support whenever I got stuck down the macro assembler rabbit hole :-)
>>> 
>>> Cheers
>>> Maurizio
>>> 
>>> [1] - http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>>> 
>>> 
>>> 
>> 
> 


From maurizio.cimadamore at oracle.com  Fri Sep 28 20:19:03 2018
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 28 Sep 2018 21:19:03 +0100
Subject: [foreign] RFR 8210757: Add binder support for direct native
 invocation strategy
In-Reply-To: <16DB46B0-D81A-480C-A64E-817F6D31F298@oracle.com>
References: <fc47a8d1-06b9-bd62-af2e-ac153486f6f2@oracle.com>
 <3e40cafe-df17-2569-8886-1b2142eb416c@oracle.com>
 <e35a0a85-591a-477a-13b1-4bbb5f44dbf1@oracle.com>
 <16DB46B0-D81A-480C-A64E-817F6D31F298@oracle.com>
Message-ID: <39f2ee95-9397-2377-3655-5493ed9b784e@oracle.com>



On 28/09/18 20:39, Henry Jen wrote:
> Looks good to me overall, although I only focus on java side and not dig into assembly.
>
> A really minor thing in the new RegisterStructTest.java, line 68 seems to be redundant?
>
>   68             checkEquals(rs.l$get(), rs.l$get());
Yep that line is bogus and probably result of cut and paste. Well 
spotted. I will fix that.

Maurizio
>
> The test passes on Mac.
>
> Cheers,
> Henry
>
>
>> On Sep 28, 2018, at 4:19 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>
>> Webrev:
>>
>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757_v2/
>>
>> Maurizio
>>
>>
>> On 28/09/18 12:19, Maurizio Cimadamore wrote:
>>> This is an updated version of the direct invocation scheme support. Very close to the last one, but there are some minor refactorings/improvements:
>>>
>>> 1) Added a @Stable annotation in DirectNativeInvoker's MH field
>>> 2) box/unbox routine used by the UniversalXYZ strategies have been moved from NativeInvoker to UniversalNativeInvoker
>>> 3) I revamped the logic which detects whether fastpath is applicable - now we create the calling sequence first, and we use that to check whether we can fast path it. Some internal benchmark have shown that with a large number of symbols, we were doing a lot of work because we were trying the fastpath always and then, in case of exception fallback to slow path; in such cases we would create calling sequence twice. This new technique might also be more friendly w.r.t. Windows and other ABIs.
>>>
>>> I'd really like to move ahead with this (as this RFR has been out for quite a while now) - if there's no other comments I'll go ahead.
>>>
>>> Maurizio
>>>
>>>
>>> On 14/09/18 19:04, Maurizio Cimadamore wrote:
>>>> Hi,
>>>> as mentioned in [1], this patch adds binder support for the so called 'direct' invocation scheme, which allows for greater native invocation downcall/upcall performances by means of specialized adapters. The core idea, also described in [1], is to define adapters of the kind:
>>>>
>>>> invokeNative_V_DDDDD
>>>> invokeNative_V_JDDDD
>>>> invokeNative_V_JJDDD
>>>> invokeNative_V_JJJDD
>>>> invokeNative_V_JJJJD
>>>> invokeNative_V_JJJJJ
>>>>
>>>> Where long arguments come before double arguments (and do this for each arity e.g. <=5).
>>>>
>>>> If all arguments are passed in register, then this reordering doesn't affect behavior, and greatly limits the number of permutations to be supported/generated.
>>>>
>>>> The downcall part (java to native) is relative straightforward: the directNativeInvoker.cpp file defines a bunch of native entry points, one per shape, which cast the input address to a function pointer of the desired shape, and then call it:
>>>>
>>>> jlong NI_invokeNative_J_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) {
>>>>      return ((jlong (*)(jlong, jdouble))addr)(arg0, arg1);
>>>> }
>>>>
>>>> The upcall business is a little trickier: first, if we are only to optimize upcalls where argument passing happens in registers, then it's crucial to note that by the time we get into the assembly stub, all the registers will have been populated by the native code to contain the right arguments in the right places. So we can avoid all the shuffling in the assembly adapter and simply jump onto a C function that looks like this:
>>>>
>>>> long specialized_upcall_helper_J(long l0, long l1, long l2, long l3,
>>>>                                        double d0, double d1, double d2, double d3,
>>>>                                         unsigned int mask, jobject rec) { ... }
>>>>
>>>> Note here that the first 8 arguments are just longs and doubles, and those will be expected to be in registers, according to the System V ABI. (In windows, the situation will be a bit different as less integer registers are available, so this will need some work there).
>>>>
>>>> So, to recap, the assembly upcall stub simply 'append' the receiver object and a 'signature mask' in the last two available C registers and then jump onto the helper function. The helper function will find all the desired arguments in the right places - there will be, in the general case, some unused arguments, but that's fine, after all it didn't cost anything to us to load them in the first place!
>>>>
>>>> Note that we have three helper variants, one for each return type { long, double, void }. This is required as we need the C helper to return a value of the right type which will generate the right assembly sequence to store the result in the right register (either integer or MMX).
>>>>
>>>> So, with three helpers we can support all the shapes with up to 8 arguments. On the Java side we have, of course, to define a specialized entry point for each shape.
>>>>
>>>> All the magic for adapting method handle to and from the specialized adapters happen in the DirectSignatureShuffler class; this class is responsible for adapting each argument e.g. from Java to native value, and then reordering the adapted method handle to match the order in which arguments are expected by the adapter (e.g. move all longs in front). The challenge was in having DirectSignatureShuffle to be fully symmetric - e.g. I did not want to have different code paths for upcalls and downcalls, so the code tries quite hard to be parametric in the shuffling direction (java->native or native->java) - which means that adapters will be applied in one way or in the inverse way depending on the shuffling direction (and as to whether we are adapting an argument or a return). Since method handle filters are composable, it all works out quite beautifully.
>>>>
>>>> Note that the resulting, adapted MH is stored in a @Stable field to tell the JIT to optimize the heck out of it (as if it were a static constant).
>>>>
>>>> This patch contains several other changes - which I discuss briefly below:
>>>>
>>>> * we need to setup a framework in which new invocation strategies can be plugged in - note that we now have essentially 4 cases:
>>>>
>>>> { NativeInvoker, UpcallHandler } x { Universal, Direct }
>>>>
>>>> When the code wants e.g. a NativeInvoker, it asks for one to the NativeInvoker::of factory (UpcallHandler work in a similar way); this factory will attempt to go down the fast path - if an error occurs when computing the fast path, the call will fallback to the universal (slow) path.
>>>>
>>>> Most of the changes you see in the Java code are associated to this refactoring - e.g. all clients of NativeInvoker/UpcallHandler should now go through the factory
>>>>
>>>> * CallbackImplGenerator had a major issue since the new factory for NativeInvoker wants to bind an address eagerly (this is required e.g. to be forward compatible with linkToNative backend); which means that at construction time we have to get the address of the callback, call the NativeInvoker factory and then stash the target method handle into a field of the anon callback class. Vlad tells me that fields of anon classes are always 'trusted' by the JIT, which means they should be treated as '@Stable' (note that I can't put a @Stable annotation there, since this code will be spinned in user-land).
>>>>
>>>> * There are a bunch of properties that can be set to either force slow path or force 'direct' path; in the latter case, if an error occurs when instantiating the direct wrapper, an exception is thrown. This mode is very useful for testing, and I indeed have tried to run all our tests with this flag enabled, to see how many places could not be optimized.
>>>>
>>>> * I've also reorganized all the native code in hotspot/prims so that we have a separate file for each scheme (and so that native Java methods could be added where they really belong). This should also help in the long run as it should make adding/removing a given scheme easier.
>>>>
>>>> * I've also added a small test which tries to pass structs of different sizes, but I will also work on a more complex test which will stress-test all invocation modes in a more complete fashion. With respect to testing, I've also done a fastdebug build and ran all tests with that (as fastdebug catches way many more hotspot assertion than the product version); everything passed.
>>>>
>>>> Webrev:
>>>>
>>>> http://cr.openjdk.java.net/~mcimadamore/panama/8210757/
>>>>
>>>> I'd like to thank Vladimir Ivanov for the prompt support whenever I got stuck down the macro assembler rabbit hole :-)
>>>>
>>>> Cheers
>>>> Maurizio
>>>>
>>>> [1] - http://mail.openjdk.java.net/pipermail/panama-dev/2018-September/002652.html
>>>>
>>>>
>>>>


From shravya.rukmannagari at intel.com  Fri Sep 28 22:58:28 2018
From: shravya.rukmannagari at intel.com (Rukmannagari, Shravya)
Date: Fri, 28 Sep 2018 22:58:28 +0000
Subject: VectorAPI shift and gather Generation Changes
Message-ID: <8D6F463991A1574A8A803B8DA605414F3A7F7357@ORSMSX111.amr.corp.intel.com>

Hi All,
I would like to contribute a patch that adds constant shift code generation support in Java for byte, short and removes Java code generation for gather(fromArray) using array indexing for byte and short.

Could you please review the patch here:

http://cr.openjdk.java.net/~srukmannagar/webrev_shift_Gather_Generation/



Thanks,
Shravya.