From ooo_saturn7 at mail.ru Fri Nov 5 16:42:55 2021 From: ooo_saturn7 at mail.ru (=?UTF-8?B?QWxleCBPcmxvdg==?=) Date: Fri, 05 Nov 2021 19:42:55 +0300 Subject: =?UTF-8?B?0J/RgNC40LLQtdGCIDM=?= Message-ID: <1636130575.19532561@f151.i.mail.ru> ? ? -- Best regards, Alex Orlov -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: temp.txt URL: From jai.forums2013 at gmail.com Sat Nov 6 04:18:21 2021 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Sat, 6 Nov 2021 09:48:21 +0530 Subject: jlink SystemModulesPlugin use of hashCode breaks reproducibility In-Reply-To: Message-ID: <01eeb954-6bc9-3741-a5dd-82783ac50c41@gmail.com> This issue has now been fixed through https://bugs.openjdk.java.net/browse/JDK-8275509 -Jaikiran From michal at kleczek.org Mon Nov 22 07:23:11 2021 From: michal at kleczek.org (=?utf-8?Q?Micha=C5=82_K=C5=82eczek?=) Date: Mon, 22 Nov 2021 08:23:11 +0100 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> Message-ID: <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> From OpenJDK discuss: > On 21 Nov 2021, at 21:55, Alan Bateman wrote: > > On 21/11/2021 13:27, Micha? K?eczek wrote: >> Hello All, >> >> I am looking for a way to associate ModuleLayer.Controller with a Module (or ModuleLayer) in such a way that it won?t disable garbage collection of Modules and ModuleLayers. >> WeakHashMap won?t work as Controller strongly references its ModuleLayer (which in turn strongly references Modules). >> WeakHashMap> won?t work either cause controller will be garbage collected. >> >> So far I only came up with subclassing a ModuleReference and keeping the reference to the controller there. This won?t work in case when ModuleLayer.configuration() is used a a parent configuration. >> >> Am I missing something obvious here? > You are correct that a module layer doesn't keep the ML.Controller alive. A ML.Controller is somewhat niche object to expand the readability graph or maybe add additional edges to open modules, and then be thrown away. It sounds like you want to keep it around in order to do further opening when additional child layers come into being. One way you could do this is create a module with a holder class to keep the reference. So rather than keeping the reference in the model world (ModuleReference), instead put it as a field in a reified module. With the right encapsulation it should give you the module layer -> Controller without compromising the integrity of the layers. If you want to discuss further then it's probably best to follow-up on jigsaw-dev rather here. > Indeed - a synthetic module providing a holder service might do the job. Seems somewhat cumbersome though. Would it be too much to ask for accepting a patch adding a reference to ML.Controller to ModuleLayer? It wouldn?t require changes to the spec I think as the lifetime of ML.Controller is unspecified. ? Michal From Alan.Bateman at oracle.com Mon Nov 22 20:37:39 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Nov 2021 20:37:39 +0000 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> Message-ID: <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> On 22/11/2021 07:23, Micha? K?eczek wrote: > > Would it be too much to ask for accepting a patch adding a reference > to ML.Controller to ModuleLayer? > It wouldn?t require changes to the spec I think as the lifetime of > ML.Controller is unspecified. > Assuming it's not exposed, and that there is no reference to the ML.Controller for the boot layer, then it might be okay but I think it would be useful to understand the use-case first. As I assume you know, the ML.Controller allows containers to do the equivalent of --add-reads, --add-exports, and --add-opens in child layers.? Since you want to retain the reference to the ML.Controller around then it may be some other reason, maybe opening packages to descendants beyond the child layer? -Alan From michal at kleczek.org Tue Nov 23 19:04:53 2021 From: michal at kleczek.org (=?utf-8?Q?Micha=C5=82_K=C5=82eczek?=) Date: Tue, 23 Nov 2021 20:04:53 +0100 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> Message-ID: Hi Alan, > On 22 Nov 2021, at 21:37, Alan Bateman wrote: > > On 22/11/2021 07:23, Micha? K?eczek wrote: >> >> Would it be too much to ask for accepting a patch adding a reference to ML.Controller to ModuleLayer? >> It wouldn?t require changes to the spec I think as the lifetime of ML.Controller is unspecified. >> > Assuming it's not exposed, and that there is no reference to the ML.Controller for the boot layer, then it might be okay but I think it would be useful to understand the use-case first. As I assume you know, the ML.Controller allows containers to do the equivalent of --add-reads, --add-exports, and --add-opens in child layers. Since you want to retain the reference to the ML.Controller around then it may be some other reason, maybe opening packages to descendants beyond the child layer? Yes. As a hobby project I am revisiting my old idea briefly described here (it?s been sooo long ago :) ): https://markmail.org/message/kte6z7kcgaqcayor In short: it is Jini/RMI with codebase annotations being ?smart? installers of code necessary to deserialise an object. At that time it wasn?t really feasible to transfer arbitrary objects graphs due to hierarchical class loading model. The general idea is that upon serialisation of an object its class is annotated with another object representing the closure of this class? module and its dependencies (subgraph of the application module graph). To annotate the annotation object I use null (actually the recursion level allowed when reading annotations is configurable). When deserialising I recreate the module graph on the receiving side. Some nodes in the dependency graph are replaced with local modules so that the receiving side can cast deserialised object to a known type. Equal annotations (as defined by Object.equals) - even received from different sources - always map to the same module on the receiving side. In non-modular pre-jigsaw world two jar files (server.jar and server-dl.jar) were provided: server-dl.jar contained a subset of server.jar classes implementing Smart proxy. In post-jigsaw world Smart proxy and Server reside in different modules. The (canonical) structure of a server application is: Server module --- depends on ---> Smart proxy module ----> depdends on ---> one or more API modules To make things more encapsulated Proxy module exports its smart proxy class package only to the Server module (Smart proxy class has to be public in contrast to pre-jigsaw world because split packages are disallowed). What I would like to support is also (rare) scenario that a superset of already instantiated module graph is received and one of the previously downloaded modules exports a package to a named module downloaded at a later time. (Imagine scenario when a server can be migrated and is transferred after its smart proxy). That?s why I need to keep ML.Controller for later. Hope that makes it clear :) -- Michal From michal at kleczek.org Wed Nov 24 12:15:56 2021 From: michal at kleczek.org (=?utf-8?Q?Micha=C5=82_K=C5=82eczek?=) Date: Wed, 24 Nov 2021 13:15:56 +0100 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> Message-ID: <1E3D9F4C-BDBB-46C9-968A-474B4964A0E2@kleczek.org> Hi Alan, Attached is a first draft of the patch. Controller is created upon ML construction and returned from static ML.defineModules methods. > On 22 Nov 2021, at 21:37, Alan Bateman wrote: >> > Assuming it's not exposed, and that there is no reference to the ML.Controller for the boot layer, I am not sure I understand this. Do you mean ML.controller should be null for the boot layer? This would require null check in static ML.defineModules methods to return new Controller for the boot layer (done only once during system init). -- Michal -------------- next part -------------- From michal at kleczek.org Wed Nov 24 12:19:08 2021 From: michal at kleczek.org (=?utf-8?Q?Micha=C5=82_K=C5=82eczek?=) Date: Wed, 24 Nov 2021 13:19:08 +0100 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> Message-ID: <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> Hi Alan, (Disregard previous message - fat fingers) Attached is a first draft of the patch: 1. Controller is created upon ML construction and returned from static ML.defineModules methods. 2. Created two controller liveness tests (they rely on System.gc() though - not sure if there are some tricks to test it reliably). > On 22 Nov 2021, at 21:37, Alan Bateman > wrote: >> > Assuming it's not exposed, and that there is no reference to the ML.Controller for the boot layer, I am not sure I understand this. Do you mean ML.controller should be null for the boot layer? This would require null check in static ML.defineModules methods to return new Controller for the boot layer (done only once during system init). -- Michal From Alan.Bateman at oracle.com Wed Nov 24 12:33:24 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 24 Nov 2021 12:33:24 +0000 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> Message-ID: <1c35091c-473f-1661-20d1-9ecef9c9ecf4@oracle.com> On 24/11/2021 12:19, Micha? K?eczek wrote: > Hi Alan, > > (Disregard previous message - fat fingers) > > Attached is a first draft of the patch: > > 1. Controller is created upon ML construction and returned from static > ML.defineModules methods. > 2. Created two controller liveness tests (they rely on System.gc() > though - not sure if there are some tricks to test it reliably). I read your mail about "smart proxy" classes and lazy downloading of modules. I don't have an opinion at this time on whether this is worth doing or not. If you want to experiment with patches then add a volatile field to ML and only set it when VM.isBooted() is true. In order words, let the ML.Controller for the boot layer be GC'ed as it is now (it would be attractive nuisance to keep it alive). There are many tests in the test/jdk tree that check that a weak ref is cleared, there is also some test infrastructure to help, so you might want to look at those. -Alan From michal at kleczek.org Wed Nov 24 18:14:13 2021 From: michal at kleczek.org (=?utf-8?Q?Micha=C5=82_K=C5=82eczek?=) Date: Wed, 24 Nov 2021 19:14:13 +0100 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <1c35091c-473f-1661-20d1-9ecef9c9ecf4@oracle.com> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> <1c35091c-473f-1661-20d1-9ecef9c9ecf4@oracle.com> Message-ID: <0068A81F-9179-4E8A-AC68-39391C26FCED@kleczek.org> > On 24 Nov 2021, at 13:33, Alan Bateman wrote: >> >> 1. Controller is created upon ML construction and returned from static ML.defineModules methods. >> 2. Created two controller liveness tests (they rely on System.gc() though - not sure if there are some tricks to test it reliably). > > I read your mail about "smart proxy" classes and lazy downloading of modules. I don't have an opinion at this time on whether this is worth doing or not. For sure it?s fun :) Regardless of ?smart proxies? the lifetime of ML.Controller should be - if not fixed - at least be documented IMHO. Right now it?s a minefield. > > If you want to experiment with patches then add a volatile field to ML and only set it when VM.isBooted() is true. In order words, let the ML.Controller for the boot layer be GC'ed as it is now (it would be attractive nuisance to keep it alive). There are many tests in the test/jdk tree that check that a weak ref is cleared, there is also some test infrastructure to help, so you might want to look at those. See attached patch. I am now curious why leaving controller for boot layer is to be avoided. -- Michal From Alan.Bateman at oracle.com Wed Nov 24 20:01:35 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 24 Nov 2021 20:01:35 +0000 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <0068A81F-9179-4E8A-AC68-39391C26FCED@kleczek.org> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> <1c35091c-473f-1661-20d1-9ecef9c9ecf4@oracle.com> <0068A81F-9179-4E8A-AC68-39391C26FCED@kleczek.org> Message-ID: On 24/11/2021 18:14, Micha? K?eczek wrote: > > > For sure it?s fun :) > > Regardless of ?smart proxies? the lifetime of ML.Controller should be > - if not fixed - at least be documented IMHO. Right now it?s a minefield. It's also somewhat unusual so I think we'll need to think if this is the right thing to do. The ML.Controller for the boot layer is not needed and I think it would be attractive nuisance to have it hanging around. -Alan From michal at kleczek.org Sun Nov 28 22:29:36 2021 From: michal at kleczek.org (=?utf-8?Q?Micha=C5=82_K=C5=82eczek?=) Date: Sun, 28 Nov 2021 23:29:36 +0100 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> <1c35091c-473f-1661-20d1-9ecef9c9ecf4@oracle.com> <0068A81F-9179-4E8A-AC68-39391C26FCED@kleczek.org> Message-ID: <3EAAFF5D-0C13-4EE0-A7C3-FC807828CED3@kleczek.org> > On 24 Nov 2021, at 21:01, Alan Bateman wrote: > > On 24/11/2021 18:14, Micha? K?eczek wrote: >> >> >> For sure it?s fun :) >> >> Regardless of ?smart proxies? the lifetime of ML.Controller should be - if not fixed - at least be documented IMHO. Right now it?s a minefield. > > It's also somewhat unusual so I think we'll need to think if this is the right thing to do. It looks like ML.Controller.addExport(source, pn, target) does not update the target module ClassLoader view of exported packages in the default ClassLoader implementation (jdk.internal.loader.Loader). What is the reasoning behind such implementation? I am wondering now how useful Controller.addExport() really is... -- Michal From Alan.Bateman at oracle.com Mon Nov 29 09:29:26 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 29 Nov 2021 09:29:26 +0000 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <3EAAFF5D-0C13-4EE0-A7C3-FC807828CED3@kleczek.org> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> <1c35091c-473f-1661-20d1-9ecef9c9ecf4@oracle.com> <0068A81F-9179-4E8A-AC68-39391C26FCED@kleczek.org> <3EAAFF5D-0C13-4EE0-A7C3-FC807828CED3@kleczek.org> Message-ID: <9dbeb58b-1be1-4afd-8748-5c2db0403a90@oracle.com> On 28/11/2021 22:29, Micha? K?eczek wrote: > : > It looks like ML.Controller.addExport(source, pn, target) does not update the target module ClassLoader view of exported packages in the default ClassLoader implementation (jdk.internal.loader.Loader). > What is the reasoning behind such implementation? I am wondering now how useful Controller.addExport() really is... > The addXXX methods are about accessibility. In the case of addExports then maybe you want to export an internal package so that code in a module in the child layer can make use of (via reflection) public classes in that package. You are right that the addXXX methods doesn't do any magic for visibility. For super advanced cases where someone is using their own class loaders then they need to ensure that the class loader delegation supports the readability graph. Same thing when they use addExports where they may need to their class loaders to adjust their delegation to support changes at run-time. -Alan From michal at kleczek.org Mon Nov 29 15:57:12 2021 From: michal at kleczek.org (=?utf-8?Q?Micha=C5=82_K=C5=82eczek?=) Date: Mon, 29 Nov 2021 16:57:12 +0100 Subject: ModuleLayer.Controller strongly references Module In-Reply-To: <9dbeb58b-1be1-4afd-8748-5c2db0403a90@oracle.com> References: <9136A4EB-3B1B-4126-8B83-94DE52134D42@kleczek.org> <8af64d7d-4602-9c50-c08c-0ce0ae1c78d6@oracle.com> <3D2F0DC8-309E-46AD-9905-F45699A9CD10@kleczek.org> <3778a45e-6d67-5e88-c01c-ce098fec8a0e@oracle.com> <8A60FD9C-AD2D-4CBF-9BAF-63FB50AE12A9@kleczek.org> <1c35091c-473f-1661-20d1-9ecef9c9ecf4@oracle.com> <0068A81F-9179-4E8A-AC68-39391C26FCED@kleczek.org> <3EAAFF5D-0C13-4EE0-A7C3-FC807828CED3@kleczek.org> <9dbeb58b-1be1-4afd-8748-5c2db0403a90@oracle.com> Message-ID: <41D484C6-3447-44C3-82ED-2BF60A4DBA4E@kleczek.org> > On 29 Nov 2021, at 10:29, Alan Bateman wrote: >> >> It looks like ML.Controller.addExport(source, pn, target) does not update the target module ClassLoader view of exported packages in the default ClassLoader implementation (jdk.internal.loader.Loader). >> What is the reasoning behind such implementation? I am wondering now how useful Controller.addExport() really is... >> > The addXXX methods are about accessibility. In the case of addExports then maybe you want to export an internal package so that code in a module in the child layer can make use of (via reflection) public classes in that package. > > You are right that the addXXX methods doesn't do any magic for visibility. For super advanced cases where someone is using their own class loaders then they need to ensure that the class loader delegation supports the readability graph. Same thing when they use addExports where they may need to their class loaders to adjust their delegation to support changes at run-time. Thanks for explanation. Unfortunately, I couldn?t find any good in-depth documentation covering Module, ModuleLayer and ClassLoader interaction. Could you point me to anything useful? In my case I wanted to avoid implementing my own custom ClassLoader but actually doing that allows for easier management of ML.Controller lifetime: the ClassLoader can keep the reference to it. In the end I ended up with one-to-one relationship between ML, Module and custom ClassLoader that form a cycle that can be GCed. Thanks for help. -- Michal