Issue in combination with JLink/JPackage

Maurizio Cimadamore maurizio.cimadamore at oracle.com
Thu Jun 9 17:00:28 UTC 2022 

On 09/06/2022 16:46, Clemens Lanthaler wrote:
> Thanks for the feedback. I must try now with a newer Jextract version 
> because I have used the latest panama JDK19 EA release version for 
> both libs. Curious is that one is working and the other not because 
> the code is completely the same for both.

That is indeed very odd - the jextract in the EA still uses the 
incubator packages, so there is zero chance it will work against a 
current Java 19 version. Probably something else is going on - and you 
end up running with 18, not 19.

Maurizio

>
>
>
> Am 08.06.22 um 22:20 schrieb Maurizio Cimadamore:
>>
>> There seems to be a combination of issues here (which seem to have 
>> little to do with library loading).
>>
>> In the "only-libraw-working" log, the failure has to do with code 
>> trying to call the SegmentAllocator.nativeAllocator method, that is 
>> no longer there (now MemorySession implements SegmentAllocator directly).
>>
>> In the "nothing-runs" log, I see references to "CLinker" - the class 
>> has been renamed to "Linker".
>>
>> It seems to me that the version of jextract you are using to work 
>> with 19 is not correct.
>>
>> You can get build a version of jextract that works with 19 here:
>>
>> https://github.com/openjdk/jextract
>>
>> (for now it has to be built manually, binaries will be made available 
>> soon - but the process is a lot simpler than building a full JDK)
>>
>> Hope this helps
>>
>> Maurizio
>>
>>
>> On 08/06/2022 19:37, Clemens Lanthaler wrote:
>>
>>> Hello Maurizio,
>>>
>>> here are the logs with all kind of setups. The name of the logfile 
>>> tells what I have tried. But basically libhheiffx is not working 
>>> also under JDK19. Librawfx is working if compiled for jdk19 or 
>>> jdk18.0.1.
>>>
>>> What I have seen is that the JVM is no longer been crashing and that 
>>> the jpackage image produces the same issues than if you run it via 
>>> maven/IDE.
>>>
>>> Hopfully that helps you.
>>>
>>> best regards,
>>> Clemens
>>>
>>>
>>> Am 05.06.22 um 23:29 schrieb Maurizio Cimadamore:
>>>>
>>>>
>>>> On 05/06/2022 11:28, Clemens Lanthaler wrote:
>>>>> Hello Maurizio,
>>>>>
>>>>> I have now updated all my panama project with the Panama Build 
>>>>> 19-panama+1-13. Now my LibrawFX library is working as expected, 
>>>>> but my second lib LibHeifFX is not working and shows the same 
>>>>> error message than before.
>>>>>
>>>>> The difference between the two native libs is that LibrawFX is 
>>>>> thread safe and LibheifFX is not. Maybe the reason for only 
>>>>> showing the issue on Libheif could be also the loading order.
>>>>>
>>>>> From the issue description the fix is already included (build 11) 
>>>>> but it seems that there is another issue behind it.
>>>>>
>>>>> What I did:
>>>>> - Generate with jextract from Panama Build 19-panama+1-13
>>>>> - Run with JDK 18.0.1
>>>>>
>>>>> Must I run it with JDK 19 to be working ? But if that is the case 
>>>>> than LibrawFX should not be working as well. Give me feedback if 
>>>>> you need any logs additionally.
>>>> Hi Clemens,
>>>>
>>>> I believe you need to run with 19 to take advantage of the fix - 
>>>> e.g. to make sure that System.loadLibrary and Panama library 
>>>> loading truly act as orthogonal.
>>>>
>>>> Let's start from there and see where we get - please post any stack 
>>>> trace that could be relevant to diagnose this.
>>>>
>>>> Thanks
>>>> Maurizio
>>>>
>>>>>
>>>>> best regards,
>>>>> Clemens
>>>>>
>>>>>
>>>>> Am 18.02.22 um 23:16 schrieb Maurizio Cimadamore:
>>>>>>
>>>>>> Hi Clemens,
>>>>>> no worries - I will let you know when we have an EA with the fix 
>>>>>> available (we'll probably wait to accumulate some other 
>>>>>> fixes/features as well).
>>>>>>
>>>>>> Thanks
>>>>>> Maurizio
>>>>>>
>>>>>> On 18/02/2022 22:04, Clemens Lanthaler wrote:
>>>>>>> Hi,
>>>>>>> Thanks allot for the info. I have seen that the fix will be part 
>>>>>>> of JDK19 and frankly speeking I have never build a jdk by 
>>>>>>> myself. I would appreciate if there is a chance to get a binary 
>>>>>>> of this pre build including panama.
>>>>>>>
>>>>>>> Thank you in advance.
>>>>>>>
>>>>>>> best regards,
>>>>>>> Clemens
>>>>>>>
>>>>>>>> Maurizio Cimadamore <maurizio.cimadamore at oracle.com> hat am 
>>>>>>>> 18.02.2022 15:30 geschrieben:
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>> quick update on this. The problem with library loading in JDK 
>>>>>>>> being too
>>>>>>>> strict for Panama has been fixed upstream (thanks Mandy!):
>>>>>>>>
>>>>>>>> https://git.openjdk.java.net/jdk/pull/7435
>>>>>>>>
>>>>>>>>
>>>>>>>> We've just merged those changes into the Panama repo - which 
>>>>>>>> means that,
>>>>>>>> if you build the latest version of the repo, you should be able to
>>>>>>>> verify that the workarounds are no longer required.
>>>>>>>>
>>>>>>>> Please let us know how that goes.
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>> On 10/02/2022 10:05, Maurizio Cimadamore wrote:
>>>>>>>>> I have managed to create a simple reproducer on Windows:
>>>>>>>>> import jdk.incubator.foreign.CLinker;
>>>>>>>>> public class TestLookup {
>>>>>>>>>     public static void main(String[] args) {
>>>>>>>>> System.load("C:\\Windows\\System32\\ucrtbase.dll");
>>>>>>>>> CLinker.systemCLinker().lookup("foo");
>>>>>>>>>     }
>>>>>>>>> }
>>>>>>>>> The call to CLinker::lookup fails with UnsatisfiedLinkError as
>>>>>>>>> ucrtbase is already loaded by the application class loader.
>>>>>>>>> This seems to be a bug in the library loading mechanism - we 
>>>>>>>>> do have a
>>>>>>>>> "raw" library loading mechanism that is not subject to JNI 
>>>>>>>>> restrictions:
>>>>>>>>> https://github.com/openjdk/jdk/blob/309acbf0e86a0d248294503fccc7a936fa0a846e/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java#L99 
>>>>>>>>> <https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/309acbf0e86a0d248294503fccc7a936fa0a846e/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java*L99__;Iw!!ACWV5N9M2RV99hQ!bOgtuoBaY3YrzwP16VXj_WbRpiIFGIzLTsiROaXth333Wb6mUx9EzcQleW7Q-4C06fylqx0$> 
>>>>>>>>>
>>>>>>>> >
>>>>>>>>> This library loader ends up calling this method which contains 
>>>>>>>>> a class
>>>>>>>>> loader check:
>>>>>>>>> https://github.com/openjdk/jdk/blob/309acbf0e86a0d248294503fccc7a936fa0a846e/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java#L203 
>>>>>>>>> <https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/309acbf0e86a0d248294503fccc7a936fa0a846e/src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java*L203__;Iw!!ACWV5N9M2RV99hQ!bOgtuoBaY3YrzwP16VXj_WbRpiIFGIzLTsiROaXth333Wb6mUx9EzcQleW7Q-4C0tN6Z3Vs$> 
>>>>>>>>>
>>>>>>>> >
>>>>>>>>> I believe this loader check should be guarded by a "isJNI" 
>>>>>>>>> flag, as we
>>>>>>>>> did for other checks - e.g. whether JNI_OnLoad should be called.
>>>>>>>>> Cheers
>>>>>>>>> Maurizio
>>>>>>>>> On 07/02/2022 12:19, Maurizio Cimadamore wrote:
>>>>>>>>>> No problem.
>>>>>>>> >> I've filed this:
>>>>>>>> >>
>>>>>>>> >> https://bugs.openjdk.java.net/browse/JDK-8281335
>>>>>>>> >>
>>>>>>>> >> Cheers
>>>>>>>> >> Maurizio
>>>>>>>> >>
>>>>>>>> >> On 07/02/2022 12:15, clemens.lanthaler at itarchitects.at wrote:
>>>>>>>> >>> Hi Maurizio,
>>>>>>>> >>>
>>>>>>>> >>> thanks allot for your help and for the investigation.
>>>>>>>> >>>
>>>>>>>> >>> best regards,
>>>>>>>> >>> Clemens
>>>>>>>> >>>
>>>>>>>> >>>
>>>>>>>> >>>> On 7. Feb 2022, at 13:03, Maurizio Cimadamore
>>>>>>>> >>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>>> >>>>
>>>>>>>> >>>>
>>>>>>>> >>>>
>>>>>>>> >>>> Hi Clemens (it seems like your message got dropped by the 
>>>>>>>> mailing
>>>>>>>> >>>> list, not sure why).
>>>>>>>> >>>>
>>>>>>>> >>>> Your message seems to point at some bad interaction between
>>>>>>>> >>>> jpackage and Panama both trying to load ucrtbase.dll (and 
>>>>>>>> only one
>>>>>>>> >>>> can win). Apparently, the Panama code isn't executed early 
>>>>>>>> enough
>>>>>>>> >>>> in the jpackage case.
>>>>>>>> >>>>
>>>>>>>> >>>> We'll need to investigate more.
>>>>>>>> >>>>
>>>>>>>> >>>> Maurizio
>>>>>>>> >>>>
>>>>>>>> >>>> On 05/02/2022 15:31, Clemens Lanthaler wrote:
>>>>>>>> >>>>> Hello everybody,
>>>>>>>> >>>>>
>>>>>>>> >>>>> I am the developer of project librawfx
>>>>>>>> >>>>> (github.com/lanthale/librawfx) and the app Photoslide
>>>>>>>> >>>>> (github.com/lanthale/photoslide).
>>>>>>>> >>>>>
>>>>>>>> >>>>> At the begining of Photoslide the whole app was 
>>>>>>>> modularized and
>>>>>>>> >>>>> therefore all was working including librawfx which is using
>>>>>>>> >>>>> project Panama. After a while I had to change to a mixed
>>>>>>>> >>>>> (modulepath for jdk+javafx libs and classpath for the rest)
>>>>>>>> >>>>> environment and deployed my app PhotoSlide again. Under 
>>>>>>>> Linux and
>>>>>>>> >>>>> OSX all is working as expected. The problem exists only 
>>>>>>>> under
>>>>>>>> >>>>> Windows. Furthermore it exists only if I am creating a
>>>>>>>> >>>>> Jlink/Jpackage distribution. Running the app from maven was
>>>>>>>> >>>>> working as expected. Only the JPackage launcher have the 
>>>>>>>> issue and
>>>>>>>> >>>>> I have no clue what the root cause is.
>>>>>>>> >>>>>
>>>>>>>> >>>>> The exception I am always seeing is:
>>>>>>>> >>>>> Feb. 03, 2022 7:44:58 PM org.librawfx.RAWImageLoader load
>>>>>>>> >>>>> SCHWERWIEGEND: null
>>>>>>>> >>>>> java.lang.UnsatisfiedLinkError: Native Library
>>>>>>>> >>>>> C:\Windows\System32\ucrtbase.dll already loaded in another
>>>>>>>> >>>>> classloader
>>>>>>>> >>>>> (Full stacktrace below)
>>>>>>>> >>>>>
>>>>>>>> >>>>> How can you reproduce the issue:
>>>>>>>> >>>>>
>>>>>>>> >>>>>  1. Git clone project 
>>>>>>>> https://github.com/lanthale/JeditFX.git 
>>>>>>>> <https://urldefense.com/v3/__https://github.com/lanthale/JeditFX.git__;!!ACWV5N9M2RV99hQ!bOgtuoBaY3YrzwP16VXj_WbRpiIFGIzLTsiROaXth333Wb6mUx9EzcQleW7Q-4C0nxVq_C0$> 
>>>>>>>>
>>>>>>>> >>>>>  2. Start the app with "mvn javafx:run at default-cli"
>>>>>>>> >>>>>  3. Click on the open icon in the toolbar and select file
>>>>>>>> >>>>> 
>>>>>>>> "...\Documents\NetBeansProjects\JeditFX\JeditFX\src\main\resources\RAW-ADOBE_DNG_Sample.dng" 
>>>>>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>>  4. The app is showing the image below the textarea
>>>>>>>> >>>>>  5. Create the jlink/jpackage app-image with "mvn 
>>>>>>>> -Ppackage clean
>>>>>>>> >>>>>     install"
>>>>>>>> >>>>>  6. Start the exe jeditfx.exe in directory 
>>>>>>>> ".\target\jeditfx"
>>>>>>>> >>>>>  7. Open the same file again -> No image is shown below 
>>>>>>>> the text
>>>>>>>> >>>>>     area and the exception is thrown in the background
>>>>>>>> >>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>> I have tried many things but it is only happening if I have
>>>>>>>> >>>>> OpenJFX+JDK Modules on the module path and the rest of 
>>>>>>>> the jar
>>>>>>>> >>>>> files on the classpath. All libs which are not having any 
>>>>>>>> Panama
>>>>>>>> >>>>> code inside of the app are working perfectly and it is only
>>>>>>>> >>>>> happening on Windows.
>>>>>>>> >>>>>
>>>>>>>> >>>>> Thank you in advance for your help!
>>>>>>>> >>>>>
>>>>>>>> >>>>> best regards,
>>>>>>>> >>>>> Clemens
>>>>>>>> >>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>> *Full stacktrace from Photoslide:*
>>>>>>>> >>>>>
>>>>>>>> >>>>> Feb. 03, 2022 7:44:41 PM org.photoslide.search.SearchIndex
>>>>>>>> >>>>> createSearchIndex
>>>>>>>> >>>>> INFORMATION: Start time create searchDB: 
>>>>>>>> 2022-02-03T19:44:41.693455
>>>>>>>> >>>>> Feb. 03, 2022 7:44:42 PM org.photoslide.SoftwareUpdater
>>>>>>>> >>>>> lambda$checkForSoftwareUpdates$2
>>>>>>>> >>>>> INFORMATION: No new version found!
>>>>>>>> >>>>> Feb. 03, 2022 7:44:43 PM org.photoslide.search.SearchIndex
>>>>>>>> >>>>> lambda$createSearchIndex$0
>>>>>>>> >>>>> INFORMATION: End time create searchDB: 
>>>>>>>> 2022-02-03T19:44:43.186194400
>>>>>>>> >>>>> Feb. 03, 2022 7:44:58 PM org.librawfx.RAWImageLoader load
>>>>>>>> >>>>> SCHWERWIEGEND: null
>>>>>>>> >>>>> java.lang.UnsatisfiedLinkError: Native Library
>>>>>>>> >>>>> C:\Windows\System32\ucrtbase.dll already loaded in another
>>>>>>>> >>>>> classloader
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> java.base/jdk.internal.loader.NativeLibraries.loadLibrary(Unknown
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> java.base/jdk.internal.loader.NativeLibraries.loadLibrary(Unknown
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> jdk.incubator.foreign/jdk.internal.foreign.SystemLookup.lambda$makeWindowsLookup$1(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> jdk.incubator.foreign/jdk.internal.foreign.SystemLookup.libLookup(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> jdk.incubator.foreign/jdk.internal.foreign.SystemLookup.makeWindowsLookup(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> jdk.incubator.foreign/jdk.internal.foreign.SystemLookup.(Unknown
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> jdk.incubator.foreign/jdk.incubator.foreign.CLinker.systemLookup(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at 
>>>>>>>> org.libraw.win.RuntimeHelper.lookup(RuntimeHelper.java:33)
>>>>>>>> >>>>> at org.libraw.win.libraw_h.(libraw_h.java:15)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> org.librawfx.LibrawImage.readPixelDataFromStream(LibrawImage.java:113) 
>>>>>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>> at 
>>>>>>>> org.librawfx.RAWImageLoader.getImageData(RAWImageLoader.java:168)
>>>>>>>> >>>>> at org.librawfx.RAWImageLoader.load(RAWImageLoader.java:84)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/com.sun.javafx.iio.ImageStorage.loadAll(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/com.sun.javafx.iio.ImageStorage.loadAll(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/com.sun.javafx.tk.quantum.PrismImageLoader2.loadAll(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/com.sun.javafx.tk.quantum.PrismImageLoader2.(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/com.sun.javafx.tk.quantum.QuantumToolkit.loadImage(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/javafx.scene.image.Image.loadImage(Unknown
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/javafx.scene.image.Image.initialize(Unknown
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at 
>>>>>>>> javafx.graphics at 18-ea/javafx.scene.image.Image.(Unknown Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> org.photoslide.datamodel.MediaFileLoader$1.call(MediaFileLoader.java:46) 
>>>>>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> org.photoslide.datamodel.MediaFileLoader$1.call(MediaFileLoader.java:43) 
>>>>>>>>
>>>>>>>> >>>>>
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> javafx.graphics at 18-ea/javafx.concurrent.Task$TaskCallable.call(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at java.base/java.util.concurrent.FutureTask.run(Unknown 
>>>>>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at java.base/java.util.concurrent.FutureTask.run(Unknown 
>>>>>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at
>>>>>>>> >>>>> 
>>>>>>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown 
>>>>>>>>
>>>>>>>> >>>>> Source)
>>>>>>>> >>>>> at java.base/java.lang.Thread.run(Unknown Source)
>>>>>
>>>>> -- 
>>>>> ITArchitects
>>>>> CEO: B.Sc. Clemens Lanthaler
>>>>> Forchachstrasse 3
>>>>> A-6166 Fulpmes
>>>>> Tel.: +43 (0)650 855 2954
>>>>> email:office at itarchitects.at
>>>>> homepage:http://www.itarchitects.at
>>>>> -------------------------------------------------
>>>>> Notice: This e-mail and any attachments are confidential and may be privileged.
>>>>> If you are not the intended recipient, notify the sender immediately, destroy all
>>>>> copies from your system and do not disclose or use the information for any purpose.
>>>>> Diese E-Mail inklusive aller Anhaenge ist vertraulich und koennte bevorrechtigtem
>>>>> Schutz unterliegen. Wenn Sie nicht der beabsichtigte Adressat sind, informieren Sie
>>>>> bitte den Absender unverzueglich, loeschen Sie alle Kopien von Ihrem System und
>>>>> veroeffentlichen Sie oder nutzen Sie die Information keinesfalls, gleich zu welchem Zweck.
>>>
>>> -- 
>>> ITArchitects
>>> CEO: B.Sc. Clemens Lanthaler
>>> Forchachstrasse 3
>>> A-6166 Fulpmes
>>> Tel.: +43 (0)650 855 2954
>>> email:office at itarchitects.at
>>> homepage:http://www.itarchitects.at
>>> -------------------------------------------------
>>> Notice: This e-mail and any attachments are confidential and may be privileged.
>>> If you are not the intended recipient, notify the sender immediately, destroy all
>>> copies from your system and do not disclose or use the information for any purpose.
>>> Diese E-Mail inklusive aller Anhaenge ist vertraulich und koennte bevorrechtigtem
>>> Schutz unterliegen. Wenn Sie nicht der beabsichtigte Adressat sind, informieren Sie
>>> bitte den Absender unverzueglich, loeschen Sie alle Kopien von Ihrem System und
>>> veroeffentlichen Sie oder nutzen Sie die Information keinesfalls, gleich zu welchem Zweck.
>
> -- 
> ITArchitects
> CEO: B.Sc. Clemens Lanthaler
> Forchachstrasse 3
> A-6166 Fulpmes
> Tel.: +43 (0)650 855 2954
> email:office at itarchitects.at
> homepage:http://www.itarchitects.at
> -------------------------------------------------
> Notice: This e-mail and any attachments are confidential and may be privileged.
> If you are not the intended recipient, notify the sender immediately, destroy all
> copies from your system and do not disclose or use the information for any purpose.
> Diese E-Mail inklusive aller Anhaenge ist vertraulich und koennte bevorrechtigtem
> Schutz unterliegen. Wenn Sie nicht der beabsichtigte Adressat sind, informieren Sie
> bitte den Absender unverzueglich, loeschen Sie alle Kopien von Ihrem System und
> veroeffentlichen Sie oder nutzen Sie die Information keinesfalls, gleich zu welchem Zweck.

More information about the panama-dev mailing list