<Swing Dev> JDK-8189938: Updated analysis

Wed Feb 21 21:54:24 UTC 2018

Hi again,

I debugged the issue a bit further and I think I tracked it down into
the visual c runtime. What got me curious, was that in any case MTA-COM 
initialization had already happend even when a new java thread was
created. So I modified the java launcher (jvm.dll) and inserted
debugging statements into some of the native methods in os_windows.cpp.
I focused on the threading methods. What I found was this:

- create_main_thread, create_os_thread, pd_start_thread all 
  run with COM not yet initialized
- the native thread initializer (thread_native_entry) is already
  running with COM initialized

Every article I read about COM insists, that your are required to
initialize the COM environment on each thread from which you do COM
calls. So there was something fishy.

I looked more into in _beginthreadex and indeed I finally found a lead
on stackoverflow. That pointed me to the sourcecode of threadex.c,
which is part of the Visual Studio 2013 Express Installation.

There you find, that _beginthreadex does not directly start the
supplied thread function, but runs a library initializer
(_threadstartex) first. Part of this initializer reads:

        _ptd->_initapartment = __crtIsPackagedApp();
        if (_ptd->_initapartment)
        {
            _ptd->_initapartment = _initMTAoncurrentthread();
        }

        _callthreadstartex();

__crtIsPackagedApp detects via a kernel function if the application is
run as a "PackagedApp" (i.e. AppX package) and if so, then the
RoInitialize is called, which from my understanding acts like the big
brother of CoInitialize.

Long story short: If your application is build with Visual Studio 2013
and run as a packaged application, you get a broken environment.

I assume that the Oracle JDK 8 binaries are build with an older version
of Visual Studio, which would explain the behavioural differences.

This looks like it supports my findings:

https://blogs.msdn.microsoft.com/vcblog/2016/07/07/using-visual-c-runtime-in-centennial-project/

Quote from the article:

        Note that the existing VC++ 12.0 libraries created during the
        Windows 8 timeframe have runtime checks to determine whether the
        app is running under the app container or not. When running
        desktop apps as a packaged app, these checks might limit the
        functionality of the desktop app or cause it to behave like a UWA
        (Universal Windows Application) (limited file system access or
        create thread initializing MTA etc.). We have fixed this behavior
        in the VC++ libraries contained in these framework packages and
        thus removing the modern app limitations from your desktop
        applications.

So the solution for packagers is:

- Read the above referenced blog
- Install the updated VC 12.0 framework packages for Desktop Bridge
- Install the appropriate contained appx packages (I tested Retail/x64)
- Package again
- Done

After the steps above my test application (a modified version of the
original version from Reinhard) reports, that the COM initialization
has not happend yet and everything is as it should be (initialization
in the JDKs ShellFolder native part is correctly initialized to STA).

=> Yay!

I suspect a java version compiled with a "fixed" version of Visual
Studio (2015 and later) should behave "correctly". It took much effort
not to write what I'm thinking about VS 2013 right now.

If I'm not mistaken the default compiler for future JDKs is about to be
bumped (I think I read something like this on one of the openjdk
mailinglists), so could someone please check if it has happend already?

After literally putting hours into this, I think I will conclude my
investigations here - it was an interesting ride through the JDK build
system, the JDK source and VS documentation.

I hope this helps someone out there

Greetings

Matthias

Am Donnerstag, den 08.02.2018, 19:32 +0100 schrieb Matthias Bläsing:
> Hi all,
> 
> last month I saw the message from Reinhard Pointer on this list:
> 
> http://mail.openjdk.java.net/pipermail/swing-dev/2018-January/008132.
> html
> 
> referring to this bug:
> 
> https://bugs.openjdk.java.net/browse/JDK-8189938
> 
> With the instructions given in:
> 
> https://bugs.openjdk.java.net/browse/JDK-8193928 (closed as duplicate
> of JDK-8189938)
> 
> I was able to reproduce the problem on a clean Windows 10
> Installation,
> as described in the initial report.
> 
> The resulting exception leads to my proposed fix:
> 
>         Could not initialize COM: HRESULT=0x80010106  
> 
> That HRESULT translates to: RPC_E_CHANGED_MODE
> 
> This means, that the COM subsystem is already initialized, but with a
> different mode. The code uses CoInitialize and this initializes the
> threading module to be apartment threaded. The above HRESULT means
> the
> thread was already initialized to be multi-threaded.
> 
> The mode can't be changed after is was set once, so instead of
> bailing
> out, I suggest to accept the situation and initialize to be multi-
> threaded.
> 
> The consequence is that calls from COM to Java could end up on the
> wrong thread. A quick look through the code suggest, that COM advises
> are not used, so this is a risk that should be taken.
> 
> The fix in this case works like this:
> 
>  * try to initialize COM as it was
>  * check the return
>  * if it is RPC_E_CHANGED_MODE, retry initialization als multi-
> threaded
>  * only if that fails raise an exception
> 
> One could argue, that if COM is already initialized, you don't need
> to
> do it yourself, but documentation states, that every successful call
> to
> CoInitialize/CoInitializeEx must be paired with CoUninitialize.
> 
> D3DPipelineManager tracks this in bComInitialized, the other two
> callers in the JDK are:
> 
> - PLATFORM_API_WinOS_DirectSound.cpp
> - ShellFolder2.cpp
> 
> Both are covered in the patches. For ShellFolder2 there is an
> initialization check, this is modified as described above. And if
> both
> initialization variant (apartment threaded and multi threaded fail)
> this raises an exception.
> 
> PLATFORM_API_WinOS_DirectSound is slightly different. The fallback in
> the initialization is still done, but no exception is raised in the
> error case. This follows the code flow before this change.
> 
> I attached the diffs against JDK10 and JDK9 to this email. If they
> are
> stripped, they can be found here:
> 
> http://www.doppel-helix.eu/JDK8189938-openjdk9.diff
> http://www.doppel-helix.eu/JDK8189938-openjdk10.diff
> 
> I hope this helps with a fix.
> 
> Greetings
> 
> Matthias