Question about vfprintf hook VM argument

[David Holmes]

Hi Thomas,

On 9/05/2017 4:54 PM, Thomas Stüfe wrote:
> Hi David,
>
> On Mon, May 8, 2017 at 11:25 PM, David Holmes <david.holmes at oracle.com
> <mailto:david.holmes at oracle.com>> wrote:
>
>     Hi Thomas,
>
>     On 8/05/2017 7:29 PM, Thomas Stüfe wrote:
>
>         Hi all,
>
>         what exactly is the purpose of the FILE* argument in the
>         vfprintf hook?
>
>
>     I see your point. :) The vfprint_hook is a replacement vfprintf
>     function to be called from jio_vfprintf:
>
>     int jio_vfprintf(FILE* f, const char *fmt, va_list args) {
>       if (Arguments::vfprintf_hook() != NULL) {
>          return Arguments::vfprintf_hook()(f, fmt, args);
>       } else {
>         return vfprintf(f, fmt, args);
>       }
>     }
>
>     so whatever gets passed to jio_vfprintf gets passed through to the hook.
>
>     But ...
>
>         We had - actually several times already - the problem that our
>         VM was
>         embedded by a customized launcher which used the vfprintf hook
>         to redirect
>         the VM output. If the launcher uses the FILE* handed over by the
>         VM to
>         write to, it must be linked against the same C-Runtime as the VM
>         itself.
>         This is not necessarily a given, especially on Windows: the
>         launcher may
>         link against the debug C-Runtime (compiled with /MDd) wheras the
>         JDK is
>         build with "/MD" and links against the release C-Runtime. Or the
>         launcher
>         may even have been linked statically against the C-Runtime. Or...
>
>         In my opinion it is not a good idea to hand over C-Runtime
>         internals - be
>         it malloced memory or FILE* pointers - to other binaries which
>         may have
>         been built with different build options. But I do not even
>         understand the
>         point of passing FILE* to the hook? If the point of the hook is
>         to give
>         embedding code the ability to write to somewhere else, why even
>         bother
>         giving it *my* file pointer?
>
>
>     ... I confess I had no idea why this vfprint hook exists, but this
>     somewhat explains it:
>
>     https://bugs.openjdk.java.net/browse/JDK-4015550
>     <https://bugs.openjdk.java.net/browse/JDK-4015550>
>
>     and yes it does suggest that although the FILE* is passed in the
>     expectation is that the function will actually write somewhere else.
>     IIUC the intent was to allow fd's 0,1 and 2 to be re-mapped by the
>     hook to match whatever the embedded app had change System.out/err
>     to. But as fd's were per-dll they couldn't pass through the fd so
>     they passed through the FILE*. But how they expected that to be
>     mapped to stdout/stderr I have no idea.
>
>
> Thanks for looking at this, interesting piece of history!
>
> Well, maybe this was just not that well thought out. Handing the va_list
> up to the hookee and letting him unwrap it is is also unconventional,
> but probably does no harm, even when done by a different C-Runtime.
>
> I guess we continue living with it. We have a checklist for potential
> embedders writing launchers (e.g. not to use the primordial thread on
> AIX), and will add "Use the same C-Runtime as the JDK on Windows" to the
> list.

I still don't understand how this is supposed to work anyway. The intent 
is to provide a means for the VM to write to System.out/err, but there 
is no general way to determine whether the FILE* the VM passes through 
represents "stdout" or "stderr". ???

Cheers,
David

> Regards, Thomas
>
>
>
>     Cheers,
>     David
>
>
>         Thanks & Kind Regards, Thomas
>
>