RFR: JDK-8266396: Allow VSCMD_DEBUG to work as intended [v2]

[John Neffenger]

On Mon, 10 May 2021 05:54:51 GMT, John Neffenger <jgneff at openjdk.org> wrote:

>> The Windows build calls a series of batch files to get the Visual Studio paths and environment variables. The batch files are a black box: any messages they print are discarded. If anything goes wrong, the only signs are a vague Gradle exception and a corrupted properties file.
>> 
>> This has been causing problems for years. There are at least 49 messages on the mailing since 2017 that discuss the exception and ways to work around it.
>> 
>> This pull request lets you enable the batch file messages, shedding light on their internal workings and allowing you to catch any errors at their source. Specifically, it adds the variable `VSCMD_DEBUG` to the environment of `genVSproperties.bat` and documents its use.
>> 
>> ### Before
>> 
>> For example, if your `PATH` environment variable is missing `C:/Windows/System32`, like mine was, you'll see the following errors:
>> 
>> 
>> $ gradle sdk
>> Dependency verification is an incubating feature.
>> 
>> FAILURE: Build failed with an exception.
>> 
>> * Where:
>> Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108
>> 
>> * What went wrong:
>> A problem occurred evaluating script.
>>> FAIL: WINSDK_DIR not defined
>> 
>>     ︙
>> 
>> BUILD FAILED in 4s
>> 
>> 
>> *Me:* What's a `WINSDK_DIR`? The Wiki says, "This means that you will have to define these paths manually." I guess this is normal. ��️
>> 
>> ### After
>> 
>> With this change, you can set the debug level to "3" in the `win.gradle` file and get a better picture of the problem:
>> 
>> 
>> $ gradle sdk
>> Dependency verification is an incubating feature.
>> 
>>> Configure project :
>> 'reg' is not recognized as an internal or external command,
>> operable program or batch file.
>> 'reg' is not recognized as an internal or external command,
>> operable program or batch file.
>> 'reg' is not recognized as an internal or external command,
>> operable program or batch file.
>> 
>>     ︙
>> 
>> 'powershell.exe' is not recognized as an internal or external command,
>> operable program or batch file.
>> 
>> FAILURE: Build failed with an exception.
>> 
>> * Where:
>> Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108
>> 
>> * What went wrong:
>> A problem occurred evaluating script.
>>> FAIL: WINSDK_DIR not defined
>> 
>>     ︙
>> 
>> BUILD FAILED in 3s
>> 
>> 
>> *Me:* Oh, it's just a "command not found" error. I'll check my `PATH`. ��
>
> John Neffenger has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - JDK-8266396: Allow VSCMD_DEBUG to work as intended
>  - Merge branch 'master' into vscmd-debug
>  - JDK-8266396: Add VSCMD_DEBUG to genVSproperties.bat environment

To understand the build error that occurred with my last commit, you need to know three things:

1. by default, the JavaFX build on Windows always sends telemetry to Microsoft each time it runs the `vcvarsall.bat` batch file,
2. by default, the process that sends the telemetry is asynchronous, and
3. the Windows build runs that batch file twice, once for `x86` (via `vcvars32.bat`) and again for `x64`.

The actual build error is way back at [Line 182][1] with the message "The process cannot access the file because it is being used by another process." It's a timing problem that occurs only when both `VSCMD_SKIP_SENDTELEMETRY` and `VSCMD_DEBUG` are undefined (or set to the empty string). In that case, the Visual Studio batch files start an asynchronous PowerShell script to send telemetry to Microsoft. Because the asynchronous process will continue to use the same standard error file descriptor as the batch file that started it, we hit the error when we try to run the batch file a second time before the PowerShell script has completed. If the PowerShell script completes before we run the batch file again, the error does not occur. I didn't hit the error because `powershell.exe` was not found on my `PATH`, essentially disabling the telemetry.

The fix is simple: skip sending the telemetry. We should probably be doing that, anyway.

An alternative, and perhaps safer, fix is not to redirect standard error when calling the `vcvarsall.bat` batch file. In that case, the trace messages would go to the log file but the error messages would go to the console, making it difficult to associate any errors with the commands that caused them.

[1]: https://github.com/jgneff/jfx/runs/2542639912#step:8:182

-------------

PR: https://git.openjdk.java.net/jfx/pull/488