RFR(XS): 8073754: Zero/PPC: StackOverflowError during build in javadoc processing.

[Severin Gehwolf]

On Thu, 2015-07-30 at 15:29 +0200, Volker Simonis wrote:
> On Thu, Jul 30, 2015 at 3:14 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > On Thu, 2015-07-30 at 14:48 +0200, Volker Simonis wrote:
> >> On Thu, Jul 30, 2015 at 1:28 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> >> > On Thu, 2015-07-30 at 10:46 +0200, Volker Simonis wrote:
> >> >> Why can't you simply use:
> >> >>
> >> >>   --with-boot-jdk-jvmargs specify JVM arguments to be passed to all java
> >> >>                           invocations of boot JDK, overriding the default
> >> >>                           values, e.g --with-boot-jdk-jvmargs="-Xmx8G
> >> >>                           -enableassertions"
> >> >>
> >> >> with the corresponding stack settings?
> >> >
> >> > See my earlier response to David. I'll do some more tests, but it didn't
> >> > work for me.
> >> >
> >>
> >> So you are saying the StackOverflow may come from a thread which has
> >> specified its own stack size at construction time and therefore the
> >> default stack size settings are ignored. But I did a fast grep through
> >> the jdk/ and langtools/ repository and couldn't find any occurrence of
> >> such a 4-argument Thread constructor (i.e. "Thread(ThreadGroup group,
> >> Runnable target, String name, long stackSize)").
> >
> > Yes. Here is one occurrence
> > (src/java.base/share/classes/java/lang/ProcessHandleImpl.java):
> >
> > final class ProcessHandleImpl implements ProcessHandle {
> >
> >     /**
> >      * The thread pool of "process reaper" daemon threads.
> >      */
> >     private static final Executor processReaperExecutor =
> >             doPrivileged((PrivilegedAction<Executor>) () -> {
> >
> >                 ThreadGroup tg = Thread.currentThread().getThreadGroup();
> >                 while (tg.getParent() != null) tg = tg.getParent();
> >                 ThreadGroup systemThreadGroup = tg;
> >
> >                 ThreadFactory threadFactory = grimReaper -> {
> >                     // Our thread stack requirement is quite modest.
> >                     Thread t = new Thread(systemThreadGroup, grimReaper,
> >                             "process reaper", 32768);
> >
> 
> OK, I've looked in the jdk8 sources and ProcessHandle is quite new in
> jdk9 (since about a month or so).

On JDK 8 I see this:
src/solaris/classes/java/lang/UNIXProcess.java-227-            ThreadFactory threadFactory = grimReaper -> {
src/solaris/classes/java/lang/UNIXProcess.java-228-                // Our thread stack requirement is quite modest.
src/solaris/classes/java/lang/UNIXProcess.java:229:                Thread t = new Thread(systemThreadGroup, grimReaper,
src/solaris/classes/java/lang/UNIXProcess.java-230-                                      "process reaper", 32768);

> Are you sure this is the root case
> of your problem?

No, but I'm pretty sure -XX:ThreadStackSize is not being honored for
those 4-arg Thread constructor cases.

> >
> > I've just tried a clean build with:
> > --with-boot-jdk-jvmargs="-XX:ThreadStackSize=2240
> > -XX:VMThreadStackSize=1600" specified to configure. It
> > stack-overflowed :(
> 
> Yes, but which thread? What's the stack trace?

Hmm, is this possible from the build.log only? The reproducer for this
build is a javadoc batch invocation:
/priv/sgehwolf/zero-builds/openjdk8u-dev-2015-07-21-release/bin/java -Xms64M -Xmx1600M -XX:ThreadStackSize=1536 -XX:ThreadStackSize=2240 -XX:VMThreadStackSize=1600 -Djava.awt.headless=true "-Xbootclasspath/p:/priv/sgehwolf/openjdk9-hs-rt/build/linux-ppc64-normal-zero-release/buildtools/interim_langtools.jar" -cp /priv/sgehwolf/openjdk9-hs-rt/build/linux-ppc64-normal-zero-release/buildtools/interim_langtools.jar com.sun.tools.javadoc.Main \
    -sourcepath ":/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.security.jgss/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.security.jgss/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.logging/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.crypto.ucrypto/solaris/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.sql.rowset/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.accessibility/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.accessibility/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.smartcardio/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.smartcardio/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.smartcardio/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.zipfs/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.xml.crypto/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.deploy.osx/macosx/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.crypto.ec/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.datatransfer/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.datatransfer/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.datatransfer/macosx/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.datatransfer/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.internal.le/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.localedata/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.base/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.base/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.base/macosx/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.base/linux/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.base/aix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.base/solaris/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.base/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.jconsole/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.sql/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.jcmd/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.naming.dns/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.sctp/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.sctp/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.sctp/macosx/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.sctp/aix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.sctp/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.charsets/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.dev/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.crypto.pkcs11/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.management/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.management/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.management/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.jdi/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.jdi/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.crypto.mscapi/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.rmi/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.security.jgss/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.jvmstat/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.naming.rmi/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.compiler/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.corba/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.hprof.agent/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.scripting/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.policytool/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.prefs/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.prefs/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.prefs/macosx/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.prefs/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.security.auth/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.httpserver/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.attach/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.attach/aix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.attach/macosx/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.attach/linux/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.attach/solaris/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.attach/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.management/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.management/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.management/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.security.sasl/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/unix/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/macosx/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/solaris/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/windows/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.instrument/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.jartool/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/jdk.rmic/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.transaction/share/classes:/priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.naming/share/classes:/priv/sgehwolf/openjdk9-hs-rt/build/linux-ppc64-normal-zero-release/support/gensrc/java.base" \
    -doclet build.tools.swingbeaninfo.GenDocletBeanInfo \
    -x false -d /priv/sgehwolf/openjdk9-hs-rt/build/linux-ppc64-normal-zero-release/support/gensrc_no_docs/java.desktop/javax/swing \
    -t /priv/sgehwolf/openjdk9-hs-rt/jdk/make/data/swingbeaninfo/SwingBeanInfo.template \
    -docletpath /priv/sgehwolf/openjdk9-hs-rt/build/linux-ppc64-normal-zero-release/buildtools/jdk_tools_classes \
    -XDignore.symbol.file=true \
    -classpath /priv/sgehwolf/openjdk9-hs-rt/build/linux-ppc64-normal-zero-release/buildtools/jdk_tools_classes /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/AbstractButton.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/Box.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JComponent.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JApplet.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JButton.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JCheckBox.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JCheckBoxMenuItem.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JComboBox.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JColorChooser.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JDesktopPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JDialog.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JEditorPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JFileChooser.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JFrame.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JFormattedTextField.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JInternalFrame.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JLabel.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JLayeredPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JList.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JMenu.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JMenuBar.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JMenuItem.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JOptionPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JPanel.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JPasswordField.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JPopupMenu.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JProgressBar.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JRadioButton.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JRadioButtonMenuItem.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JScrollBar.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JScrollPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JSeparator.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JSlider.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JSplitPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JSpinner.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JTabbedPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JTable.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JTextArea.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JTextField.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JTextPane.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JToggleButton.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JToolBar.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JTree.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/JWindow.java /priv/sgehwolf/openjdk9-hs-rt/jdk/src/java.desktop/share/classes/javax/swing/text/JTextComponent.java

> Do you see the stack arguments in the command line which produced the
> stack overflow (i.e. are they correctly propagated by the build
> system)?

Yes, as you can see above "-XX:ThreadStackSize=2240
-XX:VMThreadStackSize=1600" arguments are there.

> Does this occur for the initial build (with a jdk8 bootstrap jdk) or
> only in the self build with jdk9?

Initial build with a JDK 8 Zero bootstrap JDK.

> If you have a reproducer, can you run it with "-XX:PrintFlagsFinal" to
> see the final flag settings?

Relevant flags output with -XX:+PrintFlagsFinal:

     intx VMThreadStackSize                        := 1600                                {pd product}
     intx ThreadStackSize                          := 2240                                {pd product}

> And could you get a thread dump from the reproducer to see the actual
> stack size of the offending thread?

The best I could do is give you the stack trace from the
StackOverFlowError:
http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8073754/stack-trace-javadoc.txt

Thread dump from when the javadoc process was still running:
http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8073754/thread-dump-javadoc.txt

Another one is in the bug:
https://bugs.openjdk.java.net/browse/JDK-8073754

It's a fairly deep stack. Not sure how you dissect the actual stack size
(in bytes) from a thread dump.

> >> If a normal thread doesn't honor the -XX:ThreadStackSize settings I'd
> >> consider that a bug and fix that one first.
> >
> > By fixing it you mean in jdk or hotspot code? Any suggestions?
> >
> 
> I mean in the hostspot code. But of coarse only if the above tests
> show that the settings are not honored by the VM.

OK.

Cheers,
Severin