Sv: Re: Sv: OOME in javac when building on Windows 7

[Fredrik Öhrström]

----- philip.race at oracle.com skrev:
> I can definitely switch it to 64 bit JRE. I didn't do that because
> your original email
> subject was "build-infra now builds on 32bit windows." So I tried to 
> avoid 64 bit thinking that was what wouldn't work.

That was a reasonable assumption. :-) But the real reason it only
builds a 32 bit jvm, is because config.guess returns i686-pc-cygwin
instead of x86_64-pc-cygwin. I do not know why. Perhaps it is a
bug in config.guess, perhaps it is a side effect of cygwin being 32 bit?
Anyone out there who knows?

> Should I now infer that the configure script would have been ok with
> the client VM
> on 32 bit windows because it would have set up all options for client
> ?

The settings -client -server does not affect the possible size of the heap.
On a 32 bit machine, the available virtual memory is 2G on Windows, or 3G
since the java launcher and jvm.dll are compiled with LARGEADDRESSAWARE. 
(I think they are?)

Unfortunately, that memory is severly fragmented on Windows (dll:s are 
sprinkled around as salt and pepper), and Hotspot/OpenJDK
cannot deal with split heaps (JRockit can). So the largest contiguous memory
chunk that is possible to find on Windows 32 bit, is usually around 1.8G.
Which is the maximum heap size.
 
> But it sounds like this would lose some of the build time benefits 
> unless you can have a 64 bit JDK.

In a way, yes. Since the javacserver is a memory hog scaling in size with
the number of cores. Yes, to get full speed on all cores, you need a 64 bit machine.
When we improve javac, and move some shellscript code into a smarter javac, then
this will not be a problem.

For build systems with fewer cores (1) or very little memory. You can
use --disable-javac-multi-core --disable-javac-deps --disable-javac-server
and the memory usage will be smaller. 

I have successfully built the openjdk build-infra forest on my asus 901 netbook
with 1G of RAM using that config. Took a while though, but most of that time
was spent on C++.

> Maybe you could try "javaw.exe" to run that server ? It shouldn't need
> a console.
> "-Djava.awt.headless=true" is worth a try but I suspect won't prevent
> the console.

Ah, of course. Thanks for the tip! I'll try that. 

You can try too. Simply change in the  spec.gmk file where it says 
SERVER_JAVA:=.... to use javaw instead of java.

//Fredrik

> -phil.
> 
> On 11/7/2011 1:19 PM, Fredrik Öhrström wrote:
> > Please try with a 64 bit jvm. The configure script needs to be
> > updated to detect when you run with a 32 bit jvm to accomodate
> > for the smaller available heap size.
> >
> > The configure script calculates the needed amount of heap memory
> > for the javac server. It needs a LOT of memory for the time being
> > since each core gets its own JavaCompiler, and each JavaCompiler
> > ends up with 30% (wild guesstime) of all classes in the JDK.
> > The reason is of course that the JavaCompilers do not share any
> state.
> > In the future we will have them share state so that the memory
> usage
> > shrinks. (Look in configure.ac for "We try to limit the heap size"
> > and you will find the rudimentary calculations.)
> >
> > Now, the configure script, should stop and complain when it
> calculated
> > a heap size, which does not work. Instead the ac macro
> ADD_JVM_ARG_IF_OK
> > simply does not add the heap size, which is why the javac server
> line:
> >   
> javac=C:/build-infra/jdk7/build/windows-i586-client-release/uncygdrive.exe%20C:/jdk1.7/bin/java%20-XX:+UseParallelOldGC%20-verbosegc%20
> > does not have any -Xmx. I will fix that.
> >
> > Also, the configure script defaults to 4 cores on windows, will fix
> that too.
> >
> > As for the GUI popup. On Windows, it is impossible to run a
> background command
> > without a window, unless it is a service. Thus the window is the
> javac server.
> > I can make a copy of the javac server log scroll by in that window,
> if you like,
> > now it is stored in build/windows..../javacserver/*.stdouterr
> >
> > To get rid of the annoying pause after building a repository:
> > Remove the line BUILD_LOG_WRAPPER:=.... in your spec.gmk
> >
> > //Fredrik
> >
> > ----- philip.race at oracle.com skrev:
> >
> >> Fredrik,
> >>
> >> ['ccing the list]
> >>
> >>
> >> On 11/5/2011 2:14 AM, Fredrik Öhrström wrote:
> >>> I think you have set an explicit --with-server-java=".."
> >>> where you have not added enough heap.
> >>>
> >>> If you just run ./configure, then it should set the
> >>> heap size properly.
> >>>
> >>> Try that.
> >>>
> >>> //Fredrik
> >>>
> >>> ----- philip.race at oracle.com skrev:
> >>>
> >>>> configure is happy now but the build fails with OOM .. large
> >>>> attachment
> >>>> so sending to you directly.
> >>>>
> >>>> -phil.
> >>
> >> I never used that option. I was just using ./configure.
> >> That in fact said it would build both so I removed the server dir
> :
> >> rm -rf windows-i586-clientANDserver-release
> >>
> >> and tried again with :
> >>    ./configure --with-jvm-variant=client
> >> --with-freetype=/cygdrive/c/build-infra/freetype
> >>
> >> I still get the OOME
> >> The system is out of resources.
> >> Consult the following stack trace for details.
> >> java.lang.OutOfMemoryError: GC overhead limit exceeded
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.blockStatements(JavacParser.ja
> >> va:1551)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.block(JavacParser.java:1523)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.block(JavacParser.java:1537)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.methodDeclaratorRest(JavacPars
> >> er.java:2654)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.classOrInterfaceBodyDeclaratio
> >> n(JavacParser.java:2603)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.classOrInterfaceBody(JavacPars
> >> er.java:2531)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.classDeclaration(JavacParser.j
> >> ava:2379)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.classOrInterfaceOrEnumDeclarat
> >> ion(JavacParser.java:2320)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.typeDeclaration(JavacParser.ja
> >> va:2309)
> >>           at
> >>
> com.sun.tools.javac.parser.JavacParser.parseCompilationUnit(JavacPars
> >> er.java:2247)
> >>           at
> >> com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:630)
> >>           at
> >> com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:667)
> >>           at
> >>
> com.sun.tools.javac.main.JavaCompiler.parseFiles(JavaCompiler.java:96
> >> 4)
> >>           at
> >>
> com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:855)
> >>           at com.sun.tools.javac.main.Main.compile(Main.java:546)
> >>           at
> >>
> com.sun.tools.javac.main.JavacServer$Handler.run(JavacServer.java:660
> >> )
> >>           at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> >> java:1110)
> >>           at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
> >> .java:603)
> >>           at java.lang.Thread.run(Thread.java:722)
> >>
> >>
> >> Javac command line that failed:
> >> -XDserver:portfile=c:/build-infra/jdk7/build/win
> >>
> dows-i586-client-release/javacservers/GENERATE_NEWBYTECODE.port,poolsize=6,javac
> >>
> =C:/build-infra/jdk7/build/windows-i586-client-release/uncygdrive.exe%20C:/jdk1.
> >>
> 7/bin/java%20-XX:+UseParallelOldGC%20-verbosegc%20-Xbootclasspath/p:c:/build-inf
> >>
> ra/jdk7/build/windows-i586-client-release/langtools/dist/bootstrap/lib/javac.jar
> >>
> %20-jar%20c:/build-infra/jdk7/build/windows-i586-client-release/langtools/dist/b
> >> ootstrap/lib/javac.jar
> >> -XDdeps=file=c:/build-infra/jdk7/build/windows-i586-clien
> >>
> t-release/corba/classes/com/sun/corba/se/impl/logging/_the.package.deps,groupon=
> >> package
> >>
> -XDpubapi=file=c:/build-infra/jdk7/build/windows-i586-client-release/cor
> >>
> ba/classes/com/sun/corba/se/impl/logging/_the.package.api,notify,package=com.sun
> >> .corba.se.impl.logging
> >> -XDnativeapi=file=c:/build-infra/jdk7/build/windows-i586-
> >>
> client-release/corba/classes/com/sun/corba/se/impl/logging/_the.package.native,n
> >> otify,package=com.sun.corba.se.impl.logging -Xprefer:source
> >> -XDignore.symbol.fil
> >> e=true -cp C:/jdk1.7/lib/tools.jar
> >> -Xlint:all,-deprecation,-unchecked,-serial,-f
> >> allthrough,-cast,-rawtypes,-static,-dep-ann -implicit:none
> -sourcepath
> >>
> >> c:/build-
> >>
> infra/jdk7/corba/src/share/classes;c:/build-infra/jdk7/build/windows-i586-client
> >>
> -release/corba/gensrc;c:/build-infra/jdk7/build/windows-i586-client-release/corb
> >> a/logwrappers -d
> >> c:/build-infra/jdk7/build/windows-i586-client-release/corba/cla
> >> sses @C:\Users\prr\AppData\Local\Temp\atfile_a08860
> >>
> >>
> >> My machine is an 8-core Xeon with 6GB RAM .. running 64 bit
> Windows
> >> 7.
> >> I am using 32 bit JDK 1.7.0_01 as the bootstrap JDK, set via
> >> "JAVA_HOME".
> >>
> >> I see that the build set "UseParallelOldGC". Why ? I don't see any
> >> heap
> >> options although
> >> that may imply some but I am not sure how well it works for the
> client
> >>
> >> VM .. on
> >> the other hand it apparently worked for you.
> >> I didn't use any of the options you had in your email from a week
> or
> >> so ago
> >> "--disable-javac-multi-core --disable-javac-deps
> >> --disable-javac-server"
> >> but your email said those were just to prevent a 2 minute timeout.
> >>
> >> ====
> >>
> >> And one other thing: about a minute into the build a windows
> command
> >> tool GUI
> >> pops up, but there's no shell in the window - its empty. Its
> appears
> >> something in the
> >> build inadvertently creates this GUI to run some command.
> >> I didn't notice when it went away, but it was gone by the time the
> >> build
> >> failed.
> >>
> >> -phil.