December 2016 - quality-discuss

JDK 9 build 149 test results now available
by Muneer Kolarkunnu 23 Dec '16

23 Dec '16

JDK 9 ea build 149 test results are now available at http://www.java.net/download/openjdk/testresults/9/testresults.html The jdk test results contain 149 differences from the build 146 test results. No new testcase failures found. 0: /home/jtest/merge9/146/jdk/JTwork pass: 6,004; fail: 71; not run: 2,206 1: /home/jtest/merge9/149/jdk/JTwork pass: 6,130; fail: 7; not run: 2,195 0 1 Test pass --- com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java --- pass java/io/FilePermission/MergeName.java --- pass java/lang/Class/attributes/ClassAttributesTest.java --- pass java/lang/ClassLoader/GetResourceNullArg.java --- pass java/lang/instrument/RedefineModuleTest.java --- pass java/lang/instrument/modules/AppendToClassPathModuleTest.java --- pass java/lang/invoke/MethodHandles/privateLookupIn/Driver.java --- pass java/lang/reflect/Layer/LayerControllerTest.java --- pass java/lang/reflect/Module/AnnotationsTest.java --- pass java/lang/reflect/Module/annotation/Basic.java --- pass java/net/Authenticator/B4933582.java pass --- java/net/Authenticator/B4933582.sh --- pass java/net/Authenticator/GetAuthenticatorTest.java --- pass java/net/HttpURLConnection/SetAuthenticator/HTTPSetAuthenticatorTest.java --- pass java/net/HttpURLConnection/SetAuthenticator/HTTPTest.java --- pass java/net/InetAddress/ptr/Lookup.java pass --- java/net/InetAddress/ptr/lookup.sh --- pass java/net/ServerSocket/AcceptCauseFileDescriptorLeak.java pass --- java/net/ServerSocket/AcceptCauseFileDescriptorLeak.sh --- pass java/net/SocketOption/MinimumRcvBufferSize.java --- pass java/net/httpclient/APIErrors.java --- pass java/net/httpclient/BasicAuthTest.java --- pass java/net/httpclient/HeadersTest.java --- pass java/net/httpclient/HeadersTest1.java --- pass java/net/httpclient/HttpInputStreamTest.java --- pass java/net/httpclient/HttpRequestBuilderTest.java --- pass java/net/httpclient/ImmutableHeaders.java --- pass java/net/httpclient/ManyRequests.java --- pass java/net/httpclient/MessageHeadersTest.java --- pass java/net/httpclient/MultiAuthTest.java --- pass java/net/httpclient/ProxyAuthTest.java --- pass java/net/httpclient/RequestBodyTest.java --- pass java/net/httpclient/ShortRequestBody.java --- pass java/net/httpclient/SmokeTest.java --- pass java/net/httpclient/SplitResponse.java --- pass java/net/httpclient/TestKitTest.java --- pass java/net/httpclient/TimeoutOrdering.java --- pass java/net/httpclient/examples/WebSocketExample.java --- pass java/net/httpclient/http2/BasicTest.java --- pass java/net/httpclient/http2/ErrorTest.java --- pass java/net/httpclient/http2/HpackDriver.java --- pass java/net/httpclient/http2/HpackDriverHeaderTable.java --- pass java/net/httpclient/http2/NoBody.java --- pass java/net/httpclient/http2/RedirectTest.java --- pass java/net/httpclient/http2/ServerPush.java --- pass java/net/httpclient/http2/TLSConnection.java --- pass java/net/httpclient/http2/Timeout.java --- pass java/net/httpclient/security/Driver.java --- pass java/net/httpclient/security/Security.java --- pass java/net/httpclient/websocket/WSDriver.java --- pass java/net/httpclient/whitebox/Driver.java fail pass java/rmi/Naming/UnderscoreHost.java fail pass java/rmi/activation/Activatable/checkAnnotations/CheckAnnotations.java fail pass java/rmi/activation/Activatable/checkImplClassLoader/CheckImplClassLoader.java fail pass java/rmi/activation/Activatable/checkRegisterInLog/CheckRegisterInLog.java fail pass java/rmi/activation/Activatable/createPrivateActivable/CreatePrivateActivatable.java fail pass java/rmi/activation/Activatable/downloadParameterClass/DownloadParameterClass.java fail pass java/rmi/activation/Activatable/elucidateNoSuchMethod/ElucidateNoSuchMethod.java fail pass java/rmi/activation/Activatable/forceLogSnapshot/ForceLogSnapshot.java fail pass java/rmi/activation/Activatable/inactiveGroup/InactiveGroup.java fail pass java/rmi/activation/Activatable/lookupActivationSystem/LookupActivationSystem.java fail pass java/rmi/activation/Activatable/nestedActivate/NestedActivate.java fail pass java/rmi/activation/Activatable/nonExistentActivatable/NonExistentActivatable.java fail pass java/rmi/activation/Activatable/restartCrashedService/RestartCrashedService.java fail pass java/rmi/activation/Activatable/restartLatecomer/RestartLatecomer.java fail pass java/rmi/activation/Activatable/restartService/RestartService.java fail pass java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java fail pass java/rmi/activation/Activatable/unregisterInactive/UnregisterInactive.java fail pass java/rmi/activation/ActivateFailedException/activateFails/ActivateFails.java fail pass java/rmi/activation/ActivationGroup/downloadActivationGroup/DownloadActivationGroup.java fail pass java/rmi/activation/ActivationSystem/activeGroup/IdempotentActiveGroup.java fail pass java/rmi/activation/ActivationSystem/modifyDescriptor/ModifyDescriptor.java fail pass java/rmi/activation/ActivationSystem/stubClassesPermitted/StubClassesPermitted.java fail pass java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java fail pass java/rmi/activation/CommandEnvironment/SetChildEnv.java fail pass java/rmi/activation/rmidViaInheritedChannel/InheritedChannelNotServerSocket.java fail --- java/rmi/activation/rmidViaInheritedChannel/RmidViaInheritedChannel.java fail pass java/rmi/registry/classPathCodebase/ClassPathCodebase.java fail pass java/rmi/registry/emptyName/EmptyName.java fail pass java/rmi/registry/interfaceHash/InterfaceHash.java fail pass java/rmi/reliability/juicer/AppleUserImpl.java fail pass java/rmi/server/RMIClassLoader/delegateBeforePermissionCheck/DelegateBeforePermissionCheck.java fail pass java/rmi/server/RMIClassLoader/delegateToContextLoader/DelegateToContextLoader.java fail pass java/rmi/server/RMIClassLoader/downloadArrayClass/DownloadArrayClass.java fail pass java/rmi/server/RMIClassLoader/getClassLoader/GetClassLoader.java fail pass java/rmi/server/RMIClassLoader/loadProxyClasses/LoadProxyClasses.java fail pass java/rmi/server/RMIClassLoader/spi/ContextInsulation.java fail pass java/rmi/server/RMIClassLoader/spi/DefaultProperty.java fail pass java/rmi/server/RMIClassLoader/spi/Installed.java fail pass java/rmi/server/RMIClassLoader/spi/InvalidProperty.java fail pass java/rmi/server/RMIClassLoader/spi/Property.java fail pass java/rmi/server/RMIClassLoader/useCodebaseOnly/UseCodebaseOnly.java fail pass java/rmi/server/RMIClassLoader/useGetURLs/UseGetURLs.java fail pass java/rmi/server/RMISocketFactory/useSocketFactory/activatable/UseCustomSocketFactory.java fail pass java/rmi/server/RMISocketFactory/useSocketFactory/registry/UseCustomSocketFactory.java fail pass java/rmi/server/RMISocketFactory/useSocketFactory/unicast/UseCustomSocketFactory.java fail pass java/rmi/server/RemoteObject/verifyRemoteEquals/VerifyRemoteEquals.java fail pass java/rmi/server/UnicastRemoteObject/unexportObject/UnexportLeak.java fail pass java/rmi/server/Unreferenced/leaseCheckInterval/LeaseCheckInterval.java fail pass java/rmi/server/Unreferenced/unreferencedContext/UnreferencedContext.java fail pass java/rmi/server/clientStackTrace/ClientStackTrace.java fail pass java/rmi/transport/closeServerSocket/CloseServerSocket.java fail pass java/rmi/transport/dgcDeadLock/DGCDeadLock.java fail pass java/rmi/transport/handshakeFailure/HandshakeFailure.java fail pass java/rmi/transport/handshakeTimeout/HandshakeTimeout.java fail pass java/rmi/transport/pinClientSocketFactory/PinClientSocketFactory.java fail pass java/rmi/transport/reuseDefaultPort/ReuseDefaultPort.java --- pass java/security/ProtectionDomain/NullGetActions.java --- pass java/security/SecureRandom/NoSync.java --- pass java/security/SecureRandom/ThreadSafe.java --- pass java/util/ArrayList/IteratorMicroBenchmark.java --- pass java/util/Collection/IteratorMicroBenchmark.java --- pass java/util/Collection/RemoveMicroBenchmark.java --- pass java/util/Locale/Bug8071929.java --- pass java/util/ResourceBundle/modules/cache/CacheTest.java --- pass java/util/ResourceBundle/modules/unnamed/unnamed.sh pass --- java/util/ServiceLoader/basic.sh --- pass java/util/ServiceLoader/basic/basic.sh --- pass java/util/ServiceLoader/modules/BadProvidersTest.java --- pass java/util/ServiceLoader/modules/Basic.java pass --- java/util/ServiceLoader/modules/MiscTests.java pass --- java/util/ServiceLoader/modules/ServicesTest.java --- pass java/util/Spliterator/SpliteratorFailFastTest.java pass --- java/util/Spliterator/SpliteratorLateBindingFailFastTest.java --- pass java/util/Spliterator/SpliteratorLateBindingTest.java --- pass javax/crypto/CryptoPermissions/CryptoPolicyFallback.java --- pass javax/imageio/plugins/tiff/ReadWithoutBaselineTagSet.java fail pass javax/management/remote/mandatory/connection/RMIConnector_NPETest.java --- pass javax/net/ssl/finalize/SSLSessionFinalizeTest.java pass --- javax/net/ssl/templates/SSLSocketSample.java pass --- javax/rmi/PortableRemoteObject/8146975/RmiIiopReturnValueTest.java --- pass javax/sound/sampled/AudioFileFormat/TypeEqualsToNull.java --- pass javax/sound/sampled/AudioFormat/EncodingEqualsToNull.java --- pass javax/sound/sampled/Clip/OpenNonIntegralNumberOfSampleframes.java pass --- javax/xml/ws/clientjar/TestWsImport.java --- pass jdk/modules/open/Basic.java fail pass sun/rmi/rmic/iiopCompilation/IIOPCompilation.java fail pass sun/rmi/runtime/Log/4504153/Test4504153.java fail pass sun/rmi/runtime/Log/6409194/NoConsoleOutput.java fail pass sun/rmi/runtime/Log/checkLogging/CheckLogStreams.java fail pass sun/rmi/runtime/Log/checkLogging/CheckLogging.java fail pass sun/rmi/server/MarshalOutputStream/marshalForeignStub/MarshalForeignStub.java fail pass sun/rmi/transport/tcp/DeadCachedConnection.java --- pass tools/jar/multiRelease/Basic1.java --- pass tools/jimage/JImageBasicsTest.java --- pass tools/jimage/JImageInfoTest.java --- pass tools/jlink/basic/AllModulePath.java --- pass tools/launcher/InfoStreams.java --- pass tools/launcher/modules/addexports/manifest/AddExportsAndOpensInManifest.java 149 differences The hotspot test results contain 8 differences from the build 146 test results. No new testcase failures found. 0: /home/jtest/merge9/146/hotspot/JTwork pass: 1,340; fail: 7; error: 15; not run: 54 1: /home/jtest/merge9/149/hotspot/JTwork pass: 1,339; fail: 5; error: 19; not run: 54 0 1 Test pass --- runtime/modules/JVMCanReadModule.java pass --- runtime/modules/JVMIsExportedToModule.java --- error serviceability/jvmti/AddModuleExportsAndOpens/MyPackage/AddModuleExportsAndOpensTest.java --- error serviceability/jvmti/AddModuleReads/MyPackage/AddModuleReadsTest.java --- error serviceability/jvmti/AddModuleUsesAndProvides/MyPackage/AddModuleUsesAndProvidesTest.java fail pass serviceability/sa/TestInstanceKlassSize.java fail pass serviceability/sa/TestInstanceKlassSizeForInterface.java pass error serviceability/sa/jmap-hprof/JMapHProfLargeHeapTest.java 8 differences The langtools test results contain 21 differences from the build 146 test results. There are 4 testcase failures, these failures are under investigation. 0: /home/jtest/merge9/146/langtools/JTwork pass: 3,516; not run: 277 1: /home/jtest/merge9/149/langtools/JTwork pass: 3,527; fail: 4; not run: 290 0 1 Test --- pass tools/javac/T8170667/ParameterProcessor.java --- pass tools/javac/api/TestName.java --- pass tools/javac/boxing/QualBoxedPostOp.java --- pass tools/javac/boxing/QualBoxedPostOp2.java --- pass tools/javac/boxing/QualBoxedPostOp3.java --- pass tools/javac/desugar/BoxingAndSuper.java --- pass tools/javac/generics/inference/8170410/T8170410.java --- pass tools/javac/modules/AllDefaultTest.java --- pass tools/javac/modules/AnnotationsOnModules.java --- pass tools/javac/modules/GetLocationForModuleTest.java --- pass tools/javac/modules/OpenModulesTest.java pass --- tools/javac/modules/RequiresPublicTest.java --- pass tools/javac/modules/RequiresStaticTest.java --- pass tools/javac/modules/RequiresTransitiveTest.java pass fail tools/javac/modules/T8159439/NPEForModuleInfoWithNonZeroSuperClassTest.java --- pass tools/javac/processing/model/LocalInAnonymous.java --- pass tools/javadoc/BadOptionsTest.java --- pass tools/jdeps/modules/GenOpenModule.java pass fail tools/sjavac/ApiExtraction.java pass fail tools/sjavac/ClasspathDependencies.java pass fail tools/sjavac/IgnoreSymbolFile.java 21 differences The nashorn test result is available at http://download.java.net/openjdk/testresults/9/archives/149/emailable-repor… -- Regards, Abdul Muneer Quality Engineer Oracle, Bangalore, India

1 0

Re: Repository? -- How many lines of development?
by Martin Buchholz 23 Dec '16

23 Dec '16

On Fri, Nov 18, 2016 at 8:33 AM, joe darcy <joe.darcy(a)oracle.com> wrote: > * A master forest, serving the roles master and dev play today in 9. > > With a few exceptions, in JDK 9 master was just time-delayed copy of dev > so we can implement recording the information about which set of sources > correspond to a promoted build without using a whole other forest. > > Rather than using a separate line of development for client-libs work as > in 9, I think this should be done in the same line of development as all > other libs work in 10. > For many years, I've been advocating having a guaranteed always-working, never regressing master and also always a place for developers to submit-and-forget their (possibly slightly buggy) changes. All regressions that could be caught by a test are 100% guaranteed to be caught by a competent trusted release engineer who is the only one ever moving changes into the master forest. Based on this idea, it seems essential to have something like a jdk10-dev forest (it could also be implemented using mercurial branches, but that would be a break with many decades of tradition). I notice today the message http://mail.openjdk.java.net/pipermail/quality-discuss/2016-November/000596… where regressions have crept into a jdk9 build, which is disappointing. The whole point of regression testing is to ensure that regressions don't happen! And I recall having that job myself back in 2005!

2 2

What bug does JDK-8168389 duplicate?
by Vitaly Provodin 22 Dec '16

22 Dec '16

Good time of day Recently I detected that the issue https://bugs.openjdk.java.net/browse/JDK-8168389 <https://bugs.openjdk.java.net/browse/JDK-8168389> was closed as duplicate but there are no information about original bug. Could you pleas provide information about what bug was duplicated by JDK-8168389 <https://bugs.openjdk.java.net/browse/JDK-8168389>? Thanks, Vitaly

3 9

Re: Repository? -- How many lines of development?
by Martijn Verburg 15 Dec '16

15 Dec '16

Hi Goetz, A few FOSDEM's ago a group of us came up with a *optional* (i.e. No one should be forced to it) patch pipeline architecture where each vendor who had specific platforms could pull patches from a queue, build them on their specialised platform and return a result. This of course would require a little bit of common infrastructure which the London Java Community (as a non-profit entity) was willing to own/host on behalf of OpenJDK. At the time there wasn't unanimous support for this idea (I imagine for cost/resource reasons) from the major vendors involved in OpenJDK, but if folks think this is a good time to revisit the discussion then perhaps we could have a session on this at the upcoming 2017 FOSDEM. Cheers, Martijn On 15 December 2016 at 07:18, Lindenmaier, Goetz <goetz.lindenmaier(a)sap.com> wrote: > > well. Testing is embarrassingly parallel in principle so with enough > > hardware (or one of those "clouds" everyone is selling these days) one > > should be able to run arbitrarily many tests. No one seems to deal well > > Hmm, is there a cloud containing all the different platforms? > solarisx86_64? aixppc64? linuxs390x? linuxarm32? > I also think pre-submit testing is really helpful, but supplying the > compute power on all these platforms to do this to an extend > that you don't need any further testing is out of scope. > > Best regards, > Goetz. > > > > > > > -----Original Message----- > > From: jdk10-dev [mailto:jdk10-dev-bounces@openjdk.java.net] On Behalf Of > > Martin Buchholz > > Sent: Thursday, December 15, 2016 5:26 AM > > To: joe darcy <joe.darcy(a)oracle.com> > > Cc: jdk10-dev(a)openjdk.java.net; quality-discuss(a)openjdk.java.net > > Subject: Re: Repository? -- How many lines of development? > > > > On Wed, Dec 14, 2016 at 5:05 PM, joe darcy <joe.darcy(a)oracle.com> wrote: > > > > > > > > The tiered testing efforts made earlier in JDK 9 [1] are necessary > > > preconditions for a JDK-wide pre-push/pre-submit system. In > particular, if > > > you don't have a a set of meaningful tests that runs quickly enough and > > > passes reliably enough then a pre-push system can cause more harm than > > good > > > in introducing bottlenecks and causing changes to be spuriously > rejected, > > > say by an existing intermittent failure unrelated to the change. > > > > > > > It's a hard problem, but I think with sufficient effort presubmit can > work > > well. Testing is embarrassingly parallel in principle so with enough > > hardware (or one of those "clouds" everyone is selling these days) one > > should be able to run arbitrarily many tests. No one seems to deal well > > with flaky tests, but flakiness can be measured, and a failing test can > be > > rerun arbitrarily, so deflaking should be automatable. > > > > At Google we are starting to experiment with running jtreg tests with > > massive parallelism. > > > > Automation, presubmit testing and never-ever-broken master remain my > > goals > > for any software project. >

1 0

Re: Repository? -- How many lines of development?
by Martin Buchholz 15 Dec '16

15 Dec '16

On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy(a)oracle.com> wrote: > > For the combined dev/master forest, the most recent integration tag will > have the same stability guarantees we have today so "pull the most recent > jdk-10+XYZ tag" to get a stable snapshot. > > But ... I want better than the stability guarantees we have today! I want to obsolete those messages to quality-discuss with "new failures" because it should not even be possible for master to get into a state where regression tests have regressed (except for flakes or external dependencies). > As an aside, for JDK 10 I'd also like to see promoted builds on a more > frequent schedule than once a week. > People do "continuous testing and integration" these days. Set up your integration pipeline so that it is always running. The pipeline automatically promotes changesets to master when all tests pass. Easy! Then every master changeset is equally stable to a "promoted build". In principle, the same could be accomplished via running every single commit through a jprt-like process, but I'm afraid that would be too expensive. Back in my day, full build + test took on the order of 10 hours, so changesets would need to be batched, or you need to throw more hardware at the problem (give every changeset a dedicated machine for testing?)

4 5

Re: JDK-6253001 - Joinrowset bug - question on whether test code has been migrated?
by Martijn Verburg 06 Dec '16

06 Dec '16

Hi Lance, Thanks! We'll see if we can reproduce and if not then recommend that the issue is closed as already fixed. Cheers, Martijn On 6 December 2016 at 11:56, Lance Andersen <lance.andersen(a)oracle.com> wrote: > Hi Martjin, > > > On Dec 6, 2016, at 4:22 AM, Martijn Verburg <martijnverburg(a)gmail.com> > wrote: > > Hi all, > > The Adoption Group is having a go at collaboratively walking through the > process of resolving an open bug with the view of writing up the steps for > folks new to OpenJDK. > > We picked https://bugs.openjdk.java.net/browse/JDK-6253001 - a fairly > simple, low priority joinrowset bug reported in the sql area. Our > intention is to investigate the bug, reproduce it and *if* a fix is > required wait until the jdk 10 forests open to submit it there (as it's > only a P4). > > ----- > > Of course with older bugs there is a hidden history :-). The original test > is listed as living in: > > com.sun.rowset.tests.smoke.joinrowset.joinrowset3. > JoinRowSet3.testJoinRowSet22 > > However we're unable to find that exact test which means that either: > > 1.) It's an internal Sun/Oracle test > > The test referenced in the bug is a test in an internal test suite which > has since been retired. > > 2.) It's been migrated / replaced by one of the tests in > *jdk/javax/sql/testing/test/rowset/joinrowset/JoinRowSetTests.java* > > That is correct > > > Does anyone familiar with the tests around the SQL package know the history > here? > > > Best > Lance > > > Cheers, > Martijn > > > <http://oracle.com/us/design/oracle-email-sig-198324.gif> > <http://oracle.com/us/design/oracle-email-sig-198324.gif> > <http://oracle.com/us/design/oracle-email-sig-198324.gif> > <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| > Principal Member of Technical Staff | +1.781.442.2037 <+1%20781-442-2037> > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen(a)oracle.com > > > >

1 0

JDK-6253001 - Joinrowset bug - question on whether test code has been migrated?
by Martijn Verburg 06 Dec '16

06 Dec '16

Hi all, The Adoption Group is having a go at collaboratively walking through the process of resolving an open bug with the view of writing up the steps for folks new to OpenJDK. We picked https://bugs.openjdk.java.net/browse/JDK-6253001 - a fairly simple, low priority joinrowset bug reported in the sql area. Our intention is to investigate the bug, reproduce it and *if* a fix is required wait until the jdk 10 forests open to submit it there (as it's only a P4). ----- Of course with older bugs there is a hidden history :-). The original test is listed as living in: com.sun.rowset.tests.smoke.joinrowset.joinrowset3.JoinRowSet3.testJoinRowSet22 However we're unable to find that exact test which means that either: 1.) It's an internal Sun/Oracle test 2.) It's been migrated / replaced by one of the tests in *jdk/javax/sql/testing/test/rowset/joinrowset/JoinRowSetTests.java* Does anyone familiar with the tests around the SQL package know the history here? Cheers, Martijn

1 0