Issues running JAXP jtreg tests ("java.lang.RuntimePermission" "accessDeclaredMembers")
Daniel Fuchs
daniel.fuchs at oracle.com
Tue Nov 22 13:31:47 UTC 2016
On 22/11/16 13:01, Langer, Christoph wrote:
> Hi,
>
> we are running jtreg with something like -jdk:<build directory>/images/jdk. So would that be the exploded version or not?
Yes - that's the image. Hmm... The failures I see with the exploded
build are different than what you have anyway.
> FWIW: I think all test that fail don't have void test methods but the test methods expect input parameters and hence a tag @Test(dataProvider = "...") exists.
Good observation.
> Can it be that we are using a testng framework that is "too new" and maybe contains something which makes it not work in the jaxp scenario?
When I call jtreg -version it reports:
TestNG: version 6.9.5
This seems different to what you are using.
Can you try with
https://adopt-openjdk.ci.cloudbees.com/job/jtreg/lastSuccessfulBuild/artifact/jtreg-4.2-b03.tar.gz
without altering the version of testng?
I had no problem running the tests with that version
of testng (except if I use the exploded build).
Maybe the listener that the test uses to set up the security
manager is invoked in a different way (earlier?) with the
6.9.10 version?
best regards,
-- daniel
>
> Best,
> Christoph
>
>> -----Original Message-----
>> From: Daniel Fuchs [mailto:daniel.fuchs at oracle.com]
>> Sent: Dienstag, 22. November 2016 13:42
>> To: Volker Simonis <volker.simonis at gmail.com>; Langer, Christoph
>> <christoph.langer at sap.com>; Langer, Christoph <christoph.langer at sap.com>
>> Cc: Chris Hegarty <chris.hegarty at oracle.com>; code-tools-
>> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; jtreg-
>> use at openjdk.java.net
>> Subject: Re: Issues running JAXP jtreg tests ("java.lang.RuntimePermission"
>> "accessDeclaredMembers")
>>
>> Hi guys,
>>
>> Are you running the tests with the exploded jdk or with the image?
>>
>> I'm seeing failures with the exploded jdk.
>>
>> That could explain the difference with permission checks.
>>
>> best regards,
>>
>> -- daniel
>>
>>
>> On 22/11/16 12:32, Daniel Fuchs wrote:
>>> Hi Volker,
>>>
>>> On 22/11/16 12:25, Chris Hegarty wrote:
>>>> Volker,
>>>>
>>>> Just to add, jtreg has support in its tags to start the test VM with a
>>>> security manager and a specified policy. In the case of the test failure
>>>> you are seeing, the built-in jtreg support is not being used. Instead,
>>>> the test is executing with a test-specific system property that is being
>>>> used to trigger the programatic setting of a security manager with a
>>>> generated policy. I think this explains why you are not seeing any
>>>> policy file in the JTwork directory.
>>>>
>>>> What is odd is that the stack trace you are seeing appears to be
>>>> before the test’s main entry point, so I would not expect to the
>>>> security manager to be active at this point ( since no test code
>>>> has run ). My previous comment regarding 7901792 is not relevant
>>>> since it relates to tests executing with a security manager set
>>>> through jtreg tags.
>>>
>>> I agree with Chris - I believe CODETOOLS-7901792 was a red herring.
>>> I'm going to try the jtreg you pointed at and see if there's any
>>> difference (I'm using jtreg 4.2 fcs b03).
>>>
>>>> Is there anything in the environment that could trigger the VM
>>>> to start up with a security manager enabled?
>>>
>>> This is a good question.
>>>
>>> best regards,
>>>
>>> -- daniel
>>>
>>>>
>>>> -Chris.
>>>>
>>>>> On 22 Nov 2016, at 12:13, Volker Simonis <volker.simonis at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Daniel,
>>>>>
>>>>> thanks for your support - this problem really drives us crazy!
>>>>>
>>>>> What version of jtreg are you using?
>>>>> If you are using a central one which was configured and build by
>>>>> Oracle, could you please also try with the one from
>>>>> https://adopt-
>> openjdk.ci.cloudbees.com/job/jtreg/lastSuccessfulBuild/artifact/
>>>>>
>>>>> ?
>>>>>
>>>>> Where can we find the generated policy files? It seems they get
>>>>> deleted during test post-run cleanup phase (at least I could not find
>>>>> any of them under JTwork). Is there a way to get a more detailed trace
>>>>> on how JTreg executes testng and to leave all the generated files in
>>>>> place after the test?
>>>>>
>>>>> I'm currently running the following JAXP test and get the same error
>>>>> as described by Christoph:
>>>>>
>>>>> /tmp/jtreg/bin/jtreg -verbose:summary -jdk
>>>>> /output-jdk9-hs-dbg/images/jdk/
>>>>> /OpenJDK/jdk9-
>> hs/jaxp/test/javax/xml/jaxp/functional/catalog/CatalogReferCircularityTest.jav
>> a
>>>>>
>>>>>
>>>>> What I don't really understand is how this is supposed to work at all,
>>>>> because every JAXP test which runs with "-DrunSecMngr=true" will
>>>>> execute the following code from JAXPPolicyManager:
>>>>>
>>>>> /*
>>>>> * Install a SecurityManager along with a default Policy to allow
>>>>> testNG to
>>>>> * run when there is a security manager.
>>>>> */
>>>>> private JAXPPolicyManager() {
>>>>> // Backing up policy and security manager for restore
>>>>> policyBackup = Policy.getPolicy();
>>>>> smBackup = System.getSecurityManager();
>>>>>
>>>>> // Set customized policy
>>>>> setDefaultPermissions();
>>>>> Policy.setPolicy(policy);
>>>>> System.setSecurityManager(new SecurityManager());
>>>>> }
>>>>>
>>>>> Won't this code override the settings from the policy file which was
>>>>> potentially installed by jtreg for testng?
>>>>>
>>>>> The setDefaultPermissions() sets some special permissions for testng,
>>>>> but not "accessDeclaredMembers":
>>>>>
>>>>> private void setDefaultPermissions() {
>>>>> //Permissions to set security manager and policy
>>>>> addPermission(new SecurityPermission("getPolicy"));
>>>>> addPermission(new SecurityPermission("setPolicy"));
>>>>> addPermission(new RuntimePermission("setSecurityManager"));
>>>>> //Properties that jtreg and TestNG require
>>>>> addPermission(new
>>>>> PropertyPermission("testng.show.stack.frames", "read"));
>>>>> addPermission(new PropertyPermission("test.src", "read"));
>>>>> addPermission(new PropertyPermission("test.classes", "read"));
>>>>> addPermission(new
>>>>> PropertyPermission("dataproviderthreadcount", "read"));
>>>>> addPermission(new PropertyPermission("experimental", "read"));
>>>>> }
>>>>>
>>>>> If I add the line:
>>>>>
>>>>> addPermission(new RuntimePermission("accessDeclaredMembers"));
>>>>>
>>>>> to setDefaultPermissions(), the test will succeed.
>>>>>
>>>>> I saw that an early version of "JDK-8067170: Enable security manager
>>>>> on JAXP unit tests" contained that extra permission, but you objected
>>>>> (http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-
>> July/042520.html)
>>>>>
>>>>> and it was removed in the final version.
>>>>>
>>>>> Any more hints?
>>>>>
>>>>> Thanks,
>>>>> Volker
>>>>>
>>>>>
>>>>> On Tue, Nov 22, 2016 at 12:24 PM, Daniel Fuchs
>>>>> <daniel.fuchs at oracle.com> wrote:
>>>>>> Hi Christoph,
>>>>>>
>>>>>> Is there anything funny with the place jtreg is installed?
>>>>>> like:
>>>>>> - path contains whitespaces
>>>>>> - path is accessible through links or mount points...
>>>>>>
>>>>>> It seems clear that the issue here is that testng classes are
>>>>>> missing some permissions, so I was wondering whether that could
>>>>>> be caused by the actual path to testng.jar not matching the
>>>>>> path injected in the policy file.
>>>>>>
>>>>>> I'm using jtreg 4.2 fcs b03, and have no issues with the jaxp tests:
>>>>>>
>>>>>> $ cd jaxp/tests
>>>>>> $ rm -r JT*
>>>>>> $ jtreg -verbose:summary -ignore:quiet -jdk
>>>>>> ../../build/macosx-x86_64-normal-server-release/images/jdk javax/
>>>>>>
>>>>>> => the only test that fails is
>>>>>> javax/xml/jaxp/isolatedjdk/catalog/PropertiesTest.sh, but that's
>>>>>> expected
>>>>>> (it's in the ProblemList.txt).
>>>>>>
>>>>>> The other thing to take care of, is not to run two jtreg process
>>>>>> concurrently if they point to the same JT* directories. If you do
>>>>>> that then you might experience weird failures with permissions
>>>>>> issues (it seems to mess the policy files).
>>>>>>
>>>>>> best regards,
>>>>>>
>>>>>> -- daniel
>>>>>>
>>>>>>
>>>>>> On 22/11/16 10:52, Langer, Christoph wrote:
>>>>>>>
>>>>>>> Yes, please find it here:
>>>>>>> http://cr.openjdk.java.net/~clanger/jtreg/XSLTFunctionsTest.jtr
>>>>>>>
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Chris Hegarty [mailto:chris.hegarty at oracle.com]
>>>>>>>> Sent: Dienstag, 22. November 2016 11:03
>>>>>>>> To: Langer, Christoph <christoph.langer at sap.com>
>>>>>>>> Cc: core-libs-dev at openjdk.java.net; code-tools-dev at openjdk.java.net;
>>>>>>>> jtreg-
>>>>>>>> use at openjdk.java.net
>>>>>>>> Subject: Re: Issues running JAXP jtreg tests
>>>>>>>> ("java.lang.RuntimePermission"
>>>>>>>> "accessDeclaredMembers")
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 22 Nov 2016, at 09:43, Langer, Christoph
>>>>>>>>> <christoph.langer at sap.com>
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Chris,
>>>>>>>>>
>>>>>>>>> thanks for this hint. However, we've already seen this change and
>>>>>>>>> rebuilt
>>>>>>>>
>>>>>>>> jtreg with the latest jtreg repo. But it doesn't change a thing.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Also, the download from https://adopt-
>>>>>>>>
>>>>>>>> openjdk.ci.cloudbees.com/job/jtreg/lastSuccessfulBuild/artifact/
>>>>>>>> where I
>>>>>>>> would
>>>>>>>> suppose latest jtreg sources were used, don't help.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Am I missing something?
>>>>>>>>
>>>>>>>>
>>>>>>>> Is it possible to post, or upload to cr.o.j.n, the jtr of the failing
>>>>>>>> test?
>>>>>>>>
>>>>>>>> -Chris.
>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Christoph
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Chris Hegarty [mailto:chris.hegarty at oracle.com]
>>>>>>>>>> Sent: Dienstag, 22. November 2016 10:08
>>>>>>>>>> To: Langer, Christoph <christoph.langer at sap.com>
>>>>>>>>>> Cc: core-libs-dev at openjdk.java.net;
>>>>>>>>>> code-tools-dev at openjdk.java.net;
>>>>>>>>>> jtreg-
>>>>>>>>>> use at openjdk.java.net
>>>>>>>>>> Subject: Re: Issues running JAXP jtreg tests
>>>>>>>>>> ("java.lang.RuntimePermission"
>>>>>>>>>> "accessDeclaredMembers")
>>>>>>>>>>
>>>>>>>>>> Hi Christoph,
>>>>>>>>>>
>>>>>>>>>> Can you please ensure that your build of jtreg contains the fix for
>>>>>>>>>> 7901792
>>>>>>>>
>>>>>>>> [1].
>>>>>>>>>>
>>>>>>>>>> 7901792 grants <JTREG_HOME>/lib/testng.jar all permissions.
>>>>>>>>>>
>>>>>>>>>> -Chris.
>>>>>>>>>>
>>>>>>>>>> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7901792
>>>>>>>>>>
>>>>>>>>>>> On 22 Nov 2016, at 08:38, Langer, Christoph
>>>>>>>>>>> <christoph.langer at sap.com>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I'm currently struggling while running jtreg tests for the jaxp
>>>>>>>>>>> depot.
>>>>>>>>>>>
>>>>>>>>>>> There are several tests that fail with the same symptom. I
>>>>>>>>>>> always get
>>>>>>>>>>
>>>>>>>>>> exceptions like:
>>>>>>>>>>>
>>>>>>>>>>> java.security.AccessControlException: access denied
>>>>>>>>>>
>>>>>>>>>> ("java.lang.RuntimePermission" "accessDeclaredMembers")
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> java.base/java.security.AccessControlContext.checkPermission(AccessControlCo
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ntext.java:471)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> java.base/java.security.AccessController.checkPermission(AccessController.java
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> :894)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:5
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 48)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>> java.base/java.lang.Class.checkMemberAccess(Class.java:2595)
>>>>>>>>>>> at
>>>>>>>>>>> java.base/java.lang.Class.getDeclaredMethods(Class.java:2162)
>>>>>>>>>>> at
>>>>>>>>
>>>>>>>> org.testng.internal.ClassHelper.extractMethods(ClassHelper.java:217)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>> org.testng.internal.ClassHelper.getAvailableMethods(ClassHelper.java:182)
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>
>>>>>>>> org.testng.internal.Parameters.findDataProvider(Parameters.java:323)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>
>>>>>>>> org.testng.internal.Parameters.findDataProvider(Parameters.java:259)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>> org.testng.internal.Parameters.handleParameters(Parameters.java:419)
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>> org.testng.internal.Invoker.handleParameters(Invoker.java:1274)
>>>>>>>>>>> at
>>>>>>>>>>> org.testng.internal.Invoker.createParameters(Invoker.java:989)
>>>>>>>>>>> at
>>>>>>>>>>> org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1079)
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>> org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> java:129)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>> org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
>>>>>>>>>>>
>>>>>>>>>>> at org.testng.TestRunner.privateRun(TestRunner.java:782)
>>>>>>>>>>> at org.testng.TestRunner.run(TestRunner.java:632)
>>>>>>>>>>> at org.testng.SuiteRunner.runTest(SuiteRunner.java:366)
>>>>>>>>>>> at
>>>>>>>>>>> org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361)
>>>>>>>>>>> at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319)
>>>>>>>>>>> at org.testng.SuiteRunner.run(SuiteRunner.java:268)
>>>>>>>>>>> at
>>>>>>>>>>>
>> org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
>>>>>>>>>>> at
>>>>>>>>>>> org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
>>>>>>>>>>> at org.testng.TestNG.runSuitesSequentially(TestNG.java:1244)
>>>>>>>>>>> at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
>>>>>>>>>>> at org.testng.TestNG.run(TestNG.java:1064)
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> com.sun.javatest.regtest.TestNGAction$TestNGRunner.main(TestNGAction.java:
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 224)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> com.sun.javatest.regtest.TestNGAction$TestNGRunner.main(TestNGAction.java:
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 188)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>
>>>>>>>>>> Method)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethod
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> AccessorImpl.java:62)
>>>>>>>>>>>
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> gMethodAccessorImpl.java:43)
>>>>>>>>>>>
>>>>>>>>>>> at java.base/java.lang.reflect.Method.invoke(Method.java:537)
>>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.j
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ava:110)
>>>>>>>>>>>
>>>>>>>>>>> at java.base/java.lang.Thread.run(Thread.java:844)
>>>>>>>>>>>
>>>>>>>>>>> For instance the test
>>>>>>>>>>
>>>>>>>>>> javax/xml/jaxp/unittest/transform/XSLTFunctionsTest.java fails like
>>>>>>>>>> this.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It's calling "testng -DrunSecMngr=true" and obviously some
>>>>>>>>>>> important
>>>>>>>>>>
>>>>>>>>>> permission for testing is missing with that.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm using most current jtreg (with testng-6.9.10.jar)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks for any help.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best regards
>>>>>>>>>>> Christoph
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>
More information about the code-tools-dev
mailing list