CODETOOLS-7902083: Simplify building jtreg

Martin Buchholz martinrb at google.com
Wed Dec 27 23:17:13 UTC 2017


Because TEST.ROOT files contain lines like:

requiredVersion=4.2 b11

the BUILD_NUMBER is effectively a public API that cannot be changed.  The
architected way might be to bump the "4.2" number every time there's a
feature release, but it's probably too late for that.  So I would still put

BUILD_NUMBER = 11

into the Makefile every time it's tagged.  Extra work for jjg, but only for
jjg.


On Fri, Dec 22, 2017 at 11:40 AM, Jonathan Gibbons <
jonathan.gibbons at oracle.com> wrote:

> Volker,
>
> Thank you for noting the BUILD_NUMBER problem.
> I've pushed your fix (with a minor tweak to the syntax for consistency).
> I think there may be more changes to come, but your patch at least
> addresses the ability to pass through the BUILD_* info.
>
> I don't think that patching the code for the build number is the way
> to go. I think the "milestone" field is a better field to indicate
> deviations from a standard build. We might also want to look at
> aligning the syntax of tags and version strings with OpenJDK.
>
> Various folk have suggested some other minor improvements to the
> script in general, and I'll look at incorporating those changes as well.
>
> -- Jon
>
>
> On 12/22/2017 07:17 AM, Volker Simonis wrote:
>
>> Hi Jonathan,
>>
>> thanks a lot for improving the build. This is a huge step ahead and it
>> works great. I've just tried it and the build succeeded out of the
>> box!
>>
>> There's however one issue I still see. The default build will set the
>> version and build number to '4-2 dev b00'. Notice that such a build is
>> pretty useless, because such versions of JTreg will be rejected when
>> doing OpenJDK regression tests because the OpenJDK test have minimal
>> JTreg build requirements in their TEST.ROOT file like for example:
>>
>> # Minimum jtreg version
>> requiredVersion=4.2 b08
>>
>> Before the introduction of build-all.sh it was possible to pass
>> BUILD_NUMBER through the environment to make.
>>
>> When using build-all.sh this is not possible any more, because
>> build-all.sh calls make without honoring if BUILD_NUMBER was set in
>> the environment.
>>
>> I've opened "CODETOOLS-7902089: build-all.sh should honor BUILD_NUMBER
>> from environment (and use a reasonable default)"
>> (https://bugs.openjdk.java.net/browse/CODETOOLS-7902089) and came up
>> with the following fix:
>>
>> http://cr.openjdk.java.net/~simonis/webrevs/2017/CODETOOLS-7902089
>>
>> As an extension I propose to set the build number by default to the
>> latest available tag automatically, even if we're building tip because
>> I think that's what users actually want (and need) if they build JTreg
>> for executing the OpenJDK tests. This feature is already realized in
>> my webrev.
>>
>> Finally, we could set the build number to something like "b11+" if
>> "b11" was the latest tagged build, but we're several changes ahead
>> already (this would resemble the "hg id" output which prints a "+"
>> after the actual change hash if the working directory contains
>> additional local changes).
>>
>> I consider this to be the best solution, but in order to work we would
>> also have to update JTReg's build number parsing routine in
>> com.sun.javatest.regtest.tool.Version.getBuild() which currently only
>> accepts a plain build number (i.e. "b[0-9]+"). This feature is
>> currently not in my webrev but I'd be happy to add it if you'd
>> consider it useful.
>>
>> Can you please review and sponsor my change?
>>
>> Thank you and merry Christmas:)
>> Volker
>>
>>
>>
>> On Thu, Dec 21, 2017 at 2:12 AM, Jonathan Gibbons
>> <jonathan.gibbons at oracle.com> wrote:
>>
>>> My previous message was intended for Mani. I'm sorry I did not make that
>>> clearer.
>>>
>>> -- Jon
>>>
>>> On 12/20/2017 05:09 PM, Jonathan Gibbons wrote:
>>>
>>>> I've posted a new tag (jtreg4.2-b11) on the jtreg repo, so I would
>>>> expect
>>>> to see a new build on your CI system soon.
>>>>
>>>> If you're using the new make/build-all.sh script, then everything should
>>>> build as intended.
>>>>
>>>> If you're not using build-all.sh, note that you need to build jtreg with
>>>> variables like TESTNG_HOME, JUNIT_HOME, ASMTOOLS_HOME etc defined at the
>>>> time you run "make". It is not enough to copy the corresponding jar
>>>> files
>>>> into the lib directory; they must also be put in the Class-Path entry
>>>> of the
>>>> jtreg.jar MANIFEST.MF file (and the makefiles will take care of doing
>>>> that.)
>>>>
>>>> It occurs to me you might still be using the (old) Ant build.xml file.
>>>> If
>>>> so, I recommend you convert to using the new build-all.sh script.
>>>>
>>>> -- Jon
>>>>
>>>> On 12/19/2017 03:33 PM, Mani Sarkar wrote:
>>>>
>>>>> Hi Jon,
>>>>>
>>>>> Amended the scripts to use the latest build-all.sh from the tip, and
>>>>> got
>>>>> this - https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/.
>>>>>
>>>>> Still using our old script to build from the last tag.
>>>>>
>>>>> Both the artifacts do contain asmtools.jar - let me know if it passes
>>>>> your sanity check.
>>>>>
>>>>> Thanks again.
>>>>>
>>>>> Cheers,
>>>>> Mani
>>>>>
>>>>> On Mon, 18 Dec 2017 at 10:18 Maurizio Cimadamore
>>>>> <maurizio.cimadamore at oracle.com <mailto:maurizio.cimadamore at oracle.com
>>>>> >>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>      On 18/12/17 01:32, Jonathan Gibbons wrote:
>>>>>      > Maurizio,
>>>>>      >
>>>>>      > If you can do this in a way that makes it optional, that would
>>>>> be
>>>>>      > great.   Having put effort into pruning the set of dependencies,
>>>>>      I've
>>>>>      > no desire to see the set creep up again unnecessarily, although
>>>>> I
>>>>>      > admit the usefulness of the plugin.
>>>>>      Message understood :-)
>>>>>
>>>>>      Btw, I'm bringing this up because, currently, the only way to
>>>>>      build the
>>>>>      plugin is through the IDE itself, which, while not unreasonable
>>>>> (after
>>>>>      all you have to have an IDE if you want to use the plugin :-)), it
>>>>>      require some configuration and I've seen people getting stuck
>>>>> quite
>>>>>      frequently - a build command would probably mitigate some of those
>>>>>      concerns.
>>>>>
>>>>>      Maurizio
>>>>>      >
>>>>>      > -- Jon
>>>>>      >
>>>>>      >
>>>>>      > On 12/17/17 1:04 PM, Maurizio Cimadamore wrote:
>>>>>      >> Looks great - thanks for doing this.
>>>>>      >>
>>>>>      >> If there's interest, I could also put some effort in order to
>>>>>      >> integrate the plugins build into the system. In principle, it
>>>>>      should
>>>>>      >> be doable, by adding a bunch of env variables (to point at the
>>>>> IDEA
>>>>>      >> runtime jar). Of course that would be an optional part of the
>>>>>      build.
>>>>>      >>
>>>>>      >> Cheers
>>>>>      >> Maurizio
>>>>>      >>
>>>>>      >>
>>>>>      >> On 14/12/17 00:32, Jonathan Gibbons wrote:
>>>>>      >>> This is for folk who are interested in building jtreg from
>>>>> source.
>>>>>      >>>
>>>>>      >>> As some of you have (rightfully) commented over the past
>>>>> years,
>>>>>      >>> jtreg has not been an easy tool to build from source.
>>>>>      >>>
>>>>>      >>> And, as some of you may have noticed, there has been some
>>>>>      amount of
>>>>>      >>> activity over the past weeks and months to address this issue.
>>>>>      This
>>>>>      >>> work has been led by Erik Helin (thanks, Erik!) and we're now
>>>>>      >>> getting to the point where we can show what we have been
>>>>> working
>>>>>      >>> towards.
>>>>>      >>>
>>>>>      >>> The core of the work to build jtreg is still the Makefiles as
>>>>>      >>> before, although as was recently noted, we've been simplifying
>>>>> the
>>>>>      >>> specification of the dependencies.
>>>>>      >>>
>>>>>      >>> Separately, Erik has helped provide updates to the way that
>>>>>      some of
>>>>>      >>> the Code Tools dependencies can be built.
>>>>>      >>>
>>>>>      >>> Building on all that work, we can now get to the next stage,
>>>>> to
>>>>>      >>> provide a script that will download binaries for some
>>>>> components
>>>>>      >>> (JUnit, TestNG) and will download and build source for other
>>>>>      >>> components (AsmTools, JCov, JTHarness), for which there are no
>>>>>      >>> official binaries.
>>>>>      >>>
>>>>>      >>> To run the script, you just need to have Ant and a suitable
>>>>> "java"
>>>>>      >>> on your path, and to specify the location of an install of JDK
>>>>> 1.8
>>>>>      >>> as an argument to the script. wget is used to download files,
>>>>>      which
>>>>>      >>> honors proxy settings for those that need to use them. The
>>>>>      script is
>>>>>      >>> deliberately fairly simple, and suitable for use in a CI
>>>>> system.
>>>>>      >>>
>>>>>      >>> You can see a webrev for the script at
>>>>>      >>> http://cr.openjdk.java.net/~jjg/7902083/webrev.00/
>>>>>      <http://cr.openjdk.java.net/%7Ejjg/7902083/webrev.00/>
>>>>>      >>>
>>>>>      >>> Example of use:
>>>>>      >>>
>>>>>      >>> $ which ant
>>>>>      >>> /opt/ant/1.9.4/bin/ant
>>>>>      >>> $ which java
>>>>>      >>> /opt/jdk/1.8.0/bin/java
>>>>>      >>> $ sh make/build-all.sh /opt/jdk/1.8.0
>>>>>      >>> ... build output ...
>>>>>      >>> $ ls build/images/jtreg
>>>>>      >>> bin  COPYRIGHT  doc  legal  lib  LICENSE README release
>>>>>      >>> $
>>>>>      >>>
>>>>>      >>>
>>>>>      >>> Once this settles down a bit, I'll update the public docs on
>>>>> the
>>>>>      >>> jtreg web pages.
>>>>>      >>>
>>>>>      >>> -- Jon
>>>>>      >>>
>>>>>      >>>
>>>>>      >>
>>>>>      >
>>>>>
>>>>> --
>>>>>
>>>>> @theNeomatrix369 <http://twitter.com/theNeomatrix369> | Blog
>>>>> <http://neomatrix369.wordpress.com/> | @adoptopenjdk | Dev communities
>>>>>
>>>>> Meet-a-Project - MutabilityDetector
>>>>> <https://github.com/MutabilityDetector> | Github
>>>>> <https://github.com/neomatrix369> | Slideshare
>>>>> <https://www.slideshare.net/neomatrix369> |LinkedIn
>>>>> <http://uk.linkedin.com/pub/mani-sarkar/71/a77/39b>
>>>>>
>>>>> Come to Devoxx UK 2018:http://www.devoxx.co.uk/
>>>>>
>>>>>
>>>>> Don't chase success, rather aim for "Excellence", and success will come
>>>>> chasing after you!
>>>>>
>>>>>
>


More information about the code-tools-dev mailing list