os::javaTimeSystemUTC to call nanosecond precision OS API, so Clock.systemUTC() can give nanosecond precision UTC

Tue May 5 15:50:47 UTC 2020

Hi David,

So the issue is that while glibc 2.12 (in OEL6.4) supports
clock_gettime(CLOCK_REALTIME) it requires a runtime dependency on
librt.so, which is an optional runtime dependency, hence the dynamic
lookup you mention. The complexity of dynamic lookup makes this
enhancement unattractive. But when openjdk updates to require glibc
>=2.17 it would be trivial, because clock_gettime() was moved into the
core libc.so.

With glibc >= 2.17 there would still need to be a call to
clock_gettime() to see if the optional CLOCK_MONOTONIC is supported,
whereas CLOCK_REALTIME is required by the Posix spec and header docs.

---
This leads me to ask a different question:

What is the openjdk process for reviewing and updating its minimum dependencies?
- When are dependencies reviewed?
- What is the mailgroup to look at? (please let me know if there is a
better place to ask this question)

The info I found was last updated for JDK 13:
https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms

- Glibc 2.12 was released in 2010 and glibc 2.17 in 2012
- The current openjdk LTS release is Java SE 11 in September 2018,
supported through to October 2024 (extended support to September
2026): https://en.wikipedia.org/wiki/Java_version_history).
- Oracle Enterprise Linux 7.0 released in 2014 included glibc 2.17

These dates suggest an update of openjdk's glibc dependency might be due.
But I guess the driving factor is how the timelines of openjdk LTS
release support fits with those of Oracle Enterprise Linux versions.

---
FYI some observations on my experience of building and testing openjdk.

I installed OEL6.4, but found it very tricky to build a `real` openjdk
distribution on a current Ubuntu that would run with all features on
OEL6.4.
- The base openjdk build was easy to use, but it built a JDK with elf
dependencies based on my OS, so of course would not run on OLE6.
- The openjdk make/devkit infrastructure didn't work for me to make an
OEL6 build, after lots of attempts (including running in chroot or
docker isolation). I'm sure it works reliably, provided you have the
correct OS setup. But its not easy for an outsider to replicate that.
- The AdoptOpenJDK Docker build was super easy to use, and I liked
that it used Docker to use a reproducible and fully scripted OS build
environment. https://github.com/AdoptOpenJDK/openjdk-build
- But AdoptOpenJDK Docker build doesn't support building and using an
OEL6 devkit. AdoptOpenJDK build targets RHEL 7.4+:
https://github.com/AdoptOpenJDK/openjdk-build/wiki/%5BWIP%5D-Minimum-OS-levels

openjdk's devkit infra must pre-date the rise of Docker, and fully
reproducible build environments becoming the norm.
Maybe one day there will be time to use Docker or similar to simplify
the openjdk build, and making it easy for anyone to reproduce.

As with using newer glibc features (glibc 2.17 vs 2.12), its likely to
be drastically simpler when openjdk is able to update dependencies to
newer OS versions. Especially if openjdk can be built on its minimum
required per-platform OS version in a (Docker) container.

Mark

On Tue, 14 Apr 2020 at 23:34, David Holmes <david.holmes at oracle.com> wrote:
>
> Hi Mark,
>
> On 15/04/2020 3:09 am, Mark Kralj-Taylor wrote:
> > David, Daniel,
> > What is the oldest (lowest) version of Linux and glibc for openjdk 15?
>
> I'm not clear on that.
>
> > The availability of clock_gettime(CLOCK_REALTIME) on the oldest
> > Linux/glibc supported by openjdk 15 is likely to be the deciding
> > factor on if Hotspot Linux code can call
> > clock_gettime(CLOCK_REALTIME).
> >
> > doc/building.md suggests minimum Linux is Oracle Enterprise Linux 6.4
> > (i.e. RHEL 6.4).
> > Which I think uses glibc 2.12-1.107 (based on
> > https://yum.oracle.com/repo/OracleLinux/OL6/4/base/x86_64/index_src.html).
> > Searching for glibc sources it looks like this supports
> > clock_gettime(CLOCK_REALTIME).
> >
> > The wording of Linux docs suggests that clock_gettime(CLOCK_REALTIME)
> > should be supported if the clock_gettime() API exists. But other clock
> > sources are not mandatory. So CLOCK_REALTIME should be available even
> > if CLOCK_MONOTONIC is not.
> > - See: https://linux.die.net/man/2/clock_gettime.
> > - Also "POSIX.1-2008 marks gettimeofday() as obsolete, recommending
> > the use of clock_gettime(2) instead." from:
> > https://linux.die.net/man/2/gettimeofday
> >
> > Note that Hotspot os_posix.cpp checks for non-error return from
> > clock_gettime(CLOCK_MONOTONIC) to guard setting the _clock_gettime
> > function pointer. Which was why in the patch I called clock_gettime
> > directly for Linux specific os_linux.cpp (a subset of Posix OS-s).
> >
> > Also os_linux.cpp has:
> > #ifndef SUPPORTS_CLOCK_MONOTONIC
> > #error "Build platform doesn't support clock_gettime and related functionality"
> > #endif
> > Which made me wonder if openjdk might require CLOCK_MONOTONIC - which
> > would mean clock_gettime(CLOCK_REALTIME) is supported.
>
> So we require that the build platform supports CLOCK_MONOTONIC and
> clock_gettime, but not that the runtime platform supports
> CLOCK_MONOTONIC (without which we don't/didn't need clock_gettime().
>
> So we can switch to using clock_gettime(CLOCK_REALTIME) at build time
> with no problem.
>
> We can probably also require clock_gettime(CLOCK_REALTIME) at runtime,
> but we need to double-check that. I recall encountering a platform where
> clock_gettime was not available, but I can't recall if it was mainstream
> or on one of the other OpenJDK projects - nor do I recall exactly how
> long ago this was. Keeping the dynamic lookup of clock_gettime would be
> a conservative approach here - but we would need to make the distinction
> between the ability to use CLOCK_REALTIME and CLOCK_MONOTONIC.
>
> Or someone puts in the time and effort to establish exactly what the
> kernel and glibc dependencies are and whether we can just rely on
> everything existing on all platforms we care about. I don't have time to
> do that nor validate the results if someone else does it.
>
> Cheers,
> David
>
> > Mark
> >
> > On Tue, 14 Apr 2020 at 18:04, Mark Kralj-Taylor <kralj.mark at gmail.com> wrote:
> >>
> >> Daniel,
> >> Yes System.currentTimeMillis() and Clock.systemUTC() must be
> >> consistent, so should use the same OS time source (as shown by ).
> >>
> >> The patch to os_linux.cpp ensures this by calling the same Linux API:
> >> clock_gettime(CLOCK_REALTIME) for both, from:
> >> - os::javaTimeMillis() that backs System.currentTimeMillis()
> >> - os::javaTimeSystemUTC() that backs Clock.systemUTC()
> >>
> >> Looking at Linux / glibc source I think that gettimeofday() and
> >> clock_gettime(CLOCK_REALTIME) are supposed to be the same clocksource.
> >> i.e. that given by: cat
> >> /sys/devices/system/clocksource/clocksource0/current_clocksource
> >>
> >> Mark
> >>
> >> On Tue, 14 Apr 2020 at 13:29, Daniel Fuchs <daniel.fuchs at oracle.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On 11/04/2020 00:53, David Holmes wrote:
> >>>> Update:
> >>>>> It's a holiday weekend so I can't dig into this right now but we tried
> >>>>> using a high-precision clock source for systemUTC() in the past but it
> >>>>> didn't work because systemUTC() and currentTimeMillis() have to use
> >>>>> the same time base, and currentTimeMillis() has to use gettimeofday().
> >>>>> I thought this cross-dependency was documented somewhere but can't
> >>>>> find it right now. If gettimeofday and clock_gettime(CLOCK_REALTIME)
> >>>>> actually have the same time characteristics wrt. wall-clock time then
> >>>>> changing both as suggested may indeed work.
> >>>
> >>> Just to emphasize David's comment: System::currentTimeMillis() and
> >>> Clock::systemUTC() should use the same time source: if they don't - then
> >>> some tests will fail, and because it can be tricky to assert things
> >>> in tests, they might (and probably will) fail only intermittently.
> >>>
> >>> I'm probably the culprit here, I added those tests when I upgraded
> >>> Clock::systemUTC() to report sub millisecond granularity [1] - as was
> >>> available in the underlying clock that System::currentTimeMillis()
> >>> already used.
> >>>
> >>> However, I think I would be disturbed if System::currentTimeMillis()
> >>> and Clock::systemUTC() were using different clocks and could
> >>> report a different notion of time (by drifting away from each other).
> >>>
> >>> best regards,
> >>>
> >>> -- daniel
> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8068730