[8u] [JFR] RFR: JFR backports from 11.0.4

Fri Jan 17 11:21:22 UTC 2020

Hi!

Unfortunately, due to messed up logging code in the initial JFR backport the mentioned patch,
although fixes the problem of JFR not being able to function under certain conditions (verified),
it disables JFR logging almost completely. The reason is that logging level is in fact not initialized
because of lack of Xlog support in jdk8. In the previous implementation all the JFR logging was enabled
on the LogTag/Logger level and was controlled with -XX:+LogJFR option altogether (so no ability to selectively
enable of disable different tags or set levels).

With this, may I suggest to return "return true" statement into Logger.shouldLog method (previously
LogTag.shouldLog) so logging continues to function (the same crude

Thanks,
Andrey

> On 17 Jan 2020, at 12:54, Denghui Dong <denghui.ddh at alibaba-inc.com> wrote:
> 
> Hi,
>  Please review this backport for JDK-8209960: -Xlog:jfr* doesn't work with the JFR parser.
> 
> JIRA: https://bugs.openjdk.java.net/browse/JDK-8209960
> Webrev: http://cr.openjdk.java.net/~ddong/8209960/
> 
> The original patch is almost clean, but one place we need to check:
> The method "public boolean shouldLog(int level)" of jdk.jfr.internal.LogTag.java was removed by 8209960,
> but the content isn't same with upstream. So please help check it @Andrey
> ------------------------------------------------------------------
> From:董登辉(卓昂) <denghui.ddh at alibaba-inc.com>
> Send Time:2019年9月27日(星期五) 13:59
> To:jdk8u-dev <jdk8u-dev at openjdk.java.net>
> Cc:Mario Torre <neugens at redhat.com>; Andrey Petushkov <andrey at azul.com>
> Subject:Re: [8u] [JFR] RFR: JFR backports from 11.0.4
> 
> Hi,
>   Please review this backports for:
>     8224172: [BUG] assert(jfr_is_event_enabled(id)) failed: invariant
>     8216064: [BUG] -XX:StartFlightRecording:settings= doesn't work properly
>     8226779: [TESTBUG] Test JFR API from Java agent
>     8214750: [BUG] Unnecessary <p> tags in jfr classes
>     8227011: Starting a JFR recording in response to JVMTI VMInit and / or Java agent premain corrupts memory
> 
> http://cr.openjdk.java.net/~wzhuo/8224172/hotspot.00/
> http://cr.openjdk.java.net/~wzhuo/8216064/jdk.00/
> 
>   In fact, 8227011 is not in my plan, but test/jdk/jfr/javaagent/TestPremainAgent.java (8216064) will
> be failed without 8227011, I also checked the mail list and found no other people are backpoting it.
> 
>   Please help me to push it to jdk8u-jfr-incubator if you think there's no problem.
> 
> Thanks
> Denghui Dong
> ------------------------------------------------------------------
> From:Jaroslav Bachorík <jaroslav.bachorik at datadoghq.com>
> Send Time:2019年9月16日(星期一) 17:53
> To:Andrey Petushkov <andrey at azul.com>
> Cc:董登辉(卓昂) <denghui.ddh at alibaba-inc.com>; jdk8u-dev <jdk8u-dev at openjdk.java.net>; Ekaterina Vergizova <katya at azul.com>
> Subject:Re: [8u] [JFR] RFR: JFR backports from 11.0.4
> 
> 
> 
> On Wed, Sep 11, 2019 at 12:20 PM Jaroslav Bachorík <jaroslav.bachorik at datadoghq.com> wrote:
> Hi all,
> 
> we are planning to port also the following patches
> https://bugs.openjdk.java.net/browse/JDK-8210158 (already in 11u)
> 
> This one turned out to be not applicable to jdk8u
> 
> https://bugs.openjdk.java.net/browse/JDK-8225797 (being worked on in dev, will be backported to 11u adn jdk8u once done)
> 
> This fix has been merged to dev and I started working on the backport to 11u. So far it seems that the backport will be far from simple as it touches many places which are fundamentally different in dev, 11u and 8u :/
> 
> -JB-
> 
> 
> Cheers,
> 
> -JB-
> 
> 
> On Tue, Sep 10, 2019 at 2:53 PM Andrey Petushkov <andrey at azul.com> wrote:
> Hi Denghui,
> 
> Thank you. We'll take care of it then.
> The list of backports we're currently working on (for jdk8u incubator)
> was part of initial email. For convenience please find it below:
> 
> https://bugs.openjdk.java.net/browse/JDK-8185525: Add JFR event for DictionarySizes
> https://bugs.openjdk.java.net/browse/JDK-8213448: [TESTBUG] enhance jfr/jvm/TestDumpOnCrash
> https://bugs.openjdk.java.net/browse/JDK-8215727: Restore JFR thread sampler loop to old / previous behavior
> https://bugs.openjdk.java.net/browse/JDK-8216283: Allow shorter method sampling interval than 10 ms
> https://bugs.openjdk.java.net/browse/JDK-8217362: Emergency dump does not work when disk=false is set
> https://bugs.openjdk.java.net/browse/JDK-8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64
> https://bugs.openjdk.java.net/browse/JDK-8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero
> https://bugs.openjdk.java.net/browse/JDK-8219997: [TESTBUG] Create test for JFR events in Docker container: CPU, Memory and Process Info
> https://bugs.openjdk.java.net/browse/JDK-8220293: Deadlock in JFR string pool
> https://bugs.openjdk.java.net/browse/JDK-8220555: JFR tool shows potentially misleading message when it cannot access a file
> https://bugs.openjdk.java.net/browse/JDK-8220657: JFR.dump does not work when filename is set
> https://bugs.openjdk.java.net/browse/JDK-8221569: JFR tool produces incorrect output when both --categories and --events are specified
> https://bugs.openjdk.java.net/browse/JDK-8221711: [TESTBUG] create more tests for JFR in container environment
> https://bugs.openjdk.java.net/browse/JDK-8222888: [TESTBUG] docker/TestJFREvents.java fails due to "RuntimeException: JAVA_MAIN_CLASS_ is not defined"
> https://bugs.openjdk.java.net/browse/JDK-8223438: add VirtualizationInformation JFR event
> https://bugs.openjdk.java.net/browse/JDK-8223599: minimal build fails after JDK-8185525
> https://bugs.openjdk.java.net/browse/JDK-8224217: RecordingInfo should use textual representation of path
> https://bugs.openjdk.java.net/browse/JDK-8225310: JFR crashed in JfrPeriodicEventSet::requestProtectionDomainCacheTableStatistics()
> 
> from these there are number of issues which are not yet ported to jdk11u. We're on it,
> some of them have been pushed to jdk11u today. The rest are:
> 
> https://bugs.openjdk.java.net/browse/JDK-8185525: Add JFR event for DictionarySizes
> https://bugs.openjdk.java.net/browse/JDK-8223599: minimal build fails after JDK-8185525
> https://bugs.openjdk.java.net/browse/JDK-8225310: JFR crashed in JfrPeriodicEventSet::requestProtectionDomainCacheTableStatistics()
> https://bugs.openjdk.java.net/browse/JDK-8217362: Emergency dump does not work when disk=false is set
> https://bugs.openjdk.java.net/browse/JDK-8224217: RecordingInfo should use textual representation of path
> 
> we'll working on preparing review requests for those into jdk11u
> 
> Best Regards,
> Andrey
> 
> > On 10 Sep 2019, at 08:04, DDH <denghui.ddh at alibaba-inc.com> wrote:
> >
> > Hi Andrey,
> >
> >    Since you have already processed on 8223438([Enhancement] add VirtualizationInformation JFR event),
> > we think that we don't need to do this issue again, we will remove it from our list.
> >    By the way, can you send us a complete list that you will backport? We can double check there are any repeated issues.
> >
> > Thanks,
> > DDH
> > ------------------------------------------------------------------
> > From:Andrey Petushkov <andrey at azul.com>
> > Send Time:2019年9月9日(星期一) 20:59
> > To:Mario Torre <neugens at redhat.com>; 董登辉(卓昂) <denghui.ddh at alibaba-inc.com>
> > Cc:jdk8u-dev <jdk8u-dev at openjdk.java.net>; Ekaterina Vergizova <katya at azul.com>
> > Subject:Re: [8u] [JFR] RFR: JFR backports from 11.0.4
> >
> > Hi Denghui,
> >
> > Just a note, from the list below one backport (8223438: [Enhancement] add VirtualizationInformation JFR event)
> > is already proposed for integration as part of Azul's effort ([1]).
> > However since it's not yet integrated into jdk11u there still work to be done. We can do it, but if you'd like
> > and if you feel it's more convenient, you can take over. Anyway you might want to check implementation of
> > the backport in the respective webrev ([2]). Please let us know, thanks
> >
> > Regards,
> > Andrey
> >
> > [1] http://mail.openjdk.java.net/pipermail/jdk8u-dev/2019-September/010204.html
> > [2] http://cr.openjdk.java.net/~apetushkov/jfr_backports_katya/11.0.4/
> >
> > On 9 Sep 2019, at 12:37, Mario Torre <neugens at redhat.com> wrote:
> >
> > Hi Denghui,
> >
> > Yes, the list looks good to me. As you mentioned, we should try first
> > the 11u backports and then backport to 8u.
> >
> > The process for the backport is highlighted here:
> > https://wiki.openjdk.java.net/display/JDKUpdates/How+to+contribute+a+fix
> >
> > Cheers,
> > Mario
> >
> > On Mon, Sep 9, 2019 at 4:07 AM DDH <denghui.ddh at alibaba-inc.com> wrote:
> >
> > hi all,
> >  We(Alibaba) picked some jfr backports as follows from JBS
> > (https://bugs.openjdk.java.net/browse/JDK-8230624?jql=Subcomponent%20%3D%20jfr%20and%20resolution%20%3D%20Fixed%20and%20fixVersion%20%3E%2011.0.6%20and%20type%20!%3D%20Backport), we
> > think it is worth porting them to 8u/11u.
> >  We plan to backport them to 11u at first, and then to 8u, what's your comment?
> >  If you think it is reasonable, we can provide our webrev of some issues as soon as possible, and continue work on other issues.
> >
> > 8223396: [TESTBUG] several jfr tests do not clean up files created in /tmp
> > 8225004: Remove invalid assertion in jfr_conditional_flush()
> > 8214542: [BUG] JFR: Old Object Sample event slow on a deep heap in debug builds (Unresolved)
> > 8228834: [BUG] Regression caused by JDK-8214542 not installing complete checkpoint data to candidates
> > 8228359: [TESTBUG] jdk.jfr.e.g.c.TestGCHeapConfigurationEventWith32BitOops.java does not expect MinHeapSize to be aligned to HeapAlignment
> > 8227605: [BUG] Kitchensink fails "assert((((klass)->trace_id() & (JfrTraceIdEpoch::leakp_in_use_this_epoch_bit())) != 0)) failed: invariant"
> > 8227411: [BUG] TestTimeMultiple.java failed "assert(!lease()) failed: invariant"
> > 8224172: [BUG] assert(jfr_is_event_enabled(id)) failed: invariant
> > 8212663: [BUG] Remove conservative at_safepoint assert when JFR writes type sets during class unloading
> > 8216064: [BUG] -XX:StartFlightRecording:settings= doesn't work properly
> > 8214750: [BUG] Unnecessary <p> tags in jfr classes
> > 8213570: [TESTBUG] Update JFR sanity test set
> > 8226779: [TESTBUG] Test JFR API from Java agent
> > 8229189: [Enhancement] Improve JFR leak profiler tracing to deal with discontiguous heaps
> > 8223438: [Enhancement] add VirtualizationInformation JFR event
> >
> > ------------------------------------------------------------------
> > From:Andrey Petushkov <andrey at azul.com>
> > Send Time:2019年9月5日(星期四) 23:55
> > To:Mario Torre <neugens at redhat.com>
> > Cc:jdk8u-dev at openjdk.java.net <jdk8u-dev at openjdk.java.net>
> > Subject:Re: [8u] [JFR] RFR: JFR backports from 11.0.4
> >
> > Hi Mario,
> >
> > The following fixes apply trivially to jdk11u, so I've requested the permission to backport per process.
> >
> > https://bugs.openjdk.java.net/browse/JDK-8220555
> > https://bugs.openjdk.java.net/browse/JDK-8221711
> > https://bugs.openjdk.java.net/browse/JDK-8222888
> > https://bugs.openjdk.java.net/browse/JDK-8221569
> > https://bugs.openjdk.java.net/browse/JDK-8216283
> >
> > The rest require some rework, I'll post RFRs soon
> >
> > Thanks,
> > Andrey
> >
> > On 4 Sep 2019, at 17:47, Mario Torre <neugens at redhat.com> wrote:
> >
> > Awesome, thanks for checking zero.
> >
> > As discussed offline, we have a few backports that were directly
> > backported to 8u without first being in 11u:
> >
> > https://bugs.openjdk.java.net/browse/JDK-8185525
> > https://bugs.openjdk.java.net/browse/JDK-8223599
> > https://bugs.openjdk.java.net/browse/JDK-8225310
> > https://bugs.openjdk.java.net/browse/JDK-8221711
> > https://bugs.openjdk.java.net/browse/JDK-8222888
> > https://bugs.openjdk.java.net/browse/JDK-8216283
> > https://bugs.openjdk.java.net/browse/JDK-8220555
> > https://bugs.openjdk.java.net/browse/JDK-8217362
> > https://bugs.openjdk.java.net/browse/JDK-8221569
> > https://bugs.openjdk.java.net/browse/JDK-8224217
> >
> > A couple of those are wither being worked on or of interest for 11u,
> > so they should be fine, some aren't and while may not be critical I
> > think they are nice to have (like the container tests), so I would
> > expect all of them to be backported to 11u.
> >
> > Since this is a staging repository we may go ahead and push them and
> > work on the backport to 11 afterward, but I would prefer to not create
> > a discrepancy at this moment, so if possible we should work on the
> > backports to 11 first.
> >
> > Cheers,
> > Mario
> >
> >
> > On Wed, Sep 4, 2019 at 4:09 PM Andrey Petushkov <andrey at azul.com> wrote:
> >
> > Hi Mario,
> >
> > zero build is fine (e.g. mentioned method has default no-op implementation in vm_version.hpp)
> >
> > Andrey
> >
> > On 4 Sep 2019, at 12:52, Mario Torre <neugens at redhat.com> wrote:
> >
> > On 03/09/2019 13:53, Andrey Petushkov wrote:
> > Dear All,
> >
> > could you please consider the following set of backports of the JFR fixes from 11.0.4 release into 8u incubator baseline:
> >
> > This seems good, the only nit I have now is that some of those changes
> > may break zero again, I think it may make sense to fix it in this patch
> > set instead of filing a dedicated bug report later.
> >
> > See for example:
> >
> > JDK-8219241
> >
> > +void VM_Version::print_platform_virtualization_info(outputStream* st) {
> >
> > etc..
> >
> > Cheers,
> > Mario
> >
> > --
> > Mario Torre
> > Associate Manager, Software Engineering
> > Red Hat GmbH <https://www.redhat.com>
> > 9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898
> >
> >
> >
> > --
> > Mario Torre
> > Associate Manager, Software Engineering
> > Red Hat GmbH <https://www.redhat.com>
> > 9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898
> >
> >
> >
> > --
> > Mario Torre
> > Associate Manager, Software Engineering
> > Red Hat GmbH <https://www.redhat.com>
> > 9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898
> >
> 