On Tue, 17 May 2022 22:20:40 GMT, Erik Gahlin <egahlin@openjdk.org> wrote:

...

Could I have a review of a change that makes the 'jfr scrub' overwrite the output file if it already exists.

Testing: /test/jdk/jdk/jfr/tool (Mac, Linux, Windows)

Thanks Erik

Erik Gahlin has updated the pull request incrementally with one additional commit since the last revision:

Rename method

Marked as reviewed by mgronlun (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/8728

I'd like to propose that we add a command that filters based on time (clip perhaps) While working with large JFR files from jdk8 and jdk11 jvms I stumbled upon the 'jfr scrub' command that was introduced in 19. I used the 'jfr summary' to get a list of events and programmatically split the JFR into a JFR for each eventtype, but the data can still be unwieldy especially when converted to JSON. Given that 'jfr scrub' introduced the write(newJfrFile, predicate) method I wrote a java app to clip the JFR file based on a period of time (i hadn't seen the example in the JBS issue[1]), so now I can clip a JFR recording both horizontally (by events) and vertically (by time) and generate slices that are easier for JMC and other tools to parse (JMC would crash alot with large data) I took a stab at writing 'jfr clip' locally (based on scrub) and can share that in a new JBS issue if it's felt by this group that its a worthy addition to the jfr tool Again, I know that developers can write this themselves, but I feel it would be a nice convenience to have this in the jfr tool Alternatively, we could extend 'jfr scrub' to handle both event filters and a time region Cheers Mat [1] https://bugs.openjdk.org/browse/JDK-8271585 [JDK-8271585] JFR: Scrub recording data - Java Bug System<https://bugs.openjdk.org/browse/JDK-8271585> JDK; JDK-8271585; JFR: Scrub recording data. Log In. Export bugs.openjdk.org Sent from Outlook<http://aka.ms/weboutlook> ________________________________ From: hotspot-jfr-dev <hotspot-jfr-dev-retn@openjdk.java.net> on behalf of Erik Gahlin <egahlin@openjdk.java.net> Sent: Tuesday, May 17, 2022 9:47 PM To: hotspot-jfr-dev@openjdk.java.net <hotspot-jfr-dev@openjdk.java.net> Subject: Integrated: 8286706: JFR: 'jfr scrub' should overwrite output On Mon, 16 May 2022 14:04:43 GMT, Erik Gahlin <egahlin@openjdk.org> wrote:

Could I have a review of a change that makes the 'jfr scrub' overwrite the output file if it already exists.

Testing: /test/jdk/jdk/jfr/tool (Mac, Linux, Windows)

Thanks Erik

This pull request has now been integrated. Changeset: ab144190 Author: Erik Gahlin <egahlin@openjdk.org> URL: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.openjdk.java.net%2Fjdk%2Fcommit%2Fab144190c9951f2a9a3acf30db4b570484d5f751&data=05%7C01%7Cmatthew.carter%40microsoft.com%7C6f907133f0514919728708da3889a4f5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637884461083849921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zzPR1OA6evAeRWbmVBqXoOKgKb8wef7SVHZlxGF9xc4%3D&reserved=0 Stats: 35 lines in 2 files changed: 34 ins; 0 del; 1 mod 8286706: JFR: 'jfr scrub' should overwrite output Reviewed-by: mgronlun ------------- PR: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.openjdk.java.net%2Fjdk%2Fpull%2F8728&data=05%7C01%7Cmatthew.carter%40microsoft.com%7C6f907133f0514919728708da3889a4f5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637884461083849921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QZ1ptTTj5JDIcqDPmdmK4BWSqq3JLKHMzBhT%2BbrM%2FDs%3D&reserved=0

Hi Erik I agree that adding clip to scrub would make the commend cumbersome even though it would be performant. Answering your questions: Is the purpose to reduce the size of the recording to be able to open it in JMC?  Answer: yes this is one of the benefits Divide the recording into smaller pieces, for example, at most 5 minutes each so they are more convenient to work with?  Answer: again yes, while I considered a 'slice' command here (slice into 10 pieces or slice into 6 minute clips, this can easily be achieved with some scripting once the clip functionality exists [and is in fact what I've done locally]); also the naming of the slices likely wouldn't satisfy everyone and so to keep it simple lets not try this Pick out a certain time interval, for example where you have an indication the application/JVM didn't respond? Answer: yes, here I used three parameters to the clip command, they can be used individually and also in combinations of two; start, end and length (these where specified in s(econds), m(inutes). h(ours) and can have negative values in order to 'clip to last 5 minutes' etc). Might make sense to also support datetime whilst more cumbersome to specify manully, can help automated clipping of JFR files based on an event time Is there a need to use this in combination with options to limit the size, similar to exist today with jcmd <pid> JFR.dump Answer: I feel that if you already have a JFR file then making it smaller should not present any issues; also asking for a clip of length N and then getting a clip of length N-m makes it hard to programmatically 'slice' a clip, i..e you'd have to account for the events in the new clip to understand where to make the next slice to avoid losing data One artifact of the implementation to support 'scrub' is that the duration of the JFR file doesn't change, at first I thought this would be an issue but i actually like it esp. when visualizing in JMC (you can see where the clip was in the original recording). Hopefully that answers your questions so far Thanks again Mat Sent from Outlook From: Erik Gahlin <erik.gahlin@oracle.com> Sent: Monday, July 25, 2022 3:29 AM To: Mat Carter <Matthew.Carter@microsoft.com>; Erik Gahlin <egahlin@openjdk.java.net>; hotspot-jfr-dev@openjdk.java.net <hotspot-jfr-dev@openjdk.java.net> Subject: Re: Proposal: should we add a 'clip' command to the jfr tool?   Hi Mat, I think there is a use case for removing events with respect to time. A common use case for the 'disassemble' command today is to reduce the size of the recording, but it's unintuitive and cumbersome to use as it is based on chunks. I think the capability can be included in the existing 'scrub' command, but I can also see the use case for a separate command if 'scrub' gets too many options so it's hard for users to discover and understand how to use it. Time-based filtering has been mentioned before: https://mail.openjdk.org/pipermail/hotspot-jfr-dev/2022-February/003584.html Before discussing the implementation, it would be interesting to hear how you want to use it. Is the purpose to reduce the size of the recording to be able to open it in JMC? Divide the recording into smaller pieces, for example, at most 5 minutes each so they are more convenient to work with? Pick out a certain time interval, for example where you have an indication the application/JVM didn't respond? Is there a need to use this in combination with options to limit the size, similar to exist today with jcmd <pid> JFR.dump Thanks Erik From: Mat Carter <Matthew.Carter@microsoft.com> Sent: Tuesday, July 12, 2022 10:12 PM To: Erik Gahlin <egahlin@openjdk.java.net>; hotspot-jfr-dev@openjdk.java.net <hotspot-jfr-dev@openjdk.java.net> Subject: Proposal: should we add a 'clip' command to the jfr tool?   I'd like to propose that we add a command that filters based on time (clip perhaps) While working with large JFR files from jdk8 and jdk11 jvms I stumbled upon the 'jfr scrub' command that was introduced in 19. I used the 'jfr summary' to get a list of events and programmatically split the JFR into a JFR for each eventtype, but the data can still be unwieldy especially when converted to JSON. Given that 'jfr scrub' introduced the write(newJfrFile, predicate) method I wrote a java app to clip the JFR file based on a period of time (i hadn't seen the example in the JBS issue[1]), so now I can clip a JFR recording both horizontally (by events) and vertically (by time) and generate slices that are easier for JMC and other tools to parse (JMC would crash alot with large data) I took a stab at writing 'jfr clip' locally (based on scrub) and can share that in a new JBS issue if it's felt by this group that its a worthy addition to the jfr tool Again, I know that developers can write this themselves, but I feel it would be a nice convenience to have this in the jfr tool Alternatively, we could extend 'jfr scrub' to handle both event filters and a time region Cheers Mat [1] https://bugs.openjdk.org/browse/JDK-8271585 [JDK-8271585] JFR: Scrub recording data - Java Bug System JDK; JDK-8271585; JFR: Scrub recording data. Log In. Export bugs.openjdk.org Sent from Outlook From: hotspot-jfr-dev <hotspot-jfr-dev-retn@openjdk.java.net> on behalf of Erik Gahlin <egahlin@openjdk.java.net> Sent: Tuesday, May 17, 2022 9:47 PM To: hotspot-jfr-dev@openjdk.java.net <hotspot-jfr-dev@openjdk.java.net> Subject: Integrated: 8286706: JFR: 'jfr scrub' should overwrite output   On Mon, 16 May 2022 14:04:43 GMT, Erik Gahlin <egahlin@openjdk.org> wrote:

Could I have a review of a change that makes the 'jfr scrub' overwrite the output file if it already exists.

Testing: /test/jdk/jdk/jfr/tool  (Mac, Linux, Windows)

Thanks Erik

This pull request has now been integrated. Changeset: ab144190 Author:    Erik Gahlin <egahlin@openjdk.org> URL:       https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.openjdk.java.net%2Fjdk%2Fcommit%2Fab144190c9951f2a9a3acf30db4b570484d5f751&data=05%7C01%7Cmatthew.carter%40microsoft.com%7C6f907133f0514919728708da3889a4f5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637884461083849921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zzPR1OA6evAeRWbmVBqXoOKgKb8wef7SVHZlxGF9xc4%3D&reserved=0 Stats:     35 lines in 2 files changed: 34 ins; 0 del; 1 mod 8286706: JFR: 'jfr scrub' should overwrite output Reviewed-by: mgronlun ------------- PR: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.openjdk.java.net%2Fjdk%2Fpull%2F8728&data=05%7C01%7Cmatthew.carter%40microsoft.com%7C6f907133f0514919728708da3889a4f5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637884461083849921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QZ1ptTTj5JDIcqDPmdmK4BWSqq3JLKHMzBhT%2BbrM%2FDs%3D&reserved=0