jdk.Checkpoint

[Erik Gahlin]

Hi Michael,

See comments inline

> On 23 Feb 2020, at 00:36, Michael Bien <mbien42 at gmail.com> wrote:
> 
> Hi Erik,
> 
> thank you for the detailed response
> 
> On 22.02.20 11:34, Erik Gahlin wrote:
>> Hi Michael,
>> 
>> jdk.CheckPoint is an event users can't consume or manipulate. It is hard-wired into the file format and only visible to the ‘jfr’ tool (for debugging purposes).
>> 
>> The event contains mapping to constants. For example, if an event has a stack trace, only the ID for the stack trace is written. Later in the file, a check point event is written that maps the ID to the stack trace This is to reduces the size of the recording file.
>> 
>> With JDK 14, we introduced Event Streaming [1] which allow clients to attach to the disk repository of a running JVM (in-process and out-of-process) and continuously stream events. This feature is always on and data is flushed once every second. With each flush a check point event is written. These events adds up over a day (86 400 seconds).
> 
> Interesting. I had the feeling that it might have something to do with synchronization for event streaming. I suppose the API is only allowed to consume events up until the most recent checkpoint, then it waits for the next and so on.

The parser looks at the file header, where it can read how much data is safe to parse. It is updated by the JVM continuously. There is a protocol to make sure word tearing doesn’t happen when the value is read. 

If there are no new IDs, it would be possible to avoid checkpoint events.

That said, plan is to add support for streaming over JMX and without going into too much detail, we write the full file header (68 bytes) in a checkpoint event, so we can simplify synchronization for a future remote client that can’t read the file header on disk.

> 
>> 
>> Do you see a problem with the additional 8 MB of data, or are you just curious?
> 
> I was curious for the most part. With the default profile i probably wouldn't have even noticed it. But i came from the perspective of evaluating if JFR could be used as logging replacement. The good news is that it is static uptime footprint and it doesn't increase with load which is great. But on the other hand you could put a lot of log lines into ~8MB daily.
> 
> Maybe it would be possible to consider making the flush/checkpoint interval configurable for use cases where the realtime nature of event streaming is less important.

There is an undocumented command line flag that can be used to increase the interval. It was put in there as a workaround, if users would run into issues compared to JDK 13. Plan is to remove the flag once we know there are no issues. 

This is why I am curious about your use case.

We didn’t expect anybody to notice, especially not before JDK 14 was released :)

There are other things than can be done, for instance, only write the file header checkpoint if a JMX client has actually connected and requested a stream. 

Anyway, this is good feedback.

> Or maybe this is the wrong approach and I should only view the JFR records as log buffer and use streaming to extract what is needed for compact storage.

The file format is pretty compact, so it is unfortunate that the checkpoint event eats up so much space in your scenario. You could try to zip the recording file. If the compression algorithm is good, it should be able to reduce the 8 MB significantly due to all the redundancy.

Thanks
Erik

> 
>> 
>> Our thinking was that it would be harmless.
> 
> mostly harmless :)
> 
>> 
>> Thanks
>> Erik
> 
> thanks again,
> 
> michael
> 
>> [1] https://openjdk.java.net/jeps/349
>> 
>> 
>>> On 22 Feb 2020, at 09:30, Michael Bien <mbien42 at gmail.com> wrote:
>>> 
>>> Hello,
>>> 
>>> i was testing JFR with custom events using a mostly empty custom.jfc file and noticed a large amount of jdk.CheckPoint events in the resulting repository and dump files (both count and size wise) after a few hours of uptime (using JDK 14). I tried to explicitly disable the event in the profile config but it didn't help. It seems to be around 8MB of jdk.CheckPoint events per day for a mostly idling app server.
>>> 
>>> Accessing the event with the jfr commandline tool or API didn't work either and I couldn't find any other information on it.
>>> 
>>> What is it for? Some implementation detail needed for the file structure?
>>> 
>>> best regards,
>>> 
>>> michael
>>> 
>