<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks for the file.</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
I worry that processing the file in the JVM or creating an intuitive Java API for post-processing it will be hard. The context/event determines what needs to be redacted. If scrubbing is only necessary for these four events, hardcoding the sensitive tokens
and logic into the JVM might be a viable approach.</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Users would specify:</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
$ java -XX:FlightRecorderOptions:scrub-sensitive=true</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
or it might be enabled by default and users would need to opt-out.</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Anyway, if enabled, a jfrScrub.cpp class would do the job. Something like this:</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
EventInitialEnvironmentVariable event(UNTIMED);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_starttime(time_stamp);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_endtime(time_stamp);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_key(key);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
if (JfrScrub::is_sensitive_key(key)) {</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_value("[REDACTED]");</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
} else {</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_value(value);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
}</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.commit();</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
EventInitialSystemProperty event(UNTIMED);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_key(p->key());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
if (JfrScrub::is_sensitive_key(p->key()) {</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_value("[REDACTED]");</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
} else {</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_value(p->value());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
}</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_starttime(time_stamp);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_endtime(time_stamp);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.commit();</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
EventSystemProcess event(UNTIMED);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_pid(pid_buf);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_commandLine(JfrScrub::command_line(info));</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_starttime(start_time);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_endtime(end_time);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.commit();</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
EventJVMInformation event;</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_jvmName(VM_Version::vm_name());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_jvmVersion(VM_Version::internal_vm_info_string());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_javaArguments(JfrScrub::command_line(Arguments::java_command()));</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_jvmArguments(Arguments::jvm_args());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_jvmFlags(Arguments::jvm_flags());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_jvmStartTime(Management::vm_init_done_time());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.set_pid(os::current_process_id());</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
event.commit();</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="padding-left: 2px;"></div>
<div class="elementToProof" style="padding-left: 2px;"></div>
<div class="elementToProof" style="padding-left: 2px;"></div>
<div class="elementToProof" style="padding-left: 2px;"></div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
It's a bit ugly and not as flexible, but perhaps that's something we need to tolerate. Or will it be useless because new passwords/keys will be added all the time, or because they will match false positives, and more advanced logic is needed? Perhaps it will
give users the false(?) impression that they don't need to worry about sensitive data?</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Erik</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Erwan Viollet <erwan.viollet@gmail.com><br>
<b>Sent:</b> Thursday, June 12, 2025 5:07 PM<br>
<b>To:</b> Erik Gahlin <erik.gahlin@oracle.com><br>
<b>Cc:</b> hotspot-jfr-dev@openjdk.org <hotspot-jfr-dev@openjdk.org><br>
<b>Subject:</b> [External] : Re: JFR: Scrubbing sensitive information from events</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Hello,</div>
<div><br>
</div>
<div dir="ltr">Here is an example of the types of events we are concerned about:<br>
<br>
<font size="1"><font face="monospace">Recording</font><br>
<font face="monospace">©¦<br>
©À©¤©¤ Event (e.g. jdk.InitialSystemProperty)<br>
©¦ ©À©¤©¤ eventType: "jdk.InitialSystemProperty"<br>
©¦ ©À©¤©¤ startTime<br>
©¦ ©À©¤©¤ duration<br>
©¦ ©À©¤©¤ fields:<br>
©¦ ©¦ ©À©¤©¤ key: "javax.net.ssl.keyStorePassword"<br>
©¦ ©¦ ©À©¤©¤ value: "<b>supersecret</b>"<br>
©¦ ©¦ ©¸©¤©¤ ...<br>
©¦ ©¸©¤©¤ ...<br>
©¦<br>
©À©¤©¤ Event (e.g. jdk.JVMInformation)<br>
©¦ ©À©¤©¤ eventType: "jdk.JVMInformation"<br>
©¦ ©À©¤©¤ jvmArguments: [ "-Xmx4G", "-Djavax.net.ssl.keyStorePassword=<b>supersecret</b>", ... ]<br>
©¦ ©¸©¤©¤ ...<br>
©¦<br>
©¸©¤©¤ ...</font></font></div>
<div dir="ltr"><br>
The rules are slightly challenging as they need to account for key/value pairs, arrays and simple fields (like commandLine field). </div>
<div dir="ltr"><a href="https://urldefense.com/v3/__https://gist.github.com/r1viollet/812ed70c6410c4f62640fd792570d36c__;!!ACWV5N9M2RV99hQ!LrFg9xF9Jy2l4LW6sj6mxVPhLXr30tA_2lzstCSiBbi4SxLyh8t2wDJGc4E1b7ePKrrsivDhkoZtMsHWCKsBeBLJ$">Here</a> is a scrub file example.
I'm happy to consider ways to simplify this proposal. Storing JFR files would also be helpful to consider test cases.</div>
<div>Regards,</div>
<div><br>
</div>
<div>Erwan</div>
<div dir="ltr"><br>
</div>
<br>
<div class="x_gmail_quote x_gmail_quote_container">
<div dir="ltr" class="x_gmail_attr">Le mar. 3 juin 2025 ¨¤ 11:50, Erik Gahlin <<a href="mailto:erik.gahlin@oracle.com">erik.gahlin@oracle.com</a>> a ¨¦crit :<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div class="x_msg9147689327593888414">
<div dir="ltr">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
We have discussed it, but we don't understand all the details. We are also unsure how to best expose it to the end user. Let's say there was a command line option -XX:FlightRecorder:scrub-file=<file>.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
What would you fill that file with? I want examples that work on real data to understand how expressive the filters must be.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thanks</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Erik</div>
<div id="x_m_9147689327593888414appendonsend"></div>
<hr style="display:inline-block; width:98%">
<div id="x_m_9147689327593888414divRplyFwdMsg">
<div style="direction:ltr; font-family:Calibri,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<b>From:</b> hotspot-jfr-dev <<a href="mailto:hotspot-jfr-dev-retn@openjdk.org" target="_blank">hotspot-jfr-dev-retn@openjdk.org</a>> on behalf of Erwan Viollet <<a href="mailto:erwan.viollet@gmail.com" target="_blank">erwan.viollet@gmail.com</a>><br>
<b>Sent:</b> Monday, June 2, 2025 3:30 PM<br>
<b>To:</b> <a href="mailto:hotspot-jfr-dev@openjdk.org" target="_blank">hotspot-jfr-dev@openjdk.org</a> <<a href="mailto:hotspot-jfr-dev@openjdk.org" target="_blank">hotspot-jfr-dev@openjdk.org</a>><br>
<b>Subject:</b> JFR: Scrubbing sensitive information from events</div>
<div style="direction:ltr"> </div>
</div>
<div style="direction:ltr">Hello,</div>
<div style="direction:ltr"><br>
</div>
<div style="direction:ltr">I am currently looking into how to remove sensitive information from JFR events. The main events that typically contain sensitive information: jdk.SystemProcess, jdk.InitialSystemProperty, jdk.JVMInformation. Passwords from command
lines can typically be found in these events.</div>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">Dropping these events altogether is not ideal, as we need them to make relevant performance recommendations to users (e.g. suggesting JVM or system setting adjustments).</p>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">Dropping them or scrubbing them on the backend side (after the fact) requires decompressing and re-writing these events, which is wasteful in terms of both compute and storage. The approach is not
perfect, as we still end up intaking and temporarily storing sensitive information.</p>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">Ideally, we would like to be able to scrub or redact only the sensitive fields within these events (for example, using a simple regex or pattern-based rule), rather than dropping the whole event. We
also want to avoid handling this only after the event has already been written to the JFR file, as that does not fully mitigate the risk of exposing sensitive data.</p>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">At present, it appears there is no public API or supported mechanism to intercept or scrub JFR events in-process, before they are persisted. What would you think of an API accepting custom scrubbing
patterns so that sensitive data never leaves the JVM in an unredacted state?</p>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">Are there any plans or discussions in this area? I am fairly new to the JFR world, so it is likely that I missed previous discussions around this.</p>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">Thank you, Best regards,</p>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">Erwan Viollet, </p>
<p style="direction:ltr; margin-top:0px; margin-bottom:0px">Profiling team, Datadog</p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</body>
</html>