CDS and JVM-TI agent questions and crash

[Michael Rasmussen]

Hi

I have some questions regarding Class Data Sharing and class file transforming agents:

1. Is it possible to detect during Agent_OnLoad if CDS is used or not?
Checking for "sharing" in the "java.vm.info" system property, which seems to be how the tests are detecting it, is not an option, as this property contains bogus information during Agent_OnLoad.

2. Should the registration of a ClassFileHook, especially with can_generate_all_class_hook_events and can_generate_early_class_hook_events capabilities set, disable CDS?

3. Is it possible to explicitly disable CDS from a JVM-TI agent during Agent_OnLoad?


The reason for these questions is that we (JRebel) recently became aware of an issue if CDS is used, where transforming some of the boot classes causes the JVM to crash.
The simplest example of this is replacing Object's byte in a ClassFileLoadHook with a copy of themselves, see inlined example agent code below.
I don't know if there are more issues than that, as that's where we are currently stuck.

Running on a 11+28 fastdebug (with classes.jsa file present) it produces the following assert:
#  Internal Error (.../src/hotspot/share/oops/constantPool.cpp:325), pid=589, tid=590
#  assert(on_stack()) failed: should always be set for shared constant pools

if running with -Xshare:off (or no classes.jsa file), it works without any issue.

Kind regards
Michael Rasmussen

/* --- snip --- */
#include <string.h>
#include <jni.h>
#include <jvmti.h>

void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env,
	jclass class_being_redefined, jobject loader, const char* name, jobject protection_domain,
	jint class_data_len, const unsigned char* class_data,
	jint* new_class_data_len, unsigned char** new_class_data) {

	if (strcmp("java/lang/Object", name) == 0) {
		*new_class_data_len = class_data_len;
		(*jvmti)->Allocate(jvmti, *new_class_data_len, new_class_data);
		memcpy(*new_class_data, class_data, *new_class_data_len);
	}
}

JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) {
	jvmtiEnv *jvmti = NULL;
	jint rc = (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9);
	
	jvmtiCapabilities caps;
	memset(&caps, 0, sizeof(caps));

	caps.can_redefine_classes = 1;
	caps.can_generate_all_class_hook_events = 1;
	caps.can_generate_early_vmstart = 1;
	caps.can_generate_early_class_hook_events = 1;

	(*jvmti)->AddCapabilities(jvmti, &caps);

	jvmtiEventCallbacks callbacks;
	memset(&callbacks, 0, sizeof(callbacks));
	callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook;
	(*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks));
	(*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL);

	return 0;
}
/* --- snap --- */