RFR(S) 8020675: Trying to print to a pdf file with PDFill PDF&Image Writer 10.0 Build 4

Calvin Cheung calvin.cheung at oracle.com
Wed Aug 21 11:49:58 PDT 2013


Can someone review this?

Latest version:
http://cr.openjdk.java.net/~ccheung/8020675/webrev/

Previous version without using TRAPS:
http://cr.openjdk.java.net/~ccheung/8020675/webrev.00/

thanks,
Calvin

On 8/12/2013 11:19 PM, Calvin Cheung wrote:
> Hi David,
>
> I cloned the hotspot repo corresponding to 7u25 b15 and debugged a bit 
> more and found that the error was thrown at a very early stage within 
> the init_globals() called from create_vm(). Below is the call stack:
>      jvm.dll!ClassLoader::create_class_path_entry(char * path, stat 
> st, ClassPathEntry * * new_entry, bool lazy)  Line 508    C++
>      jvm.dll!LazyClassPathEntry::resolve_entry()  Line 303 + 0x24 
> bytes    C++
>      jvm.dll!LazyClassPathEntry::open_stream(const char * name) Line 
> 322 + 0x8 bytes C++
>      jvm.dll!ClassLoader::load_classfile(Symbol * h_name, Thread * 
> __the_thread__)  Line 898 + 0x17 bytes    C++
>      jvm.dll!SystemDictionary::load_instance_class(Symbol * 
> class_name, Handle class_loader, Thread * __the_thread__)  Line 1362 + 
> 0x14 bytes    C++
> jvm.dll!SystemDictionary::resolve_instance_class_or_null(Symbol * 
> name, Handle class_loader, Handle protection_domain, Thread * 
> __the_thread__)  Line 758 + 0x18 bytes    C++
>      jvm.dll!SystemDictionary::resolve_or_null(Symbol * class_name, 
> Handle class_loader, Handle protection_domain, Thread * 
> __the_thread__)  Line 206 + 0x15 bytes    C++
>      jvm.dll!SystemDictionary::resolve_or_null(Symbol * class_name, 
> Thread * __the_thread__)  Line 211 + 0x23 bytes    C++
> jvm.dll!SystemDictionary::initialize_wk_klass(SystemDictionary::WKID 
> id, int init_opt, Thread * __the_thread__)  Line 1935 + 0xd bytes    C++
> jvm.dll!SystemDictionary::initialize_wk_klasses_until(SystemDictionary::WKID 
> limit_id, SystemDictionary::WKID & start_id, Thread * __the_thread__)  
> Line 1949 + 0x11 bytes    C++
>      jvm.dll!SystemDictionary::initialize_preloaded_classes(Thread * 
> __the_thread__)  Line 2011 + 0xf bytes    C++
>      jvm.dll!SystemDictionary::initialize(Thread * __the_thread__)  
> Line 1904 + 0x9 bytes    C++
>      jvm.dll!Universe::genesis(Thread * __the_thread__)  Line 353 + 
> 0x9 bytes    C++
>      jvm.dll!universe2_init()  Line 1009 + 0x9 bytes    C++
> >   jvm.dll!init_globals()  Line 114    C++
>      jvm.dll!Threads::create_vm(JavaVMInitArgs * args, bool * 
> canTryAgain)  Line 3220 + 0x5 bytes    C++
>      jvm.dll!JNI_CreateJavaVM(JavaVM_ * * vm, void * * penv, void * 
> args)  Line 5133 + 0xd bytes    C++
>      java.exe!011013bd()
>      [Frames below may be incorrect and/or missing, no symbols loaded 
> for java.exe]
>      java.exe!01101e2f()
>      java.exe!0110c807()
>      java.exe!0110a5a1()
>      java.exe!0110c845()
>      java.exe!0110a62b()
>      kernel32.dll!767633aa()
>      ntdll.dll!77739ef2()
>      ntdll.dll!77739ec5()
>
> The error was thrown in the method below:
> bool Exceptions::special_exception(Thread* thread, const char* file, 
> int line, Symbol* h_name, const char* message) {
>   // bootstrapping check
>   if (!Universe::is_fully_initialized()) {
>     if (h_name == NULL) {
>       // atleast an informative message.
>       vm_exit_during_initialization("Exception", message);
>     } else {
>       vm_exit_during_initialization(h_name, message);               
> <=====here
>     }
>     ShouldNotReachHere();
>   }
> }
>
> Note that it happened in the early stage during create_vm(), so the 
> (!Universe::is_fully_initialized()) was true.
>
> The class it was trying to load was sun/misc/AtomicLongCSImpl which I 
> couldn't find in 7u25.
> It was going through all the jar files in the bootclasspath and 
> finally got to the foo.jar which has 0-byte and got "error in opening 
> jar file" and then THROW_MSG().
> So I think it's just a coincident in 7u25 where the correct error was 
> thrown.
>
> In hs25, it didn't try to load the sun/misc/AtomicLongCSImpl class and 
> thus it didn't hit the "error in opening jar file" (for the foo.jar) 
> early until it passed the point when the vm is considered 
> "initialized" - is_init_completed() is true. So when THROW_MSG() was 
> called when it tried to load the class specified by the test case 
> ("xxx"), it triggered the fatal error in ~ExceptionMark(). Call stack 
> up to the THROWN_MSG() as follows:
>      jvm.dll!ClassLoader::create_class_path_entry(char * path, stat 
> st, ClassPathEntry * * new_entry, bool lazy)  Line 508    C++
>      jvm.dll!LazyClassPathEntry::resolve_entry()  Line 303 + 0x24 
> bytes    C++
> >    jvm.dll!LazyClassPathEntry::open_stream(const char * name)  Line 
> 322 + 0x8 bytes    C++
>      jvm.dll!ClassLoader::load_classfile(Symbol * h_name, Thread * 
> __the_thread__)  Line 900 + 0x17 bytes C++
>      jvm.dll!SystemDictionary::load_instance_class(Symbol * 
> class_name, Handle class_loader, Thread * __the_thread__)  Line 1301 + 
> 0x14 bytes    C++
> jvm.dll!SystemDictionary::resolve_instance_class_or_null(Symbol * 
> name, Handle class_loader, Handle protection_domain, Thread * 
> __the_thread__)  Line 779 + 0x18 bytes    C++
>      jvm.dll!SystemDictionary::resolve_or_null(Symbol * class_name, 
> Handle class_loader, Handle protection_domain, Thread * 
> __the_thread__)  Line 232 + 0x15 bytes    C++
>      jvm.dll!SystemDictionary::resolve_or_null(Symbol * class_name, 
> Thread * __the_thread__)  Line 237 + 0x23 bytes    C++
>      jvm.dll!JVM_FindClassFromBootLoader(JNIEnv_ * env, const char * 
> name)  Line 773 + 0x12 bytes    C++
>      java.dll!69461e8c()
>      [Frames below may be incorrect and/or missing, no symbols loaded 
> for java.dll]
>      jvm.dll!GrowableArray<Metadata *>::append(Metadata * const & 
> elem)  Line 207    C++
>      jvm.dll!os::write_memory_serialize_page(JavaThread * thread) Line 
> 375 + 0x11 bytes    C++
>      jvm.dll!InterfaceSupport::serialize_memory(JavaThread * thread)  
> Line 40 + 0x9 bytes    C++
>
> I'll try using TRAPS all the way through the call chain probably 
> starting from ClassLoader::load_classfile().
>
> thanks,
> Calvin
>
> On 8/11/2013 10:40 PM, David Holmes wrote:
>> Hi Calvin,
>>
>> I don't think this works. As I said previously the expectations of 
>> this code with regard to Java exceptions seems to be that it doesn't 
>> expect them at all. If you are using TRAPS etc then it should be used 
>> all the way through the call chain. What we have now is this call 
>> sequence:
>>
>> void ClassLoader::initialize() {
>>   assert(_package_hash_table == NULL, "should have been initialized 
>> by now.");
>>   EXCEPTION_MARK;
>>   ...
>>   setup_bootstrap_search_path();
>>     -> update_class_path_entry_list(path, false);
>>       ->  create_class_path_entry((char *)path, st, &new_entry, 
>> LazyBootClassLoader, CHECK);
>>
>> So if we return after the call to create_class_path_entry with an 
>> exception pending we will crash when we hit the EXCEPTION_MARK.
>>
>> It may be that the EXCEPTION_MARK needs replacing with an explicit 
>> exception check and a vm_exit_during_initialization, if this 
>> exception can readily occur at runtime. But further analysis of all 
>> this code needs to be undertaken.
>>
>> David
>> -----
>>
>> On 10/08/2013 8:11 AM, Calvin Cheung wrote:
>>> I've updated the webrev with 2 changes:
>>> 1) added the TRAPS as the last parameter to the
>>> create_class_path_entry() method;
>>> 2) added a jtreg test case.
>>>
>>> http://cr.openjdk.java.net/~ccheung/8020675/webrev/
>>>
>>> Please take a look again.
>>>
>>> One more response in-lined below...
>>>
>>> On 8/8/2013 4:11 PM, Ioi Lam wrote:
>>>> |I found one version of JDK7 that does not crash:||
>>>> ||
>>>> sc11136754 ~$
>>>> /net/jre.us.oracle.com/p/v31/jdk/7/all/b24/binaries/linux-amd64/bin/java 
>>>>
>>>> -showversion -cp . -Xbootclasspath/a:foo.jar test2
>>>> Error occurred during initialization of VM
>>>> java/lang/ClassNotFoundException: error in opening JAR file foo.jar
>>>> |
>>>
>>> |I noticed that it started breaking in 7u40 b01; it was still working
>>> with 7u25.
>>> It may have something to do with when the set_init_completed() is 
>>> called:
>>> ExceptionMark::~ExceptionMark() {
>>>    if (_thread->has_pending_exception()) {
>>>      Handle exception(_thread, _thread->pending_exception());
>>>      _thread->clear_pending_exception(); // Needed to avoid infinite
>>> recursion
>>>      if (is_init_completed()) {
>>>        exception->print();
>>>        fatal("ExceptionMark destructor expects no pending exceptions");
>>>      } else {
>>>        vm_exit_during_initialization(exception);
>>>      }
>>>    }
>>> }
>>>
>>> Before 7u40, I think the code path got to the "else" branch of
>>> ~ExceptionMark() and we just printed an error and exit.
>>>
>>> set_init_completed() is only called from Threads::create_vm() and that
>>> method didn't change much between 7u25 and 7u40 except for some NMT
>>> initialization - MemTracker::bootstrap_single_thread(),
>>> MemTracker::start(), etc. But I thought NMT isn't enabled by default?
>>>
>>> Debugging with hs25, the set_init_completed() always happened before
>>> create_class_path_entry() and both calls were done within the same
>>> thread - "vm startup".
>>>
>>> thanks,
>>> Calvin
>>>
>>>
>>> |
>>>> |||
>>>> ||
>>>> ||- Ioi
>>>> |
>>>> On 08/08/2013 02:26 PM, Ioi Lam wrote:
>>>>> |I found an official version that's closest to the Redhat version
>>>>> (OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode). It still
>>>>> crashes.||
>>>>> ||
>>>>> ||sc11136754 ~$
>>>>> /java/re/jdk/6_25/latest/binaries/linux-amd64/bin/java -showversion
>>>>> -cp . -Xbootclasspath/a:foo.jar test2||
>>>>> ||java version "1.6.0_25"||
>>>>> ||Java(TM) SE Runtime Environment (build 1.6.0_25-b06)||
>>>>> ||Java HotSpot(TM) 64-Bit Server VM (build 20.0-b11, mixed mode)||
>>>>> ||
>>>>> ||java.lang.ClassNotFoundException ||
>>>>> || - klass: 'java/lang/ClassNotFoundException'||
>>>>> ||#||
>>>>> ||# A fatal error has been detected by the Java Runtime 
>>>>> Environment:||
>>>>> ||#||
>>>>> ||#  Internal Error (exceptions.cpp:397), pid=29955,
>>>>> tid=140608485558016||
>>>>> ||#  fatal error: ExceptionMark destructor expects no pending
>>>>> exceptions||
>>>>> ||#||
>>>>> ||# JRE version: 6.0_25-b06||
>>>>> ||# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode
>>>>> linux-amd64 compressed oops)||
>>>>> ||# An error report file with more information is saved as:||
>>>>> ||# /home/iklam/hs_err_pid29955.log||
>>>>> ||#||
>>>>> ||# If you would like to submit a bug report, please visit:||
>>>>> ||# http://java.sun.com/webapps/bugreport/crash.jsp||
>>>>> ||#||
>>>>> ||Aborted (core dumped)||
>>>>> |
>>>>> - Ioi
>>>>>
>>>>> On 08/08/2013 02:18 PM, David Holmes wrote:
>>>>>> On 9/08/2013 4:52 AM, Ioi Lam wrote:
>>>>>>> |Hmmm, I used JDK6 from OEL 6.0 (derived from RedHat), which
>>>>>>> apparently
>>>>>>> works differently
>>>>>>> (better?) than the official JDK6 build. Maybe Redhat fixed this in
>>>>>>> their
>>>>>>> own repo??||
>>>>>>
>>>>>> Their hotspot version is much later - hs20 vs hs17
>>>>>>
>>>>>> David
>>>>>> -----
>>>>>>
>>>>>>> ||
>>>>>>> |
>>>>>>> |==OEL6================================================
>>>>>>> ||
>>>>>>> ||sc11136754 ~$ uname -a||
>>>>>>> ||Linux sc11136754 2.6.39-100.5.1.el6uek.x86_64 #1 SMP Tue Mar 6
>>>>>>> 20:26:00 EST 2012 x86_64 x86_64 x86_64 GNU/Linux||
>>>>>>> ||sc11136754 ~$ type java||
>>>>>>> ||java is hashed (/usr/bin/java)||
>>>>>>> ||sc11136754 ~$ java -version||
>>>>>>> ||java version "1.6.0_22"||
>>>>>>> ||OpenJDK Runtime Environment (IcedTea6 1.10.4)
>>>>>>> (rhel-1.41.1.10.4.el6-x86_64)||
>>>>>>> ||OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)||
>>>>>>> ||sc11136754 ~$ java -showversion -cp . -Xbootclasspath/a:foo.jar
>>>>>>> test2||
>>>>>>> ||Error occurred during initialization of VM||
>>>>>>> ||java/lang/ClassNotFoundException: error in opening JAR file
>>>>>>> foo.jar||
>>>>>>> ||
>>>>>>> ==OFFICIAL============================================
>>>>>>> ||
>>>>>>> sc11136754 ~$ 
>>>>>>> /java/re/jdk/6_22/latest/binaries/linux-amd64/bin/java
>>>>>>> -showversion -cp . -Xbootclasspath/a:foo.jar test2
>>>>>>> java version "1.6.0_22"
>>>>>>> Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode)
>>>>>>>
>>>>>>> #
>>>>>>> # A fatal error has been detected by the Java Runtime Environment:
>>>>>>> #
>>>>>>> #  Internal Error (exceptions.cpp:367), pid=25167, 
>>>>>>> tid=140510319580928
>>>>>>> #  Error: ExceptionMark destructor expects no pending exceptions
>>>>>>> #
>>>>>>> # JRE version: 6.0_22-b04
>>>>>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (17.1-b03 mixed mode
>>>>>>> linux-amd64 )
>>>>>>> # An error report file with more information is saved as:
>>>>>>> # /home/iklam/hs_err_pid25167.log
>>>>>>> #
>>>>>>> # If you would like to submit a bug report, please visit:
>>>>>>> # http://java.sun.com/webapps/bugreport/crash.jsp
>>>>>>> #
>>>>>>> |
>>>>>>> |======================================================
>>>>>>> ||
>>>>>>> ||- Ioi|
>>>>>>>
>>>>>>> On 08/08/2013 11:08 AM, Calvin Cheung wrote:
>>>>>>>> Hi Ioi, David,
>>>>>>>>
>>>>>>>> On 8/7/2013 7:15 PM, Ioi Lam wrote:
>>>>>>>>> |JDK6 seems to handle this better. I have written a simple
>>>>>>>>> stand-alone test case that doesn't require truncating JAR 
>>>>>>>>> files in
>>>>>>>>> the JDK:||
>>>>>>>>> ||
>>>>>>>>> ||=========================||
>>>>>>>>> ||/*||
>>>>>>>>> ||javac test2.java||
>>>>>>>>> ||rm -f foo.jar||
>>>>>>>>> ||java -cp . -Xbootclasspath/a:foo.jar test2||
>>>>>>>>> ||touch foo.jar||
>>>>>>>>> ||java -cp . -Xbootclasspath/a:foo.jar test2||
>>>>>>>>> ||*/||
>>>>>>>>> ||
>>>>>>>>> ||public class test2 {||
>>>>>>>>> ||    public static void main(String args[]) {||
>>>>>>>>> ||        try {||
>>>>>>>>> ||System.out.println(Class.forName("javax.crypto.MacSpi"));||
>>>>>>>>> ||System.out.println(Class.forName("xxx"));||
>>>>>>>>> ||        } catch (Throwable t) {||
>>>>>>>>> ||            t.printStackTrace();||
>>>>>>>>> ||        }||
>>>>>>>>> ||    }||
>>>>>>>>> ||}||
>>>>>>>>> ||=========================||
>>>>>>>>> | |
>>>>>>>>> ||JDK6 works fine, but JDK7 would crash with the second "java 
>>>>>>>>> -cp .
>>>>>>>>> -Xbootclasspath/a:foo.jar test2" command.||
>>>>>>>>> |
>>>>>>>> |I saw crash with the latest 6 update release with both test 
>>>>>>>> cases.
>>>>>>>> The crash seems to be at the same spot.
>>>>>>>> |
>>>>>>>>> |||
>>>>>>>>> ||So I think the correct fix is to restore the JDK6 behavior (not
>>>>>>>>> sure if it's already the same with your patch, but worth 
>>>>>>>>> checking),
>>>>>>>>> and also add a new jtreg test into hotspot.||
>>>>>>>>> |
>>>>>>>> |I checked the jdk 6 code and those 3 methods which I changed look
>>>>>>>> the
>>>>>>>> same as jdk 8.
>>>>>>>> Sure, I can add a jtreg test.
>>>>>>>> |
>>>>>>>>> |||
>>>>>>>>> ||Thanks||
>>>>>>>>> ||- Ioi||
>>>>>>>>> ||
>>>>>>>>> ||On 08/07/2013 06:47 PM, David Holmes wrote:||
>>>>>>>>> |
>>>>>>>>>> |Hi Calvin, ||
>>>>>>>>>> | |
>>>>>>>>>> ||It seems to me that this code has a general problem with
>>>>>>>>>> exceptions. If exceptions can be posted then methods should be
>>>>>>>>>> declared with a last parameter of TRAPS and called with a CHECK_
>>>>>>>>>> macro. The way this code is written it does not seem to 
>>>>>>>>>> expect any
>>>>>>>>>> exceptions to be possible. I would check with Karen on this. ||
>>>>>>>>>> |
>>>>>>>> |I was thinking about that but that would involves a bit more
>>>>>>>> changes.
>>>>>>>> I can give it a try.
>>>>>>>> |
>>>>>>>>>> || |
>>>>>>>>>> ||This addition seems unused: ||
>>>>>>>>>> | |
>>>>>>>>>> ||+   Thread* THREAD = thread; ||
>>>>>>>>>> |
>>>>>>>> |It is required for the THROW_MSG().
>>>>>>>> It won't be needed if the method is declared with TRAPS as the 
>>>>>>>> last
>>>>>>>> parameter (I think).
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Calvin
>>>>>>>> |
>>>>>>>>>> || |
>>>>>>>>>> ||Thumbs up on the removal of the EXCEPTION_MARKs prior to 
>>>>>>>>>> throwing
>>>>>>>>>> exceptions - don't know what the thought process was there :) ||
>>>>>>>>>> | |
>>>>>>>>>> ||David ||
>>>>>>>>>> | |
>>>>>>>>>> ||On 8/08/2013 10:25 AM, Calvin Cheung wrote: ||
>>>>>>>>>> |
>>>>>>>>>>> |Please review this small fix for fixing a fatal error in ||
>>>>>>>>>>> ||~ExceptionMark() triggered by ClassNotFoundException in ||
>>>>>>>>>>> ||ClassLoader::create_class_path_entry(). I could reproduce
>>>>>>>>>>> similar
>>>>>>>>>>> crash ||
>>>>>>>>>>> ||by trying to load a class from an empty jar which is in 
>>>>>>>>>>> the ||
>>>>>>>>>>> ||bootclasspath. The following program can trigger the crash if
>>>>>>>>>>> the ||
>>>>>>>>>>> ||jce.jar is invalid (e.g. 0-byte): ||
>>>>>>>>>>> | |
>>>>>>>>>>> ||public class TestForName { ||
>>>>>>>>>>> ||     public static void main(String[] args) { ||
>>>>>>>>>>> ||         try { ||
>>>>>>>>>>> ||             Class cls =
>>>>>>>>>>> Class.forName("javax.crypto.KeyGenerator"); ||
>>>>>>>>>>> ||             System.out.println("Class = " + 
>>>>>>>>>>> cls.getName()); ||
>>>>>>>>>>> ||         } catch (ClassNotFoundException cnfe) { ||
>>>>>>>>>>> ||             cnfe.printStackTrace(); ||
>>>>>>>>>>> ||         } ||
>>>>>>>>>>> ||     } ||
>>>>>>>>>>> ||} ||
>>>>>>>>>>> | |
>>>>>>>>>>> ||The fix also changes the assert() in
>>>>>>>>>>> LazyClassPathEntry::resolve_entry() ||
>>>>>>>>>>> ||to a NULL check. Otherwise, the assert() will fail under the
>>>>>>>>>>> above test ||
>>>>>>>>>>> ||scenario. ||
>>>>>>>>>>> | |
>>>>>>>>>>> ||With the fix, the following ClassNotFoundException will be
>>>>>>>>>>> thrown ||
>>>>>>>>>>> ||instead of a crash: ||
>>>>>>>>>>> ||java.lang.ClassNotFoundException: 
>>>>>>>>>>> javax.crypto.KeyGenerator ||
>>>>>>>>>>> ||         at
>>>>>>>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:365) ||
>>>>>>>>>>> ||         at
>>>>>>>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:354) ||
>>>>>>>>>>> ||         at 
>>>>>>>>>>> java.security.AccessController.doPrivileged(Native
>>>>>>>>>>> Method) ||
>>>>>>>>>>> ||         at
>>>>>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:353) ||
>>>>>>>>>>> ||         at
>>>>>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:423) ||
>>>>>>>>>>> ||         at
>>>>>>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) || 
>>>>>>>>>>>
>>>>>>>>>>> ||         at
>>>>>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:356) ||
>>>>>>>>>>> ||         at java.lang.Class.forName0(Native Method) ||
>>>>>>>>>>> ||         at java.lang.Class.forName(Class.java:258) ||
>>>>>>>>>>> ||         at TestForName.main(TestForName.java:6) ||
>>>>>>>>>>> | |
>>>>>>>>>>> ||webrev: 
>>>>>>>>>>> http://cr.openjdk.java.net/~ccheung/8020675/webrev/ ||
>>>>>>>>>>> | |
>>>>>>>>>>> ||JBS: https://jbs.oracle.com/bugs/browse/JDK-8020675 ||
>>>>>>>>>>> ||bug: http://bugs.sun.com/view_bug.do?bug_id=8020675 ||
>>>>>>>>>>> | |
>>>>>>>>>>> ||Tests: ||
>>>>>>>>>>> ||     jprt, vm.quick.testlist ||
>>>>>>>>>>> | |
>>>>>>>>>>> ||thanks, ||
>>>>>>>>>>> ||Calvin ||
>>>>>>>>>>> | |
>>>>>>>>>>> | |
>>>>>>>>>>> |
>>>>>>>>> |
>>>>>>>>> |
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>



More information about the hotspot-runtime-dev mailing list