jtreg testing integrated

[Joe Darcy]

Mark Wielaard wrote:
> Hi Joe,
>
> On Wed, 2008-05-21 at 11:37 -0700, Joseph D. Darcy wrote:
>   
>

[snip]

>> On a related note, there are times where it could be useful for a test 
>> to have somewhat different behavior on Sun's production JDK versus 
>> OpenJDK.  For example, I'm the author of the 
>> java/lang/reflect/Generics/Probe.java test which fails on OpenJDK since 
>> it references some production-only classes.  There are a few options to 
>> fix this.  One way would be to remove all reference to the 
>> production-only classes, which would reduce the usefulness of the test 
>> for the production code; conversely, the whole test could be moved to 
>> closed, reducing the test coverage of OpenJDK.  The test could be split 
>> into open and closed parts, complicating maintenance.  So instead I 
>> changed the test to only look at the production classes for a production 
>> build.  I test for a production build using
>>
>> System.getProperty("java.runtime.name").startsWith("Java(TM)")
>>
>> Out of the box, our OpenJDK builds have "java.runtime.name" start with 
>> "OpenJDK"; is that true in your IcedTea/OpenJDK builds too?
>>     
>
> It depends on the version and the distribution. IcedTea6 now calls
> itself OpenJDK, but IcedTea7 calls itself IcedTea. Ubuntu has a patch to
> return "Java(TM)" here. And we might actually might want to make that
> the default. 

I doubt Sun's lawyers would be happy with that!

> There are already bug reports about applications and
> applets testing for this string to be exactly in the way as the
> proprietary implementations (you could argue that is a bug in that
> application/applet, but users don't really care unfortunately).
>
> Whenever possible I think we should make this configurable on a specific
> test property. That is more work, but allows all derivatives, not just
> Sun's ClosedJDK to use the same tests with their own settings. The
> defaults should of course be what works with generic upstream OpenJDK.
>
> I was looking at this particular test and I didn't immediately see why
> javax.crypto.SunJCE_x couldn't be replaced with a commonly available
> class in OpenJDK. What is special about those classes that they need to
> be explicitly tested against?
>   

The crypto classes in Sun's product are obfuscated, US government 
requirement, and the obfuscators have been know to corrupt the signature 
attribute used by the reflection methods in question.

>   
>> For the javax/script tests, I was going to add "if (no javascript engine 
>> and production JDK) fail else note/warning" logic so that our production 
>> build gets the same coverage and if an OpenJDK build come with a 
>> javascript engine, it would get testing too.
>>     
>
> What do you precisely mean when you say that you have an "if production
> JDK"? What functional feature does that test against?
>   

I use the value in that system property.  In a helper class I define

  32     public static ScriptEngine getJsEngine(ScriptEngineManager m) {
  33         ScriptEngine e  = m.getEngineByName("js");
  34         if (e == null &&
  35             
System.getProperty("java.runtime.name").startsWith("Java(TM)")) {
  36             // A js engine is requied for Sun's product JDK
  37             throw new RuntimeException("no js engine found");
  38         }
  39         return e;
  40     }

and then the tests in question use

  40             ScriptEngine jsengine = Helper.getJsEngine(manager);
  41             if (jsengine == null) {
  42                 System.out.println("Warning: No js engine found; 
test vacuously passes.");
  43                 return;
  44             }
  45  // Rest of test...

-Joe