Valhalla EG minutes June 05, 2019

[Karen Kinnear]

Attendees: Remi, Tobi, Dan H, Karen
AI: Remi - a request for a specific use case for indy with expression mode BSM if you could

I. Valhalla Generics
Remi: Are there any intermediate steps between LW10 Erased Generics and LW100 Specialized Generics?
e.g Can we get reification without specialization of constant pools or bytecodes?

Karen: Who needs to know? Is the generic signature sufficient as available to Reflection?
Dan: Current strategy: type arguments create a specialized class
  Is this similar to the token passed in with unsafe.DefineAnonymousClass?
Remi: (ed note - I can’t parse my notes)
Dan: At callsites, we do not have the type arguments
Remi: At creation site we need the type arguments. If we propagate the type arg when we create a new instance, that is enough by itself
Dan: We don’t have sufficient information to propagate the full type argument, the plan was to create an instance of a species
Remi: Still need a species, some values are not accessible from the constant pool
Karen: What do you mean by “without specialization” if you have species?
Remi: With full specialization, reference to a method requires the fully specialized type descriptor.
   The step before: There is access to the type arguments in the species, but no constant pool constants
depend on them. Think of the type args as stored in the CP.
Karen: If we don’t have species, then we need a side channel, and this is not an intermediate step.
Dan: It would be a lot of work to pass side channel information with “same type”. This work is not an intermediate step on the path.
Karen: What is the use case if you don’t have different fields and different methods?

Remi:
goal: support for other languages - so they can encode the type parameter system in
the constant pool and model other type systems
concern: specialization models java specific semantics in the bytecode

II. Records and Sealed Types - JEP https://bugs.openjdk.java.net/browse/JDK-8222777
Exploring for post-JDK13

IIa. Records
Karen summarized JVM perspective - but read the JEP for the real information
Records:
  data record
  compact syntax - class which is a transparent holder of shallowly immutable data
  declare state - looks like a constructor signature
      - automatically derive:
         private final field for each component, public constructor, equals/hashCode:(same type & same state), toString - with all components and names

JVMS changes:

Format of record_params attribute - initial prototype
  record_params_count
  Record_param record_params[count]

Record_param
   name_index
   access_flags
   descriptor_index
   signature_index

Expect to also allow annotations on components

Karen questions: 
   Are there any required consistencies between e.g. component access and e.g. field access?
   Are there any duplications of flags or attributes?

JVM only needs to know about record class information from the class file in order to pass this through to Reflection. There is no JVM enforcement at this time.

Remi: Will the Record attribute have any notion of order or guarantee order of fields?
Karen: NO - runtime determines layout
Remi: Will the order of components of the Record be the same as you get for Reflection?
Dan: Reflection spec says we do not guarantee a specific order
Remi: Not for the fields and methods, but for the components
   Can Reflection see the components at runtime?
Karen: yes
Remi: Can you get components both by order and by name?
  - like Scala/Kotlin/C#
Dan: Record declaration is syntactic sugar
Remi: But it is recorded as an attribute in the bytecode
Dan: mostly for javac
Remi: also fo Reflection
Remi: Want pattern matching - way to extract a value from a record, want to encode in bytecode by name or position (which requires guaranteed order)
    if only by name - we have binary compatibility, not source compatibility
Karen: I doubt the intention here is to provide positional referencing

     - Brian, do you have an answer for Remi? 

IIb. Sealed Types: same JEP: JEP https://bugs.openjdk.java.net/browse/JDK-8222777

Karen summary:

Sealed types: subclasses restricted according to guidance specified with the type’s declaration

User Model:
  “sealed” modifier with optional permits list
   - permits must be in the same module. If this is the unnamed module, permits must be in the same package
ed note: goal of this is to reduce separate compilation inconsistencies
   - if permits list is omitted: compiler can infer from the current compilation unit

Defaults:
   abstract subtypes of sealed types are implicitly sealed
   concrete subtypes of sealed types are implicitly final
You can explicitly modify the subtype with a “non-sealed” modifier

Clarification: sealed is shallow, only applies to direct subtypes

Prohibits subtypes which are nameless - i.e. java anonymous subclasses and lambdas (and future Lookup.defineClass - hidden/non-findable - whatever we end up calling it)

Remi: What about separate compilation? Can we ensure that the language and JVM constraints are the same?
Remi: Propose - must be in the same nest - since vm has no notion of compilation unit
Dan: It might not be intended that all the sealed subtypes have access to private members
Remi: Agreed - orthogonal: sealed and nestmates
Remi: Might want to restrict sealed to same compilation unit - or might not

Remi: Can you dynamically update the permit list?
Karen: I had the same question - today the answer is no
Remi: ok with no, please be explicit in the JEP not planning to support mocking

Karen: Propose we consider disallowing JVMTI redefineclasses to modify permitted subtype list

Karen: prototype - use ACC_FINAL and a new permitted attribute
   I was concerned about the concept of final for restricted subtyping
   I believe John was concerned that if a third-party tool loses the permitted attribute because they don’t yet support it, they are not likely to lose ACC_FINAL, so less risk, euro on side of secure.
Dan: Lost 1/2 of your information. If you add agents into the picture, this is not really a security argument
Remi: Asm: default keeps all attributes
Dan: Any new feature, tools must be updated
Remi: If a tool doesn’t recognize a class file version, it should not operate on it

Karen: runtime: reject at defineClass time attempts to load a subclass not in the permitted attribute
Remi: Not eagerly load permitted subtypes

III. Condy and Indy changes - BootstrapMethod new modes

Karen summary: From November 2017, we were exploring two new BSM modes. This is the current update.
Exploring for post-JDK13

User model goals:

New uses of BSMs:
I. Lookup/BSCI request: indy and condy
(BootStrapCallInfo (vm uses BCI as bytecode index))
Use cases:
1. Allow BSM authors to design extensible APIs in a binary compatible way, without having to have more and more bootstrap variants.
2. Allow a BSM that works with either indy/condy
  e.g. replace generation of indy with ldc for the case in which a lambda does not capture
  and the BSM would have returned a ConstantCallSite

  VM Request:
    rather than Lookup/String/TypeDescriptor/Object prefers BSM(Lookup/BSCI) for Condy
    with a static BSM method.
    (today - with eager resolution of static arguments)

II. Expression Mode BSM: No Lookup: condy only
1. Allow BSM authors to use regular method signatures, with no Lookup so APIs can be used in other places as well
   e.g. javac: generate condy for references to pattern extractors

III. Farther Future - lazy resolution of arguments (if asked)
use cases:
 - e.g. performance: BSM generates bytecodes, and BSM needs to reflect over e.g. MH/MT arguments. Could be useful to just deliver unresolved types (rather than resolve/revealDirect)
  - sometimes not all arguments are resolvable
We will explore possibilities here in future. ConstantDesc and subtypes offer a
way to handle this, as long as you can get a Lookup for resolution, so one approach
would be for the author to use Lookup/BSCI and return a ConstantDesc for lazy resolution.

===
JVMS Changes for BSM modes:

0. existing BSM(lookup, name, type, ...)
   always eager

1. BSCI
  Lookup, BSCI
  identified by 2 arguments, first is a Lookup
  for Condy and Indy
  initially all arguments statically resolved

2. BSM with no Lookup, name, type - Expression Mode
   For condy only
   identified by no initial lookup
   goal: ordinary static methods for BSMs

Dan: Do both modes apply to condy and indy?
Karen: BSM with no Lookup, name, type is currently targeted for condy only.

Remi: Does not like having the expression mode be for condy only.
use case for indy: Clojure runtime - would like to do more lazy loading of BSM static arguments for indy
another use case for indy for a constant - might want to add constraints or security checks each time you resolve
Karen: I think the only reason we did not plan expression mode for indy was no use cases.
Remi: For the goal of sharing a BSM for both condy and indy, you can do that today using Object as super class of MethodType

Karen: We are proposing to phase in BSM mode handling. The next step does not have a lazy resolution mode.
Note that the author of the BSM has a workaround to support lazy resolution now that ConstantDesc
exists, so we would like to see how far that can go.

Dan: What about class redefinition?
Karen: might want to think about disallowing removing BSM attribute entries with class redefinition.
Dan: Only allow adding BSM attribute entries?
Karen: we could either enforce this with the redefinition JVMTI spec or the vm might need
to implement it internally
Karen: We also need to think about add-exports which can change an error return into a success case
   Both the redefinition and add-exports issues get more complex with lazy resolution.
Dan: Last conversation with John about lazy resolution via Constable, the results were not written back to the constant pool.
Ed. note - still the case, although there is caching - need to study in more detail relative to result consistency expectations

Remi: Would like a Constable for String - so we could have a lazy string
Karen: I thought String was resolved like any primitive today. Why would you want that lazily resolved?
Remi: e.g. if the String is a secret that you don’t want stored in memory before resolved
Karen: you could use a class or in future an inline class - since you want a binary blob in the constant pool, rather than a String
Dan: This is not a feature you want


IV. Value Types static init factories and withfield
Tobi: what is the scope/restrictions for withfield?
Karen: nestmates I think
ed. note - double-checked: yes, restricted to nestmates
Remi: yes, you want withfield to work inside instance methods not just in factories

Corrections welcome,
thanks,
Karen