Behavior Proposal v2

Andy Goryachev andy.goryachev at oracle.com
Fri Nov 17 20:48:51 UTC 2023 

Thank you, John, for a productive discussion!  I am very happy to see that we both gained a better understanding of the problems we are trying to solve.

For me, some of the insights came to light after our conversation.  To complement John’s list:

- in cases where the behavior part is stateless (or state is fully contained within the control), the behavior can indeed be implemented statically, potentially saving memory.

- we might still have a problem with event handling priority, so addressing that (along the lines of https://github.com/openjdk/jfx/pull/1266 or in some other way) is likely a prerequisite

- perhaps the InputMap can be split into two parts (user- and skin-), providing a better API and enforcing the separation of concerns.

I would like to thank you both, John and Michael, for your patience and discussion!  I am going to re-do my proposal to incorporate your feedback, hopefully before the Thanksgiving break next week.

-andy


From: openjfx-dev <openjfx-dev-retn at openjdk.org> on behalf of John Hendrikx <john.hendrikx at gmail.com>
Date: Friday, November 17, 2023 at 05:35
To: openjfx-dev at openjdk.org <openjfx-dev at openjdk.org>
Subject: Behavior Proposal v2

I'm working on a new version of the behavior proposal after some fruitful discussions here on this list, and a meeting with Andy.

In this meeting a few problems I was unable to address came to light, and which I've now addressed in this post:

1) TextAreaBehavior is relying heavily on internal functions provided by the corresponding Skin. These are all related to how Text is visually laid out and actions that you can take that would need this layout information of which the Control is unaware; the simplest example is pressing the "HOME" or "END" key which needs to move to the start or end of current line depending on its visual bounds.

2) Having Behaviors as a separate concept didn't seem all that useful.

3) The potential for having a HUGE number of semantic events in Skin -> Behavior communication

TextAreaBehavior
==============

I think its fair to say that this is definitely one of the more unique behaviors in JavaFX.  Together with its Skin it is doing things that I don't think any of the other skins do.  The Skin for example provides styleable CSS properties, and provides a lot of call backs for the behavior.  This latter part I've not seen in any other skins (I did a quick search).  I didn't investigate if any other Skins are also providing CSS stylable properties.

Now I think this can mean that perhaps the TextArea control is lacking some functions that it should be providing. TextArea is aware that its content will be wrapped and split into lines, that it has a view port, and provides ways to move the caret.  What it doesn't provide is more precise caret control.  It doesn't seem like a stretch to also provide for functions that can navigate the caret to the start or end of the current line, or to the start or end of the current visible page, etc.

Lacking that, we could also accept that perhaps a Behavior should have some more awareness of a related Skin.  In MVC, the Controller (Behavior) has a reference to the View (Skin) and also has some knowledge of how the View operates. A solution here can be to have Skins with very specific needs implement an interface.  The Behavior can then be attuned to that interface, which would allow a new Skin to be constructed which reuses a lot of an existing Behavior without the requirement to subclass a specific Skin.  Note that the Behavior can fairly easily access the Skin already through Control.  A simple instanceof check will suffice to see if it can expand its behavior to support visual operations.

    void moveCaretToEndOfLine(TextArea control) {  // called in response to a KeyEvent
          if (control.getSkinnable() instanceof VisuallyCaretAwareInterface x) {
               x.moveCaret(...);
          }
          // do nothing, wanted behavior not supported by Skin
    }

Usefulness of Behavior Concept
==========================

The point was made that creating a new Behavior is still very limited in what it allows the user to change how a control "feels".  This is because the Skin does a lot of filtering when it is passing things to the associated behavior.  For example, the SpinnerSkin will call the behavior when the spinner buttons are pressed (with any mouse button(!!!)), but the Behavior can't limit this to just the left mouse button (like Spinners on Windows do). Other Skins will only call the Behavior for specific mouse buttons, or for specific events.  In effect, the Skin is already dictating part of the behavior (either by omission or by generalization) which should be part of the Behavior impementation.

For example, if I want to create a SpinnerBehavior that would react to the scroll wheel when it is hovering above the Spinner's text field, I'm out of luck; the Skin is not calling the Behavior for SCROLL events.

So perhaps we can leverage something here that is already public knowledge: the Substructure of Controls. Spinner for example has the substructure:

  *   text-field — TextField
  *   increment-arrow-button — StackPane

     *   increment-arrow — Region

  *   decrement-arrow-button — StackPane

     *   decrement-arrow — Region

The names here are part of the CSS public API and indicate interesting parts of the spinner control.

What if we completely forbid the Skin from installing event handlers, even on its children (or at least the ones that are part of the substructure), and let those events bubble up for Behavior to catch?  An immediate problem would be how the Behavior would know whether I pressed the mouse button on the UP or DOWN button, or if I pressed the button on the Text Field portion.  We could look at the event's target, but even though you can see which specific Node was targetted, the Behavior doesn't know what that node means.  So, instead we could ask the Skin this (or alternatively, put this in a fixed Key on the Node in its mappings, or use the CSS id field for this purpose):

     interface Skin<T> {
          default String determineSubstructureElement(Event<?> event) { return null; }
     }
The Skin will reply here with the publically known style class name that it assigns to its substructure elements, or `null` if it doesn't support this feature or if the Event passed in doesn't match to any (public) substructure element or if the Event just didn't belong to this Skins substructure at all.

The Behavior can then use this "name" or "annotation" as an extra piece of information to determine the desired action:

     void mousePressed(MouseEvent event, Spinner control) {
           if (event.getButton() == MouseButton.PRIMARY) {
               control.requestFocus();

               switch (control.getSkinnable().determineSubstructureElement(event)) {
                    case "increment-arrow-button" -> startSpinningUp();
                    case "increment-arrow" -> startSpinningUp();
                    case "decrement-arrow-button" -> startSpinningDown();
                    case "decrement-arrow" -> startSpinningDown();
               }
          }
     }

In this way Behaviors can support many more things than the Skin initially thought of providing.  You could have the Spinner buttons only react on LMB, have the text field respond to SCROLL events, have the spinner buttons react differently on long or short presses, on drags or gestures, etc...

Note, the above suggested API fpr determineSubstructureElement returns a String with the element name; it could also be a simple object that is aware of the nested substructure, so that it can be queried "did this Event involve the 'increment-arrow-button' or any of its children?" more easily.

Semantic Events
==============

Note that the above change sort of eliminates the need for Semantic Events between Skins and Behavior, which I think will severely cut down on how many Semantic Events will be needed.  As the events are now limited to specifying the communication needed between Behavior and Control (or only an indirection between Behavior and itself), they can be more directly related to state changes.  Instead of having 4 events for START/STOP_SPINNING_UP/DOWN, we could just have a SPINNER_VALUE_CHANGE event which is parameterized.

The value in still having the semantic events lies in:

- Being able to indirect several low-level sets of events to a single high-level action, allowing the high-level function to be remapped or accessed directly
- Being able to intercept and block events using the normal event system before they're acted upon, or even to generate a high level event directly

Who does what?
================

This changed a little bit from my earlier definitions, but I'll adjust them here:
# Controls (no changes)

- Controls refer to a Skin and Behavior
- Controls define semantic events
- Controls can generate semantic events (via mappings)
- Controls never modify their own (user writable) state on their own accord (the user is in full control)
- Controls provide an override based key mapping system

# Skins (stop acting on events)

- Skins refer to a Control (legacy inheritance) but are limited in their allowed interactions (unwritten rule since the beginning)
- Skins never interprete any events, not even on their children, with perhaps the one exception if the handling of the event only changes Skin internal state
- Skins never act upon semantic events
- Skins provide information about their substructure, either on the Node or when asked

# Behaviors (also generates semantic events, and reacts on more events)

- Behaviors refer to nothing
    - Control (and indirectly Skin) reference is received via event handler and listeners only
    - Skins can be checked to see if they support special behavior via an Interface
- Behaviors define base key mappings
    - Controls refer to these, after first checking for any user overrides
    - Base key mappings can be global and immutable, user overrides (maintained in Control) are mutable
- Behaviors generate semantic events in response to (combinations of) low-level events (or via timers)
- Behaviors can act upon events of any type (if unconsumed by the user) that are targetted at (or bubble up to) the control (enforced as that's the only place they can install handlers)
- Behaviors are allowed to modify control state and their own state

The above definitions seperate the Look & Feel part even better than my earlier proposal, and makes Behaviors far more powerful than before at only a slight cost to Skins.  Skins now truly only provide visuals, and never interprete events, while Behaviors have more options to determine what events they are interested in and how to respond to them.

The balance between the amount of code in Skins and Behaviors will shift a bit.  More event handling will move to Behaviors (where it actually belongs), which may go a long way to also making Skins easier to subclass or reimplement.

--John












-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20231117/1604bf70/attachment-0001.htm>

More information about the openjfx-dev mailing list