<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Iosevka Fixed SS16";
panose-1:2 0 5 9 3 0 0 0 0 4;}
@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Iosevka Fixed SS16";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16"">Dear colleagues:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16"">I would like to thank John and Michael for a lively discussion, during which we clarified a number of concepts, specifically the roles of Control (C), Skin (S), and Behavior
(B). There is still a bit of difference in how we see things, but I am pleased to say we do have a lot in common.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16"">Especially valuable is definition of Behavior as a layer that translates the user input into state changes of the control. How these changes are effected is where we have
different ideas, but I really like this definition.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16"">We now have a clear separation of concerns when it comes to C, S, B. We also acknowledge that effort required to migrate from the current implementation to a new one should
be minimized within reason, and could be done gradually and without breaking compatibility.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16"">Another positive development is realization that FX has an issue with event handling priority, which cannot be solved using existing APIs. This problem can be decoupled from
the behavior/input map discussion and solved in a separate PR ( <a href="https://github.com/openjdk/jfx/pull/1266">
https://github.com/openjdk/jfx/pull/1266</a> ), so maybe we should look at it first (or in parallel)?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16"">On the other hand, we do have some disagreement. The good thing is, we share the same goal - to give application developers a platform and the APIs that are both useful and
convenient. This makes me think we can get to a mutually acceptable design. What should be the process to arrive at the common solution?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16"">-andy<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Iosevka Fixed SS16""><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">openjfx-dev <openjfx-dev-retn@openjdk.org> on behalf of John Hendrikx <john.hendrikx@gmail.com><br>
<b>Date: </b>Sunday, November 12, 2023 at 16:13<br>
<b>To: </b>openjfx-dev@openjdk.org <openjfx-dev@openjdk.org>, Michael Strauß <michaelstrau2@gmail.com><br>
<b>Subject: </b>Re: Public Behavior API proposal<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Hi everyone, and specifically Andy and Michael,<br>
<br>
I'm working on updating the Behavior API proposal, and I've been <br>
thinking about the semantic events a lot. I would really like to hear <br>
what you think, and how it matches with your ideas.<br>
<br>
Quick recap, Semantic Events are high level events (like the ActionEvent <br>
from Button) that can be triggered by a combination of low level events. <br>
They represent an action to be achieved, but are not tied to any <br>
specific means of triggering it. For example, the ActionEvent can be <br>
triggered with the mouse, or with the keyboard; it is irrelevant which <br>
one it was. Semantic events can be generated by Skins (as a result of <br>
interactions with the Skin's managed children), by Controls (see below) <br>
and users directly. You can compare these with Andy's FunctionTags or <br>
Actions from various systems.<br>
<br>
Let me describe exactly each part's role as I see it currently:<br>
<br>
# Controls<br>
<br>
Controls define semantic events, provides infrastructure for handling <br>
events that is separated from internal needs (user comes first). User <br>
installed event handlers always have priority to make the user feel in <br>
control. The Control also provides for another new infrastructure, the <br>
managing of key mappings. The mapping system can respond directly to <br>
Key events (after the user had their chance) to generate a semantic <br>
event. This means that both Control and Skin are allowed to generate <br>
semantic events, although for Control this is strictly limited to the <br>
mapping system. The key mappings are only overrideable, and their base <br>
configuration is provided by whatever Behavior is installed. Exchanging <br>
the Behavior does not undo user key mapping overrides, but instead <br>
provides a new base line upon which the overrides are applied. So if a <br>
Behavior provides a mapping for SPACE, and the user removed it, <br>
installing a different behavior that also provides a mapping for SPACE <br>
will still see that mapping as removed. If a behavior doesn't define <br>
SPACE, and the user removed it, then nothing special happens (but it is <br>
remembered).<br>
<br>
- Controls refer to a Skin and Behavior<br>
- Controls define semantic events<br>
- Controls can generate semantic events (via mappings)<br>
- Controls never modify their own (user writable) state on their own <br>
accord (the user is in full control)<br>
- Controls provide an override based key mapping system<br>
<br>
# Skins<br>
<br>
Skins provide the visuals, and although they get a Control reference, <br>
they are restricted to only adding property listeners (not event <br>
handlers) and modifying the children list (which is read only for users <br>
as Control extends from Region). This keeps the user fully in control <br>
when it comes to any writable properties and events on Control. Most <br>
Skins already do this as I think it was an unwritten rule from the <br>
beginning. Skins then install event handlers on their children (but <br>
never the top level Control) where translation takes place to semantic <br>
events. Skins have no reference to the Behavior to ensure that all <br>
communication has to go through (interceptable) semantic events. Not <br>
all events a Skin receives must be translated; if some events only <br>
result in the Skin's internal state changing, and does not need to be <br>
reflected in the Control's state then Skins can handle these directly <br>
without going through a Behavior. Examples might be the position of the <br>
caret, or the exact scroll location of a View, if such things aren't <br>
part of the Control state.<br>
<br>
- Skins refer to a Control (legacy inheritance) but are limited in their <br>
allowed interactions (unwritten rule since the beginning)<br>
- Better would be to provide skins with only a Context object that <br>
only allows installing of listeners to ensure they can't do nasty things <br>
(and to track all changes a Skin did for when it is replaced, similar <br>
idea to BehaviorContext)<br>
- Skins interprete normal events of their CHILDREN only (after the user <br>
did not consume them), and either:<br>
- Translates them to semantic events (targetted at the top level <br>
Control)<br>
- Acts upon them directly if only Skin internal state is involved<br>
- Skins never act upon semantic events<br>
<br>
# Behaviors<br>
<br>
Behaviors provide a standard set of key mappings, and provide standard <br>
ways of dealing with semantic events. Installing a new Behavior on a <br>
control can make it "feel" completely different, from how it reacts to <br>
keys, and which keys, how it deals with the high level semantic events, <br>
as well as how it handles mouse interactions. Behaviors can act upon <br>
both normal and semantic events, but don't generate any events <br>
themselves. Again, they only act upon events after the user had a <br>
chance to act upon them first. A behavior is free to act upon Key events <br>
directly (for things too complicated for a simple key mapping), but it <br>
would be better to indirect them as much as possible via a semantic <br>
event that is triggered by a key mapping in the Control. Mouse events <br>
are more complicated and don't need to be indirected to be handled (not <br>
100% sure here yet). When receiving a semantic event that the user <br>
didn't care about, the Behavior consumes it and does its thing by <br>
calling methods on the Control and modifying control state.<br>
<br>
- Behaviors refer to nothing<br>
- Control reference is received via event handler and listeners only<br>
- Behaviors define base key mappings<br>
- Controls refer to these, after first checking for any user overrides<br>
- Base key mappings can be global and immutable, user overrides <br>
(maintained in Control) are mutable<br>
- Behaviors never generate events<br>
- Behaviors can act upon events of any type (if unconsumed by the user) <br>
that are targetted at the control (enforced as that's the only place <br>
they can install handlers)<br>
- Behaviors are allowed to modify control state and their own state<br>
<br>
# Interaction with Andy's proposal<br>
<br>
I think the above works pretty nicely together with Andy's proposal. By <br>
moving the responsibility managing the key mappings to Control, but <br>
leaving the responsibility of defining the mappings with Behaviors, I <br>
see a nice path forward to opening up a key mapping system for simple <br>
overrides, as well as having a public Behavior API for more advanced <br>
behaviorial modifications.<br>
<br>
Notice that I didn't provide for a FunctionTag remapping system; as <br>
these would be semantic events in this proposal, they can be <br>
filtered/handled before the Behavior gets them. So to change a <br>
function, just consume it and fire a different one. To completely block <br>
it, just consume it. To replace a single function with two functions, <br>
consume it and fire two new ones, etc. So to globally swap the <br>
increment/decrement functions of all Spinners, at the Scene level you <br>
could have a handler do this.<br>
<br>
To also answer Andy's 10 questions:<br>
<br>
Q1. Changing an existing key binding from one key combination to another.<br>
<br>
-> Control provides a small API to override base mappings (remap)<br>
<br>
Q2. Remapping an existing key binding to a different function.<br>
<br>
-> Control provides a small API to override base mappings, including to <br>
which semantic event they translate<br>
<br>
Q3. Unmapping an existing key binding.<br>
<br>
-> Control provides a small API to override base mappings (disable)<br>
<br>
Q4. Adding a new key binding mapped to a new function.<br>
<br>
-> Still debatable if this should be provided, but I don't see much <br>
blockers for this; someone will have to interpret the new function <br>
though; this could be a user event handler that knows the event (which <br>
can be a custom one) or a customized Behavior.<br>
<br>
Q5. (Q1...Q4) scenarios, at run time.<br>
<br>
-> All possible at runtime. With a "-fx-behavior" CSS feature, this <br>
could also be provided via CSS selectors, allowing far reaching changes <br>
without having to modify each control individually.<br>
<br>
Q6. How the set behavior handles a change from the default skin to a <br>
custom skin with some visual elements that expects input removed, and <br>
some added.<br>
<br>
-> Behaviors act only upon events targetted at the Control. Skins that <br>
don't provide some events means they will just not be picked up by <br>
Behaviors. Skins that provide unknown semantic events require a <br>
corresponding Behavior upgrade. Skins actions that don't require <br>
Control state changes (only Skin state changes) can ignore this system <br>
altogether.<br>
<br>
Q7. Once the key binding has been modified, is it possible to invoke the <br>
default functionality?<br>
<br>
-> Yes, just fire the appropriate semantic event at the Control<br>
<br>
Q8. How are the platform-specific key bindings created?<br>
<br>
-> Debatable if this is needed, but probably something similar to your <br>
proposal will be possible; Platforms don't change at runtime, so why <br>
they are even added as bindings (instead of just skip adding them if not <br>
on the right platform) is a mystery to me. A simple tool (perhaps on <br>
the Platform class) to check the platform should be sufficient; no need <br>
to interweave this with key mappings themselves.<br>
<br>
Q9. How are the skin-specific (see Q6) handlers removed when changing <br>
the skins?<br>
<br>
-> Skins clean up after themselves, and they're not allowed to install <br>
handlers on the control (only on their children)<br>
<br>
Q10. When a key press happens, does it cause a linear search through <br>
listeners or just a map lookup?<br>
<br>
-> No, Controls have freedom to optimize how they do this; Behaviors <br>
provide base mappings in some kind of Map form, or have an API to <br>
quickly look up a base mapping (probably the latter to encapsulate it <br>
better).<br>
<br>
Thanks for reading,<br>
<br>
--John<br>
<br>
<br>
On 07/11/2023 08:09, Michael Strauß wrote:<br>
> Hi John,<br>
><br>
> I like that you clearly define the terms Control, Skin and Behavior,<br>
> as well as their roles within the control architecture.<br>
><br>
> However, I don't see how your proposal scales to non-trivial controls,<br>
> and I agree with Andy that the Button example doesn't help (because a<br>
> Button lacks substructure and provides only a single interaction).<br>
><br>
> I'm missing your previous idea of using the event system for<br>
> higher-level semantic events, because I think they're required to make<br>
> this work. Here's how I see these parts working together:<br>
><br>
> 1) A control is an opaque node in the scene graph, which defines the<br>
> API for a particular interactive element. It also defines the<br>
> interactions afforded by its implementation. For example, a Spinner<br>
> will usually consist of a text field and two buttons, but a skin might<br>
> choose to implement these components differently. The interactions<br>
> afforded by a control are exposed as semantic events:<br>
><br>
> class SpinnerEvent extends Event {<br>
> static EventType<SpinnerEvent> COMMIT_TEXT;<br>
> static EventType<SpinnerEvent> START_INCREMENT;<br>
> static EventType<SpinnerEvent> STOP_INCREMENT;<br>
> static EventType<SpinnerEvent> START_DECREMENT;<br>
> static EventType<SpinnerEvent> STOP_DECREMENT;<br>
> }<br>
><br>
> 2) Skins are responsible for generating semantic events, and sending<br>
> those events to the control. Since we don't need those events to have<br>
> a tunneling/bubbling behavior, we could have a flag on the event that<br>
> indicates a "direct event", one that is dispatched directly to its<br>
> target.<br>
><br>
> 3) Behaviors listen for semantic events on the control, and convert<br>
> these events into state changes of the control. This part would<br>
> probably be quite similar to some of the things that have already been<br>
> proposed.<br>
><br>
> In this way, controls, skins, and behaviors would end up as loosely<br>
> coupled parts. In particular, I don't see the value in making<br>
> behaviors public API if they are so tightly coupled to skins that they<br>
> end up as being basically implementation details.<br>
><br>
> Andy:<br>
>> Imagine a specific skin that has a Node that accepts a user input. A scroll bar, a button, or a region with a some function. Unless this element is proclaimed as must-have for any skin and codified via some new public API (MySkin.getSomeElement()), it
is specific to that particular skin and that particular behavior.<br>
> I think that's a very important observation. A skin can't just be<br>
> anything it wants to be, it must be suitable for its control. So we<br>
> need a place where we define the API and the interactions afforded by<br>
> that control. In my opinion, this place is the Control. Its<br>
> functionality is exposed via properties and methods, and its<br>
> interactions are specified using semantic events.<br>
><br>
> Now skins are free to be implemented in any imaginable way, provided<br>
> that they interact with the control using semantic events. This gives<br>
> us very straightforward restrictions:<br>
> * A skin can never add interactions that the control didn't specify.<br>
> * If additional interactions are required, the control must be<br>
> subclassed and the interactions must be specified by the control.<br>
> Additionally, the behavior must be extended to account for the<br>
> additional interactions.<br>
><br>
><br>
><br>
><br>
> On Mon, Nov 6, 2023 at 4:50 AM John Hendrikx <john.hendrikx@gmail.com> wrote:<br>
>> As promised, a public Behavior API proposal.<br>
>><br>
>> Summary:<br>
>><br>
>> Introduce a new Behavior interface that can be set on a control to replace its current behavior. The new behavior can be fully custom or composed (or later subclassed) from a default behavior. Some default behaviors will be provided as part of this proposal,
but not all.<br>
>><br>
>> See here: <a href="https://gist.github.com/hjohn/293f3b0ec98562547d49832a2ce56fe7">
https://gist.github.com/hjohn/293f3b0ec98562547d49832a2ce56fe7</a><br>
>><br>
>> --John<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>