[External] : Re: Proof of concept pull request for Behavior API (PR 1265)
John Hendrikx
john.hendrikx at gmail.com
Sat Oct 28 19:55:21 UTC 2023
Hi Andy,
On 27/10/2023 19:10, Andy Goryachev wrote:
>
> Dear John:
>
> I think our goal is for all of us to agree on a solution which solves
> the problems. We are still talking, right?
>
> I have to say - it is very difficult to have a meaningful conversation
> when questions are not being answered. It is even more difficult to
> do over email and time zones, as the context can often be lost.
>
I'm really a bit surprised, as I think I responded quite quickly to a
lot of the posts surrounding the proposals, and also answered quite a
lot of questions. The mailinglist format seems to have served Java
quite well for this purpose for years now, and if memory serves, earlier
FX proposals also were discussed here. I'm unsure what the JEP format
would contribute here, given that it does not allow for inline comments
or threads, but I'm not unwilling to try my hand at one.
> To reiterate, a proposal in a JEP format would be nice, so we can see
> the public API.
>
Alright, this will take a bit of time. I have several proposals, and
some are too big for a single proposal (IMHO) which would you like me to
focus on?
1. A public Behavior API with the initial focus on being able to reuse
and replace default behaviors
This proposal would like to achieve a clear definition of a Behavior and
clear separation (to aid in reusability and creation). It would define
a Behavior interface, and a clean way of installing/uninstalling a
behavior on controls. This would be primarily the
Behavior/BehaviorContext part of my sample PR
https://github.com/openjdk/jfx/pull/1265, leaving the more controversial
event definition parts out for now
1.1. A possible extension of the above Behavior API to allow changing
high level behavior
I'm not sure yet what would be a good approach here. I would probably
either lean towards reusing the Event system for this, or doing this
with overridable methods; ie. in order to override a function, listen
for its event and call different code or trigger a different event; or,
ensure there is an overridable method so it can be overriden directly in
code.
1.2. An extension to the above behavior API to allow for more user
friendly key rebinding
The idea here would be to create a custom behavior (allowed by 1.), call
into a behavior you want to modify to install its defaults, and then
make changes after. There is more than one possible approach here. One
I raised earlier was offering more specific methods on BehaviorContext.
Another possiblity is to make it specific to certain behaviors only
(MappableBehavior), or behaviors that can somehow provide an InputMap
(without entangling everything).
2. An improved event handling system (Michael Strauss already did some
work there) that would allow users to override or disable default event
processing
This would be a low-level improvement that would allow 3rd parties to
override large parts of JavaFX in a supported manner. It would open the
way to a 3rd party behavior system or navigation system, but also
simpler things like just changing a key mapping, even ones claimed by
behaviors currently. As it is a low level API, this would be somewhat
cumbersome for seemingly simple tasks, and the various platforms would
need to be supported manually.
In essence the above proposal would allow user installed event handlers
to receive any event before a default handler can get to it, so that say
remapping the LEFT_ARROW key is actualy possible and doesn't magically
disappear (many new users, including me at the time were/are somewhat
surprised that an event handler installed on the control is not
receiving all events). The root cause of this is the sharing of the
event handling lists on the control with (magically) installed
behaviors. The above proposal would change this (in a backwards
compatible way) to work more like how default exception handlers work --
only unconsumed events that bubble up to the root level are considered
for default behavior actions.
------
As you can see, one my problems with answering the key rebinding
questions is that IMHO this more of a later extension on a Behavior API;
this means to get to the key remapping design, there first would need to
be a Behavior API design. How key bindings are done is IMHO more of an
implementation detail of **specific** behaviors, as there are probably
more ways to do this. So for my proposals, a somewhat fleshed out
Behavior API design is an important prerequisite before offering key
rebinding.
If we can advance this design far enough, we may see a way to do this
without needing a Behavior API first; for example, we could have a
Behavior interface, and a subtype, MappableBehavior; only behaviors of
that sub type support key remapping, while general behaviors don't have
to. This would make the key rebinding just something that **some**
behaviors support, and not a general feature if it instead was added to
say Control, Behavior or BehaviorContext. This may be a short cut that
we could take to get to key rebinding quickly, without closing off a
future behavior API.
Again, thanks for reading, I look forward to some feedback, and as said,
I will try put some time towards writing a JEP. I'm also happy to
collaborate on this once a design direction becomes a bit more clear.
--John
> Please, if you have time, answer these questions. A short pseudo-code
> example will be fine.
>
> Q1. Changing an existing key binding from one key combination to another.
>
> Q2. Remapping an existing key binding to a different function.
>
> Q3. Unmapping an existing key binding.
>
> Q4. Adding a new key binding mapped to a new function.
>
> Q5. (Q1...Q4) scenarios, at run time.
>
> Q6. How the set behavior handles a change from the default skin to a
> custom skin with some visual elements that expects input removed, and
> some added.
>
> Q7. Once the key binding has been modified, is it possible to invoke
> the default functionality?
>
> Q8. How are the platform-specific key bindings created?
>
> Q9. How are the skin-specific (see Q6) handlers removed when changing
> the skins?
>
> Q10. When a key press happens, does it cause a linear search through
> listeners or just a map lookup?
>
> Lastly, I do think prototyping the alternative proposal using simple
> control like Button is insufficient. TextArea would be much better, as
> it has a ton of key bindings, platform-specific logic, various
> handlers that do and do not consume events by default.
>
> Thank you
>
> -andy
>
> P.S. I noticed that I switched my PR to Open by mistake. Sorry, it’s
> back in Draft.
>
> *From: *John Hendrikx <john.hendrikx at gmail.com>
> *Date: *Thursday, October 26, 2023 at 02:15
> *To: *Andy Goryachev <andy.goryachev at oracle.com>,
> openjfx-dev at openjdk.org <openjfx-dev at openjdk.org>, Kevin Rushforth
> <kevin.rushforth at oracle.com>
> *Subject: *Re: [External] : Re: Proof of concept pull request for
> Behavior API (PR 1265)
>
> The normal procedure I think is also to first provide a JEP for
> review, before starting on the implementation...
>
> Given the doubts raised, feedback given and potential alternatives
> proposed, I don't see why you are still moving forward with your own
> proposal. The critiques I've given have been mostly hand waved with
> arguments that have no place in JEP evaluation (time restrictions,
> existing code already works this way, false equivalency with MVC
> pattern), and therefore have IMHO not been taken serious at all.
>
> This leaves me in the position of putting in a lot of work that will
> essentially be ignored as I feel an (internal) decision has already
> been reached, regardless of the feedback on the mailinglist.
>
> The (partial) proposal I've made, and also simpler proposals so that
> 3rd parties could do a keybinding implementation, should be sufficient
> to reconsider the current proposal that is being moved forward.
>
> I'll reiterate my problems with your proposal:
>
> - Introduces a lot of API for what is essentially the configuration of
> internal event handlers
> - The proposed API partially overlaps with the existing event handler
> API, meaning that some keys could be changed with just event handlers,
> while some can only be changed with the BaseBehavior API; it also
> provides for creating new functions and assigning them to keys,
> essentially a new (very limited) API for what was already possible in
> the much more flexible event handling API
> - Introduces the term "Behavior" in public API without clearly
> specifying what that is, nor showing enough forethought that may make
> it possible in the future to have public Behaviors
> - Introduces the term "InputMap" in public API, which is just an
> implementation detail of the internal event handlers
> - Doesn't address the real issue IMHO, which is that JavaFX
> Skins/Behaviors install their Event Handlers directly on Controls,
> mixing them with user event handlers leading to all sorts of
> unpredictable behavior due to call order and internal handlers
> essentially stealing and consuming events before the user has a chance
> to look at them (and thus blocking any 3rd party key alterations)
> which leads to the (false) need to change key bindings and Behaviors
> directly...
>
> So if you want me to work on such a proposal, fully fleshing it out, I
> would like to know if it will be given consideration. I would also
> like some more feedback on what is already there, as I think it is
> sufficient to decide if a full proposal is worth it or not.
>
> My proposals in short:
>
> 1.
>
> - Fix the issues with Events being stolen before users can get a them
> - Users should be able to have priority on Events, Michael Strauss
> already has a PR that fixes the issue in part
> - Events should not be consumed when not used (navigation does
> this) as this precludes the user being able to change their meaning
> - Even better would be if internal event handlers were isolated
> and did not mix themselves with user event handlers at all
>
> The above can be done separately, and should already make it possible
> to do a lot of things that were close to impossible before when it
> comes to changing key handling, but certainly not everything.
>
> - Building on top of the improved event handling system, introduce a
> flag to indicate an event is not to be consumed by internal event handlers
>
> These two together can form the basis for a 3rd party Behavior
> implementation as standard behavior can be prevented from occurring.
> It leaves platform dependent behavior to be addressed by such a 3rd
> party / user implementation as it is a very low level API. Any key
> remapping logic would be provided by the 3rd party API.
>
> 2.
>
> I also have a more fleshed out alternative proposal that attempts to
> introduce Behaviors into JavaFX as a first class concept, instead of a
> potential 3rd party add-on. Recap:
>
> - Introduce a Behavior interface with a single method "install" to be
> called by a Control
> - The "install" method is provided a context object, BehaviorContext.
> This indirects any changes the Behavior can make to a Control, so the
> Control is fully aware of all changes and can uninstall them without
> further co-operation from the behavior.
> - The BehaviorContext provides low level functions to add/remove event
> handlers and listeners, but can also provide higher level functions
> (in perhap a later PR) to allow for some kind of control provided
> input map system
> - Standard Behaviors can be made public and can be easily subclassed
> or composed as they need not have any state. State is tracked inside
> the behavorial installed listeners and handlers themselves (either
> directly or by referring to some shared State object).
> - Clear separation of concerns; Behaviors, a resuable concept that can
> be applied to a control; BehaviorContext, manages behavior lifecycle
> by abstracting away direct Control access; behavior state management
> left up to the implementation and created (on demand and as needed)
> when "install" is called.
> - Indirection from key mapping to semantic meaning is provided by
> introducing control specific events. These semantic events can be
> handled, filtered and consumed like all other events, allowing for
> changing/remapping/blocking or ignoring; this part can be left out
> from an initial implementation to further evaluate how such events
> might interact with Skins that need specific events (there is nothing
> stopping us from having some of these semantic events be handled by
> the Control directly, and some by the specific needs of the Skin)
>
> To get at the internal key mappings, you'd need to subclass or compose
> a Behavior. The Behaviors are setup to allow this easily. To modify
> the bindings of a Control, one would install such a modified Behavior
> as a whole; overkill perhaps for one binding change, but convenient
> when multiple bindings are changed, and reusable accross controls (the
> Behavior only need to be created once).
>
> The proposal also includes an indirection between Key/Mouse event and
> its semantic meaning. This is achieved by firing higher level more
> meaningful events, but that's not the only option; it could also be
> done with overridable methods on the Behavior, or a behavior specific
> interface if the Event based proposal is seen as too audacious.
>
> This proposal advocates a clear seperation of the Behavior from the
> Skin, essentially making them Controller and View, where the View has
> no knowledge of the Controller. I see no reason why this wouldn't be
> possible, given that it is a standard pattern. That existing controls
> may be difficult to untangle is IMHO irrelevant, especially when this
> can be done one at a time. I realize that Controllers (Behaviors) may
> have functions that are sort of View (Skin) specific; this is not an
> issue, as it should be fine to trigger a behavior without it being
> consumed; unconsumed behaviorial events just bubble up. This allows
> Behaviors to have events specific to a Skin without them interfering
> if they're unused by an alternative Skin. An alternative Skin that
> also needs new behavior will also need to create a new behavior to go
> along with it (or when paired with the standard one, accept that those
> new behaviors won't be triggered).
>
> Thanks for reading.
>
> --John
>
> On 26/10/2023 00:59, Andy Goryachev wrote:
>
> Dear John:
>
> It is difficult to review the alternative proposal for a number of
> reasons. A prototype is a good start, but for any proposal to go
> forward we need a bit more work. Let me enumerate the steps that
> we expect:
>
> 1. Provide an overview of the proposal following a JEP outline:
>
> *Summary*
>
> *Goals*
>
> *Non-Goals*
>
> *Motivation*
>
> *Description*
>
> *Alternatives*
>
> *Risks and Assumptions*
>
> *Dependencies*
>
> 2. A draft PR that provides a proof of concept, using, in this
> case, a few complex controls like TextArea, TableView, ComboBox.
>
> 3. Address the question raised earlier, perhaps by providing code
> examples (pseudo code is acceptable, I think).
>
> More specifically, I’d like to know how the following concerns
> will be addressed by the new proposal:
>
> Q1. Changing an existing key binding from one key combination to
> another.
>
> Q2. Remapping an existing key binding to a different function.
>
> Q3. Unmapping an existing key binding.
>
> Q4. Adding a new key binding mapped to a new function.
>
> Q5. (Q1...Q4) scenarios, at run time.
>
> Q6. How the set behavior handles a change from the default skin to
> a custom skin with some visual elements that expects input
> removed, and some added.
>
> Q7. Once the key binding has been modified, is it possible to
> invoke the default functionality?
>
> Q8. How are the platform-specific key bindings created?
>
> Q9. How are the skin-specific (see Q6) handlers removed when
> changing the skins?
>
> Q10. When a key press happens, does it cause a linear search or a
> map lookup?
>
> Thank you
>
> -andy
>
> *From: *John Hendrikx <john.hendrikx at gmail.com>
> <mailto:john.hendrikx at gmail.com>
> *Date: *Tuesday, October 24, 2023 at 04:58
> *To: *Andy Goryachev <andy.goryachev at oracle.com>
> <mailto:andy.goryachev at oracle.com>, openjfx-dev at openjdk.org
> <openjfx-dev at openjdk.org> <mailto:openjfx-dev at openjdk.org>
> *Subject: *Re: [External] : Re: Proof of concept pull request for
> Behavior API (PR 1265)
>
> On 23/10/2023 23:57, Andy Goryachev wrote:
>
> You'd create a new class, `MyBehavior`,
>
> By “customizing” I also mean at run time. Creating new
> classes wouldn’t work.
>
> This would also work at runtime, as the class you create can be
> instantiated with parameters that control its key binding
> behavior. Even though the standard Behaviors should probably be
> singletons (so they can be reused and composed) or have public
> well documented constructors, a custom behavior created by the
> user has no such re-usability restrictions.
>
> coupling
>
> I don’t think it is our choice - it is up to the skin
> designed. If they add a node that needs to take input, or if
> the behavior is drastically different, it is almost impossible
> to create a common interface. So skin and behaviors are
> coupled, besides we have to design for the worst case (of a
> totally different skin). The division between S and B comes
> mostly from the division between V and C in MVC. From a
> distance, the user does not see it at all - all they see is a
> control.
>
> JavaFX is not doing MVC.
>
> In MVC, the 3 components are not entangled; Model refers View,
> Controller refers View and Model, View refers nothing; in JavaFX
> the View (Skin) creates the Controller (Behavior); the View
> especially normally can be created without any dependencies, and
> can be tested as such; with Skins being tightly coupled to both
> Behaviors and Controls, that doesn't even come close.
>
> For it to be MVC you'd need to:
>
> - Remove reference from Skin to Control
> - Do not let Skins create Behaviors
> - Instantation order should be, create a Skin first (with no
> Control reference), then create the Control (with Skin as
> parameter or setter), then create a Behavior (with Control as
> parameter, and one or more Views (Skins))
>
> What JavaFX is exactly, I don't know. It doesn't follow MVC (even
> though it claims to) because in the current setup the Skin is both
> V and C; that's not MVC. At most it is MS (Model Skin), and so
> there is no reason to expose anything beyond the Skin then, as
> that would just be pretending to be something that it is not.
>
> This suggest another metric at judging the usefulness of a
> design - how easy it is to understand and perform 80% of most
> common tasks.
>
> Now that I explained how key remappings would work, I don't see
> how this would disqualify the alternative proposal.
>
>
> There are more interesting ideas at the end of the message I
> am replying to - fxml, css, global changes - these go far
> beyond the simple input map improvement. I did mention this
> already, but neither open source community, nor my employer
> might have the resources to make such drastic changes.
>
> I didn't mention FXML, but yes, I gave some other things to think
> about. As for how drastic any of those are, that remains to be
> seen. Certainly the global changes would not be that hard at
> all. The CSS proposal would need some research if there is some
> will to go there; it assumes that the information needed can be
> transported in a reasonable manner to the key binding system using
> the existing CSS infrastructure.
>
>
> So we have to be realistic, I think. We are travelling to a
> different planet in a small spaceship and we only have so much
> material and oxygen to play with. A simple improvement that
> helps 80% of use cases might be better than a major redesign
> (I still think the event proposal involves major redesign).
>
> I think that if that's the case that we'd better focus on making
> it possible for 3rd parties to deliver these features, and do the
> simplest thing that would allow them to do so. That would be
> prioritized event handlers (so a 3rd party can always intercept
> before the Skin/Behavior gets to it) + a flag to skip system event
> handlers (ala consumed) to allow bubbling up.
>
> On top of that any key remapping or behavior change system can be
> constructed already.
>
> --John
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20231028/91f1191c/attachment-0001.htm>
More information about the openjfx-dev
mailing list