Touch Events, Scrolling, and Windows Pen
Anthony Petrov
anthony.petrov at oracle.com
Tue Jun 4 03:22:21 PDT 2013
Hi Danno,
I've just pushed your fix:
http://hg.openjdk.java.net/openjfx/8/graphics/rt/rev/1dd4bb7090a2
Thank you for the contribution.
--
best regards,
Anthony
On 06/03/2013 06:47 PM, Artem Ananiev wrote:
>
> On 6/3/2013 5:52 PM, Anthony Petrov wrote:
>> Hi Danno,
>>
>> Thank you for the patch. I've published a webrev at:
>>
>> http://cr.openjdk.java.net/~anthony/g-301-penEvents-RT-30869.0/
>>
>> The fix looks fine to me. Could anyone else please review it?
>
> +1.
>
> Thanks,
>
> Artem
>
>> --
>> best regards,
>> Anthony
>>
>> On 06/03/13 04:59, Danno Ferrin wrote:
>>> I created a new bug for the pen support bug, with a tested patch. I
>>> spent way more time getting the build up than doing the fix, such is
>>> native work.
>>> https://javafx-jira.kenai.com/browse/RT-30869
>>> For the pan-select, I attached a fix for list views to the existing
>>> bug. If that approach passes muster I can do the same for tree and
>>> table.
>>> https://javafx-jira.kenai.com/browse/RT-21091
>>>
>>> On Tue, May 28, 2013 at 10:46 AM, Anthony Petrov
>>> <anthony.petrov at oracle.com <mailto:anthony.petrov at oracle.com>> wrote:
>>>
>>> Sounds good to me.
>>>
>>> Danno, could you please prepare a patch for this bug, test it, and
>>> post it for review here?
>>>
>>> --
>>> best regards,
>>> Anthony
>>>
>>>
>>> On 05/28/2013 07:46 PM, Danno Ferrin wrote:
>>>
>>> For 8.0 that would be the best solution IMHO.
>>> On May 28, 2013 8:41 AM, "Pavel Safrata"
>>> <pavel.safrata at oracle.com <mailto:pavel.safrata at oracle.com>>
>>> wrote:
>>>
>>> So it seems to me that in short term (FX8) the pen should
>>> produce mouse
>>> events with isSynthesized==false. We can introduce something
>>> more advanced
>>> in the future, but right now it should generally work and
>>> the synthesized
>>> flag really seems to be misused here.
>>> Pavel
>>>
>>> On 28.5.2013 16:09, Danno Ferrin wrote:
>>>
>>> On Tue, May 28, 2013 at 2:44 AM, Anthony Petrov
>>> <anthony.petrov at oracle.com
>>> <mailto:anthony.petrov at oracle.com>>**__wrote:
>>>
>>> Hi Danno,
>>>
>>>
>>>
>>> On 05/27/2013 07:27 PM, Danno Ferrin wrote:
>>>
>>> My next problem is the pen device on Windows.
>>> The only events captured
>>>
>>> in
>>> JavaFX from a pen are mouse events. To
>>> complicate things the
>>> "Synthesized"
>>> flag is set. This is a bit disconcerting
>>> because to the view of a JFX
>>> app
>>> there is no other event for it to be synthesized
>>> to. So I would propose
>>> the isPenEvent method in
>>> /glass/glass-lib-windows/src/*__**
>>>
>>> *ViewContainer.cpp
>>> be
>>> added to check that the 0x80 bit is set by
>>> changing the signature
>>> to 0xFF515780 and the mask to 0xFFFFFF80, making
>>> a pen event look just
>>> like
>>> a mouse event, since it only generates mouse
>>> events. (the linked MSDN
>>> article in the code explains the lower 8 bits,
>>> and the high bit is a
>>> pen/touch flag, the logic may need to be more
>>> complex). To distinguish
>>> a
>>> pen tap perhaps we set the direct flag but not
>>> the synthesized flag,
>>> since
>>> a pen is direct on the screen (usually).
>>>
>>> So am I way off base on these? Should I work up
>>> a patch for these? Or
>>> must the current behavior be maintained.
>>>
>>> AFAIK, we haven't tested Glass/FX with a Pen
>>> device and I doubt we even
>>>
>>> have proper hardware to test it at the moment. So
>>> such a patch would
>>> certainly be welcome. Please feel free to post it on
>>> this mailing list
>>> for
>>> a review.
>>>
>>> I think the QA and dev team should invest a few
>>> thousand and buy one or
>>>
>>> two
>>> each of a Surface Pro, a Samsung Ativ 500T, and maybe an
>>> Asus VivoTab (but
>>> only the Asus if you can get the pen with it on the same
>>> order). The
>>> "triple threat" input devices (mouse/pen/touch) will
>>> cover a lot of bases.
>>>
>>>
>>> However, we should first decide how we want to
>>> process Pen events. They
>>>
>>> are not regular mouse events, nor are they similar
>>> to regular touch
>>> events
>>> (though they have a lot in common with the latter).
>>> E.g. suppose you can
>>> control your Surface Pro with a finger using its
>>> touch screen. These are
>>> touch events. Now, you could also connect a USB
>>> mouse to this device, and
>>> that would have to generate regular mouse events.
>>> Additionally, you could
>>> connect a graphic tablet such as a Wacom Bamboo for
>>> example, and this
>>> device would have to generate some special Pen
>>> events, right?
>>>
>>> As far as I can tell pen evens are just the same
>>> as mouse events unless
>>>
>>> you
>>> use microsofts "Ink" APIs to insert "Ink" into your
>>> application. So
>>> unless
>>> JavaFX adds support for digital ink, there is vey little
>>> difference. The
>>> only one that comes to mind is pressure sensitivity, and
>>> the minority of
>>> windows pen screens or pads support pressure
>>> sensitivity.
>>>
>>>
>>>
>>> Should we introduce a new kind of events for Pen
>>> events in FX? Or does it
>>>
>>> make sense to use touch events for this purpose? How
>>> would we distinguish
>>> between finger- and pen- based input events in the
>>> above scenario then?
>>>
>>> Behavior is a big difference in pen vs touch.
>>> Tap and drag for touch
>>>
>>> gets interpreted as a scroll pan, while tap and drag for
>>> a pen
>>> gets interpreted as a mouse click and drag, for example
>>> selecting text.
>>>
>>> Pen events don't generate touch gestures like the touch
>>> screens do.
>>> Unless
>>> you count the "flicks" api in microsoft land which just
>>> generates events
>>> at
>>> the windows level for stuff like delete, cut, copy,
>>> paste, etc. Much like
>>> the multimedia keyboard.
>>>
>>> A new boolean field could be added isPen could be added
>>> on the mouseEvent,
>>> or a new enum property at InputEvent called Input
>>> Device, that would
>>> enumerate if it was mouse, touch, pen, keyboard, or
>>> whatever standard
>>> input
>>> devies show up in the future. These enums could then be
>>> loaded with all
>>> sorts of interesting queries in case a user wants to
>>> write their app in a
>>> future proof way, such as are drag events principally
>>> pan-scrolling, is
>>> the
>>> gesture device direct/indirect. Maybe not an enum, but
>>> a separate HCI
>>> object rather than pushing the random flags into the
>>> event system.
>>>
>>>
>>> What other options do we have?
>>>
>>>
>>>
>>> Keeping pen input second class is fine I think,
>>> as long as it is
>>>
>>> consistent
>>> with the differences between touch and mouse in non-ink
>>> areas. If we have
>>> to confuse them with one type or the other, confuse them
>>> with mouse events
>>> because they are closer to mouse events then touch
>>> events. There are more
>>> important areas for JavaFX to go to before it goes down
>>> the digital ink
>>> path.
>>>
>>>
>>> --
>>>
>>> best regards,
>>> Anthony
>>>
>>>
>>>
>>>
More information about the openjfx-dev
mailing list