Touch Events, Scrolling, and Windows Pen

[Pavel Safrata]

So it seems to me that in short term (FX8) the pen should produce mouse 
events with isSynthesized==false. We can introduce something more 
advanced in the future, but right now it should generally work and the 
synthesized flag really seems to be misused here.
Pavel

On 28.5.2013 16:09, Danno Ferrin wrote:
> On Tue, May 28, 2013 at 2:44 AM, Anthony Petrov
> <anthony.petrov at oracle.com>wrote:
>
>> Hi Danno,
>>
>>
>> On 05/27/2013 07:27 PM, Danno Ferrin wrote:
>>
>>> My next problem is the pen device on Windows.  The only events captured in
>>> JavaFX from a pen are mouse events.  To complicate things the
>>> "Synthesized"
>>> flag is set.  This is a bit disconcerting because to the view of a JFX app
>>> there is no other event for it to be synthesized to.  So I would propose
>>> the isPenEvent method in /glass/glass-lib-windows/src/**ViewContainer.cpp
>>> be
>>> added to check that the 0x80 bit is set by changing the signature
>>> to 0xFF515780 and the mask to 0xFFFFFF80, making a pen event look just
>>> like
>>> a mouse event, since it only generates mouse events. (the linked MSDN
>>> article in the code explains the lower 8 bits, and the high bit is a
>>> pen/touch flag, the logic may need to be more complex).  To distinguish a
>>> pen tap perhaps we set the direct flag but not the synthesized flag, since
>>> a pen is direct on the screen (usually).
>>>
>>> So am I way off base on these?  Should I work up a patch for these?  Or
>>> must the current behavior be maintained.
>>>
>> AFAIK, we haven't tested Glass/FX with a Pen device and I doubt we even
>> have proper hardware to test it at the moment. So such a patch would
>> certainly be welcome. Please feel free to post it on this mailing list for
>> a review.
>>
> I think the QA and dev team should invest a few thousand and buy one or two
> each of a Surface Pro, a Samsung Ativ 500T, and maybe an Asus VivoTab (but
> only the Asus if you can get the pen with it on the same order).  The
> "triple threat" input devices (mouse/pen/touch) will cover a lot of bases.
>
>
>> However, we should first decide how we want to process Pen events. They
>> are not regular mouse events, nor are they similar to regular touch events
>> (though they have a lot in common with the latter). E.g. suppose you can
>> control your Surface Pro with a finger using its touch screen. These are
>> touch events. Now, you could also connect a USB mouse to this device, and
>> that would have to generate regular mouse events. Additionally, you could
>> connect a graphic tablet such as a Wacom Bamboo for example, and this
>> device would have to generate some special Pen events, right?
>>
> As far as I can tell pen evens are just the same as mouse events unless you
> use microsofts "Ink" APIs to insert "Ink" into your application.  So unless
> JavaFX adds support for digital ink, there is vey little difference. The
> only one that comes to mind is pressure sensitivity, and the minority of
> windows pen screens or pads support pressure sensitivity.
>
>
>
>> Should we introduce a new kind of events for Pen events in FX? Or does it
>> make sense to use touch events for this purpose? How would we distinguish
>> between finger- and pen- based input events in the above scenario then?
>>
> Behavior is a big difference in pen vs touch.  Tap and drag for touch
> gets interpreted as a scroll pan, while tap and drag for a pen
> gets interpreted as a mouse click and drag, for example selecting text.
>
> Pen events don't generate touch gestures like the touch screens do.  Unless
> you count the "flicks" api in microsoft land which just generates events at
> the windows level for stuff like delete, cut, copy, paste, etc.  Much like
> the multimedia keyboard.
>
> A new boolean field could be added isPen could be added on the mouseEvent,
> or a new enum property at InputEvent called Input Device, that would
> enumerate if it was mouse, touch, pen, keyboard, or whatever standard input
> devies show up in the future.  These enums could then be loaded with all
> sorts of interesting queries in case a user wants to write their app in a
> future proof way, such as are drag events principally pan-scrolling, is the
> gesture device direct/indirect.  Maybe not an enum, but a separate HCI
> object rather than pushing the random flags into the event system.
>
>
>> What other options do we have?
>>
>>
> Keeping pen input second class is fine I think, as long as it is consistent
> with the differences between touch and mouse in non-ink areas.  If we have
> to confuse them with one type or the other, confuse them with mouse events
> because they are closer to mouse events then touch events. There are more
> important areas for JavaFX to go to before it goes down the digital ink
> path.
>
>
>> --
>> best regards,
>> Anthony
>>