Touch Events, Scrolling, and Windows Pen

[Anthony Petrov]

Sounds good to me.

Danno, could you please prepare a patch for this bug, test it, and post 
it for review here?

--
best regards,
Anthony

On 05/28/2013 07:46 PM, Danno Ferrin wrote:
> For 8.0 that would be the best solution IMHO.
> On May 28, 2013 8:41 AM, "Pavel Safrata" <pavel.safrata at oracle.com> wrote:
>
>> So it seems to me that in short term (FX8) the pen should produce mouse
>> events with isSynthesized==false. We can introduce something more advanced
>> in the future, but right now it should generally work and the synthesized
>> flag really seems to be misused here.
>> Pavel
>>
>> On 28.5.2013 16:09, Danno Ferrin wrote:
>>
>>> On Tue, May 28, 2013 at 2:44 AM, Anthony Petrov
>>> <anthony.petrov at oracle.com>**wrote:
>>>
>>>   Hi Danno,
>>>>
>>>>
>>>> On 05/27/2013 07:27 PM, Danno Ferrin wrote:
>>>>
>>>>   My next problem is the pen device on Windows.  The only events captured
>>>>> in
>>>>> JavaFX from a pen are mouse events.  To complicate things the
>>>>> "Synthesized"
>>>>> flag is set.  This is a bit disconcerting because to the view of a JFX
>>>>> app
>>>>> there is no other event for it to be synthesized to.  So I would propose
>>>>> the isPenEvent method in /glass/glass-lib-windows/src/***
>>>>> *ViewContainer.cpp
>>>>> be
>>>>> added to check that the 0x80 bit is set by changing the signature
>>>>> to 0xFF515780 and the mask to 0xFFFFFF80, making a pen event look just
>>>>> like
>>>>> a mouse event, since it only generates mouse events. (the linked MSDN
>>>>> article in the code explains the lower 8 bits, and the high bit is a
>>>>> pen/touch flag, the logic may need to be more complex).  To distinguish
>>>>> a
>>>>> pen tap perhaps we set the direct flag but not the synthesized flag,
>>>>> since
>>>>> a pen is direct on the screen (usually).
>>>>>
>>>>> So am I way off base on these?  Should I work up a patch for these?  Or
>>>>> must the current behavior be maintained.
>>>>>
>>>>>   AFAIK, we haven't tested Glass/FX with a Pen device and I doubt we even
>>>> have proper hardware to test it at the moment. So such a patch would
>>>> certainly be welcome. Please feel free to post it on this mailing list
>>>> for
>>>> a review.
>>>>
>>>>   I think the QA and dev team should invest a few thousand and buy one or
>>> two
>>> each of a Surface Pro, a Samsung Ativ 500T, and maybe an Asus VivoTab (but
>>> only the Asus if you can get the pen with it on the same order).  The
>>> "triple threat" input devices (mouse/pen/touch) will cover a lot of bases.
>>>
>>>
>>>   However, we should first decide how we want to process Pen events. They
>>>> are not regular mouse events, nor are they similar to regular touch
>>>> events
>>>> (though they have a lot in common with the latter). E.g. suppose you can
>>>> control your Surface Pro with a finger using its touch screen. These are
>>>> touch events. Now, you could also connect a USB mouse to this device, and
>>>> that would have to generate regular mouse events. Additionally, you could
>>>> connect a graphic tablet such as a Wacom Bamboo for example, and this
>>>> device would have to generate some special Pen events, right?
>>>>
>>>>   As far as I can tell pen evens are just the same as mouse events unless
>>> you
>>> use microsofts "Ink" APIs to insert "Ink" into your application.  So
>>> unless
>>> JavaFX adds support for digital ink, there is vey little difference. The
>>> only one that comes to mind is pressure sensitivity, and the minority of
>>> windows pen screens or pads support pressure sensitivity.
>>>
>>>
>>>
>>>   Should we introduce a new kind of events for Pen events in FX? Or does it
>>>> make sense to use touch events for this purpose? How would we distinguish
>>>> between finger- and pen- based input events in the above scenario then?
>>>>
>>>>   Behavior is a big difference in pen vs touch.  Tap and drag for touch
>>> gets interpreted as a scroll pan, while tap and drag for a pen
>>> gets interpreted as a mouse click and drag, for example selecting text.
>>>
>>> Pen events don't generate touch gestures like the touch screens do.
>>>   Unless
>>> you count the "flicks" api in microsoft land which just generates events
>>> at
>>> the windows level for stuff like delete, cut, copy, paste, etc.  Much like
>>> the multimedia keyboard.
>>>
>>> A new boolean field could be added isPen could be added on the mouseEvent,
>>> or a new enum property at InputEvent called Input Device, that would
>>> enumerate if it was mouse, touch, pen, keyboard, or whatever standard
>>> input
>>> devies show up in the future.  These enums could then be loaded with all
>>> sorts of interesting queries in case a user wants to write their app in a
>>> future proof way, such as are drag events principally pan-scrolling, is
>>> the
>>> gesture device direct/indirect.  Maybe not an enum, but a separate HCI
>>> object rather than pushing the random flags into the event system.
>>>
>>>
>>>   What other options do we have?
>>>>
>>>>
>>>>   Keeping pen input second class is fine I think, as long as it is
>>> consistent
>>> with the differences between touch and mouse in non-ink areas.  If we have
>>> to confuse them with one type or the other, confuse them with mouse events
>>> because they are closer to mouse events then touch events. There are more
>>> important areas for JavaFX to go to before it goes down the digital ink
>>> path.
>>>
>>>
>>>   --
>>>> best regards,
>>>> Anthony
>>>>
>>>>
>>