<Swing Dev> <AWT Dev> [10] Review request for 8166772: Touch keyboard is not shown for text components on a screen touch

Thu Sep 14 12:20:55 UTC 2017

Hello Shashi,

Thank you for attention to my review request. Win32 API provides 
different API sets dedicated to handling various types of user's touch 
input in different degree of deepness. For example:
1.  Windows Touch Input (WM_TOUCH messages)
2.  Windows Touch Gestures (WM_GESTURE messages)
     Official information about 2 API sets ab: 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd562100(v=vs.85).aspx
3.  Touch Hit Testing - 
https://msdn.microsoft.com/en-us/library/windows/desktop/hh437255(v=vs.85).aspx

WM_POINTER may be yet another approach.

I chose to exploit API related to "WM_TOUCH" message, because this API, 
as I understand, provides more opportunities to control the touch input 
then other 2 APIs specified above, and it allows to handle multiple 
touch points, what may become necessary in future.

RESPONSE TO YOUR REMARK:
I cannot agree that "WM_TOUCH" is officially deprecated. In the official 
Win32 API specification there is no information that functions from this 
API are deprecated or obsolete. For example:
1.  RegisterTouchWindow - 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd317326(v=vs.85).aspx
2.  GetTouchInputInfo - 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd371582(v=vs.85).aspx
3.  TOUCHINPUT - 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd317334(v=vs.85).aspx
4.  WM_TOUCH - 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd317341(v=vs.85).aspx
     (There is some community comment, but community comments are 
written by anybody, deprecation is not announced in community comments. 
For example, the really deprecated function is "GetVersion" - 
(https://msdn.microsoft.com/en-us/library/windows/desktop/ms724439(v=vs.85).aspx)).

The web page, whose reference you provided is just a forum, is not Win32 
API specification or official announcement from Microsoft. So I would 
not rely on this information.

Additionally approach based on "WM_TOUCH" showed itself in my test 
environment reliably, WM_TOUCH are generated stably in MS Windows 10 OS, 
also I was able to successfully port the fix to JDK 8 and verify that it 
works. Currently I am working on addressing remarks from Sergey and do 
not plan to refuse from currently developed real basement of the whole 
fix, which uses WM_TOUCH.

Thank you,
Anton

On 14/09/2017 10:55, Shashidhara Veerabhadraiah wrote:
> RE: <AWT Dev> <Swing Dev> [10] Review request for 8166772: Touch 
> keyboard is not shown for text components on a screen touch
>
> Hi Anton, I have a question to ask for. I was working on another bug 
> and found that the WM_TOUCHhas been deprecated and instead we should 
> be using the WM_POINTER? Here is some info on 
> that:_https://stackoverflow.com/questions/23790602/wm-touch-vs-wm-pointer_
>
> Typically, the WM_POINTER types can handle following type of/_pointers_/:
>
> typedef enum tagPOINTER_INPUT_TYPE {
>
> PT_POINTER   = 0x00000001,
>
> PT_TOUCH     = 0x00000002,
>
> PT_PEN       = 0x00000003,
>
> PT_MOUSE     = 0x00000004,
>
> PT_TOUCHPAD  = 0x00000005
>
> }POINTER_INPUT_TYPE;
>
> Hope this is useful foryourfix.
>
> Thanks and regards,
>
> Shashi
>
> -----Original Message-----
> From:Sergey Bylokhov
> Sent:Tuesday, September 5, 2017 10:45 PM
> To:Anton Litvinov <anton.litvinov at oracle.com>; 
> awt-dev at openjdk.java.net; swing-dev at openjdk.java.net
> Subject:Re: <AWT Dev> <Swing Dev> [10] Review request for 8166772: 
> Touch keyboard is not shown for text components on a screen touch
>
> Hi, Anton.
>
> The fix looks good.
>
>   - But can you please recheck that is is not necessary to use 
> additional synchronization in showOrHideTouchKeyboard() if a few EDT 
> will be used.
>
>   - I suggest to invert the awt.touchKeyboardAutoShowIsEnabled and use 
> true as default value, we will have more coverage and feedback in this 
> case. This property will be used as a workaround if some bugs will be 
> found.
>
> On 8/30/17 11:51, Anton Litvinov wrote:
>
> > Hello dear reviewers,
>
> > 
>
> > Could anybody please look at this review request?
>
> > 
>
> > Thank you,
>
> > Anton
>
> > 
>
> > On 17/08/2017 13:20, Anton Litvinov wrote:
>
> >> Hello,
>
> >>
>
> >> Could you please review the following fix for the bug.
>
> >>
>
> >> Bug:https://bugs.openjdk.java.net/browse/JDK-8166772
>
> >> Webrev:http://cr.openjdk.java.net/~alitvinov/8166772/jdk10/webrev.00<http://cr.openjdk.java.net/%7Ealitvinov/8166772/jdk10/webrev.00>
>
> >>
>
> >> The bug is the fact that, when a user touches any Swing or AWT text
>
> >> component, for example "JTextField", "JTextArea", "TextField",
>
> >> "TextArea", by means of a touch screen on a host with MS Windows
>
> >> 10/8.1/8 OS, the system touch keyboard is not shown automatically.
>
> >> Please find a detailed description of the bug, screenshots depicting
>
> >> the touch keyboard and a compilable test case with Swing/AWT text
>
> >> components in JBS bug record. Also a summary of the done research of
>
> >> the issue with a description of identified approaches for its
>
> >> resolution are reported in my last comment in the bug record.
>
> >>
>
> >> THE FIX:
>
> >> On a very abstract level the fix functioning can be presented by the
>
> >> next 3 stages:
>
> >>
>
> >> Stage 1.
>
> >> The fix adds support of "WM_TOUCH" system window messages to AWT
>
> >> native peer windows through C++ class "AwtComponent". It "processes"
>
> >> "WM_TOUCH" message and marks "java.awt.event.MouseEvent", which is
>
> >> created as a result of handling of the further coming
>
> >> "WM_LBUTTONDOWN", "WM_LBUTTONUP" messages sent by the system in
>
> >> substitution for this "WM_TOUCH" message, by the new private field
>
> >> flag "MouseEvent.causedByTouchEvent".
>
> >>
>
> >> Stage 2.
>
> >> Then on Java level the fix handles "MouseEvent", "FocusEvent"
>
> >> received only by the instances of "java.awt.TextComponent",
>
> >> "javax.swing.text.TextComponent" in
>
> >> "WToolkit.showOrHideTouchKeyboard()" called from
>
> >> "Component.dispatchEventImpl()" and shows or hidesthe touch keyboard
>
> >> on "MouseEvent.MOUSE_RELEASED" and "FocusEvent.FOCUS_LOST" events by
>
> >> calling corresponding native methods of "WToolkit" class.
>
> >>
>
> >> Stage 3.
>
> >> Showing of the touch keyboard is implemented in C++ class "AwtToolkit"
>
> >> and is done by launching the system application "TabTip.exe" which
>
> >> implements the system touch keyboard. This approach is described in
>
> >> the bug record.
>
> >>
>
> >> FEATURES OF THE FIX:
>
> >> 1. By default all native and Java parts of the fix do not function at
>
> >> all - the fix is disabled. To enable the fix the application should
>
> >> be run with "-Dawt.touchKeyboardAutoShowIsEnabled=true" option.
>
> >> Handling of this new property is implemented in "sun.awt.SunToolkit" 
> class.
>
> >>
>
> >> 2. Native parts of the fix functions only on MS Window 8 or later.
>
> >>
>
> >> 3. The fix implements automatic showing of the touch keyboard for the
>
> >> following 2 use cases:
>
> >> ??? a.? The user touches the text components using the touch screen.
>
> >> ??? b.? The user does mouse clicks on the text components, while no
>
> >> any keyboard is attached to the host.
>
> >>
>
> >> FIX LOGICAL STRUCTURE BY SOURCE CODE:
>
> >> 1. Core of the fix:
>
> >> ??? Native code:? awt_Toolkit.[h/cpp], awt_Component.[h/cpp],
>
> >> awt_MouseEvent.[h/cpp], awt.h
>
> >> ??? Java:? SunToolkit.java, WToolkit.java, Component.java,
>
> >> MouseEvent.java, AWTAccessor.java
>
> >>
>
> >> 2. Changes in all remaining Java files are connected with retaining
>
> >> of the flag value "MouseEvent.causedByTouchEvent" during creation of
>
> >> the new instances of "MouseEvent" class based on the original "MouseEvent"
>
> >> instances.
>
> >>
>
> >> Work of the fix was verified both in the environment with the real
>
> >> touch screen device and in the environment with the emulated touch
>
> >> screen.
>
> >>
>
> >> Thank you,
>
> >> Anton
>
> > 
>
>
> --
>
> Best regards, Sergey.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/swing-dev/attachments/20170914/30ba989f/attachment.html>