<Swing Dev> <AWT Dev> [10] Review request for 8166772: Touch keyboard is not shown for text components on a screen touch

Fri Sep 15 03:56:17 UTC 2017

Thank you Anton for that information.

Thanks and regards,

Shashi

From: Anton Litvinov 
Sent: Thursday, September 14, 2017 5:51 PM
To: Shashidhara Veerabhadraiah <shashidhara.veerabhadraiah at oracle.com>; Sergey Bylokhov <sergey.bylokhov at oracle.com>; awt-dev at openjdk.java.net; swing-dev at openjdk.java.net
Subject: Re: <AWT Dev> <Swing Dev> [10] Review request for 8166772: Touch keyboard is not shown for text components on a screen touch

Hello Shashi,

Thank you for attention to my review request. Win32 API provides different API sets dedicated to handling various types of user's touch input in different degree of deepness. For example:
1.  Windows Touch Input (WM_TOUCH messages)
2.  Windows Touch Gestures (WM_GESTURE messages)
    Official information about 2 API sets ab: https://msdn.microsoft.com/en-us/library/windows/desktop/dd562100(v=vs.85).aspx
3.  Touch Hit Testing - https://msdn.microsoft.com/en-us/library/windows/desktop/hh437255(v=vs.85).aspx

WM_POINTER may be yet another approach.

I chose to exploit API related to "WM_TOUCH" message, because this API, as I understand, provides more opportunities to control the touch input then other 2 APIs specified above, and it allows to handle multiple touch points, what may become necessary in future.

RESPONSE TO YOUR REMARK:
I cannot agree that "WM_TOUCH" is officially deprecated. In the official Win32 API specification there is no information that functions from this API are deprecated or obsolete. For example:
1.  RegisterTouchWindow - https://msdn.microsoft.com/en-us/library/windows/desktop/dd317326(v=vs.85).aspx
2.  GetTouchInputInfo - https://msdn.microsoft.com/en-us/library/windows/desktop/dd371582(v=vs.85).aspx
3.  TOUCHINPUT - https://msdn.microsoft.com/en-us/library/windows/desktop/dd317334(v=vs.85).aspx
4.  WM_TOUCH - https://msdn.microsoft.com/en-us/library/windows/desktop/dd317341(v=vs.85).aspx
    (There is some community comment, but community comments are written by anybody, deprecation is not announced in community comments. For example, the really deprecated function is "GetVersion" - (https://msdn.microsoft.com/en-us/library/windows/desktop/ms724439(v=vs.85).aspx)).

The web page, whose reference you provided is just a forum, is not Win32 API specification or official announcement from Microsoft. So I would not rely on this information.

Additionally approach based on "WM_TOUCH" showed itself in my test environment reliably, WM_TOUCH are generated stably in MS Windows 10 OS, also I was able to successfully port the fix to JDK 8 and verify that it works. Currently I am working on addressing remarks from Sergey and do not plan to refuse from currently developed real basement of the whole fix, which uses WM_TOUCH.

Thank you,
Anton

On 14/09/2017 10:55, Shashidhara Veerabhadraiah wrote:

Hi Anton, I have a question to ask for. I was working on another bug and found that the WM_TOUCH has been deprecated and instead we should be using the WM_POINTER? Here is some info on that: https://stackoverflow.com/questions/23790602/wm-touch-vs-wm-pointer

Typically, the WM_POINTER types can handle following type of pointers:

typedef enum tagPOINTER_INPUT_TYPE { 

  PT_POINTER   = 0x00000001,

  PT_TOUCH     = 0x00000002,

  PT_PEN       = 0x00000003,

  PT_MOUSE     = 0x00000004,

  PT_TOUCHPAD  = 0x00000005

} POINTER_INPUT_TYPE;

Hope this is useful for your fix.

Thanks and regards,

Shashi

-----Original Message-----
From: Sergey Bylokhov
Sent: Tuesday, September 5, 2017 10:45 PM
To: Anton Litvinov HYPERLINK "mailto:anton.litvinov at oracle.com"<anton.litvinov at oracle.com>; HYPERLINK "mailto:awt-dev at openjdk.java.net"awt-dev at openjdk.java.net; HYPERLINK "mailto:swing-dev at openjdk.java.net"swing-dev at openjdk.java.net
Subject: Re: <AWT Dev> <Swing Dev> [10] Review request for 8166772: Touch keyboard is not shown for text components on a screen touch

Hi, Anton.

The fix looks good.

  - But can you please recheck that is is not necessary to use additional synchronization in showOrHideTouchKeyboard() if a few EDT will be used.

  - I suggest to invert the awt.touchKeyboardAutoShowIsEnabled and use true as default value, we will have more coverage and feedback in this case. This property will be used as a workaround if some bugs will be found.

On 8/30/17 11:51, Anton Litvinov wrote:

> Hello dear reviewers,

> 

> Could anybody please look at this review request?

> 

> Thank you,

> Anton

> 

> On 17/08/2017 13:20, Anton Litvinov wrote:

>> Hello,

>> 

>> Could you please review the following fix for the bug.

>> 

>> Bug: https://bugs.openjdk.java.net/browse/JDK-8166772

>> Webrev: HYPERLINK "http://cr.openjdk.java.net/%7Ealitvinov/8166772/jdk10/webrev.00"http://cr.openjdk.java.net/~alitvinov/8166772/jdk10/webrev.00

>> 

>> The bug is the fact that, when a user touches any Swing or AWT text 

>> component, for example "JTextField", "JTextArea", "TextField", 

>> "TextArea", by means of a touch screen on a host with MS Windows

>> 10/8.1/8 OS, the system touch keyboard is not shown automatically. 

>> Please find a detailed description of the bug, screenshots depicting 

>> the touch keyboard and a compilable test case with Swing/AWT text 

>> components in JBS bug record. Also a summary of the done research of 

>> the issue with a description of identified approaches for its 

>> resolution are reported in my last comment in the bug record.

>> 

>> THE FIX:

>> On a very abstract level the fix functioning can be presented by the 

>> next 3 stages:

>> 

>> Stage 1.

>> The fix adds support of "WM_TOUCH" system window messages to AWT 

>> native peer windows through C++ class "AwtComponent". It "processes"

>> "WM_TOUCH" message and marks "java.awt.event.MouseEvent", which is 

>> created as a result of handling of the further coming 

>> "WM_LBUTTONDOWN", "WM_LBUTTONUP" messages sent by the system in 

>> substitution for this "WM_TOUCH" message, by the new private field 

>> flag "MouseEvent.causedByTouchEvent".

>> 

>> Stage 2.

>> Then on Java level the fix handles "MouseEvent", "FocusEvent" 

>> received only by the instances of "java.awt.TextComponent", 

>> "javax.swing.text.TextComponent" in 

>> "WToolkit.showOrHideTouchKeyboard()" called from 

>> "Component.dispatchEventImpl()" and shows or hides the touch keyboard 

>> on "MouseEvent.MOUSE_RELEASED" and "FocusEvent.FOCUS_LOST" events by 

>> calling corresponding native methods of "WToolkit" class.

>> 

>> Stage 3.

>> Showing of the touch keyboard is implemented in C++ class "AwtToolkit" 

>> and is done by launching the system application "TabTip.exe" which 

>> implements the system touch keyboard. This approach is described in 

>> the bug record.

>> 

>> FEATURES OF THE FIX:

>> 1. By default all native and Java parts of the fix do not function at 

>> all - the fix is disabled. To enable the fix the application should 

>> be run with "-Dawt.touchKeyboardAutoShowIsEnabled=true" option. 

>> Handling of this new property is implemented in "sun.awt.SunToolkit" class.

>> 

>> 2. Native parts of the fix functions only on MS Window 8 or later.

>> 

>> 3. The fix implements automatic showing of the touch keyboard for the 

>> following 2 use cases:

>> ??? a.? The user touches the text components using the touch screen.

>> ??? b.? The user does mouse clicks on the text components, while no 

>> any keyboard is attached to the host.

>> 

>> FIX LOGICAL STRUCTURE BY SOURCE CODE:

>> 1. Core of the fix:

>> ??? Native code:? awt_Toolkit.[h/cpp], awt_Component.[h/cpp], 

>> awt_MouseEvent.[h/cpp], awt.h

>> ??? Java:? SunToolkit.java, WToolkit.java, Component.java, 

>> MouseEvent.java, AWTAccessor.java

>> 

>> 2. Changes in all remaining Java files are connected with retaining 

>> of the flag value "MouseEvent.causedByTouchEvent" during creation of 

>> the new instances of "MouseEvent" class based on the original "MouseEvent"

>> instances.

>> 

>> Work of the fix was verified both in the environment with the real 

>> touch screen device and in the environment with the emulated touch 

>> screen.

>> 

>> Thank you,

>> Anton

> 

--

Best regards, Sergey.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/swing-dev/attachments/20170915/0c0ff69b/attachment.html>