[External] : Re: Question: bidi navigation

[Andy Goryachev]

Maybe, just maybe, we should implement the dual carets with directionality indicators (highlighted here for illustration purposes only, the actual indicator might look like a flag or a “ г ”):

[cid:image001.png at 01DA0280.E8714DE0]

What do you think?

-andy

From: Nir Lisker <nlisker at gmail.com>
Date: Thursday, October 19, 2023 at 00:28
To: Philip Race <philip.race at oracle.com>
Cc: Andy Goryachev <andy.goryachev at oracle.com>, openjfx-dev at openjdk.org <openjfx-dev at openjdk.org>
Subject: Re: [External] : Re: Question: bidi navigation
Thanks Andy, the new link works. I read pages 33-36. Page 35 details the two approaches: the dual caret you have described (which I had never seen before in my life), and the single "jumping" caret, which is what I advised to do. The document doesn't show, however, a direction indicator for the caret based on the selected language, and I think that that's something that should be shown.

Phil, I'm not sure how the split caret can tell a user on its own where the text will appear. The user will have to look at which language is currently selected to know which of the carets is the real indicator. That is, unless I misunderstand how the split caret works.
The "why did it appear over there?" question is common in UI's that don't handle the caret indication properly, for example when there is no visual difference between the RTL and LTR carets (the direction indicator). A jumping caret with a visual direction indicator solves 100% of the ambiguity and the surprise factor without needing to look outside of the text control to check which language is selected.

- Nir

On Wed, Oct 18, 2023 at 10:31 PM Philip Race <philip.race at oracle.com<mailto:philip.race at oracle.com>> wrote:
So it seems Swing never calls Java2D's TextLayout.getCaretShapes() API which is what would provide the split carets.

Swing's caret is an instance of javax.swing.text.DefaultCaret which has support for rendering a "flag" that indicates
the direction of the caret bias.

Split caret is however useful to tell a user where the text of the other direction would appear.
Otherwise even to someone familiar with editing bi-di text I except some cases of "oh, why did what I typed appear way over there"?
They might still wonder, but at least they'd know where it was going to be rendered ahead of time.

-phil
On 10/18/23 10:47 AM, Andy Goryachev wrote:
Thank you, Nir.

Try this link:
https://web.archive.org/web/20120802192035/http://developer.apple.com/legacy/mac/library/documentation/Carbon/Conceptual/ATSUI_Concepts/atsui.pdf<https://web.archive.org/web/20120802192035/http:/developer.apple.com/legacy/mac/library/documentation/Carbon/Conceptual/ATSUI_Concepts/atsui.pdf>

Curiously, the dual (split) caret capability is present in java2d, but apparently it is not used in Swing as far as I can tell:

https://docs.oracle.com/javase/tutorial/2d/text/textlayoutbidirectionaltext.html#moving_carets

-andy

From: Nir Lisker <nlisker at gmail.com><mailto:nlisker at gmail.com>
Date: Wednesday, October 18, 2023 at 09:15
To: Andy Goryachev <andy.goryachev at oracle.com><mailto:andy.goryachev at oracle.com>
Cc: openjfx-dev at openjdk.org<mailto:openjfx-dev at openjdk.org> <openjfx-dev at openjdk.org><mailto:openjfx-dev at openjdk.org>
Subject: Re: [External] : Re: Question: bidi navigation
I've never seen this dual caret system. The link you gave leads to a 404 error.

I can't comment as to the plan without knowing what Prism was designed to do, but it's rather unusual. The logical navigation choice seems correct regardless.

On Tue, Oct 17, 2023 at 7:13 PM Andy Goryachev <andy.goryachev at oracle.com<mailto:andy.goryachev at oracle.com>> wrote:
Dear Nir:

Thank you so much for the information.  I spoke to several people none of whom, unfortunately, use an environment configured for RTL mode (but who have keyboard settings for RTL languages).  Based on the very small sample, it appears that logical navigation is a way to go - which means the FX behavior (or rather lack thereof due to https://bugs.openjdk.org/browse/JDK-8296266) needs to change relative to jfx8.

Regarding the direction indicator - FX implements a dual caret the logic of which I am still trying to decipher:


[cid:18b46c3eddd4cff311]


(sliptCaret in PrismTextLayout:354)

The closest description I was able to find is in Apple ATSUI Programming guide
http://developer.apple.com/legacy/mac/library/documentation/Carbon/Conceptual/ATSUI_Concepts/atsui.pdf<https://urldefense.com/v3/__http:/developer.apple.com/legacy/mac/library/documentation/Carbon/Conceptual/ATSUI_Concepts/atsui.pdf__;!!ACWV5N9M2RV99hQ!LM7co48XfI4StmMZQ9TBX3emuN_Y_WM2CEdQi3PZQQ7LfjCCzhZRd1m4jTSc2n4tv6lQD49_YAhItF5cU8ZH$>
on page 35, though I am not sure FX works as described in that document.

The expectation is that the primary (high) caret is where the character having the same directionality as the “primary line direction” is inserted.  It is possible that the text layout determines the primary line direction based on the text as opposed to taking a hint from NodeOrientation.

Since this behavior is baked into FX prism text layout, I think it’s unlikely to change.

To summarize, I think we should switch FX TextInputControl hierarchy to logical navigation.  What do you think?

-andy



From: Nir Lisker <nlisker at gmail.com<mailto:nlisker at gmail.com>>
Date: Tuesday, October 17, 2023 at 03:44
To: Andy Goryachev <andy.goryachev at oracle.com<mailto:andy.goryachev at oracle.com>>
Cc: openjfx-dev at openjdk.org<mailto:openjfx-dev at openjdk.org> <openjfx-dev at openjdk.org<mailto:openjfx-dev at openjdk.org>>
Subject: Re: [External] : Re: Question: bidi navigation
I have just tested on Win10: Notepad, Wordpad, MSWord 2021, Discord desktop, WhatsApp desktop, Opera, Eclipse, Gimp 2, Audacity 2.1.3, and MS VisualStudio 2022 all use logical. Edge uses logical in text areas and text fields, but visual in the address bar (seems like a bug, but you can't have spaces in web addresses anyway). I don't remember ever using a visual navigation application, maybe it was very long ago. If there ever was a decision there, it was made long ago, at least on Windows.

It's very important to show the cursor direction because it resolves ambiguities. It's available in most applications.

My Windows UI is in standard English LTR. I just have an RTL language installed.

Logical navigation is a bit easier to work with I think. The behavior at the edge of a word that changes the direction can be surprising (see the ambiguities above), but that can be helped with the cursor direction indication. If we can do a custom implementation, I would go a step further and actually resolve the position ambiguities by positioning the cursor in accordance with the selected insertion method (RTL or LTR). This means that the cursor will jump when switching the language, but it will make life easier because you will easily know in which direction you're about to type.

On Mon, Oct 16, 2023 at 6:16 PM Andy Goryachev <andy.goryachev at oracle.com<mailto:andy.goryachev at oracle.com>> wrote:
Nir, thank you for responding!

The behavior you describe (“logical” navigation) is what can be seen in many, but not all applications, and that is what puzzles me.  What’s more interesting, the applications that use a “visual” kind of navigation, that is the RIGHT ARROW key always moving the cursor right regardless of the text, is used by javafx8 (it’s totally broken in jfx17, but it looks from the code that it is supposed to be the same as in javafx8), java swing, MS Word 2007 on Windows 10, macOS Notes app, macOS TextEdit, and Mozilla Thunderbird.  Also, this is the kind of navigation that some users prefer (based on a very, very limited sample I was able to contact).

What puzzles me is that there is no apparent standard even among the modern applications (bundled macOS apps), although the transition from visual to logical navigation in MS Word might indicate that the logical navigation is winning.

The appearance of caret is another aspect that seem to have no standard.  In many apps the caret does not change at all, very rarely we see a flag indicating direction (java swing), and only javafx8 and some obsolete mac Carbon reference doc shows a split caret.

More questions for you:

  1.  it looks like you are on Windows, and are you using (or have you seen) a fully localized version of Windows with all the UI set to RTL mode?
  2.  Have you seen any native applications that use the visual navigation model?

Getting back to the problem at hand: if we were to retain the backward compatibility in FX, we would need to fix the “visual” navigation.  FX uses the split caret which some users find confusing but we probably are stuck with it.  If we were to assume that the “logical” navigation is a standard everyone is slowly converging to, then my fix for https://bugs.openjdk.org/browse/JDK-8296266 is the right one and we should declare a change in behavior.

What do you think?

P.S. I wonder if the logical navigation was chosen because of ease of implementation, or is there a deeper reason?




From: Nir Lisker <nlisker at gmail.com<mailto:nlisker at gmail.com>>
Date: Monday, October 16, 2023 at 04:52
To: Andy Goryachev <andy.goryachev at oracle.com<mailto:andy.goryachev at oracle.com>>
Cc: openjfx-dev at openjdk.org<mailto:openjfx-dev at openjdk.org> <openjfx-dev at openjdk.org<mailto:openjfx-dev at openjdk.org>>
Subject: [External] : Re: Question: bidi navigation
This is a tricky one. All applications I have seen, and I think that's what people expect, is that the cursor changes direction during traversal.

A key point is where the paragraph is aligned to (in Windows adjusted with left CTRL+SHIFT and right CTRL+SHIFT). This sets the forward and backward direction: if the paragraph is left-aligned, pressing the right arrow moves the cursor forward, and for a right aligned, the right arrow moves the cursor backward. Then the actual movement of the cursor is relative to the paragraph alignment: in RTL alignment, traversing RTL text moves the cursor forward, while traversing LTR moves the cursor backward.

Examples
In a left-aligned paragraph, pressing the right arrow will move the cursor (|) like this:
|ab אבג cd
a|b אבג cd
ab| אבג cd
ab |אבג cd      OR     ab אבג| cd    (there is ambiguity because the space character can be both RTL or LTR)
ab א|בג cd
ab אב|ג cd
ab אבג| cd      OR     ab |אבג cd
ab אבג |cd
ab אבג c|d
ab אבג cd|

To help with navigation, the cursor has a line attached to its top showing which direction it's facing.

Hope this helps.

On Thu, Oct 12, 2023 at 3:42 AM Andy Goryachev <andy.goryachev at oracle.com<mailto:andy.goryachev at oracle.com>> wrote:
Hi.

I have a question for people who routinely use right-to-left RTL languages (Arabic, Hebrew, etc.):

What is your expectation for navigating text using left/right arrow keys when the text contains a mixture of RTL and LTR?

It looks like there is no standard when it comes to modern applications – see a small sample:
https://gist.github.com/andy-goryachev-oracle/4802f9380fb03ec2be7ac36bd98a2059<https://urldefense.com/v3/__https:/gist.github.com/andy-goryachev-oracle/4802f9380fb03ec2be7ac36bd98a2059__;!!ACWV5N9M2RV99hQ!P_TgGd02CrA1gNF2bW5yHBRJHFkdDqluPJmHDwIcAQ-DR_NWNd-JMkTn0x9d1m5azgCucompGMSgi7PqR7TS$>

In javafx, the navigation of bidirectional (bidi) text might have been broken sometime after jfx8, and even jfx8 might have issues, see
https://bugs.openjdk.org/browse/JDK-8296266

It looks like the most modern applications use logical navigation and logical selection (that is, when navigating using left/right arrow keys, the cursor position reflects previous/next insertion indexes in the text, rather than visual position).  This causes the cursor to change the direction of movement when it crosses the bidi boundary.  Would you say this is the expected behavior?

Thank you
-andy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20231019/9bb745fe/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 12656 bytes
Desc: image001.png
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20231019/9bb745fe/image001-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 9982 bytes
Desc: image001.png
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20231019/9bb745fe/image001-0003.png>