[OpenJDK 2D-Dev] HiDPI support issues on Windows

[Sergey Bylokhov]

Immediately after I sent a message I realized that we had similar 
discussion about clip already.
It seems this is the same bug as discussed here:
http://mail.openjdk.java.net/pipermail/2d-dev/2016-July/007299.html

And those solution fix this bug as well. I will file a separate bug and 
send a review request soon.

On 06.10.16 16:02, Sergey Bylokhov wrote:
> hi, Jim.
> Can you please take a look to the small example(attached) which was
> created by Alex.
> The test uses 3 operations: fillRect, drawImage, drawRect.
> Each time we draw something to the left rectangle [0,0,w,h] and to the
> right rectangle[w,0,w,h].
> I can understand why fillrect draw the shapes one after another, w/o
> overlapping and gaps.
> I also understand why results of drawRect are overlaps, sine w,h is a
> middle of the line.
>
> But why the images are overlapped? So results on the screen will be
> depending on what image was painted last time. Note that the clip is set
> for both images, but this does not help. Also the clips cannot prevent
> overlapping of drawrect(I guess it should).
>
>
> On 05.10.16 22:23, Jim Graham wrote:
>> Hi Sergey,
>>
>> I'd be interested in the details of that analysis.  Are you saying that
>> drawImage can violate the clip?  That would be a big problem if true.
>>
>> I looked at the clip validation code and did spot an issue.  When the
>> clip is a Rectangle2D or a transformed Rectangle (which produces an R2D
>> after transformation in some cases), the clip is set from
>> r2d.getBounds() (line 1908 in SG2D) which performs a floor/ceil
>> operation, but it should be governed by the center-of-pixel inclusion
>> rule that governs fills.  It should instead be something like:
>>
>> x0 = ceil(r2d.x - 0.5);
>> y0 = ceil(r2d.y - 0.5);
>> x1 = ceil(r2d.getMaxX() - 0.5);
>> y1 = ceil(r2d.getMaxY() - 0.5);
>> (see the code in copyArea for an example of this)
>>
>> We could probably add code like that to the Region class in case it is
>> needed in other places...
>>
>>             ...jim
>>
>> On 10/5/16 5:34 AM, Sergey Bylokhov wrote:
>>> Looking to this bug I found something new for me. At least rounding of
>>> the image is non-intuitive. Usually our animation
>>> is a sequence of draw/drawImage+fillRect/clearRect. But in case of
>>> fractional scale we transform drawImage and fillRect
>>> differently, which can cause some artifacts, because drawImage can
>>> fill more pixels than fillRect(even if the clip is set).
>>>
>>> Some other root of artifacts is a usage of vectors API like
>>> drawLine/drawRect, which can produce artifacts outside the
>>> component if the clip is incorrectly set and if it set properly such
>>> API can produce too thin lines.
>>>
>>> On 02.10.16 22:10, Jim Graham wrote:
>>>> After looking into the code in RepaintManager and re-reading
>>>> Alexander's
>>>> message again I can see how it describes what is going on more clearly.
>>>>
>>>> Fixing the rounding errors doesn't necessarily require avoiding use of
>>>> the intermediate image for damage repair, you just have to make sure
>>>> that you use the incoming xywh as suggestions for what needs to be
>>>> redrawn, but instead determine exact pixels that you will repaint
>>>> (usually floor,floor,ceil,ceil to "round out" the area), and then use
>>>> those pixel-precise locations instead of passing along the integers
>>>> that
>>>> came from the repaint requests and hoping for the right rounding.  The
>>>> problem is that a number of the interfaces used by the RepaintManager
>>>> take integers and hide a scale from the caller so we need to either
>>>> work
>>>> around their implicit scale, or possible create internal variants that
>>>> let us work in pixels.
>>>>
>>>> In other words, the typical boilerplate for intermediate image damage
>>>> repair would be:
>>>>
>>>> // repainting x,y,w,h
>>>> img = make image (w,h)
>>>> g = img.getGraphics()
>>>> g.setClip(x,y,w,h)
>>>> g.translate(-x,-y)
>>>> component.paint(g)
>>>> destination.drawImage(img, x,y)
>>>>
>>>> but that boilerplate only works if x,y are exact pixel coordinates, but
>>>> since it is all being doing on a scaled graphics then x,y will
>>>> transform
>>>> to arbitrary not-necessarily-integer locations and then all bets are
>>>> off.
>>>>
>>>> Fixing this could either rely on using float interfaces wherever
>>>> available, or by undoing all of the implicit scales and working in
>>>> pixels, but being aware of the scale that is required for the
>>>> destination.  Something like one of these boilerplates instead:
>>>>
>>>> // repainting x,y,w,h integers using floats
>>>> float pixelx1 = floor(x * scaleX)
>>>> float pixely1 = floor(y * scaleY)
>>>> float pixelx2 = ceil((x+w) * scaleX)
>>>> float pixely2 = ceil((y+h) * scaleY)
>>>> int pixelw = (int) (pixelx2 - pixelx1)
>>>> int pixelh = (int) (pixely2 - pixely1)
>>>> // Note that the code currently asks the destination to make
>>>> // a compatible image of a virtual pixel size that is then
>>>> // scaled to match.  A "make me an image of this many pixels"
>>>> // might be less cumbersome.
>>>> img = make image (ceil(pixelw / scaleX),
>>>>                   ceil(pixelh / scaleY))
>>>> g = img.getGraphics() // will be scaled already
>>>> // The following will use the translate(double, double) method
>>>> g.setClip(new Rectangle2D.Double(pixel* / scale*))
>>>> g.translate(-pixelx1 / scaleX, -pixely1 / scaleY)
>>>> component.paint(g)
>>>> // Since there is no drawImage(img, float x, float y)...
>>>> destination.translate(pixelx1 / scaleX, pixely1 / scaleY)
>>>> destination.drawImage(img, 0, 0)
>>>> // (restore transforms where needed)
>>>>
>>>> That version uses floating point interfaces in a number of key places
>>>> (notably translate() calls are available as either int or double in the
>>>> Graphics and have to use the setClip(Shape) method to specify a
>>>> floating
>>>> point rectangle), but a down side is that using those interfaces means
>>>> that you have a value that you know is at a pixel boundary and you pass
>>>> it in as "number / scale" only to have the code in the Graphics
>>>> immediately apply that scale and you end up with the final result of
>>>> "number / scale * scale" which might incur round-off errors and end up
>>>> being slightly off of a pixel.
>>>>
>>>> In another approach, you could also kill all of the transforms and
>>>> do it
>>>> more directly in pixels as in the following:
>>>>
>>>> // repainting x,y,w,h integers using unscaled operations
>>>> // Some parts more cumbersome to undo the implicit scaling
>>>> // but it won't suffer from round-off errors when constantly
>>>> // scaling and unscaling through the various interfaces
>>>> // that have transforms built in
>>>> int pixelx1 = (int) floor(x * scaleX)
>>>> int pixely1 = (int) floor(y * scaleY)
>>>> int pixelx2 = (int) ceil((x+w) * scaleX)
>>>> int pixely2 = (int) ceil((y+h) * scaleY)
>>>> int pixelw = pixelx2 - pixelx1;
>>>> int pixelh = pixely2 - pixely1;
>>>> // Not sure if there is a mechanism for this since I think
>>>> // all of the interfaces to get a compatible image are
>>>> // designed to assume that the caller is not scale-aware.
>>>> img = make pixel-sized image (pixelw, pixelh)
>>>> g = img.getGraphics()
>>>> // assuming that g would be unscaled in this case, but
>>>> // if g is scaled, then g.setTransform(IDENTITY)
>>>> // translate by an integer amount, and then scale
>>>> g.setClip(pixelx1, pixely1, pixelw, pixelh)
>>>> g.translate(pixelx1, pixely1)
>>>> g.scale(scaleX, scaleY);
>>>> component.paint(g)
>>>> destinationg.setTransform(IDENTITY)
>>>> destinationg.drawImage(img, pixelx1, pixely1)
>>>> // (restore transforms where needed)
>>>>
>>>>             ...jim
>>>>
>>>> On 9/30/2016 1:30 PM, Jim Graham wrote:
>>>>>
>>>>>
>>>>> On 9/30/16 3:22 AM, Alexandr Scherbatiy wrote:
>>>>>> The problem is that the RepaintManager draws a region to a buffered
>>>>>> image at first and draws the image after that to the
>>>>>> window.
>>>>>> Suppose the image has int coordinates and size (x, y, w, h) in the
>>>>>> user space. It should be drawn into the region with
>>>>>> coordinates (x, y, x+width, y+height) = (x1, y1, x2, y2).
>>>>>> If floating point UI scale is used (like 1.5) the region coordinates
>>>>>> are converted to values (1.5 * x1, 1.5 * y1, 1.5 *
>>>>>> x2, 1.5 * y2) in the dev space.
>>>>>> Now these coordinates need to be rounded and the process really
>>>>>> depends on the evenness or oddness of the start and end
>>>>>> coordinates. They both can be rounded to one side or to opposite.
>>>>>> Depending on this some lines near the drawn image
>>>>>> region can be not filled or just wrongly filled.
>>>>>
>>>>> The repaint manager should compute the nearest pixel bounds outside of
>>>>> the scaled repaint area, and then adjust the rendering to repaint
>>>>> all of
>>>>> those pixels.  You don't "round" here, you "floor,floor,ceil,ceil"
>>>>> (and
>>>>> then worry how to present the clip region to the app so it can do the
>>>>> right thing - probably by clipping to a Rect2D.Float() and letting the
>>>>> integer g.getClipBounds() further round out the coordinates which
>>>>> means
>>>>> extra paint calls, but at least you'll repaint all the dirty pixels
>>>>> and
>>>>> they will be blitted to the right destination pixels if the code that
>>>>> sends them to the screen is aware of the full resolution...)
>>>>>
>>>>>                 ...jim
>>>
>>>
>
>


-- 
Best regards, Sergey.