[OpenJDK 2D-Dev] HiDPI support issues on Windows

[Sergey Bylokhov]

hi, Jim.
Can you please take a look to the small example(attached) which was 
created by Alex.
The test uses 3 operations: fillRect, drawImage, drawRect.
Each time we draw something to the left rectangle [0,0,w,h] and to the 
right rectangle[w,0,w,h].
I can understand why fillrect draw the shapes one after another, w/o 
overlapping and gaps.
I also understand why results of drawRect are overlaps, sine w,h is a 
middle of the line.

But why the images are overlapped? So results on the screen will be 
depending on what image was painted last time. Note that the clip is set 
for both images, but this does not help. Also the clips cannot prevent 
overlapping of drawrect(I guess it should).


On 05.10.16 22:23, Jim Graham wrote:
> Hi Sergey,
>
> I'd be interested in the details of that analysis.  Are you saying that
> drawImage can violate the clip?  That would be a big problem if true.
>
> I looked at the clip validation code and did spot an issue.  When the
> clip is a Rectangle2D or a transformed Rectangle (which produces an R2D
> after transformation in some cases), the clip is set from
> r2d.getBounds() (line 1908 in SG2D) which performs a floor/ceil
> operation, but it should be governed by the center-of-pixel inclusion
> rule that governs fills.  It should instead be something like:
>
> x0 = ceil(r2d.x - 0.5);
> y0 = ceil(r2d.y - 0.5);
> x1 = ceil(r2d.getMaxX() - 0.5);
> y1 = ceil(r2d.getMaxY() - 0.5);
> (see the code in copyArea for an example of this)
>
> We could probably add code like that to the Region class in case it is
> needed in other places...
>
>             ...jim
>
> On 10/5/16 5:34 AM, Sergey Bylokhov wrote:
>> Looking to this bug I found something new for me. At least rounding of
>> the image is non-intuitive. Usually our animation
>> is a sequence of draw/drawImage+fillRect/clearRect. But in case of
>> fractional scale we transform drawImage and fillRect
>> differently, which can cause some artifacts, because drawImage can
>> fill more pixels than fillRect(even if the clip is set).
>>
>> Some other root of artifacts is a usage of vectors API like
>> drawLine/drawRect, which can produce artifacts outside the
>> component if the clip is incorrectly set and if it set properly such
>> API can produce too thin lines.
>>
>> On 02.10.16 22:10, Jim Graham wrote:
>>> After looking into the code in RepaintManager and re-reading Alexander's
>>> message again I can see how it describes what is going on more clearly.
>>>
>>> Fixing the rounding errors doesn't necessarily require avoiding use of
>>> the intermediate image for damage repair, you just have to make sure
>>> that you use the incoming xywh as suggestions for what needs to be
>>> redrawn, but instead determine exact pixels that you will repaint
>>> (usually floor,floor,ceil,ceil to "round out" the area), and then use
>>> those pixel-precise locations instead of passing along the integers that
>>> came from the repaint requests and hoping for the right rounding.  The
>>> problem is that a number of the interfaces used by the RepaintManager
>>> take integers and hide a scale from the caller so we need to either work
>>> around their implicit scale, or possible create internal variants that
>>> let us work in pixels.
>>>
>>> In other words, the typical boilerplate for intermediate image damage
>>> repair would be:
>>>
>>> // repainting x,y,w,h
>>> img = make image (w,h)
>>> g = img.getGraphics()
>>> g.setClip(x,y,w,h)
>>> g.translate(-x,-y)
>>> component.paint(g)
>>> destination.drawImage(img, x,y)
>>>
>>> but that boilerplate only works if x,y are exact pixel coordinates, but
>>> since it is all being doing on a scaled graphics then x,y will transform
>>> to arbitrary not-necessarily-integer locations and then all bets are
>>> off.
>>>
>>> Fixing this could either rely on using float interfaces wherever
>>> available, or by undoing all of the implicit scales and working in
>>> pixels, but being aware of the scale that is required for the
>>> destination.  Something like one of these boilerplates instead:
>>>
>>> // repainting x,y,w,h integers using floats
>>> float pixelx1 = floor(x * scaleX)
>>> float pixely1 = floor(y * scaleY)
>>> float pixelx2 = ceil((x+w) * scaleX)
>>> float pixely2 = ceil((y+h) * scaleY)
>>> int pixelw = (int) (pixelx2 - pixelx1)
>>> int pixelh = (int) (pixely2 - pixely1)
>>> // Note that the code currently asks the destination to make
>>> // a compatible image of a virtual pixel size that is then
>>> // scaled to match.  A "make me an image of this many pixels"
>>> // might be less cumbersome.
>>> img = make image (ceil(pixelw / scaleX),
>>>                   ceil(pixelh / scaleY))
>>> g = img.getGraphics() // will be scaled already
>>> // The following will use the translate(double, double) method
>>> g.setClip(new Rectangle2D.Double(pixel* / scale*))
>>> g.translate(-pixelx1 / scaleX, -pixely1 / scaleY)
>>> component.paint(g)
>>> // Since there is no drawImage(img, float x, float y)...
>>> destination.translate(pixelx1 / scaleX, pixely1 / scaleY)
>>> destination.drawImage(img, 0, 0)
>>> // (restore transforms where needed)
>>>
>>> That version uses floating point interfaces in a number of key places
>>> (notably translate() calls are available as either int or double in the
>>> Graphics and have to use the setClip(Shape) method to specify a floating
>>> point rectangle), but a down side is that using those interfaces means
>>> that you have a value that you know is at a pixel boundary and you pass
>>> it in as "number / scale" only to have the code in the Graphics
>>> immediately apply that scale and you end up with the final result of
>>> "number / scale * scale" which might incur round-off errors and end up
>>> being slightly off of a pixel.
>>>
>>> In another approach, you could also kill all of the transforms and do it
>>> more directly in pixels as in the following:
>>>
>>> // repainting x,y,w,h integers using unscaled operations
>>> // Some parts more cumbersome to undo the implicit scaling
>>> // but it won't suffer from round-off errors when constantly
>>> // scaling and unscaling through the various interfaces
>>> // that have transforms built in
>>> int pixelx1 = (int) floor(x * scaleX)
>>> int pixely1 = (int) floor(y * scaleY)
>>> int pixelx2 = (int) ceil((x+w) * scaleX)
>>> int pixely2 = (int) ceil((y+h) * scaleY)
>>> int pixelw = pixelx2 - pixelx1;
>>> int pixelh = pixely2 - pixely1;
>>> // Not sure if there is a mechanism for this since I think
>>> // all of the interfaces to get a compatible image are
>>> // designed to assume that the caller is not scale-aware.
>>> img = make pixel-sized image (pixelw, pixelh)
>>> g = img.getGraphics()
>>> // assuming that g would be unscaled in this case, but
>>> // if g is scaled, then g.setTransform(IDENTITY)
>>> // translate by an integer amount, and then scale
>>> g.setClip(pixelx1, pixely1, pixelw, pixelh)
>>> g.translate(pixelx1, pixely1)
>>> g.scale(scaleX, scaleY);
>>> component.paint(g)
>>> destinationg.setTransform(IDENTITY)
>>> destinationg.drawImage(img, pixelx1, pixely1)
>>> // (restore transforms where needed)
>>>
>>>             ...jim
>>>
>>> On 9/30/2016 1:30 PM, Jim Graham wrote:
>>>>
>>>>
>>>> On 9/30/16 3:22 AM, Alexandr Scherbatiy wrote:
>>>>> The problem is that the RepaintManager draws a region to a buffered
>>>>> image at first and draws the image after that to the
>>>>> window.
>>>>> Suppose the image has int coordinates and size (x, y, w, h) in the
>>>>> user space. It should be drawn into the region with
>>>>> coordinates (x, y, x+width, y+height) = (x1, y1, x2, y2).
>>>>> If floating point UI scale is used (like 1.5) the region coordinates
>>>>> are converted to values (1.5 * x1, 1.5 * y1, 1.5 *
>>>>> x2, 1.5 * y2) in the dev space.
>>>>> Now these coordinates need to be rounded and the process really
>>>>> depends on the evenness or oddness of the start and end
>>>>> coordinates. They both can be rounded to one side or to opposite.
>>>>> Depending on this some lines near the drawn image
>>>>> region can be not filled or just wrongly filled.
>>>>
>>>> The repaint manager should compute the nearest pixel bounds outside of
>>>> the scaled repaint area, and then adjust the rendering to repaint
>>>> all of
>>>> those pixels.  You don't "round" here, you "floor,floor,ceil,ceil" (and
>>>> then worry how to present the clip region to the app so it can do the
>>>> right thing - probably by clipping to a Rect2D.Float() and letting the
>>>> integer g.getClipBounds() further round out the coordinates which means
>>>> extra paint calls, but at least you'll repaint all the dirty pixels and
>>>> they will be blitted to the right destination pixels if the code that
>>>> sends them to the screen is aware of the full resolution...)
>>>>
>>>>                 ...jim
>>
>>


-- 
Best regards, Sergey.
-------------- next part --------------
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;

public final class FPDrawImageTest {

    static final Color LEFT_COLOR = new Color(255, 0, 0, 100);

    static final Color RIGHT_COLOR = new Color(0, 0, 255, 100);

    public static void main(final String[] args) throws Exception {

        int w = 7;
        int h = 7;
        // Init image
        BufferedImage img = new BufferedImage(70, 70, BufferedImage.TYPE_INT_ARGB);
        Graphics2D g = img.createGraphics();
        g.setColor(Color.WHITE);
        g.fillRect(0, 0, img.getWidth(), img.getHeight());

        //g.scale(1, 1);
        g.scale(1.5, 1.5);

        // FillRect
        g.setColor(LEFT_COLOR);
        g.fillRect(0, 0, w, h);

        g.setColor(RIGHT_COLOR);
        g.fillRect(w, 0, w, h);

        // MoveTo the next line
        g.translate(0, h + 3);

        // DrawImage
        BufferedImage img1 = createImage(w, h, LEFT_COLOR);
        g.setClip(0, 0, w, h);
        g.drawImage(img1, 0, 0, null);
        BufferedImage img2 = createImage(w, h, RIGHT_COLOR);
        g.setClip(w, 0, w, h);
        g.drawImage(img2, w, 0, null);
        g.setClip(null);

        // MoveTo the next line
        g.translate(0, h + 3);

        // DrawRect
        g.setColor(LEFT_COLOR);
        g.setClip(0, 0, w, h);
        g.drawRect(0, 0, w, h);
        g.setColor(RIGHT_COLOR);
        g.setClip(w, 0, w, h);
        g.drawRect(w, 0, w, h);

        g.dispose();

        ImageIO.write(img, "png", new File("two-images-clip-s150.png"));
    }

    private static BufferedImage createImage(int w, int h, Color c1) {
        BufferedImage img = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
        Graphics2D g = img.createGraphics();
        g.setColor(c1);
        g.fillRect(0, 0, w, h);
        return img;
    }
}