<AWT Dev> [OpenJDK 2D-Dev] [9] Review request for 8029339 Custom MultiResolution image support on HiDPI displays
Alexander Scherbatiy
alexandr.scherbatiy at oracle.com
Fri Apr 4 11:53:38 UTC 2014
On 4/3/2014 2:23 AM, Jim Graham wrote:
> Hi Alexandr,
>
> The back and forth is getting confusing here, so I thought I'd try to
> summarize and start fresh(ish):
>
> 1. We need to support @2x internally for MacOS compatibility (done).
>
> 2. We will need to support _DPI images for Win-DPI compatibility (TBD).
>
> 3. Customers may have their own collection of images to bundle
> together into an MR image (working on that here). What is the push
> for this? Is this simply parity with Mac interfaces?
----------
Image[] resolutionVariants = // get sorted by sizes array of
resolution variants;
Image mrImage =
Toolkit.getDefaultToolkit().createMRImage(baseImageIndex,
resolutionVariants);
----------
Here is the proposed patch:
http://cr.openjdk.java.net/~alexsch/8029339/webrev.04/
> 4. Customers may want to synthetically generate images at arbitrary
> resolutions (a variation that is impacting this solution). What is
> the push for this?
----------
Image mrImage =
Toolkit.getDefaultToolkit().createMRImage(baseImageWidth, baseImageHeight,
new float[][]{{100, 100}, {150, 150}, {200, 200}}, //
resolution variants sizes
(rvWidth, rvHeight) -> { /* generate a resolution
variant */ });
----------
>
> 5. I'm guessing that customers might want to override the logic to
> choose from among multiple resolutions. That came from me based on
> seeing Mac and Win using different selection logic and our history of
> developers split between those wanting cross-platform consistency and
> those wanting consistency with native apps on each platform. Also,
> the needs of an animator may differ from the needs of a
> resolution-settable-document editor as to how dynamically the images
> shift between resolution variants.
----------
Image[] resolutionVariants = // get sorted by sizes array of
resolution variants;
Image mrImage = ImageResolutionHelper.createMRImage(
(rvWidth, rvHeight, resolutionVariants) -> { /*use a
custom logic to choose a resolution variant from an array of images*/},
(logicalDPI, baseImageSize, destImageSize) ->
destImageSize, // calculate the custom aware resolution variant size
baseImageIndex, resolutionVariants);
----------
or just extend the CustomMultiResolutionImage which has Image as the
parent class:
--------------------
public class CustomMultiResolutionImage extends
AbstractMultiResolutionImage {
@Override
public Image getResolutionVariant(float logicalDPIX, float
logicalDPIY,
float baseImageWidth, float baseImageHeight,
float destImageWidth, float destImageHeight) {
// return a resolution variant based on the given logical DPI,
// base image size, or destination image size
}
@Override
public List<Image> getResolutionVariants() {
// return a list of resolution variants
}
@Override
protected Image getBaseImage() {
// return the base image
}
}
--------------------
>
> Is that a fair summary of all of the considerations so far, or did I
> miss something?
I think it should cover the main needs.
Thanks,
Alexandr.
>
> ...jim
>
> On 3/27/14 7:43 AM, Alexander Scherbatiy wrote:
>>
>> Below are some thoughts about TK.createMRImage(...) method
>>
>> On 3/24/2014 4:52 PM, Alexander Scherbatiy wrote:
>>> Hello,
>>>
>>> Could you review the updated fix:
>>> http://cr.openjdk.java.net/~alexsch/8029339/webrev.03/
>>>
>>> - baseImageWidth/Height arguments are added to the
>>> getResolutionVariant(...) method
>>> - dest image sizes are reverted to included DPI scale
>>> - AbstractMultiResolutionImage is added. It needs only to implement
>>> only 3 methods from the AbstractMultiResolutionImage class
>>> to create a custom multi-resolution image. For example:
>>>
>>> On 3/22/2014 3:57 AM, Jim Graham wrote:
>>>>
>>>> Your code example below can be expressed as an implementation of the
>>>> single-method, lambda-compatible interface that expresses just the
>>>> getRV() method. They could easily do:
>>>>
>>>> final Image baseImage = ...;
>>>> TK.createMRImage(new RVInterface() {
>>>> public Image getRV(...) {
>>>> // calculate rvWidth and rvHeight
>>>> // look up rvWidth/rvHeight in a database of images
>>>> // possibly contruct a new image
>>>> return rvImage;
>>>> }
>>>> }, baseImage);
>>>>
>> The RVInterface mixes the logic that construct an image and
>> chooses the necessary resolution variant.
>> It is ok if a developer always implements this interface. If it
>> needs to have DPI/Transform/Platform aware RVInterface the image
>> construction logic should be separated.
>>
>> Does TK.createMRImage() method implies that Platform aware logic
>> should be used for a resolution-variant choosing?
>> If so, may be general createMRImage() can be placed in the
>> ImageResolutionHelper.
>>>> The main issue I see is if you might want the newly constructed
>>>> variants to appear in the List returned from the getVariants()
>>>> method. I'm not sure what value that would have beyond simply
>>>> returning the base media that the object uses from which to construct
>>>> its variants...?
>>
>> It can be solved by using something like array of image sizes or
>> other seeds and a mapper that can create an image from the given seed.
>>
>> It can look like:
>> -------------------------
>> public class ImageResolutionHelper {
>> public interface RVChooser {
>>
>> public Image getRV(
>> float logicalDPIX, float logicalDPIY,
>> float baseImageWidth, float baseImageHeight,
>> float destImageWidth, float destImageHeight,
>> final Image... resolutionVariants);
>> }
>>
>> public static final RVChooser DPI_AWARE = ...;
>> public static final RVChooser TRANSFORM_AWARE = ...;
>>
>> // resolutionVariants is an array of sorted by width/height images
>> static Image createMRImage(final RVChooser rvChooser,
>> final int baseImageIndex, final Image...
>> resolutionVariants) { ... }
>>
>> // sorted by width/height images should be generated from seeds
>> static <Type> Image createMRImage(final RVChooser rvChooser,
>> final Type baseImageSeed, final Function<Type, Image>
>> mapper, final Type... rvSeeds) {...}
>> }
>>
>> public abstract class Toolkit {
>> public abstract Image createMRImage(int baseImageIndex, Image...
>> resolutionVariants); // Platform aware rv chooser is used
>> public abstract RVChooser getPlatformRVChooser() ;
>> }
>> --------------------------
>> Thanks,
>> Alexandr.
>>
>>>>
>>>>> I think it is better to provide both the MultiResolutionImage
>>>>> and
>>>>> its implementation based on the given resolution variants array.
>>>>
>>>> It occurs to me that even if we don't go with a lambda-factory-based
>>>> approach like what I'm describing, it might make sense to provide a
>>>> baseMR implementation that they can subclass to keep them from trying
>>>> to subclass off of BufferedImage instead. I really would like to
>>>> avoid "custom MR images are subclasses of BufImg" if we can as I
>>>> think the mix of concepts is a little jarring...
>>>>
>>>> ...jim
>>>>
>>>>> The implementation could look like:
>>>>> ---------------------------------
>>>>> public class CustomMultiResolutionImage extends Image implements
>>>>> MultiResolutionImage {
>>>>>
>>>>> int baseImageIndex;
>>>>> Image[] resolutionVariants;
>>>>>
>>>>> public CustomMultiResolutionImage(int baseImageIndex,
>>>>> Image... resolutionVariants) {
>>>>> this.baseImageIndex = baseImageIndex;
>>>>> this.resolutionVariants = resolutionVariants;
>>>>> }
>>>>>
>>>>> @Override
>>>>> public int getWidth(ImageObserver observer) {
>>>>> return getBaseImage().getWidth(null);
>>>>> }
>>>>>
>>>>> @Override
>>>>> public int getHeight(ImageObserver observer) {
>>>>> return getBaseImage().getHeight(null);
>>>>> }
>>>>>
>>>>> @Override
>>>>> public ImageProducer getSource() {
>>>>> return getBaseImage().getSource();
>>>>> }
>>>>>
>>>>> @Override
>>>>> public Graphics getGraphics() {
>>>>> return getBaseImage().getGraphics();
>>>>> }
>>>>>
>>>>> @Override
>>>>> public Object getProperty(String name, ImageObserver observer) {
>>>>> return getBaseImage().getProperty(name, observer);
>>>>> }
>>>>>
>>>>> @Override
>>>>> public Image getResolutionVariant(float logicalDPIX, float
>>>>> logicalDPIY,
>>>>> float destinationImageWidth, float
>>>>> destinationImageHeight) {
>>>>> // calculate resolution variant width/height
>>>>> return getResolutionVariant(rvWidth, rvHeight);
>>>>> }
>>>>>
>>>>> @Override
>>>>> public List<Image> getResolutionVariants() {
>>>>> return Arrays.asList(resolutionVariants);
>>>>> }
>>>>>
>>>>> private Image getResolutionVariant(float rvWidth, float
>>>>> rvHeight) {
>>>>> // return a resolution variant based on the given width and
>>>>> height
>>>>> }
>>>>>
>>>>> private Image getBaseImage() {
>>>>> return resolutionVariants[baseImageIndex];
>>>>> }
>>>>> }
>>>>> ---------------------------------
>>>>>
>>>>> Thanks,
>>>>> Alexandr.
>>>>>
>>>>>>
>>>>>> Then we provide one of these from TK.get/createImage() when the
>>>>>> platform detects @2x, or Win8-style variants.
>>>>>>
>>>>>> For custom images we provide TK.createMRImage(lambda getRV, Image
>>>>>> variants...) and TK.createMRImage(Image variants...);
>>>>>>
>>>>>> Since the get<List> method is just bookkeeping, I don't see them
>>>>>> needing to override it, so the getRV() method is really the only
>>>>>> thing
>>>>>> they might want to override, and we can tie into the new Lambda
>>>>>> capabilities by making a single-method interface for it that they
>>>>>> supply in a factory method.
>>>>>>
>>>>>> I realize that the interface you created is more fundamentally
>>>>>> OO, but
>>>>>> the Image class has always been special in this regard in the AWT
>>>>>> ecosystem (in so far as we do not support someone implementing their
>>>>>> own Image subclass even though it is technically possible).
>>>>>> Because of
>>>>>> this special nature of Image, we end up with the situation that if
>>>>>> someone were given a need to create a subclass of Image, then they
>>>>>> would turn to BufImg as their superclass even though BufImg is
>>>>>> essentially an implementation-specific leaf node on the Image class
>>>>>> hierarchy. This approach with a factory method to create an
>>>>>> internal
>>>>>> subclass of the new MRI class mirrors the existing cases of Image
>>>>>> objects that come from factories as well.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> ...jim
>>>>>>
>>>>>>
>>>>>> On 3/20/14 7:52 AM, Alexander Scherbatiy wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Could you review the updated version of the fix:
>>>>>>> http://cr.openjdk.java.net/~alexsch/8029339/webrev.01/
>>>>>>>
>>>>>>> - The "getResolutionVariant(int width, int height)" method from
>>>>>>> MultiResolutionImage class is changed to
>>>>>>> Image getResolutionVariant(float logicalDPIX, float
>>>>>>> logicalDPIY,
>>>>>>> float width, float height, AffineTransform transform);
>>>>>>>
>>>>>>> - sun.awt.image.ImageResolutionHelper class is added. The
>>>>>>> sun.awt.image.MultiResolutionToolkitImage and
>>>>>>> sun.awt.image.MultiResolutionBufferedImage classes are used
>>>>>>> PLATFORM ImageResolutionHelper.
>>>>>>>
>>>>>>> The MultiResolutionImage interface implementation could look
>>>>>>> like:
>>>>>>> ------------------------------
>>>>>>> public class CustomMultiResolutionImage extends BufferedImage
>>>>>>> implements
>>>>>>> MultiResolutionImage {
>>>>>>>
>>>>>>> private final Image[] resolutionVariants;
>>>>>>>
>>>>>>> public CustomMultiResolutionImage(int baseIndex, Image...
>>>>>>> images) {
>>>>>>> super(images[baseIndex].getWidth(null),
>>>>>>> images[baseIndex].getHeight(null),
>>>>>>> BufferedImage.TYPE_INT_RGB);
>>>>>>> this.resolutionVariants = images;
>>>>>>> Graphics g = getGraphics();
>>>>>>> g.drawImage(images[baseIndex], 0, 0, null);
>>>>>>> g.dispose();
>>>>>>> }
>>>>>>>
>>>>>>> @Override
>>>>>>> public Image getResolutionVariant(float logicalDPIX, float
>>>>>>> logicalDPIY,
>>>>>>> float width, float height, AffineTransform
>>>>>>> transform) {
>>>>>>> return getResolutionVariant(logicalDPIX * width,
>>>>>>> logicalDPIY *
>>>>>>> height);
>>>>>>> }
>>>>>>>
>>>>>>> @Override
>>>>>>> public List<Image> getResolutionVariants() {
>>>>>>> return Arrays.asList(resolutionVariants);
>>>>>>> }
>>>>>>>
>>>>>>> public Image getResolutionVariant(double width, double
>>>>>>> height) {
>>>>>>> for (Image image : resolutionVariants) {
>>>>>>> if (width <= image.getWidth(null) && height <=
>>>>>>> image.getHeight(null)) {
>>>>>>> return image;
>>>>>>> }
>>>>>>> }
>>>>>>> return this;
>>>>>>> }
>>>>>>> }
>>>>>>> ------------------------------
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Alexandr.
>>>>>>>
>>>>>>> On 2/27/2014 4:54 PM, Alexander Scherbatiy wrote:
>>>>>>>> On 2/22/2014 3:54 AM, Jim Graham wrote:
>>>>>>>>> Hi Alexandr,
>>>>>>>>>
>>>>>>>>> On 2/18/14 7:33 AM, Alexander Scherbatiy wrote:
>>>>>>>>>> Hi Jim,
>>>>>>>>>>
>>>>>>>>>> Let's divide the discussion into two part.
>>>>>>>>>>
>>>>>>>>>> 1. Where it is better to hold resolution variants?
>>>>>>>>>> Putting resolution variants in Image class brings some
>>>>>>>>>> questions like:
>>>>>>>>>> - Some type of images do not need to have resolution variants
>>>>>>>>>> - Should resolution variants have the same type as the base
>>>>>>>>>> image?
>>>>>>>>>> - getResolutionVariants() method can return copy of the
>>>>>>>>>> original
>>>>>>>>>> list
>>>>>>>>>> so add/removeRV methods should be also added.
>>>>>>>>>>
>>>>>>>>>> There are pros and cons for placing resolution variants to
>>>>>>>>>> Image
>>>>>>>>>> class or to a separate intreface.
>>>>>>>>>
>>>>>>>>> I agree that this could be a separate interface. In my examples
>>>>>>>>> below I was just sticking them inside an "Image{}" to show where
>>>>>>>>> they
>>>>>>>>> lived in the set of involved objects, not a specific
>>>>>>>>> recommendation
>>>>>>>>> that they actually be new methods on the base class itself. I
>>>>>>>>> probably should have put a comment there about that.
>>>>>>>>>
>>>>>>>>> With respect to add/remove - that is assuming a need for manual
>>>>>>>>> construction of an image set, right? Forgive me if I'm
>>>>>>>>> forgetting
>>>>>>>>> something, but I seem to recall that manual Multi-Res images was
>>>>>>>>> proposed as a way for developers to introduce @2x support
>>>>>>>>> themselves,
>>>>>>>>> but if we are internally managing @2x and -DPI variants for them,
>>>>>>>>> then I'm not sure if there is actual developer need to manually
>>>>>>>>> construct their own. Am I forgetting something?
>>>>>>>> The NSImage has addRepresentation/removeRepresentation
>>>>>>>> methods to
>>>>>>>> work with image representations on Mac OS X.
>>>>>>>> The java.awt.Image class should provide similar
>>>>>>>> functionality to
>>>>>>>> have the possibilities as Cocoa on HiDPI displays.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> 2. Using scale factor/image sizes/scaled image sizes to
>>>>>>>>>> retreive a
>>>>>>>>>> resolution variant.
>>>>>>>>>>
>>>>>>>>>> May be it is better to have a structure that provide all
>>>>>>>>>> necessary
>>>>>>>>>> information to query the resolution variant: scale factor,
>>>>>>>>>> draw area
>>>>>>>>>> width/height, transformed area width/height?
>>>>>>>>>>
>>>>>>>>>> For example:
>>>>>>>>>> ---------------------
>>>>>>>>>> public interface MultiResolutionImage {
>>>>>>>>>>
>>>>>>>>>> interface DrawAreaInfo {
>>>>>>>>>>
>>>>>>>>>> float getScaleFactor();
>>>>>>>>>> float getAreaWidth();
>>>>>>>>>> float getAreaHeight();
>>>>>>>>>> float getTransformedAreaWidth();
>>>>>>>>>> float getTransformedAreaHeight();
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> public Image getResolutionVariant(DrawAreaInfo
>>>>>>>>>> drawAreaInfo) ;
>>>>>>>>>> public List<Image> getResolutionVariants();
>>>>>>>>>> }
>>>>>>>>>> ---------------------
>>>>>>>>>
>>>>>>>>> The problem with a constructor is that this is something that is
>>>>>>>>> (potentially) done on every drawImage() call, which means we are
>>>>>>>>> inviting GC into the equation. If we can come up with a simple
>>>>>>>>> "just
>>>>>>>>> a couple/3/4 numbers" way to embed that data into a method call
>>>>>>>>> argument list then we can make this lighter weight.
>>>>>>>>>
>>>>>>>>> What about simply having floating point (double) dimensions on
>>>>>>>>> the
>>>>>>>>> rendered size
>>>>>>>> There should be a way to choose a resolution variant
>>>>>>>> based on
>>>>>>>> requested drawing size or transformed drawing size.
>>>>>>>> At least a current transformation should be included too.
>>>>>>>>> plus a single floating point "logical DPI" for the screen?
>>>>>>>> There is the ID2D1Factory::GetDesktopDpi method which returns
>>>>>>>> dpiX and dpiY.
>>>>>>>> http://msdn.microsoft.com/en-us/library/windows/apps/dd371316
>>>>>>>>
>>>>>>>> That means that logicalDPIX/Y can have different values.
>>>>>>>> At least it is described in the
>>>>>>>> http://msdn.microsoft.com/en-us/library/windows/apps/ff684173
>>>>>>>> "To get the DPI setting, call the ID2D1Factory::GetDesktopDpi
>>>>>>>> method. The DPI is returned as two floating-point values, one for
>>>>>>>> the
>>>>>>>> x-axis and one for the y-axis. In theory, these values can differ.
>>>>>>>> Calculate a separate scaling factor for each axis."
>>>>>>>>
>>>>>>>> The getResolutionVariant method could look like:
>>>>>>>> --------------------------------------
>>>>>>>> public Image getResolutionVariant(float logicalDPIX, float
>>>>>>>> logicalDPIY,
>>>>>>>> float widthX, float widthY, AffineTransform
>>>>>>>> transform);
>>>>>>>> --------------------------------------
>>>>>>>>
>>>>>>>>> If the image is known (either passed as an argument or the
>>>>>>>>> method is
>>>>>>>>> called on the image), then it can provide the original WH.
>>>>>>>>>
>>>>>>>>>> The MultiResolutionImage default implementation could allow
>>>>>>>>>> to use
>>>>>>>>>> different strategies like scale factor/transfom/OS based
>>>>>>>>>> to query a resolution variant. The OS based strategy can be
>>>>>>>>>> used by
>>>>>>>>>> default.
>>>>>>>>>
>>>>>>>>> For Mac policy, all we need is the transformed dimensions, which
>>>>>>>>> can
>>>>>>>>> be passed in as FP for generality. For Windows policy, all we
>>>>>>>>> need
>>>>>>>>> is logical DPI for the screen. What other information would we
>>>>>>>>> need, or would an algorithm like to use, that can't be computed
>>>>>>>>> from
>>>>>>>>> those 2 pieces?
>>>>>>>> The aim is to provide a base class that can be used to
>>>>>>>> create a
>>>>>>>> MultiResolutionImage like:
>>>>>>>> http://hg.openjdk.java.net/jdk9/client/jdk/diff/ae53ebce5fa3/src/share/classes/sun/awt/image/MultiResolutionBufferedImage.java
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> A developer should be able to implement a custom algorithm to
>>>>>>>> query a resolution variant.
>>>>>>>>
>>>>>>>> It can be done by overriding the getResolutionVariant image:
>>>>>>>> -----------------------
>>>>>>>> Image mrImage = new MultiResolutionBufferedImage(){
>>>>>>>> @Override
>>>>>>>> public Image getResolutionVariant(...) {
>>>>>>>> // Custom logic here
>>>>>>>> }
>>>>>>>> };
>>>>>>>> -----------------------
>>>>>>>>
>>>>>>>> Or it can be done by using resolution variant choosers so a
>>>>>>>> developer can implement custom resolution variant query:
>>>>>>>> -----------------------
>>>>>>>> public class MultiResolutionBufferedImage implements
>>>>>>>> MultiResolutionImage{
>>>>>>>>
>>>>>>>> interface ResolutionVariantChooser{
>>>>>>>> Image getResolutionVariant(dpi, size,..., List<Image>
>>>>>>>> resolutionVariants);
>>>>>>>> }
>>>>>>>> ResolutionVariantChooser TRANSFORM_BASED = null;
>>>>>>>> ResolutionVariantChooser DPI_BASED = null;
>>>>>>>>
>>>>>>>> ResolutionVariantChooser rvChooser;
>>>>>>>>
>>>>>>>> @Override
>>>>>>>> public Image getResolutionVariant(dpi, size,...,) {
>>>>>>>> return rvChooser.getResolutionVariant(dpi, size,...,
>>>>>>>> getResolutionVariants());
>>>>>>>> }
>>>>>>>> }
>>>>>>>> -----------------------
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Alexandr.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> ...jim
>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alexandr.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2/13/2014 4:42 AM, Jim Graham wrote:
>>>>>>>>>>> On 2/12/14 5:59 AM, Alexander Scherbatiy wrote:
>>>>>>>>>>>> On 2/8/2014 4:19 AM, Jim Graham wrote:
>>>>>>>>>>>>> The primary thing that I was concerned about was the
>>>>>>>>>>>>> presence of
>>>>>>>>>>>>> integers in the API when Windows uses non-integer multiples
>>>>>>>>>>>> It would make sense to pass real numbers to the
>>>>>>>>>>>> getResolutionVariant() method if the difference between
>>>>>>>>>>>> resolution
>>>>>>>>>>>> variants sizes is 1.
>>>>>>>>>>>> It seems that it is not a common case.
>>>>>>>>>>>
>>>>>>>>>>> I was thinking of other API that is related to this, such as
>>>>>>>>>>> the API
>>>>>>>>>>> that queries the scaling factor from a SurfaceManager. I
>>>>>>>>>>> seem to
>>>>>>>>>>> remember some integer return values in that, but Windows might
>>>>>>>>>>> have
>>>>>>>>>>> the answer 1.4 or 1.8, depending on the screen scaling factor
>>>>>>>>>>> that was
>>>>>>>>>>> determined from the UI.
>>>>>>>>>>>
>>>>>>>>>>> In terms of the getResolutionVariant() method here, those
>>>>>>>>>>> non-integer
>>>>>>>>>>> screen scaling factors don't directly impact this API. But, we
>>>>>>>>>>> have
>>>>>>>>>>> some issues with the use of integers there from other sources:
>>>>>>>>>>>
>>>>>>>>>>> - That API assumes that the caller will determine the pixel
>>>>>>>>>>> size
>>>>>>>>>>> needed, but the actual media choice is determined with
>>>>>>>>>>> different
>>>>>>>>>>> techniques on Mac and Windows so this means that the caller
>>>>>>>>>>> will
>>>>>>>>>>> have
>>>>>>>>>>> to worry about platform conventions. Is that the right
>>>>>>>>>>> tradeoff?
>>>>>>>>>>>
>>>>>>>>>>> - The technique recommended for Mac involves computing the
>>>>>>>>>>> precise
>>>>>>>>>>> size desired using the current transform, which may be a
>>>>>>>>>>> floating
>>>>>>>>>>> point value, so the integer values used in this API are already
>>>>>>>>>>> approximations and there is no documentation on how to
>>>>>>>>>>> generate the
>>>>>>>>>>> proper integer. In particular, the current code in SG2D
>>>>>>>>>>> naively
>>>>>>>>>>> uses
>>>>>>>>>>> a cast to integer to determine the values to supply which
>>>>>>>>>>> means a
>>>>>>>>>>> transformed size of W+0.5 will be truncated to W and the lower
>>>>>>>>>>> resolution image will be used. Does that conform to Mac
>>>>>>>>>>> guidelines? Do
>>>>>>>>>>> they require the truncated size to reach W+1 before the next
>>>>>>>>>>> size is
>>>>>>>>>>> used? Passing in float or double values would sidestep all of
>>>>>>>>>>> that
>>>>>>>>>>> since then the comparisons would be done with full precision,
>>>>>>>>>>> but as
>>>>>>>>>>> long as we can determine a "best practices compatible with all
>>>>>>>>>>> platforms" rule on how to round to integers, then integers
>>>>>>>>>>> are OK
>>>>>>>>>>> there.
>>>>>>>>>>>
>>>>>>>>>>> - The Windows document you cite below suggests that the
>>>>>>>>>>> determination
>>>>>>>>>>> should be made by looking at the Screen DPI and choosing the
>>>>>>>>>>> next
>>>>>>>>>>> higher media variant based on that screen DPI. They do not
>>>>>>>>>>> specify
>>>>>>>>>>> choosing media based on the current transform as is done for
>>>>>>>>>>> Mac. If
>>>>>>>>>>> we stick with supplying values that are used to determine which
>>>>>>>>>>> media
>>>>>>>>>>> to use, then on Windows we should not take the transform into
>>>>>>>>>>> account,
>>>>>>>>>>> but instead query the SurfaceManager for the scale factor and
>>>>>>>>>>> only
>>>>>>>>>>> transform by those values (even if the current transform was
>>>>>>>>>>> manually
>>>>>>>>>>> overridden to identity).
>>>>>>>>>>>
>>>>>>>>>>> There are pros and cons to both approaches.
>>>>>>>>>>>
>>>>>>>>>>> Mac ensures that you are always using the best media for any
>>>>>>>>>>> given
>>>>>>>>>>> render operation.
>>>>>>>>>>>
>>>>>>>>>>> But, Windows ensure more consistency in the face of other
>>>>>>>>>>> scaling.
>>>>>>>>>>>
>>>>>>>>>>> The thing to consider is that if you have a 500x500 image
>>>>>>>>>>> with a
>>>>>>>>>>> 1000x1000 variant and you rendering it at 500x500 and then
>>>>>>>>>>> 501x501,
>>>>>>>>>>> that first jump will be fairly jarring as the scaled version
>>>>>>>>>>> of the
>>>>>>>>>>> 1000x1000 will not look precisely like the original 500x500
>>>>>>>>>>> did.
>>>>>>>>>>> With
>>>>>>>>>>> @2x images only, this effect is minimized so the advantage of
>>>>>>>>>>> always
>>>>>>>>>>> using "the best media for a given render operation" may
>>>>>>>>>>> outweigh the
>>>>>>>>>>> inconsistency issue. But, on Windows where the media are
>>>>>>>>>>> 1.4x or
>>>>>>>>>>> 1.8x
>>>>>>>>>>> in size, a downscaled image will start to show more
>>>>>>>>>>> interpolation
>>>>>>>>>>> noise and so the balance of the two choices may shift more
>>>>>>>>>>> towards not
>>>>>>>>>>> wanting a jarring shift.
>>>>>>>>>>>
>>>>>>>>>>> We might want one or more of the following:
>>>>>>>>>>>
>>>>>>>>>>> - Developer chooses policy (TX_AWARE, DPI_AWARE,
>>>>>>>>>>> ALWAYS_LARGEST,
>>>>>>>>>>> NONE,
>>>>>>>>>>> PLATFORM) where the last policy would use TX_AWARE on Mac and
>>>>>>>>>>> DPI_AWARE on Windows
>>>>>>>>>>>
>>>>>>>>>>> - We create our own policy and always use it (TX_AWARE? or
>>>>>>>>>>> DPI_AWARE?)
>>>>>>>>>>>
>>>>>>>>>>> - We create our own policy that dynamically chooses one of the
>>>>>>>>>>> above
>>>>>>>>>>> strategies depending on platform or available media or ???
>>>>>>>>>>>
>>>>>>>>>>> - We could create an optional interface for them to install
>>>>>>>>>>> their
>>>>>>>>>>> own
>>>>>>>>>>> algorithm as well. I think it would work best as a delegate
>>>>>>>>>>> interface
>>>>>>>>>>> that one installs into Image so that it can be used with any
>>>>>>>>>>> image
>>>>>>>>>>> without having to subclass (it wouldn't really have much to do
>>>>>>>>>>> for
>>>>>>>>>>> BufferedImages or VolatileImages, though):
>>>>>>>>>>>
>>>>>>>>>>> class Image {
>>>>>>>>>>> void setResolutionHelper(ImageResolutionHelper foo);
>>>>>>>>>>> List<Image> getResolutionVariants();
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> or:
>>>>>>>>>>>
>>>>>>>>>>> class Graphics {
>>>>>>>>>>> void setResolutionHelper(ImageResolutionHelper foo);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> or - anywhere else it could be installed more centrally (per
>>>>>>>>>>> App
>>>>>>>>>>> context)?
>>>>>>>>>>>
>>>>>>>>>>> and the interface would be something like one of these
>>>>>>>>>>> variants:
>>>>>>>>>>>
>>>>>>>>>>> interface ImageResolutionHelper {
>>>>>>>>>>> // This version would prevent substituting a random image:
>>>>>>>>>>> // They have to return an index into the List<Image> for
>>>>>>>>>>> that
>>>>>>>>>>> image...
>>>>>>>>>>> public int chooseVariant(Image img, double dpi, number w,
>>>>>>>>>>> number h);
>>>>>>>>>>>
>>>>>>>>>>> or:
>>>>>>>>>>>
>>>>>>>>>>> // This version would allow substituting an arbitrary
>>>>>>>>>>> image:
>>>>>>>>>>> public Image getVariant(Image img, double dpi, number w,
>>>>>>>>>>> number
>>>>>>>>>>> h);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> Since they would be in full control of the policy, though, we
>>>>>>>>>>> would
>>>>>>>>>>> unfortunately always have to call this, there would be no more
>>>>>>>>>>> testing
>>>>>>>>>>> in SG2D to see "if" we need to deal with DPI, though perhaps we
>>>>>>>>>>> could
>>>>>>>>>>> document some internal conditions in which we do not call it
>>>>>>>>>>> for
>>>>>>>>>>> common cases (but that would have to be well agreed not to
>>>>>>>>>>> get in
>>>>>>>>>>> the
>>>>>>>>>>> way of reasonable uses of the API and well documented)?
>>>>>>>>>>>
>>>>>>>>>>> Note that we would have to do a security audit to make sure
>>>>>>>>>>> that
>>>>>>>>>>> random image substitution couldn't allow any sort of "screen
>>>>>>>>>>> phishing"
>>>>>>>>>>> exploit.
>>>>>>>>>>>
>>>>>>>>>>> ...jim
>>>>>>>>>>>
>>>>>>>>>>>>> and also what policy they use for choosing scaled images.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't see a mention of taking the current transform into
>>>>>>>>>>>>> account,
>>>>>>>>>>>>> just physical issues like screen DPI and form factor. They
>>>>>>>>>>>>> talk
>>>>>>>>>>>>> about
>>>>>>>>>>>>> resolution plateaus and in their recommendations section they
>>>>>>>>>>>>> tell the
>>>>>>>>>>>>> developer to use a particular property that tells them the
>>>>>>>>>>>>> screen
>>>>>>>>>>>>> resolution to figure out which image to load if they are
>>>>>>>>>>>>> loading
>>>>>>>>>>>>> manually. There is no discussion about dynamically loading
>>>>>>>>>>>>> multiple
>>>>>>>>>>>>> versions of the image based on a dynamic program-applied
>>>>>>>>>>>>> transform
>>>>>>>>>>>>> factor as is done on MacOS.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, they tell developers to draw images to a specific size
>>>>>>>>>>>>> rather
>>>>>>>>>>>>> than using auto-sizing. That begs the question as to how
>>>>>>>>>>>>> they
>>>>>>>>>>>>> interpret a call to draw an image just using a location in
>>>>>>>>>>>>> the
>>>>>>>>>>>>> presence of various DPI factors.
>>>>>>>>>>>> There is an interesting doc that describes how to write
>>>>>>>>>>>> DPI-aware
>>>>>>>>>>>> Win32 applications:
>>>>>>>>>>>> http://msdn.microsoft.com/en-us/library/dd464646%28v=vs.85%29.aspx
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> It is suggested to handle WM_DPICHANGED message, load
>>>>>>>>>>>> the
>>>>>>>>>>>> graphic
>>>>>>>>>>>> that has slightly greater resolution to the current DPI and
>>>>>>>>>>>> use
>>>>>>>>>>>> StretchBlt
>>>>>>>>>>>> to scale down the image.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Alexandr.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ...jim
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 2/7/14 3:00 AM, Alexander Scherbatiy wrote:
>>>>>>>>>>>>>> On 1/22/2014 6:40 AM, Jim Graham wrote:
>>>>>>>>>>>>>>> Hi Alexander,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Before we get too far down the road on this API, I think we
>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>> the way in which MacOS processes multi-resolution images
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> HiDPI
>>>>>>>>>>>>>>> screens, but have we investigated the processes that
>>>>>>>>>>>>>>> Windows
>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>> under Windows 8? My impression is that Windows 8 has
>>>>>>>>>>>>>>> included a
>>>>>>>>>>>>>>> number of new techniques for dealing with the high
>>>>>>>>>>>>>>> resolution
>>>>>>>>>>>>>>> displays
>>>>>>>>>>>>>>> that it will run on in the Windows tablet and mobile
>>>>>>>>>>>>>>> industries
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> that these will also come into play as 4K displays (already
>>>>>>>>>>>>>>> available)
>>>>>>>>>>>>>>> become more common on the desktop. We should make sure
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> what we
>>>>>>>>>>>>>>> come up with here can provide native compatibility with
>>>>>>>>>>>>>>> either
>>>>>>>>>>>>>>> platform's policies and standard practices.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you've investigated the MS policies I'd like to see a
>>>>>>>>>>>>>>> summary so
>>>>>>>>>>>>>>> that we can consider them as we review this API...
>>>>>>>>>>>>>> There is the Windows Guidelines for scaling to pixel
>>>>>>>>>>>>>> density:
>>>>>>>>>>>>>> http://msdn.microsoft.com/en-us/library/windows/apps/hh465362.aspx
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> which says that Windows has automatic resource loading
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>> three version of images scaling (100%, 140%, and 180%)
>>>>>>>>>>>>>> --------------------------------
>>>>>>>>>>>>>> Without scaling, as the pixel density of a display device
>>>>>>>>>>>>>> increases, the
>>>>>>>>>>>>>> physical sizes of objects on screen get smaller.
>>>>>>>>>>>>>> When UI would otherwise be too small to touch and when text
>>>>>>>>>>>>>> gets
>>>>>>>>>>>>>> too
>>>>>>>>>>>>>> small to read,
>>>>>>>>>>>>>> Windows scales the system and app UI to one of the following
>>>>>>>>>>>>>> scaling
>>>>>>>>>>>>>> plateaus:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1.0 (100%, no scaling is applied)
>>>>>>>>>>>>>> 1.4 (140% scaling)
>>>>>>>>>>>>>> 1.8 (180% scaling)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Windows determines which scaling plateau to use based on the
>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>> screen size, the screen resolution, the DPI of the
>>>>>>>>>>>>>> screen, and
>>>>>>>>>>>>>> form
>>>>>>>>>>>>>> factor.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Use resource loading for bitmap images in the app package
>>>>>>>>>>>>>> For
>>>>>>>>>>>>>> bitmap
>>>>>>>>>>>>>> images stored
>>>>>>>>>>>>>> in the app package, provide a separate image for each
>>>>>>>>>>>>>> scaling
>>>>>>>>>>>>>> factor(100%, 140%, and 180%),
>>>>>>>>>>>>>> and name your image files using the "scale" naming
>>>>>>>>>>>>>> convention
>>>>>>>>>>>>>> described
>>>>>>>>>>>>>> below.
>>>>>>>>>>>>>> Windows loads the right image for the current scale
>>>>>>>>>>>>>> automatically.
>>>>>>>>>>>>>> --------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The image name convention for the various scales is:
>>>>>>>>>>>>>> images/logo.scale-100.png
>>>>>>>>>>>>>> images/logo.scale-140.png
>>>>>>>>>>>>>> images/logo.scale-180.png
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The 'ms-appx:///images/logo.png' uri is used to load the
>>>>>>>>>>>>>> image
>>>>>>>>>>>>>> in an
>>>>>>>>>>>>>> application.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If we want to support this in the same way as it is done
>>>>>>>>>>>>>> for Mac
>>>>>>>>>>>>>> OS X
>>>>>>>>>>>>>> the WToolkit should return MultiResolution image in
>>>>>>>>>>>>>> case if
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> loaded image has .scale-* qualifiers.
>>>>>>>>>>>>>> The Graphics class can request an image with necessary
>>>>>>>>>>>>>> resolution
>>>>>>>>>>>>>> from the MultiResolution image.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It seems that nothing should be changed in the
>>>>>>>>>>>>>> MultiResolution
>>>>>>>>>>>>>> interface in this case.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Alexandr.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ...jim
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 1/14/14 2:54 AM, Alexander Scherbatiy wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Could you review the fix:
>>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8029339
>>>>>>>>>>>>>>>> webrev:
>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~alexsch/8029339/webrev.00
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is a proposal to introduce an API that allows to
>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>> multi resolution image.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I. It seems reasonable that the API should provide two
>>>>>>>>>>>>>>>> basic
>>>>>>>>>>>>>>>> operations:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. Get the resolution variant based on the requested
>>>>>>>>>>>>>>>> image
>>>>>>>>>>>>>>>> width and
>>>>>>>>>>>>>>>> height:
>>>>>>>>>>>>>>>> - Image getResolutionVariant(int width, int height)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Usually the system provides the scale factor which
>>>>>>>>>>>>>>>> represents
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> number of pixels corresponding to each linear unit on the
>>>>>>>>>>>>>>>> display.
>>>>>>>>>>>>>>>> However, it has sense to combine the scale factor and
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>>> transformations to get the actual image size to be
>>>>>>>>>>>>>>>> displayed.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2. Get all provided resolution variants:
>>>>>>>>>>>>>>>> - List<Image> getResolutionVariants()
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There are several uses cases:
>>>>>>>>>>>>>>>> - Create a new multi-resolution image based on the
>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>> multi-resolution image.
>>>>>>>>>>>>>>>> - Pass to the native system the multi-resolution
>>>>>>>>>>>>>>>> image. For
>>>>>>>>>>>>>>>> example,
>>>>>>>>>>>>>>>> a use can set to the system the custom multi-resolution
>>>>>>>>>>>>>>>> cursor.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> II. There are some possible ways where the new API can be
>>>>>>>>>>>>>>>> added
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. java.awt.Image.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The 2 new methods can be added to the Image class. A
>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>> override
>>>>>>>>>>>>>>>> the getResolutionVariant() and getResolutionVariants()
>>>>>>>>>>>>>>>> methods to
>>>>>>>>>>>>>>>> provide the resolution variants
>>>>>>>>>>>>>>>> or there can be default implementations of these
>>>>>>>>>>>>>>>> methods
>>>>>>>>>>>>>>>> if a
>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>> puts resolution variants
>>>>>>>>>>>>>>>> to the list in the sorted order.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To check that the image has resolution variants the
>>>>>>>>>>>>>>>> following
>>>>>>>>>>>>>>>> statement can be used:
>>>>>>>>>>>>>>>> image.getResolutionVariants().size()
>>>>>>>>>>>>>>>> != 1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The disadvantage is that there is an overhead that the
>>>>>>>>>>>>>>>> Image
>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>> should contain the List object and not all
>>>>>>>>>>>>>>>> images can have resolution variants.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2. Introduce new MultiResolutionImage interface.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> A user should extend Image class and implement the
>>>>>>>>>>>>>>>> MultiResolutionImage interface.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> For example:
>>>>>>>>>>>>>>>> ---------------------
>>>>>>>>>>>>>>>> public class CustomMultiResolutionImage extends
>>>>>>>>>>>>>>>> BufferedImage
>>>>>>>>>>>>>>>> implements MultiResolutionImage {
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Image highResolutionImage;
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> public CustomMultiResolutionImage(BufferedImage
>>>>>>>>>>>>>>>> baseImage,
>>>>>>>>>>>>>>>> BufferedImage highResolutionImage) {
>>>>>>>>>>>>>>>> super(baseImage.getWidth(),
>>>>>>>>>>>>>>>> baseImage.getHeight(),
>>>>>>>>>>>>>>>> baseImage.getType());
>>>>>>>>>>>>>>>> this.highResolutionImage = highResolutionImage;
>>>>>>>>>>>>>>>> Graphics g = getGraphics();
>>>>>>>>>>>>>>>> g.drawImage(baseImage, 0, 0, null);
>>>>>>>>>>>>>>>> g.dispose();
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> @Override
>>>>>>>>>>>>>>>> public Image getResolutionVariant(int width, int
>>>>>>>>>>>>>>>> height) {
>>>>>>>>>>>>>>>> return ((width <= getWidth() && height <=
>>>>>>>>>>>>>>>> getHeight()))
>>>>>>>>>>>>>>>> ? this : highResolutionImage;
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> @Override
>>>>>>>>>>>>>>>> public List<Image> getResolutionVariants() {
>>>>>>>>>>>>>>>> return Arrays.asList(this,
>>>>>>>>>>>>>>>> highResolutionImage);
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>> ---------------------
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The current fix adds the MultiResolutionImage interface
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> public
>>>>>>>>>>>>>>>> resolution variant rendering hints.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Alexandr.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>
More information about the awt-dev
mailing list