From linuxhippy at gmail.com Thu Mar 3 05:19:29 2011 From: linuxhippy at gmail.com (Clemens Eisserer) Date: Thu, 3 Mar 2011 14:19:29 +0100 Subject: Input-Handling Message-ID: Hi, I am working on a small university-project where the goal is to execute awt/java2d applications directly on an application server and re-direct its output to a web-browser. My plan would be to render to a BufferedImage and send only updated areas using dirty tracking. Caciocavallo seems like a perfect fit with its lightweight peer-implementation and the PlatformScreen abstraction, dirty tracking could be done through the surface-locking functions. I had a look at the SDL implementation, but I haven't found how input-handling is done. If this isn't a part Cacio cares about, do you have some pointers where to start? Thanks, Clemens From neugens.limasoftware at gmail.com Thu Mar 3 12:16:30 2011 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Thu, 03 Mar 2011 21:16:30 +0100 Subject: Input-Handling In-Reply-To: References: Message-ID: <1299183390.2757.8.camel@galactica> Il giorno gio, 03/03/2011 alle 14.19 +0100, Clemens Eisserer ha scritto: > Hi, > > I am working on a small university-project where the goal is to > execute awt/java2d applications directly on an application server and > re-direct its output to a web-browser. > My plan would be to render to a BufferedImage and send only updated > areas using dirty tracking. > Caciocavallo seems like a perfect fit with its lightweight > peer-implementation and the PlatformScreen abstraction, dirty tracking > could be done through the surface-locking functions. > > I had a look at the SDL implementation, but I haven't found how > input-handling is done. > If this isn't a part Cacio cares about, do you have some pointers > where to start? > > Thanks, Clemens Hi Clemens, The SDL backend was never done, because all the backends were (sadly) developed under closed doors at aicas. We wanted to revive the SDL backend but so far we didn't have time. Anyway, Cacio handles the events by passing back and forth around the "fake" AWT widget and the peer widget. You should look at two things, CacioComponentPeer and CacioEventPump. Anyway, for a managed window component, things are really easy, search for SDLScreen and in particular, search for the native implementation, it's very basic in the sdl peer, but it dispatches mouse events: Java_net_java_openjdk_awt_peer_sdl_SDLScreen_nativeGetEvent This is where we dispatch the events. The screen implements the CacioEventSource, which is taken care by the FullScreenEventPump. For non managed windows things are a bit more complicated in the sense that you have to provide what this event pump does (see also dispatchEvent in AbstractManagedWindowContainer). We need to fix up things so that they work on current JDK though... Cheers, Mario -- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF IcedRobot: www.icedrobot.org Proud GNU Classpath developer: http://www.classpath.org/ Read About us at: http://planet.classpath.org OpenJDK: http://openjdk.java.net/projects/caciocavallo/ Please, support open standards: http://endsoftpatents.org/ From linuxhippy at gmail.com Sat Mar 5 10:14:55 2011 From: linuxhippy at gmail.com (Clemens Eisserer) Date: Sat, 5 Mar 2011 19:14:55 +0100 Subject: Input-Handling In-Reply-To: <1299183390.2757.8.camel@galactica> References: <1299183390.2757.8.camel@galactica> Message-ID: Hi Mario, > We need to fix up things so that they work on current JDK though... I had success getting cacio to work with openjdk head, changes required were using AWTAccessor instead of ComponentAccessor. Unfourtunatly font-support is more broken, I had to implement my own Font-Manager (simply took X11FontManager and messed arround) and make a cast conditional somewhere in the Motif-Peers. All in all it works quite well: http://linuxhippy.blogspot.com/2011/03/caciocavallo-goes-web.html > Anyway, for a managed window component, things are really easy, ?search > for ?SDLScreen and in particular, search for the native implementation, > it's very basic in the sdl peer, but it dispatches mouse events: > Java_net_java_openjdk_awt_peer_sdl_SDLScreen_nativeGetEvent Thanks for the pointer - I now can pass simple events, although it doesn't work completly (e.g. a mouse-click only selects a JCheckBox but does not change its state), I guess I'll have to learn a bit more about AWT's event handling and when which events should be fired. > This is where we dispatch the events. The screen implements the > CacioEventSource, which is taken care by the FullScreenEventPump. For > non managed windows things are a bit more complicated in the sense that > you have to provide what this event pump does (see also dispatchEvent in > AbstractManagedWindowContainer). There's one big (huge) problem left - on the server there should me many apps running side-by-side without disturbing each other. I thought about running it under different ClassLoaders, however the native libraries can only be loaded once, so no luck. Do you think this can be solved with multiple AWT-Screens which are mapped to multiple PlatformScreens? How would you start, is it feasable at all? Would it be enough to extend makeScreenDevice/getNumScreens in SDLGraphicsEnvironment, extend the CacioEventPumps a bit and somehow glue that together with multiple SDLScreens or should I expect serious trouble? - Clemens PS: Thanks for all the hard work on Caciocavallo! It was amazing how quick that prototyp was implemented thanks to Cacio:) From neugens.limasoftware at gmail.com Sat Mar 5 10:31:06 2011 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Sat, 05 Mar 2011 19:31:06 +0100 Subject: Input-Handling In-Reply-To: References: <1299183390.2757.8.camel@galactica> Message-ID: <1299349866.2954.44.camel@galactica> Il giorno sab, 05/03/2011 alle 19.14 +0100, Clemens Eisserer ha scritto: > Hi Mario, > > > We need to fix up things so that they work on current JDK though... > > I had success getting cacio to work with openjdk head, changes > required were using AWTAccessor instead of ComponentAccessor. > Unfourtunatly font-support is more broken, I had to implement my own > Font-Manager (simply took X11FontManager and messed arround) and make > a cast conditional somewhere in the Motif-Peers. > All in all it works quite well: > http://linuxhippy.blogspot.com/2011/03/caciocavallo-goes-web.html Wowowow! This is so cool! ;) We would love to merge those changes in if possible. > > Anyway, for a managed window component, things are really easy, search > > for SDLScreen and in particular, search for the native implementation, > > it's very basic in the sdl peer, but it dispatches mouse events: > > Java_net_java_openjdk_awt_peer_sdl_SDLScreen_nativeGetEvent > > Thanks for the pointer - I now can pass simple events, although it > doesn't work completly (e.g. a mouse-click only selects a JCheckBox > but does not change its state), I guess I'll have to learn a bit more > about AWT's event handling and when which events should be fired. Maybe the SDL backend is really limited, I think we don't dispatch events at all, other than a very small subset fo them (long time I wrote this). I believe there's an X11 one that is a little more complete, this was our first example and we showcased it at J1 two years ago. > There's one big (huge) problem left - on the server there should me > many apps running side-by-side without disturbing each other. > I thought about running it under different ClassLoaders, however the > native libraries can only be loaded once, so no luck. > Do you think this can be solved with multiple AWT-Screens which are > mapped to multiple PlatformScreens? How would you start, is it > feasable at all? Mmmm... You could create different instances of the Screen, although we never really did this, so I'm not sure it work out of the box (but I don't see why it shouldn't to be honest). Or you could use non managed windows (Screens are for quick prototyping). I think it's not a problem to share the same library, unless you internally share resources. Maybe SDL is not the perfect library for this though. > Would it be enough to extend makeScreenDevice/getNumScreens in > SDLGraphicsEnvironment, extend the CacioEventPumps a bit and somehow > glue that together with multiple SDLScreens or should I expect serious > trouble? This should not really be needed, this guy works the same as any other AWT code, so maybe it's enough to use different AppContex (maybe we need to make Cacio AppContex aware, but I think it's easier than hacking complicated workarounds). > - Clemens > > PS: Thanks for all the hard work on Caciocavallo! It was amazing how > quick that prototyp was implemented thanks to Cacio:) Thanks! I still think Oracle should invest in this a little, fix the bugs, and use Cacio for the OSX version of Java. If you have more questions ask us, it's nice to see you working on the project! :) I'll make sure to give you commit access (it's a little bit tricky, but for you this shouldn't be an issue). Cheers, Mario -- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF IcedRobot: www.icedrobot.org Proud GNU Classpath developer: http://www.classpath.org/ Read About us at: http://planet.classpath.org OpenJDK: http://openjdk.java.net/projects/caciocavallo/ Please, support open standards: http://endsoftpatents.org/ From linuxhippy at gmail.com Mon Mar 7 07:26:44 2011 From: linuxhippy at gmail.com (Clemens Eisserer) Date: Mon, 7 Mar 2011 16:26:44 +0100 Subject: Input-Handling In-Reply-To: <1299349866.2954.44.camel@galactica> References: <1299183390.2757.8.camel@galactica> <1299349866.2954.44.camel@galactica> Message-ID: Hi Mario, > Mmmm... > > You could create different instances of the Screen, although we never > really did this, so I'm not sure it work out of the box (but I don't see > why it shouldn't to be honest). Didn't realize Cacio was so well-suited for multi-screen environments, I now simply create seperate screens and GraphicsConfigurations. Works really well, even when rendering is triggered by non-input threads :) > Wowowow! > This is so cool! ;) > We would love to merge those changes in if possible. Glad you like it - however many features are still missing, I'll have to do some clean-up before I'll release it in the wild. > Maybe the SDL backend is really limited, I think we don't dispatch > events at all, other than a very small subset fo them (long time I wrote > this). I believe there's an X11 one that is a little more complete, this > was our first example and we showcased it at J1 two years ago. I got mouse-events mostly working (same state as the SDL backend), however key-events remain a mystery. Even the X11 backend doesn't seem to implement key-events. Is there some support in Cacio for synthetisizing events? I read a few comments about it, but didn't find code doing it. >From the canvas I only get button_pressed/released - when I sythetisize the clicked myself I suddenly can click buttons and checkboxes. However keeping double-clicking etc in mind, I fear this will be quite tiresome. Thanks, Clemens From curtcox at gmail.com Mon Mar 7 18:05:56 2011 From: curtcox at gmail.com (Curt Cox) Date: Mon, 7 Mar 2011 20:05:56 -0600 Subject: Input-Handling In-Reply-To: References: <1299183390.2757.8.camel@galactica> <1299349866.2954.44.camel@galactica> Message-ID: Count me among those that are eager to learn more and get my hands on something I can play with. One of my first thoughts was using this to deploy Swing apps to the iPad. - Curt On Mon, Mar 7, 2011 at 9:26 AM, Clemens Eisserer wrote: > Hi Mario, > > > Mmmm... > > > > You could create different instances of the Screen, although we never > > really did this, so I'm not sure it work out of the box (but I don't see > > why it shouldn't to be honest). > > Didn't realize Cacio was so well-suited for multi-screen environments, > I now simply create seperate screens and GraphicsConfigurations. Works > really well, even when rendering is triggered by non-input threads :) > > > Wowowow! > > This is so cool! ;) > > We would love to merge those changes in if possible. > > Glad you like it - however many features are still missing, I'll have > to do some clean-up before I'll release it in the wild. > > > > Maybe the SDL backend is really limited, I think we don't dispatch > > events at all, other than a very small subset fo them (long time I wrote > > this). I believe there's an X11 one that is a little more complete, this > > was our first example and we showcased it at J1 two years ago. > > I got mouse-events mostly working (same state as the SDL backend), > however key-events remain a mystery. > Even the X11 backend doesn't seem to implement key-events. > > Is there some support in Cacio for synthetisizing events? > I read a few comments about it, but didn't find code doing it. > > >From the canvas I only get button_pressed/released - when I > sythetisize the clicked myself I suddenly can click buttons and > checkboxes. However keeping double-clicking etc in mind, I fear this > will be quite tiresome. > > > Thanks, Clemens > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/caciocavallo-dev/attachments/20110307/cded3146/attachment.html From neugens.limasoftware at gmail.com Mon Mar 7 22:44:04 2011 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Tue, 08 Mar 2011 07:44:04 +0100 Subject: Input-Handling In-Reply-To: References: <1299183390.2757.8.camel@galactica> <1299349866.2954.44.camel@galactica> Message-ID: <1299566644.2755.1.camel@galactica> Il giorno lun, 07/03/2011 alle 20.05 -0600, Curt Cox ha scritto: > Count me among those that are eager to learn more and get my hands on > something I can play with. One of my first thoughts was using this to > deploy Swing apps to the iPad. > This is not difficult at all, but it's impossible to officially support because it breaks the Apple use requirements (you can do it for yourself or under some jail broken environment that I understand are legal now, but IANAL). Cheers, Mario -- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF IcedRobot: www.icedrobot.org Proud GNU Classpath developer: http://www.classpath.org/ Read About us at: http://planet.classpath.org OpenJDK: http://openjdk.java.net/projects/caciocavallo/ Please, support open standards: http://endsoftpatents.org/ From neugens.limasoftware at gmail.com Mon Mar 7 22:48:26 2011 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Tue, 08 Mar 2011 07:48:26 +0100 Subject: Input-Handling In-Reply-To: References: <1299183390.2757.8.camel@galactica> <1299349866.2954.44.camel@galactica> Message-ID: <1299566906.2755.5.camel@galactica> Il giorno lun, 07/03/2011 alle 16.26 +0100, Clemens Eisserer ha scritto: > Hi Mario, > > > Mmmm... > > > > You could create different instances of the Screen, although we never > > really did this, so I'm not sure it work out of the box (but I don't see > > why it shouldn't to be honest). > > Didn't realize Cacio was so well-suited for multi-screen environments, > I now simply create seperate screens and GraphicsConfigurations. Works > really well, even when rendering is triggered by non-input threads :) Hehe, it's a nice thing, I know ;) > I got mouse-events mostly working (same state as the SDL backend), > however key-events remain a mystery. > Even the X11 backend doesn't seem to implement key-events. I'll send you later an example, it's a little bit tricky because this is a gray area that is really platform dependent and the mapping is tedious (and always basically the same but slightly different so you cannot really abstract it much). > Is there some support in Cacio for synthetisizing events? > I read a few comments about it, but didn't find code doing it. > > >From the canvas I only get button_pressed/released - when I > sythetisize the clicked myself I suddenly can click buttons and > checkboxes. However keeping double-clicking etc in mind, I fear this > will be quite tiresome. Some of them are... The problem is that each platform has different rules so it was not really obvious how we could make it less boring to do. This is usually something you want to pass over your trainees >;) Cheers, Mario -- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF IcedRobot: www.icedrobot.org Proud GNU Classpath developer: http://www.classpath.org/ Read About us at: http://planet.classpath.org OpenJDK: http://openjdk.java.net/projects/caciocavallo/ Please, support open standards: http://endsoftpatents.org/ From curtcox at gmail.com Tue Mar 8 19:30:30 2011 From: curtcox at gmail.com (Curt Cox) Date: Tue, 8 Mar 2011 21:30:30 -0600 Subject: Input-Handling In-Reply-To: <1299566644.2755.1.camel@galactica> References: <1299183390.2757.8.camel@galactica> <1299349866.2954.44.camel@galactica> <1299566644.2755.1.camel@galactica> Message-ID: Mario, I see basically four reasons to favor the web approach over jailbreaking. 1) Ubiquity. It will work on all iPads. Most Swing applications are internal corporate applications. Jailbreaking will be very hard, if not impossible, to sell in this setting. 2) Zero install. As long as the proper browser is installed, there is no separate application install. 3) Application footprint. Many Swing applications are architected to require much more in terms of RAM and CPU than an iPad has to offer. 4) Application transportability. It would be really useful to launch an application in a browser on one device, and then switch to a browser on another. None of these advantages are new. They can all be achieved using something like VNC. However, I've never found a remoting technology that produces a high quality user experience for Swing applications unless you have a very high-bandwidth and low-latency connection. VNC, X, NX, and RDP all handle native applications far better than they handle Swing applications. Plus, this seems more useful for publishing applications, or even publishing parts of applications. The other remoting technologies are aimed at publishing desktops. As an application developer, I want to publish applications. I believe Citrix and Sun Secure Global desktop have some support for application publishing, but I have little experience with either of them. - Curt On Tue, Mar 8, 2011 at 12:44 AM, Mario Torre wrote: > Il giorno lun, 07/03/2011 alle 20.05 -0600, Curt Cox ha scritto: > > Count me among those that are eager to learn more and get my hands on > > something I can play with. One of my first thoughts was using this to > > deploy Swing apps to the iPad. > > > > This is not difficult at all, but it's impossible to officially support > because it breaks the Apple use requirements (you can do it for yourself > or under some jail broken environment that I understand are legal now, > but IANAL). > > Cheers, > Mario > -- > pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF > Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF > > IcedRobot: www.icedrobot.org > Proud GNU Classpath developer: http://www.classpath.org/ > Read About us at: http://planet.classpath.org > OpenJDK: http://openjdk.java.net/projects/caciocavallo/ > > Please, support open standards: > http://endsoftpatents.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/caciocavallo-dev/attachments/20110308/cddc5ab7/attachment.html From proetel at aicas.de Mon Mar 21 02:45:36 2011 From: proetel at aicas.de (Ingo Proetel) Date: Mon, 21 Mar 2011 10:45:36 +0100 Subject: [PATCH] Delegat off-screen image creation to the CacioToolkit. This allows different platforms to provide optimized off-screen images Message-ID: <00f4f494fe678b7d7f84.1300700736@crowley> # HG changeset patch # User Ingo Proetel # Date 1300698958 -3600 # Node ID 00f4f494fe678b7d7f84199c9cbae4e2024ffe7c # Parent ec33131d820f9c7f3bfbb10c0871c674a6ad73a0 Delegat off-screen image creation to the CacioToolkit. This allows different platforms to provide optimized off-screen images. Added a flag to indicate that the background needs to be cleared during the next paint operation. diff -r ec33131d820f -r 00f4f494fe67 src/share/classes/sun/awt/peer/cacio/CacioComponentPeer.java --- a/src/share/classes/sun/awt/peer/cacio/CacioComponentPeer.java Thu Mar 17 15:39:43 2011 +0100 +++ b/src/share/classes/sun/awt/peer/cacio/CacioComponentPeer.java Mon Mar 21 10:15:58 2011 +0100 @@ -120,6 +120,12 @@ private RepaintArea paintArea; private Rectangle viewRect; + + /** + * Flag to indicate that a change requires a repaint + * of the background. + */ + private boolean needsClearBackground = false; /** * Creates a new CacioComponentPeer. @@ -345,7 +351,9 @@ case PaintEvent.UPDATE: case PaintEvent.PAINT: if (! isLayouting()) { - paintArea.paint(getAWTComponent(), false); + boolean tmp = needsClearBackground; + needsClearBackground = false; + paintArea.paint(getAWTComponent(), tmp); } break; case MouseEvent.MOUSE_PRESSED: @@ -611,6 +619,7 @@ // laid out correctly. swingComponent.validate(); } + needsClearBackground = true; } public void setEnabled(boolean enable) { @@ -647,7 +656,7 @@ } public void setVisible(boolean b) { - + needsClearBackground = b; if (proxy != null) { proxy.setVisible(b); } @@ -680,13 +689,7 @@ public Image createImage(int width, int height) { - GraphicsConfiguration gc = getGraphicsConfiguration(); - ColorModel model = gc.getColorModel(Transparency.OPAQUE); - WritableRaster wr = - model.createCompatibleWritableRaster(width, height); - return new OffScreenImage(awtComponent, model, wr, - model.isAlphaPremultiplied()); - + return ((CacioToolkit)Toolkit.getDefaultToolkit()).createOffScreenImage(awtComponent,width,height); } public VolatileImage createVolatileImage(int width, int height) { diff -r ec33131d820f -r 00f4f494fe67 src/share/classes/sun/awt/peer/cacio/CacioToolkit.java --- a/src/share/classes/sun/awt/peer/cacio/CacioToolkit.java Thu Mar 17 15:39:43 2011 +0100 +++ b/src/share/classes/sun/awt/peer/cacio/CacioToolkit.java Mon Mar 21 10:15:58 2011 +0100 @@ -25,15 +25,19 @@ package sun.awt.peer.cacio; + import java.awt.Button; import java.awt.Canvas; import java.awt.Checkbox; import java.awt.CheckboxMenuItem; import java.awt.Choice; +import java.awt.Component; import java.awt.Dialog; import java.awt.FileDialog; import java.awt.Frame; +import java.awt.GraphicsConfiguration; import java.awt.HeadlessException; +import java.awt.Image; import java.awt.KeyboardFocusManager; import java.awt.Label; import java.awt.List; @@ -46,7 +50,10 @@ import java.awt.Scrollbar; import java.awt.TextArea; import java.awt.TextField; +import java.awt.Transparency; import java.awt.Window; +import java.awt.image.ColorModel; +import java.awt.image.WritableRaster; import java.awt.peer.ButtonPeer; import java.awt.peer.CanvasPeer; import java.awt.peer.CheckboxMenuItemPeer; @@ -70,6 +77,7 @@ import java.awt.peer.WindowPeer; import sun.awt.SunToolkit; +import sun.awt.image.OffScreenImage; public abstract class CacioToolkit extends SunToolkit { @@ -323,4 +331,22 @@ public abstract PlatformWindowFactory getPlatformWindowFactory(); + /** + * Create an off-screen image base on the given component. + * + * @param component The component to base the off-screen image on. + * @param width The width of the image. + * @param hight The height of the image. + * + * @return New off-screen image. + */ + public Image createOffScreenImage(Component component, int width, int height) { + GraphicsConfiguration gc = component.getGraphicsConfiguration(); + ColorModel model = gc.getColorModel(Transparency.OPAQUE); + WritableRaster wr = + model.createCompatibleWritableRaster(width, height); + return new OffScreenImage(component, model, wr, + model.isAlphaPremultiplied()); + + } } From neugens.limasoftware at gmail.com Tue Mar 22 05:29:07 2011 From: neugens.limasoftware at gmail.com (neugens.limasoftware at gmail.com) Date: Tue, 22 Mar 2011 12:29:07 +0000 Subject: hg: caciocavallo/ng: Delegat off-screen image creation to the CacioToolkit. Message-ID: <20110322122907.820D84737D@hg.openjdk.java.net> Changeset: 38ec32bc6f11 Author: Mario Torre Date: 2011-03-22 13:28 +0100 URL: http://hg.openjdk.java.net/caciocavallo/ng/rev/38ec32bc6f11 Delegat off-screen image creation to the CacioToolkit. This allows different platforms to provide optimized off-screen images. Added a flag to indicate that the background needs to be cleared during the next paint operation. Contributed by: Ingo Proetel ! src/share/classes/sun/awt/peer/cacio/CacioComponentPeer.java ! src/share/classes/sun/awt/peer/cacio/CacioToolkit.java From neugens.limasoftware at gmail.com Thu Mar 31 11:39:56 2011 From: neugens.limasoftware at gmail.com (neugens.limasoftware at gmail.com) Date: Thu, 31 Mar 2011 18:39:56 +0000 Subject: hg: caciocavallo/ng: Add missing closing-comment marker. Message-ID: <20110331183957.9F3A847688@hg.openjdk.java.net> Changeset: f79650214a1b Author: Mario Torre Date: 2011-03-31 20:39 +0200 URL: http://hg.openjdk.java.net/caciocavallo/ng/rev/f79650214a1b Add missing closing-comment marker. ! src/share/classes/sun/awt/peer/cacio/CacioToolkit.java