From Jim.A.Graham at Sun.COM Tue May 15 16:53:47 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Tue, 15 May 2007 16:53:47 -0700 Subject: [OpenJDK Rasterizer] Getting started - some background and status Message-ID: <0JI3003S5X18W19S@d1-sfbay-09.sun.com> It looks like this mailing list has attracted a few subscribers now and so I'd like to get some discussion off the ground here. At JavaOne we mentioned the (Antialiasing) Graphics Rasterizer project and didn't give many details other than the fact that we were looking at Pisces, a renderer used for some of our recent ME products as a possible replacement for our encumbered AA rasterizer. If you've downloaded the OpenJDK sources and built them you may have noticed that not only is the Ductus Rendering engine supplied only as a binary plug, but many other classes and object files need to be supplied in binary form even though their sources are shipped with OpenJDK because they rely on interfaces and header files defined by the Ductus product. There are basically two main pieces of functionality that the Ductus library supplied us with that are used in various places. The first is, obviously, the AA rasterizer which turned paths into tiles of coverage values that we could apply to destinations using operations like MaskFill and MaskBlit, depending on whether the source color was a simple color or a raster from a Paint object. The second piece of functionality is the code that widens paths. We never developed our own code to widen paths because the Ductus library came with that functionality built-in so we leveraged it in other places in our code - including the BasicStroke object in its implementation of the createStrokedShape method and also for our non-AA rasterizer when it needs to deal with a widened path. It's the second part which causes most of the incidental encumbrances which cause much of our open code to be unbuildable, but both represent encumbrances of some sort. The Pisces renderer has some advantages to look at for a replacement: - It was developed to implement Java rendering (though at the ME level) - It is already open source - It is a fairly complete implementation - It has been tuned a bit already Unfortunately it also has some disadvantages: - It is tuned for tiny phone screens - It deals primarily with limited ranges of coordinates - It may not meet our quality goals - It was not developed to fit into our desktop rendering architecture - It's fairly large and requires a bit of porting to fit in - (What if we get done porting it and it just gets replaced anyway?) We are still working on getting Pisces adapted to our internal architecture, but until we do that we still have no non-binary solution for AA rendering and line widening in the OpenJDK build tree. In my next message I'll solicit some feedback on how to proceed... ...jim From Jim.A.Graham at Sun.COM Tue May 15 17:18:17 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Tue, 15 May 2007 17:18:17 -0700 Subject: [OpenJDK Rasterizer] Some questions Message-ID: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> Hopefully you've all had time to read my getting started message. Now some details on where we might go from here. We're continuing to work on getting the Pisces renderer into shape to have it potentially be the long-term replacement for our encumbered Ductus binary plug, but that effort will take a little more time. In the meantime, yesterday I hacked up a quick AA rasterizer replacement just to see if we could get something simple and expedient in place to disencumber our sources. It is a small piece of code, but it isn't the best performing alternative. I basically render the path at N times larger into a Region, similar to the way we turn a Shape into a clipping Region, and then generate tiles of alpha coverage from the Region at about the same relative size as the tiles that were generated by the Ductus code (32x32 pixels at a time). I can get anywhere from 50% down to 10% of the performance of the Ductus binary plug with this code at 16x16 oversampling - which is similar quality to what the encumbered library produces. If I drop it down to 4x4 oversampling the performance gets up to about 30% to 70% of the binary plug at the expense of some rendering quality. The variations in performance relate to the complexity of the path and how much area it covers. What I think I'd like to do at this point is to create a pluggable interface and use this quick and dirty solution as a temporary open source plug for AA rendering in order to get our code more buildable sooner rather than later so that others could start contributing towards the eventual replacement. It may not have the best performance, but it doesn't represent a lot of work that may eventually be obsoleted by the community and it gets us weaned off of the closed sources quicker. The remaining work then would be to create a "quick and dirty" line widener that plugs into the rest of our code reasonably well. I can talk more about that in another message. Any thoughts on getting this quick and dirty AA plugin out in the public sources sooner vs. waiting for a replacement that might have more longer term potential? Is anyone out there already looking at AA rendering alternatives that they want to plug into the OpenJDK sources? ...jim From Jim.A.Graham at Sun.COM Tue May 15 17:31:04 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Tue, 15 May 2007 17:31:04 -0700 Subject: [OpenJDK Rasterizer] Task 2 - line widening Message-ID: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> Path widening algorithms are nothing new, but they can be a pain to work out all of the corner cases. The widener in the Ductus library that we've licensed has some nice properties that we'd like to keep even as we replace it: - It preserves curves in the widened outlines - It can be used as a filter in an existing rendering pipeline - It's fairly stable and has been working fine for several years now - It has provisions for dropout control (minimum line widths) It also has a couple of issues: - Obviously it is encumbered by a license - It's native code so can suffer from JNI performance issues - There are still some cases where it can generate unexpected output I'd love to just start hacking on a replacement, but for a couple of issues: - I've seen the encumbered sources so I could probably be considered tainted - The main area where I might be considered tainted is in how they preserve curves in the output - I'd likely make all of the standard mistakes someone makes in their first draft of a line widening engine - It's one step slightly more complicated to make one that acts like a filter like the current code rather than doing one that simple takes an input path and generates an output path as in BasicStroke.createStrokedShape. - The sources that represent the interfaces that one needs to implement to use the existing line widener like a filter are part of the licensed sources. I don't have any quick and dirty answers here... It would be fairly easy to create a standalone polygonal line widener and run all paths through it flattened, but that would give up some quality (at least in the short term). I could probably do that without even tainting the sources since such algorithms are very straightforward. It would be fairly easy to integrate this into the code by doing the fairly obvious and straightforward step of calling BasicStroke.createStrokedShape on all paths, but we'd lose some performance there too in having to save the widened path in a temporary path structure before feeding it into the path filling machinery. Also, creating such a variation might make it hard to have our product code continue to work like a filter - thereby either creating lots of differences between the OpenJDK and the production code or threatening a performance loss in the production code. Has anyone started looking into replacing the line widening code yet at all? ...jim From roman at kennke.org Wed May 16 00:29:50 2007 From: roman at kennke.org (Roman Kennke) Date: Wed, 16 May 2007 09:29:50 +0200 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> Message-ID: <1179300590.5970.6.camel@mercury> Hi there, > Any thoughts on getting this quick and dirty AA plugin out in the public > sources sooner vs. waiting for a replacement that might have more longer > term potential? IMO, get the stuff out as soon as possible, so that others can work and look at it. > Is anyone out there already looking at AA rendering alternatives that > they want to plug into the OpenJDK sources? I was implementing a AA rasterizer as part of GNU Classpath. I am the only one hacking on this, so it should be easy for me to SCA it to OpenJDK if feasible. It's a Java solution (I suppose your stuff is native?) and so far I have no idea if it would fit anywhere in the OpenJDK rasterizer architecture, but it could be a starting point. It's quite flexible (can do oversampling at variable rates) and relatively fast for my taste. I am still tuning it though to have minimum memory and cpu footprint. /Roman From roman at kennke.org Wed May 16 00:36:17 2007 From: roman at kennke.org (Roman Kennke) Date: Wed, 16 May 2007 09:36:17 +0200 Subject: [OpenJDK Rasterizer] Task 2 - line widening In-Reply-To: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> References: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> Message-ID: <1179300977.5970.13.camel@mercury> Hi again, if you think that a 100% java solution for BasicStroke could be feasible, you may have a look at GNU Classpath's BasicStroke implementation. IIRC, the guy who did this, made some benchmarks of this implementation against JDK5 back then, and the performance was pretty ok, comparable with the JDK's BasicStroke. AFAIK, it preservers curves too. If you think this could be worth a try, maybe have a look at the sources for this: http://developer.classpath.org/doc/java/awt/BasicStroke-source.html (it references classes elsewhere, so you'd better pull yourself the CVS or TAR.GZ package) Or I can try how it fits with OpenJDK. /Roman From Jim.A.Graham at Sun.COM Wed May 16 03:27:48 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Wed, 16 May 2007 03:27:48 -0700 Subject: [OpenJDK Rasterizer] Task 2 - line widening In-Reply-To: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> References: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> Message-ID: <0JI4003S8QBIW1CV@d1-sfbay-09.sun.com> I want to clarify here that my statements below about whether or not and how I might be tainted with respect to creating some quick and dirty alternate implementations of line widening are personal speculation and need to be evaluated by our legal department before we can treat them as fact. My intention was to get the ball rolling and lay out possible obstacles that might affect the evaluation of alternative action items. It was my secondary intention to scare up someone external to offer up some code and given Roman's responses I think that may have been the result... ;-) ...jim Jim Graham wrote: > - I've seen the encumbered sources so I could probably be considered > tainted > - The main area where I might be considered tainted is in how they > preserve curves in the output > > It would be fairly easy to create a standalone polygonal line widener > and run all paths through it flattened, but that would give up some > quality (at least in the short term). I could probably do that without > even tainting the sources since such algorithms are very straightforward. From Jim.A.Graham at Sun.COM Wed May 16 03:38:00 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Wed, 16 May 2007 03:38:00 -0700 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <1179300590.5970.6.camel@mercury> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> Message-ID: <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> Thanks Roman, How does your AA rasterizer plug in? In the case of our encumbered code it doesn't do final rendering, it only takes in geometry and then hands back tiles of alpha coverage values in the range 0-255 stored in a byte array. We then hand those coverage tiles on to one of several types of final rendering stage which deals with how to apply the source colors to the destination with whatever compositing is in effect. The sun.java2d.pipe.CompositePipe is the typical interface implemented by the final rendering stages. Most of the parts of our pipeline can deal with tiles representing any rectangular size set of coverage values, but some parts, such as the OpenGL pipeline, only really work well if the rectangles of coverage values are about 32x32 pixels each. We could fix the pipelines to allow more generality in all cases, but if you can return tiles of about that size of coverage values then it would be an almost direct drop-in replacement. Can your rasterizer plug in at that level and in that format? ...jim Roman Kennke wrote: > Hi there, > > >> Any thoughts on getting this quick and dirty AA plugin out in the public >> sources sooner vs. waiting for a replacement that might have more longer >> term potential? > > IMO, get the stuff out as soon as possible, so that others can work and > look at it. > >> Is anyone out there already looking at AA rendering alternatives that >> they want to plug into the OpenJDK sources? > > I was implementing a AA rasterizer as part of GNU Classpath. I am the > only one hacking on this, so it should be easy for me to SCA it to > OpenJDK if feasible. It's a Java solution (I suppose your stuff is > native?) and so far I have no idea if it would fit anywhere in the > OpenJDK rasterizer architecture, but it could be a starting point. It's > quite flexible (can do oversampling at variable rates) and relatively > fast for my taste. I am still tuning it though to have minimum memory > and cpu footprint. > > /Roman > > From Jim.A.Graham at Sun.COM Wed May 16 03:40:11 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Wed, 16 May 2007 03:40:11 -0700 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <1179300590.5970.6.camel@mercury> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> Message-ID: <0JI4003BOQW5W1FV@d1-sfbay-09.sun.com> Also, how fast is "relatively fast"? Do you have some comparative benchmarks of Classpath vs. production JDK? (Note that there is a benchmark shipped with the OpenJDK in src/share/demo/java2d/J2DBench that can evaluate the speed of various AA primitives if you check the right boxes...) ...jim Roman Kennke wrote: > I was implementing a AA rasterizer as part of GNU Classpath. I am the > only one hacking on this, so it should be easy for me to SCA it to > OpenJDK if feasible. It's a Java solution (I suppose your stuff is > native?) and so far I have no idea if it would fit anywhere in the > OpenJDK rasterizer architecture, but it could be a starting point. It's > quite flexible (can do oversampling at variable rates) and relatively > fast for my taste. I am still tuning it though to have minimum memory > and cpu footprint. > > /Roman > > From roman at kennke.org Wed May 16 03:41:39 2007 From: roman at kennke.org (Roman Kennke) Date: Wed, 16 May 2007 12:41:39 +0200 Subject: [OpenJDK Rasterizer] Task 2 - line widening In-Reply-To: <0JI4003S8QBIW1CV@d1-sfbay-09.sun.com> References: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> <0JI4003S8QBIW1CV@d1-sfbay-09.sun.com> Message-ID: <1179312099.5970.51.camel@mercury> Hi, > I want to clarify here that my statements below about whether or not and > how I might be tainted with respect to creating some quick and dirty > alternate implementations of line widening are personal speculation and > need to be evaluated by our legal department before we can treat them as > fact. > > My intention was to get the ball rolling and lay out possible obstacles > that might affect the evaluation of alternative action items. It was my > secondary intention to scare up someone external to offer up some code > and given Roman's responses I think that may have been the result... ;-) Nicely formulated :-) Anyway, I didn't want to taint anybody, nor do I intent to push classpath's stuff over you. I only want to offer some help. So let me reformulate my suggestions as question and explanations: Could a java only implementation for the stroking and AA rendering be feasible? I've worked on both and found that it's possible to implement with good performance. The stroking (BasicStroke) on which I (and others) worked preserves curves and is comparable in performance (afaict) to the JDK5/6 BasicStroke. The rasterizer is basically a couple of classes, the base class of which is ScanlineConverter, which can render Shape objects both AA and non-AA to some surface (defined by an interface). The performance is quite good and it doesn't need any allocations except some initial table setup. /Roman > > - I've seen the encumbered sources so I could probably be considered > > tainted > > - The main area where I might be considered tainted is in how they > > preserve curves in the output > > > > It would be fairly easy to create a standalone polygonal line widener > > and run all paths through it flattened, but that would give up some > > quality (at least in the short term). I could probably do that without > > even tainting the sources since such algorithms are very straightforward. > -- http://kennke.org/blog/ From roman at kennke.org Wed May 16 04:00:34 2007 From: roman at kennke.org (Roman Kennke) Date: Wed, 16 May 2007 13:00:34 +0200 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> Message-ID: <1179313234.5970.68.camel@mercury> Hi, > How does your AA rasterizer plug in? The ScanlineConverter class basically takes one Shape object (the actual Shape to be rendered), another one (the clip), an AffineTransform and a parameter which says how sub-pixel resolution to render. It also takes an object of an interface type to which the actual rendering is then delegated. Objects implementing this interface basically fill lines with pixels according to the coverage values passed into them. It doesn't create tiles of larger areas yet, but that shouldn't be a big problem to add. This means more or less aggregating the data of multiple lines. Also, I don't deal with the data in a pixel-by-pixel format, but with ranges on a scanline. Basically I store the x location, length and coverage value for any scanline segment with one coverage value. I haven't tested performance on JDK yet, Classpath can't easily plugged into hotspot as is. Rendering some shapes like glyphs with classpath, cacaovm and the Escher backend showed a performance difference of around 16x slower for non-AA rendering and 4x slower for AA rendering. The performance difference between hotspot and cacaovm is also around the factor 4x (cacao beeing 4x slower than hotspot). But takes these numbers with a large portion of salt, it's only been a very naive simplistic benchmark. If you think this could be feasible and I should try to wire it to OpenJDK, let me know, and give me some pointers where to start. /Roman -- http://kennke.org/blog/ From fkung at redhat.com Wed May 16 09:05:39 2007 From: fkung at redhat.com (Francis Kung) Date: Wed, 16 May 2007 12:05:39 -0400 Subject: [OpenJDK Rasterizer] Task 2 - line widening In-Reply-To: <1179312099.5970.51.camel@mercury> References: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> <0JI4003S8QBIW1CV@d1-sfbay-09.sun.com> <1179312099.5970.51.camel@mercury> Message-ID: <464B2BD3.1040002@redhat.com> > The stroking (BasicStroke) on which I (and others) worked preserves > curves and is comparable in performance (afaict) to the JDK5/6 > BasicStroke. I hate to rain on this parade, but I'm fairly sure that we (Classpath) flatten cubic curves in our stroker =( I remember reading at some point that it is mathematically impossible to derive a bezier curve parallel to another bezier curve, so it has to approximate somewhere... I assume that the preferred behaviour would be to produce an approximated curve, rather than flattening, though. Anyways, I haven't looked at the openjdk source in this area yet, but I will soon and see how feasible the Classpath code is (at least temporarily) - something is still better than nothing. Cheers, Francis From Jim.A.Graham at Sun.COM Wed May 16 11:41:55 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Wed, 16 May 2007 11:41:55 -0700 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <1179313234.5970.68.camel@mercury> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> <1179313234.5970.68.camel@mercury> Message-ID: <0JI500343D70W13Z@d1-sfbay-09.sun.com> Hi Roman, Does the code which does the scan conversion also directly modify the pixels? The reason I ask is that our rasterizers simply calculate data about how the pixels are to be modified (spans of pixels to fill for non-AA and buffers of coverage values for AA), but leave it to another piece of code to do the actual pixel modification. From your description it sounds like your AA rasterizer goes all the way from geometry down to actually modifying the destination pixels in one indivisible module - is that really the case? ...jim Roman Kennke wrote: > Hi, > >> How does your AA rasterizer plug in? > > The ScanlineConverter class basically takes one Shape object (the actual > Shape to be rendered), another one (the clip), an AffineTransform and a > parameter which says how sub-pixel resolution to render. It also takes > an object of an interface type to which the actual rendering is then > delegated. Objects implementing this interface basically fill lines with > pixels according to the coverage values passed into them. > > It doesn't create tiles of larger areas yet, but that shouldn't be a big > problem to add. This means more or less aggregating the data of multiple > lines. Also, I don't deal with the data in a pixel-by-pixel format, but > with ranges on a scanline. Basically I store the x location, length and > coverage value for any scanline segment with one coverage value. > > I haven't tested performance on JDK yet, Classpath can't easily plugged > into hotspot as is. Rendering some shapes like glyphs with classpath, > cacaovm and the Escher backend showed a performance difference of around > 16x slower for non-AA rendering and 4x slower for AA rendering. The > performance difference between hotspot and cacaovm is also around the > factor 4x (cacao beeing 4x slower than hotspot). But takes these numbers > with a large portion of salt, it's only been a very naive simplistic > benchmark. > > If you think this could be feasible and I should try to wire it to > OpenJDK, let me know, and give me some pointers where to start. > > /Roman > From Jim.A.Graham at Sun.COM Wed May 16 11:46:57 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Wed, 16 May 2007 11:46:57 -0700 Subject: [OpenJDK Rasterizer] Task 2 - line widening In-Reply-To: <464B2BD3.1040002@redhat.com> References: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> <0JI4003S8QBIW1CV@d1-sfbay-09.sun.com> <1179312099.5970.51.camel@mercury> <464B2BD3.1040002@redhat.com> Message-ID: <0JI50045ODFEK4I4@d1-sfbay-10.sun.com> Francis Kung wrote: > I remember reading at some point that it is mathematically impossible to > derive a bezier curve parallel to another bezier curve, so it has to > approximate somewhere... I assume that the preferred behaviour would be > to produce an approximated curve, rather than flattening, though. That is likely true, but it is also mathematically impossible to derive a set of straight lines that are exactly parallel to a bezier curve as well, so both techniques are producing approximations. Would you rather approximate a parallel bezier with lines that vary from the true curve by no more than epsilon or with a bezier that varies from the desired curve by no more than epsilon? The advantage of curves in the widened path is that there are fewer segments to pass on to the final renderer. If anyone were to call BS.createStrokedShape and then stroke the widened outline, it is also prettier, but the performance advantage in reducing the number of segments for rendering when it is invoked internally as part of a rendering operation is probably the more important benefit... ...jim From roman at kennke.org Wed May 16 11:55:53 2007 From: roman at kennke.org (Roman Kennke) Date: Wed, 16 May 2007 20:55:53 +0200 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <0JI500343D70W13Z@d1-sfbay-09.sun.com> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> <1179313234.5970.68.camel@mercury> <0JI500343D70W13Z@d1-sfbay-09.sun.com> Message-ID: <1179341753.5970.92.camel@mercury> Hi Jim, > Does the code which does the scan conversion also directly modify the > pixels? No. The actual pixel modification is done elsewhere, depending on the compositing and backend etc. The scanline converter only calculates the spans and coverage values of scanlines and passes this data downward to the 'backend' for actual pixel level modification. /Roman -- http://kennke.org/blog/ From fkung at redhat.com Wed May 16 12:25:07 2007 From: fkung at redhat.com (Francis Kung) Date: Wed, 16 May 2007 15:25:07 -0400 Subject: [OpenJDK Rasterizer] Task 2 - line widening In-Reply-To: <0JI50045ODFEK4I4@d1-sfbay-10.sun.com> References: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> <0JI4003S8QBIW1CV@d1-sfbay-09.sun.com> <1179312099.5970.51.camel@mercury> <464B2BD3.1040002@redhat.com> <0JI50045ODFEK4I4@d1-sfbay-10.sun.com> Message-ID: <464B5A93.7080701@redhat.com> >> I remember reading at some point that it is mathematically impossible >> to derive a bezier curve parallel to another bezier curve > > That is likely true, but it is also mathematically impossible to derive > a set of straight lines that are exactly parallel to a bezier curve as > well, so both techniques are producing approximations. ... > The advantage of curves in the widened path is that there are fewer > segments to pass on to the final renderer. If anyone were to call > BS.createStrokedShape and then stroke the widened outline, it is also > prettier, but the performance advantage in reducing the number of > segments for rendering when it is invoked internally as part of a > rendering operation is probably the more important benefit... Agreed - of the two approximation techniques, an approximated curve is better than flattening. I'm guessing that the encumbered code you've seen mostly deals with finding a good way to create such a curve... and unfortunately Classpath went the route of flattening in this area. Francis From roman.kennke at aicas.com Fri May 18 09:14:17 2007 From: roman.kennke at aicas.com (Roman Kennke) Date: Fri, 18 May 2007 18:14:17 +0200 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <0JI500343D70W13Z@d1-sfbay-09.sun.com> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> <1179313234.5970.68.camel@mercury> <0JI500343D70W13Z@d1-sfbay-09.sun.com> Message-ID: <1179504857.6911.21.camel@mercury> Hi Jim, Is there anything I can do right now to make progress? You could point me to the pieces where the rasterizer has to be plugged in, or I could send you my code for review (currently it's FSF copyrighted code, but I can relatively easily make use of the copyright grantback clause because it's all written by myself). /Roman > Does the code which does the scan conversion also directly modify the > pixels? The reason I ask is that our rasterizers simply calculate data > about how the pixels are to be modified (spans of pixels to fill for > non-AA and buffers of coverage values for AA), but leave it to another > piece of code to do the actual pixel modification. > > From your description it sounds like your AA rasterizer goes all the > way from geometry down to actually modifying the destination pixels in > one indivisible module - is that really the case? > > ...jim > > Roman Kennke wrote: > > Hi, > > > >> How does your AA rasterizer plug in? > > > > The ScanlineConverter class basically takes one Shape object (the actual > > Shape to be rendered), another one (the clip), an AffineTransform and a > > parameter which says how sub-pixel resolution to render. It also takes > > an object of an interface type to which the actual rendering is then > > delegated. Objects implementing this interface basically fill lines with > > pixels according to the coverage values passed into them. > > > > It doesn't create tiles of larger areas yet, but that shouldn't be a big > > problem to add. This means more or less aggregating the data of multiple > > lines. Also, I don't deal with the data in a pixel-by-pixel format, but > > with ranges on a scanline. Basically I store the x location, length and > > coverage value for any scanline segment with one coverage value. > > > > I haven't tested performance on JDK yet, Classpath can't easily plugged > > into hotspot as is. Rendering some shapes like glyphs with classpath, > > cacaovm and the Escher backend showed a performance difference of around > > 16x slower for non-AA rendering and 4x slower for AA rendering. The > > performance difference between hotspot and cacaovm is also around the > > factor 4x (cacao beeing 4x slower than hotspot). But takes these numbers > > with a large portion of salt, it's only been a very naive simplistic > > benchmark. > > > > If you think this could be feasible and I should try to wire it to > > OpenJDK, let me know, and give me some pointers where to start. > > > > /Roman > > -- Dipl.-Inf. Roman Kennke, Software Engineer, http://kennke.org aicas Allerton Interworks Computer Automated Systems GmbH Haid-und-Neu-Stra?e 18 * D-76131 Karlsruhe * Germany http://www.aicas.com * Tel: +49-721-663 968-0 USt-Id: DE216375633, Handelsregister HRB 109481, AG Karlsruhe Gesch?ftsf?hrer: Dr. James J. Hunt From Jim.A.Graham at Sun.COM Fri May 18 15:50:38 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Fri, 18 May 2007 15:50:38 -0700 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <1179504857.6911.21.camel@mercury> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> <1179313234.5970.68.camel@mercury> <0JI500343D70W13Z@d1-sfbay-09.sun.com> <1179504857.6911.21.camel@mercury> Message-ID: <0JI9006ARE3YRCI5@d1-sfbay-09.sun.com> I'm making good headway on ridding our open code from build dependencies on the licensed Ductus library and package. I'll try to send out some javadocs of the interface as soon as I get it all in order... ...jim Roman Kennke wrote: > Hi Jim, > > Is there anything I can do right now to make progress? You could point > me to the pieces where the rasterizer has to be plugged in, or I could > send you my code for review (currently it's FSF copyrighted code, but I > can relatively easily make use of the copyright grantback clause because > it's all written by myself). > > /Roman > >> Does the code which does the scan conversion also directly modify the >> pixels? The reason I ask is that our rasterizers simply calculate data >> about how the pixels are to be modified (spans of pixels to fill for >> non-AA and buffers of coverage values for AA), but leave it to another >> piece of code to do the actual pixel modification. >> >> From your description it sounds like your AA rasterizer goes all the >> way from geometry down to actually modifying the destination pixels in >> one indivisible module - is that really the case? >> >> ...jim >> >> Roman Kennke wrote: >>> Hi, >>> >>>> How does your AA rasterizer plug in? >>> The ScanlineConverter class basically takes one Shape object (the actual >>> Shape to be rendered), another one (the clip), an AffineTransform and a >>> parameter which says how sub-pixel resolution to render. It also takes >>> an object of an interface type to which the actual rendering is then >>> delegated. Objects implementing this interface basically fill lines with >>> pixels according to the coverage values passed into them. >>> >>> It doesn't create tiles of larger areas yet, but that shouldn't be a big >>> problem to add. This means more or less aggregating the data of multiple >>> lines. Also, I don't deal with the data in a pixel-by-pixel format, but >>> with ranges on a scanline. Basically I store the x location, length and >>> coverage value for any scanline segment with one coverage value. >>> >>> I haven't tested performance on JDK yet, Classpath can't easily plugged >>> into hotspot as is. Rendering some shapes like glyphs with classpath, >>> cacaovm and the Escher backend showed a performance difference of around >>> 16x slower for non-AA rendering and 4x slower for AA rendering. The >>> performance difference between hotspot and cacaovm is also around the >>> factor 4x (cacao beeing 4x slower than hotspot). But takes these numbers >>> with a large portion of salt, it's only been a very naive simplistic >>> benchmark. >>> >>> If you think this could be feasible and I should try to wire it to >>> OpenJDK, let me know, and give me some pointers where to start. >>> >>> /Roman >>> From Jim.A.Graham at Sun.COM Fri May 18 16:00:49 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Fri, 18 May 2007 16:00:49 -0700 Subject: [OpenJDK Rasterizer] Task 2 - line widening In-Reply-To: <464B5A93.7080701@redhat.com> References: <0JI3003LYYRDW1JS@d1-sfbay-09.sun.com> <0JI4003S8QBIW1CV@d1-sfbay-09.sun.com> <1179312099.5970.51.camel@mercury> <464B2BD3.1040002@redhat.com> <0JI50045ODFEK4I4@d1-sfbay-10.sun.com> <464B5A93.7080701@redhat.com> Message-ID: <0JI90034FEKXW2MW@d1-sfbay-09.sun.com> Yes, that's definitely an area where I would probably be considered tainted if I created code that widened curves - hopefully someone on the ClassPath team might consider this a challenge to come up with a novel way to do it? ;-) ...jim Francis Kung wrote: >>> I remember reading at some point that it is mathematically impossible >>> to derive a bezier curve parallel to another bezier curve >> >> That is likely true, but it is also mathematically impossible to >> derive a set of straight lines that are exactly parallel to a bezier >> curve as well, so both techniques are producing approximations. > ... >> The advantage of curves in the widened path is that there are fewer >> segments to pass on to the final renderer. If anyone were to call >> BS.createStrokedShape and then stroke the widened outline, it is also >> prettier, but the performance advantage in reducing the number of >> segments for rendering when it is invoked internally as part of a >> rendering operation is probably the more important benefit... > > Agreed - of the two approximation techniques, an approximated curve is > better than flattening. I'm guessing that the encumbered code you've > seen mostly deals with finding a good way to create such a curve... and > unfortunately Classpath went the route of flattening in this area. > > Francis From Phil.Race at Sun.COM Fri May 18 17:26:05 2007 From: Phil.Race at Sun.COM (Phil Race) Date: Fri, 18 May 2007 17:26:05 -0700 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <1179504857.6911.21.camel@mercury> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> <1179313234.5970.68.camel@mercury> <0JI500343D70W13Z@d1-sfbay-09.sun.com> <1179504857.6911.21.camel@mercury> Message-ID: <464E441D.3020009@sun.com> Roman, Roman Kennke wrote: > Hi Jim, > > Is there anything I can do right now to make progress? You could point > me to the pieces where the rasterizer has to be plugged in, or I could > send you my code for review (currently it's FSF copyrighted code, but I > can relatively easily make use of the copyright grantback clause because > it's all written by myself). I am told you need to send in code as per this procedure .. http://openjdk.java.net/contribute/ .. until we get a better process. -phil. From Jim.A.Graham at Sun.COM Fri May 18 19:40:50 2007 From: Jim.A.Graham at Sun.COM (Jim Graham) Date: Fri, 18 May 2007 19:40:50 -0700 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <1179313234.5970.68.camel@mercury> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> <1179313234.5970.68.camel@mercury> Message-ID: <0JI9006T0ORMRC96@d1-sfbay-09.sun.com> As I've said, right now I'm working on creating a cleaner separation of our open code and the encumbered libraries that do the pixel coverage calculations. In that effort, I'm creating an interface behind which the encumbered code can live and be replaced. Since our current library is tile-based, that is the direction I'm taking the interface. But, that begs the question - is that the right direction to choose? Short term or long run? In the short term, I can get the interfaces in place and the code factored out fairly quickly with this approach - in a week or two. Also, in the short, or short to intermediate term, we focus the replacement effort on a smaller piece and we can keep all of the rendering back end loops in several pipelines (software, OpenGL, D3D, X11) which have all been tuned. In the long run, maybe we want to investigate different intermediate formats, but doing so will involve not only an analysis of how efficiently we can compute them, but also how efficiently they can be rendered across all of the different pipelines. It would be a shame to integrate a new run-length-coverage based rasterizer only to find out that OpenGL renders such a format 10x as slow as it can manage a tile, for instance. On the other hand, if I get this new interface out there in the next update and we find that the ClassPath rasterizer is slower by 4x or worse because it is not designed for that interface, we haven't really created a viable replacement yet anyway. I think getting this straw man separate of "tile generation" into the workspace is a good starting point, though. The fact that this might mean that your renderer will have to be shoe-horned in or converted into a new format isn't optimal, but we can get started on the road fairly quickly and at least achieve a 100% unencumbered milestone in this area sooner. The alternative would be to start looking to replace the renderer at a higher level - at the "ShapeDrawPipe" level and doing so would involve completely replacing not only the "geometry to coverage" library, but also the backend renderers and possibly disabling the OpenGL pipelines until they can be updated... ...jim Roman Kennke wrote: > It doesn't create tiles of larger areas yet, but that shouldn't be a big > problem to add. This means more or less aggregating the data of multiple > lines. Also, I don't deal with the data in a pixel-by-pixel format, but > with ranges on a scanline. Basically I store the x location, length and > coverage value for any scanline segment with one coverage value. > > I haven't tested performance on JDK yet, Classpath can't easily plugged > into hotspot as is. Rendering some shapes like glyphs with classpath, > cacaovm and the Escher backend showed a performance difference of around > 16x slower for non-AA rendering and 4x slower for AA rendering. The > performance difference between hotspot and cacaovm is also around the > factor 4x (cacao beeing 4x slower than hotspot). But takes these numbers > with a large portion of salt, it's only been a very naive simplistic > benchmark. From roman.kennke at aicas.com Sat May 19 00:55:43 2007 From: roman.kennke at aicas.com (Roman Kennke) Date: Sat, 19 May 2007 09:55:43 +0200 Subject: [OpenJDK Rasterizer] Some questions In-Reply-To: <0JI9006T0ORMRC96@d1-sfbay-09.sun.com> References: <0JI3004HXY61K4C3@d1-sfbay-10.sun.com> <1179300590.5970.6.camel@mercury> <0JI4003TEQSIW1EV@d1-sfbay-09.sun.com> <1179313234.5970.68.camel@mercury> <0JI9006T0ORMRC96@d1-sfbay-09.sun.com> Message-ID: <1179561343.5967.5.camel@mercury> Hi, > As I've said, right now I'm working on creating a cleaner separation of > our open code and the encumbered libraries that do the pixel coverage > calculations. In that effort, I'm creating an interface behind which > the encumbered code can live and be replaced. Great. I will use the time to polish up my implementation and resolve the legal issues so that I can actually send you my code. When you're done with the interfaces, I'll look at implementing a conversion class to allow us to easily plug in the rasterizer. > Since our current library is tile-based, that is the direction I'm > taking the interface. But, that begs the question - is that the right > direction to choose? Short term or long run? > > In the short term, I can get the interfaces in place and the code > factored out fairly quickly with this approach - in a week or two. > Also, in the short, or short to intermediate term, we focus the > replacement effort on a smaller piece and we can keep all of the > rendering back end loops in several pipelines (software, OpenGL, D3D, > X11) which have all been tuned. > > In the long run, maybe we want to investigate different intermediate > formats, but doing so will involve not only an analysis of how > efficiently we can compute them, but also how efficiently they can be > rendered across all of the different pipelines. Sure. In my experience, a run-length format is more efficient, because for the tile format (when I understand it correctly) in order to find the coverage for a given pixel, you have to actually look at each pixel. While with a run-length form you directly find consecutive chunks which can probably be rendered in one go. But then again, my primary targets so far have been framebuffers (often a memset() for consecutive chunks) and X11. > I think getting this straw man separate of "tile generation" into the > workspace is a good starting point, though. The fact that this might > mean that your renderer will have to be shoe-horned in or converted into > a new format isn't optimal, but we can get started on the road fairly > quickly and at least achieve a 100% unencumbered milestone in this area > sooner. Agreed. > The alternative would be to start looking to replace the renderer at a > higher level - at the "ShapeDrawPipe" level and doing so would involve > completely replacing not only the "geometry to coverage" library, but > also the backend renderers and possibly disabling the OpenGL pipelines > until they can be updated... Has to be investigated then. Kind regards, Roman -- Dipl.-Inf. Roman Kennke, Software Engineer, http://kennke.org aicas Allerton Interworks Computer Automated Systems GmbH Haid-und-Neu-Stra?e 18 * D-76131 Karlsruhe * Germany http://www.aicas.com * Tel: +49-721-663 968-0 USt-Id: DE216375633, Handelsregister HRB 109481, AG Karlsruhe Gesch?ftsf?hrer: Dr. James J. Hunt From Tom.Marble at Sun.COM Sat May 19 14:06:04 2007 From: Tom.Marble at Sun.COM (Tom Marble) Date: Sat, 19 May 2007 16:06:04 -0500 Subject: [OpenJDK Rasterizer] test message Message-ID: <464F66BC.4030002@sun.com> All: Please ignore this test message. --Tom From roman.kennke at aicas.com Thu May 24 14:23:31 2007 From: roman.kennke at aicas.com (Roman Kennke) Date: Thu, 24 May 2007 23:23:31 +0200 Subject: [OpenJDK Rasterizer] Rasterizer replacement proposal Message-ID: <1180041811.6039.35.camel@mercury> Hello, I finished up what I wanted to add to my rasterizer and want to make it available to anybody who would like to review it or play with it. You can download the code here: http://kennke.org/~roman/rasterizer-rkennke-2007-05-24.zip Legal note: Right now this is ripped straight out of GNU Classpath, to which this code originally belongs. It is licensed under GPL+exception. The FSF holds the copyright to that code through their copyright assignment thingy. However, I am the original author of all of the code and will do the copyright grantback procedure and sign the SCA (I think my employer (aicas) already did so for me even) to make this code available to OpenJDK. I would like to hear your opinion on the code first, though. If that doesn't look feasible at all, then I can avoid doing all that. Technical notes: - The entry point is the ScanlineConverter class. I have an instance of this in my Graphics2D implementation and store it as a thread local variable. The 'frontend' is ScanlineConverter.renderShape(), the 'backend' is specified by the interface Pixelizer. The Pixelizer implementation is responsible for actually modifying the surface pixels. It gets called once for each scanline. The coverage information is carried by ScanlineCoverage instances. Use ScanlineCoverage.iterate() to get the actual coverage information, and use getMaximumCoverage() to get the scaling right. - In order to convert this information to the tiles that you need, it shouldn't be too hard to implement a Pixelizer that aggregates a couple of scanlines into tiles. The interesting task here is to do this efficiently. - The rasterizer is a generic Shape rasterizer. I believe it is very efficient for rasterizing arbitrary Shape objects, plus transform and clipping. It uses a sophisticated scanline algorithm (if you want it, I can write more about this, but this is going to be a small essay then.. ;-) ). However, it is not (yet) optimized for certain frequent special cases. Most importantly, I think lines and rectangles could be done much faster in certain settings. I'll add that in the future. - Most calculations are done using fixed point arithmetics (that's because the original target have been platforms without FPU). I don't know if that does any good for performance on usual desktop systems, but it won't do any harm either. - Datastructures are reused in almost all cases. That means that the scanline converter doesn't allocate any new memory once it has its datastructures set up after a couple of rendering cycles. New allocations only happen when the datastructures need to be extended (e.g. for extraordinary complex shapes). I will still be around tomorrow, and starting from saturday I will be offline for 2 weeks (or, at best be online sporadically) because I'm going to marry, yay! If you have urgent questions it's a good idea to ask them ASAP. Otherwise, I'll be curious to hear your opinions in two weeks then. Cheers, Roman -- Dipl.-Inform. Roman Kennke, Software Engineer, http://kennke.org aicas Allerton Interworks Computer Automated Systems GmbH Haid-und-Neu-Stra?e 18 * D-76131 Karlsruhe * Germany http://www.aicas.com * Tel: +49-721-663 968-0 USt-Id: DE216375633, Handelsregister HRB 109481, AG Karlsruhe Gesch?ftsf?hrer: Dr. James J. Hunt From Phil.Race at Sun.COM Thu May 24 16:27:01 2007 From: Phil.Race at Sun.COM (Phil Race) Date: Thu, 24 May 2007 16:27:01 -0700 Subject: [OpenJDK Rasterizer] Rasterizer replacement proposal In-Reply-To: <1180041811.6039.35.camel@mercury> References: <1180041811.6039.35.camel@mercury> Message-ID: <46561F45.60709@sun.com> Roman, Thanks for this. Jim Graham is the best person to comment on this but he's on vacation until next week some time. And congrats on getting married. -phil. Roman Kennke wrote: > Hello, > > I finished up what I wanted to add to my rasterizer and want to make it > available to anybody who would like to review it or play with it. You > can download the code here: > > http://kennke.org/~roman/rasterizer-rkennke-2007-05-24.zip > > Legal note: Right now this is ripped straight out of GNU Classpath, to > which this code originally belongs. It is licensed under GPL+exception. > The FSF holds the copyright to that code through their copyright > assignment thingy. However, I am the original author of all of the code > and will do the copyright grantback procedure and sign the SCA (I think > my employer (aicas) already did so for me even) to make this code > available to OpenJDK. I would like to hear your opinion on the code > first, though. If that doesn't look feasible at all, then I can avoid > doing all that. > > Technical notes: > - The entry point is the ScanlineConverter class. I have an instance of > this in my Graphics2D implementation and store it as a thread local > variable. The 'frontend' is ScanlineConverter.renderShape(), the > 'backend' is specified by the interface Pixelizer. The Pixelizer > implementation is responsible for actually modifying the surface pixels. > It gets called once for each scanline. The coverage information is > carried by ScanlineCoverage instances. Use ScanlineCoverage.iterate() to > get the actual coverage information, and use getMaximumCoverage() to get > the scaling right. > - In order to convert this information to the tiles that you need, it > shouldn't be too hard to implement a Pixelizer that aggregates a couple > of scanlines into tiles. The interesting task here is to do this > efficiently. > - The rasterizer is a generic Shape rasterizer. I believe it is very > efficient for rasterizing arbitrary Shape objects, plus transform and > clipping. It uses a sophisticated scanline algorithm (if you want it, I > can write more about this, but this is going to be a small essay > then.. ;-) ). However, it is not (yet) optimized for certain frequent > special cases. Most importantly, I think lines and rectangles could be > done much faster in certain settings. I'll add that in the future. > - Most calculations are done using fixed point arithmetics (that's > because the original target have been platforms without FPU). I don't > know if that does any good for performance on usual desktop systems, but > it won't do any harm either. > - Datastructures are reused in almost all cases. That means that the > scanline converter doesn't allocate any new memory once it has its > datastructures set up after a couple of rendering cycles. New > allocations only happen when the datastructures need to be extended > (e.g. for extraordinary complex shapes). > > I will still be around tomorrow, and starting from saturday I will be > offline for 2 weeks (or, at best be online sporadically) because I'm > going to marry, yay! If you have urgent questions it's a good idea to > ask them ASAP. Otherwise, I'll be curious to hear your opinions in two > weeks then. > > Cheers, Roman >