From mike at bfo.com Tue Feb 2 18:26:44 2016 From: mike at bfo.com (Mike Bremford) Date: Tue, 2 Feb 2016 18:26:44 +0000 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 Message-ID: Hi All I was steered to this list after initially posting here: https://community.oracle.com/message/13632163 We've been doing some initial testing of our product with Java 9 (build 9-ea+100-2016-01-06-195905.javare.4235.nc on multi-core Xeon running Linux) and managed to get a consistent segfault out of it. Stack: [0x00007ffb7f351000,0x00007ffb7f452000], sp=0x00007ffb7f44f1c0, free space=1016k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) J 9056 C2 sun.java2d.marlin.Renderer._endRendering(II)V (2400 bytes) @ 0x00007ffb6def265b [0x00007ffb6def1e20+0x000000000000083b] J 3982 C2 sun.java2d.marlin.Renderer.endRendering(I)V (53 bytes) @ 0x00007ffb6d4d3be0 [0x00007ffb6d4d3b20+0x00000000000000c0] J 4993 C2 sun.java2d.marlin.MarlinTileGenerator.getAlphaNoRLE([BII)V (401 bytes) @ 0x00007ffb6d6f9744 [0x00007ffb6d6f92a0+0x00000000000004a4] J 8725 C2 sun.java2d.pipe.AAShapePipe.renderTiles(Lsun/java2d/SunGraphics2D;Ljava/awt/Shape;Lsun/java2d/pipe/AATileGenerator;[ILsun/java2d/pipe/AAShapePipe$TileState;)V (302 bytes) @ 0x00007ffb6d321600 [0x00007ffb6d321420+0x00000000000001e0] J 13257 C2 sun.java2d.pipe.AAShapePipe.renderPath(Lsun/java2d/SunGraphics2D;Ljava/awt/Shape;Ljava/awt/BasicStroke;)V (99 bytes) @ 0x00007ffb6d9f0ce4 [0x00007ffb6d9efe20+0x0000000000000ec4] J 13391 C2 sun.java2d.pipe.ValidatePipe.draw(Lsun/java2d/SunGraphics2D;Ljava/awt/Shape;)V (20 bytes) @ 0x00007ffb6dc032b8 [0x00007ffb6dc03120+0x0000000000000198] J 12857 C2 sun.java2d.SunGraphics2D.draw(Ljava/awt/Shape;)V (64 bytes) @ 0x00007ffb6d8a6298 [0x00007ffb6d8a60e0+0x00000000000001b8] I have narrowed it down to a self-contained test case which you can run - there's a lot going on under the hood in our Jar which I cant supply the source for, but I imagine for a SEGV the source won't make much difference. The testcase is 3.2MB, if anyone wants it please drop me an email directly or let me know and I'll post it to this list. Cheers... Mike -- ----------------------------------------------------- Mike Bremford - CTO mike at bfo.com Big Faceless Organization http://bfo.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourges.laurent at gmail.com Tue Feb 2 19:02:00 2016 From: bourges.laurent at gmail.com (=?UTF-8?Q?Laurent_Bourg=C3=A8s?=) Date: Tue, 2 Feb 2016 20:02:00 +0100 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: References: Message-ID: Hi, Please send me your test case at bourges.laurent (gmail). I need it to reproduce your issue. Thanks, Laurent -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike at bfo.com Tue Feb 2 19:39:29 2016 From: mike at bfo.com (Mike Bremford) Date: Tue, 2 Feb 2016 19:39:29 +0000 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: References: Message-ID: Hi Laurent - gmail are blocking my attachments for security reasons, but you can download at http://bfo.com/misc/test.tar.gz - let me know when you've got it and I'll delete it. Cheers... Mike -- ----------------------------------------------------- Mike Bremford - CTO mike at bfo.com Big Faceless Organization http://bfo.com On 2 February 2016 at 19:02, Laurent Bourg?s wrote: > Hi, > > Please send me your test case at bourges.laurent (gmail). > > I need it to reproduce your issue. > > Thanks, > Laurent > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourges.laurent at gmail.com Tue Feb 2 19:58:58 2016 From: bourges.laurent at gmail.com (=?UTF-8?Q?Laurent_Bourg=C3=A8s?=) Date: Tue, 2 Feb 2016 20:58:58 +0100 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: References: Message-ID: Mike, I download your test code, thanks ! I will look later at your problem. Laurent 2016-02-02 20:39 GMT+01:00 Mike Bremford : > Hi Laurent - gmail are blocking my attachments for security reasons, but > you can download at http://bfo.com/misc/test.tar.gz - let me know when > you've got it and I'll delete it. > > Cheers... Mike > -- > ----------------------------------------------------- > Mike Bremford - CTO mike at bfo.com > Big Faceless Organization http://bfo.com > > On 2 February 2016 at 19:02, Laurent Bourg?s > wrote: > >> Hi, >> >> Please send me your test case at bourges.laurent (gmail). >> >> I need it to reproduce your issue. >> >> Thanks, >> Laurent >> > > -- -- Laurent Bourg?s -------------- next part -------------- An HTML attachment was scrubbed... URL: From philip.race at oracle.com Tue Feb 2 20:22:40 2016 From: philip.race at oracle.com (Phil Race) Date: Tue, 02 Feb 2016 12:22:40 -0800 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: References: Message-ID: <56B11010.8@oracle.com> I have filed a bug https://bugs.openjdk.java.net/browse/JDK-8148886 I also downloaded the test case and see the crash. FYI if you want to continue to test JDK9 builds and this is a blocker for you (I suppose it is), then until it is fixed you can do -Dsun.java2d.renderer=sun.dc.DuctusRenderingEngine or -Dsun.java2d.renderer=sun.java2d.pisces.PiscesRenderingEngine -phil. On 02/02/2016 11:58 AM, Laurent Bourg?s wrote: > Mike, > > I download your test code, thanks ! > > I will look later at your problem. > > Laurent > > > 2016-02-02 20:39 GMT+01:00 Mike Bremford >: > > Hi Laurent - gmail are blocking my attachments for security > reasons, but you can download at http://bfo.com/misc/test.tar.gz - > let me know when you've got it and I'll delete it. > > Cheers... Mike > -- > ----------------------------------------------------- > Mike Bremford - CTO mike at bfo.com > Big Faceless Organization http://bfo.com > > On 2 February 2016 at 19:02, Laurent Bourg?s > > wrote: > > Hi, > > Please send me your test case at bourges.laurent (gmail). > > I need it to reproduce your issue. > > Thanks, > Laurent > > > > > > -- > -- > Laurent Bourg?s -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourges.laurent at gmail.com Wed Feb 3 13:28:40 2016 From: bourges.laurent at gmail.com (=?UTF-8?Q?Laurent_Bourg=C3=A8s?=) Date: Wed, 3 Feb 2016 14:28:40 +0100 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: <56B11010.8@oracle.com> References: <56B11010.8@oracle.com> Message-ID: Phil, bug https://bugs.openjdk.java.net/browse/JDK-8148886 Thanks for creating the bug ! I spent some time tracking down the bug and make the diagnostic: 1. I enabled diagnostics: *-Dsun.java2d.renderer.log=true* *-Dsun.java2d.renderer.doChecks=true* -Dsun.java2d.renderer.doStats=true -Dsun.java2d.renderer.logUnsafeMalloc=true I also enabled MarlinConst.doLogBounds in the Marlin code to have more details ... but it is not necessary. 2. The array checks detect invalid arrays (edgeBuckets / edgeBucketCounts) WARNING: Invalid array value at: *16438* = 24 from: 0 to: 312 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...] java.lang.Throwable at org.marlin.pisces.IntArrayCache.check(IntArrayCache.java:139) at org.marlin.pisces.IntArrayCache.fill(IntArrayCache.java:128) at org.marlin.pisces.Renderer.dispose(Renderer.java:649) at org.marlin.pisces.MarlinTileGenerator.dispose(MarlinTileGenerator.java:65) * at sun.java2d.pipe.AAShapePipe.renderPath(AAShapePipe.java:150)* at sun.java2d.pipe.AAShapePipe.fill(AAShapePipe.java:75) at sun.java2d.pipe.PixelToParallelogramConverter.fill(PixelToParallelogramConverter.java:164) at sun.java2d.pipe.ValidatePipe.fill(ValidatePipe.java:160) * at sun.java2d.SunGraphics2D.fill(SunGraphics2D.java:2527)* at org.faceless.pdf2.af$b.a(af$b.java:178) at org.faceless.pdf2.af.a(af.java:843) at org.faceless.pdf2.af.f(af.java:666) at org.faceless.pdf2.am.a(am.java:726) at org.faceless.pdf2.am.a(am.java:237) at org.faceless.pdf2.b1.b(b1.java:441) at org.faceless.pdf2.b1.a(b1.java:415) at org.faceless.pdf2.b_.createContext(b_.java:101) at sun.java2d.pipe.AlphaPaintPipe.startSequence(AlphaPaintPipe.java:84) at sun.java2d.pipe.AAShapePipe.renderTiles(AAShapePipe.java:163) * at sun.java2d.pipe.AAShapePipe.renderPath(AAShapePipe.java:149)* at sun.java2d.pipe.AAShapePipe.fill(AAShapePipe.java:75) at sun.java2d.pipe.PixelToParallelogramConverter.fill(PixelToParallelogramConverter.java:164) at sun.java2d.pipe.ValidatePipe.fill(ValidatePipe.java:160) * at sun.java2d.SunGraphics2D.fill(SunGraphics2D.java:2527)* at org.faceless.pdf2.af$b.a(af$b.java:178) at org.faceless.pdf2.af.a(af.java:843) at org.faceless.pdf2.af.f(af.java:666) at org.faceless.pdf2.am.a(am.java:726) at org.faceless.pdf2.am.a(am.java:237) at org.faceless.pdf2.am.a(am.java:941) at org.faceless.pdf2.am.parsePage(am.java:154) at org.faceless.pdf2.PagePainter.a(PagePainter.java:638) at org.faceless.pdf2.PagePainter.a(PagePainter.java:362) at org.faceless.pdf2.PagePainter.getSubImage(PagePainter.java:341) at org.faceless.pdf2.PagePainter.getImage(PagePainter.java:310) at PDFToImage.main(PDFToImage.java:155) Conclusion: There is a reentrance issue in Marlin when it uses thread local storage (default): *SunGraphics2D.fill()* -> AAShapePipe.renderTiles() -> AlphaPaintPipe.startSequence() -> org.faceless.pdf2.b_.createContext() -> *SunGraphics2D.fill() ...* 1. Reentrance is not supported when the RendererContext is stored in a thread-local variable (and reused among invocations). The segfault happens as the edgeBuckets / edgeBucketCounts arrays are empty (not zero-filled) and these arrays store "java pointers" (integer) to the Unsafe edge array ! 2. I tested CLQ storage (concurrent linked queue) (~10% slower): it is not affected by this problem as the RendererContext is removed / added to the pool (not reused in recursion). Please use the simple workaround (CLQ): -Dsun.java2d.renderer.useThreadLocal=false I will then look at a proper solution: detect reentrance and use CLQ to store child RendererContexts ? Regards, Laurent Le 2 f?vr. 2016 21:26, "Phil Race" a ?crit : > I have filed a bug https://bugs.openjdk.java.net/browse/JDK-8148886 > > I also downloaded the test case and see the crash. > > FYI if you want to continue to test JDK9 builds and this is a blocker > for you (I suppose it is), then until it is fixed you can do > > -Dsun.java2d.renderer=sun.dc.DuctusRenderingEngine > or > -Dsun.java2d.renderer=sun.java2d.pisces.PiscesRenderingEngine > > -phil. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.graham at oracle.com Thu Feb 4 01:55:05 2016 From: james.graham at oracle.com (Jim Graham) Date: Wed, 3 Feb 2016 17:55:05 -0800 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: References: <56B11010.8@oracle.com> Message-ID: <56B2AF79.2030701@oracle.com> Interesting, so this can happen with any custom Paint or Composite. Yes, re-entrance detection should be done. Just out of curiosity - the 10% performance hit for the CLQ storage vs. thread-local - that was specifically only for the time it takes to retreive the Renderer, right? I can't imagine why the entire rendering operation would take 10% longer for any mechanism that retrieves a cached Renderer - unless perhaps it was missing the cache and allocating a new one every time? I would think a simple synchronized block around a small cache of Renderers would be nearly imperceptible compared to the amount of time spent inside the rendering process... ...jim On 2/3/2016 5:28 AM, Laurent Bourg?s wrote: > Phil, > > bug https://bugs.openjdk.java.net/browse/JDK-8148886 > > Thanks for creating the bug ! > > I spent some time tracking down the bug and make the diagnostic: > > 1. I enabled diagnostics: > > *-Dsun.java2d.renderer.log=true* > > *-Dsun.java2d.renderer.doChecks=true* > > ** > > -Dsun.java2d.renderer.doStats=true > > -Dsun.java2d.renderer.logUnsafeMalloc=true > > I also enabled MarlinConst.doLogBounds in the Marlin code to have more > details ... but it is not necessary. > > 2. The array checks detect invalid arrays (edgeBuckets / edgeBucketCounts) > > WARNING: Invalid array value at: *16438* = 24 from: 0 to: 312 > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, > 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, > 0, ...] > > java.lang.Throwable > at > org.marlin.pisces.IntArrayCache.check(IntArrayCache.java:139) > at org.marlin.pisces.IntArrayCache.fill(IntArrayCache.java:128) > at org.marlin.pisces.Renderer.dispose(Renderer.java:649) > > at > org.marlin.pisces.MarlinTileGenerator.dispose(MarlinTileGenerator.java:65) > * at sun.java2d.pipe.AAShapePipe.renderPath(AAShapePipe.java:150) > * at sun.java2d.pipe.AAShapePipe.fill(AAShapePipe.java:75) > at > sun.java2d.pipe.PixelToParallelogramConverter.fill(PixelToParallelogramConverter.java:164) > at sun.java2d.pipe.ValidatePipe.fill(ValidatePipe.java:160) > * at sun.java2d.SunGraphics2D.fill(SunGraphics2D.java:2527)* > > at org.faceless.pdf2.af > $b.a(af$b.java:178) > at org.faceless.pdf2.af.a(af.java:843) > at org.faceless.pdf2.af.f(af.java:666) > at org.faceless.pdf2.am.a(am.java:726) > at org.faceless.pdf2.am.a(am.java:237) > at org.faceless.pdf2.b1.b(b1.java:441) > at org.faceless.pdf2.b1.a(b1.java:415) > at org.faceless.pdf2.b_.createContext(b_.java:101) > > at > sun.java2d.pipe.AlphaPaintPipe.startSequence(AlphaPaintPipe.java:84) > at sun.java2d.pipe.AAShapePipe.renderTiles(AAShapePipe.java:163) > * at sun.java2d.pipe.AAShapePipe.renderPath(AAShapePipe.java:149) > * at sun.java2d.pipe.AAShapePipe.fill(AAShapePipe.java:75) > at > sun.java2d.pipe.PixelToParallelogramConverter.fill(PixelToParallelogramConverter.java:164) > at sun.java2d.pipe.ValidatePipe.fill(ValidatePipe.java:160) > * at sun.java2d.SunGraphics2D.fill(SunGraphics2D.java:2527)* > > at org.faceless.pdf2.af > $b.a(af$b.java:178) > at org.faceless.pdf2.af.a(af.java:843) > at org.faceless.pdf2.af.f(af.java:666) > at org.faceless.pdf2.am.a(am.java:726) > at org.faceless.pdf2.am.a(am.java:237) > at org.faceless.pdf2.am.a(am.java:941) > at org.faceless.pdf2.am.parsePage(am.java:154) > at org.faceless.pdf2.PagePainter.a(PagePainter.java:638) > at org.faceless.pdf2.PagePainter.a(PagePainter.java:362) > at org.faceless.pdf2.PagePainter.getSubImage(PagePainter.java:341) > at org.faceless.pdf2.PagePainter.getImage(PagePainter.java:310) > at PDFToImage.main(PDFToImage.java:155) > > Conclusion: > > There is a reentrance issue in Marlin when it uses thread local storage > (default): > > *SunGraphics2D.fill()* -> AAShapePipe.renderTiles() -> > AlphaPaintPipe.startSequence() -> > org.faceless.pdf2.b_.createContext() -> *SunGraphics2D.fill() ... > * > > 1. Reentrance is not supported when the RendererContext is stored in a > thread-local variable (and reused among invocations). > > The segfault happens as the edgeBuckets / edgeBucketCounts arrays are > empty (not zero-filled) and these arrays store "java pointers" (integer) > to the Unsafe edge array ! > > 2. I tested CLQ storage (concurrent linked queue) (~10% slower): it is > not affected by this problem as the RendererContext is removed / added > to the pool (not reused in recursion). > > Please use the simple workaround (CLQ): > > -Dsun.java2d.renderer.useThreadLocal=false > > > I will then look at a proper solution: detect reentrance and use CLQ to > store child RendererContexts ? > > Regards, > > Laurent > > > Le 2 f?vr. 2016 21:26, "Phil Race" > a ?crit : > > I have filed a bug https://bugs.openjdk.java.net/browse/JDK-8148886 > > I also downloaded the test case and see the crash. > > FYI if you want to continue to test JDK9 builds and this is a blocker > for you (I suppose it is), then until it is fixed you can do > > -Dsun.java2d.renderer=sun.dc.DuctusRenderingEngine > or > -Dsun.java2d.renderer=sun.java2d.pisces.PiscesRenderingEngine > > -phil. > From bourges.laurent at gmail.com Thu Feb 4 22:43:04 2016 From: bourges.laurent at gmail.com (=?UTF-8?Q?Laurent_Bourg=C3=A8s?=) Date: Thu, 4 Feb 2016 23:43:04 +0100 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: <56B2AF79.2030701@oracle.com> References: <56B11010.8@oracle.com> <56B2AF79.2030701@oracle.com> Message-ID: Jim, Jim, here are my comments: 2016-02-04 2:55 GMT+01:00 Jim Graham : > Interesting, so this can happen with any custom Paint or Composite. Yes, > re-entrance detection should be done. > FYI, I proposed a bug fix and also checked the output images are now correct: http://mail.openjdk.java.net/pipermail/2d-dev/2016-February/006272.html Please also check twice AAShapePipe as it may also have problem with reentrancy as the TileState is in thread-local storage too: Should I also use another TileState ? I fixed an issue with int[] abox but I eventually see another issue with the int[] alpha if outpipe.renderPathTile() make reentrant calls too ! > Just out of curiosity - the 10% performance hit for the CLQ storage vs. > thread-local - that was specifically only for the time it takes to retreive > the Renderer, right? I can't imagine why the entire rendering operation > would take 10% longer for any mechanism that retrieves a cached Renderer - > unless perhaps it was missing the cache and allocating a new one every > time? I would think a simple synchronized block around a small cache of > Renderers would be nearly imperceptible compared to the amount of time > spent inside the rendering process... > Let me try to explain: In my MapBench tool, I have a very complex map [135 000 shapes]: dc_shp_alllayers_2013-00-30-07-00-47.ser - Marlin [TL]: Test Threads Ops Med Pct95 Avg StdDev Min Max FPS(med) TotalOps [ms/op] dc_shp_alllayers_2013-00-30-07-00-47.ser 1 25 778.610 779.885 778.876 0.622 777.904 780.050 1.284 25 dc_shp_alllayers_2013-00-30-07-00-47.ser 2 50 780.596 781.677 780.532 0.733 779.091 782.623 1.281 50 dc_shp_alllayers_2013-00-30-07-00-47.ser 4 100 781.663 788.781 783.098 4.548 779.113 805.189 1.279 100 - Marlin [CLQ]: Test Threads Ops Med Pct95 Avg StdDev Min Max FPS(med) TotalOps [ms/op] dc_shp_alllayers_2013-00-30-07-00-47.ser 1 25 787.245 789.810 787.370 1.556 784.139 790.589 1.270 25 dc_shp_alllayers_2013-00-30-07-00-47.ser 2 50 846.023 877.742 853.157 19.231 818.292 885.293 1.182 50 dc_shp_alllayers_2013-00-30-07-00-47.ser 4 100 826.174 851.683 831.681 12.548 814.943 871.630 1.210 100 As you can see, the 8 to 10% difference corresponds to the ratio between total times taken to render the map [877ms vs 780ms] (on the 95th percentile) that happens with 2 or 4 concurrent threads. I guess: 1. threads are rendering many shapes in parallel (135k) so there is a high pressure on the underlying ConcurrentLinkedQueue to add/remove the RendererContext instances. I tried also with a simple synchronized(ArrayDeque) that is slightly slower than CLQ (high concurrency) as all threads are busy. 2. More garbage is produced as every CLQ.offer() will create a CLQ.Node as seen with jmap -histo : num #instances #bytes class name ---------------------------------------------- 1: 3370992 107871744 java.awt.geom.Path2D$Float$CopyIterator * 2: 3370996 80903904 java.util.concurrent.ConcurrentLinkedQueue$Node* 3: 135226 9736272 java.awt.geom.AffineTransform 4: 858 8430464 [I 5: 135309 7770696 [F 6: 135213 6490224 it.geosolutions.java2d.DrawingCommand 7: 135213 4326816 java.awt.geom.GeneralPath 8: 99122 3964880 java.awt.BasicStroke 9: 99118 3964720 it.geosolutions.java2d.SerializableBasicStroke 10: 135391 3500728 [B 11: 135213 3245112 it.geosolutions.java2d.SerializableAlphaComposite PS: the most important garbage corresponds to the Path2D.Float.CopyIterator instances : that could also be improved ... Regards, Laurent -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.graham at oracle.com Fri Feb 5 01:17:33 2016 From: james.graham at oracle.com (Jim Graham) Date: Thu, 4 Feb 2016 17:17:33 -0800 Subject: [OpenJDK Rasterizer] Segfault in marlin code for Java 9 ea100 In-Reply-To: References: <56B11010.8@oracle.com> <56B2AF79.2030701@oracle.com> Message-ID: <56B3F82D.8050300@oracle.com> Ah, interesting, Yes, TileState in AAShapePipe is also a problem for reentrancy. It has a lower resulting damage potential given the nature of the data it holds, but it can still cause problems. The defensive copy of the abox values is one step to help here, but it isn't complete. Now that you point it out, I just noticed a bug in the way you cache the abox values - they might be modified between when you cache them and when you use them in the for loops... :( ...jim On 2/4/2016 2:43 PM, Laurent Bourg?s wrote: > Jim, > Jim, here are my comments: > > 2016-02-04 2:55 GMT+01:00 Jim Graham >: > > Interesting, so this can happen with any custom Paint or Composite. > Yes, re-entrance detection should be done. > > > FYI, I proposed a bug fix and also checked the output images are now > correct: > http://mail.openjdk.java.net/pipermail/2d-dev/2016-February/006272.html > > Please also check twice AAShapePipe as it may also have problem with > reentrancy as the TileState is in thread-local storage too: > Should I also use another TileState ? > I fixed an issue with int[] abox but I eventually see another issue with > the int[] alpha if outpipe.renderPathTile() make reentrant calls too ! > > Just out of curiosity - the 10% performance hit for the CLQ storage > vs. thread-local - that was specifically only for the time it takes > to retreive the Renderer, right? I can't imagine why the entire > rendering operation would take 10% longer for any mechanism that > retrieves a cached Renderer - unless perhaps it was missing the > cache and allocating a new one every time? I would think a simple > synchronized block around a small cache of Renderers would be nearly > imperceptible compared to the amount of time spent inside the > rendering process... > > > Let me try to explain: > In my MapBench tool, I have a very complex map [135 000 shapes]: > dc_shp_alllayers_2013-00-30-07-00-47.ser > > - Marlin [TL]: > Test Threads Ops > Med Pct95 Avg StdDev Min Max FPS(med) TotalOps > [ms/op] > dc_shp_alllayers_2013-00-30-07-00-47.ser 1 25 778.610 > 779.885 778.876 0.622 777.904 780.050 1.284 25 > dc_shp_alllayers_2013-00-30-07-00-47.ser 2 50 780.596 > 781.677 780.532 0.733 779.091 782.623 1.281 50 > dc_shp_alllayers_2013-00-30-07-00-47.ser 4 100 781.663 > 788.781 783.098 4.548 779.113 805.189 1.279 100 > > - Marlin [CLQ]: > Test Threads Ops Med Pct95 Avg StdDev Min Max > FPS(med) TotalOps [ms/op] > dc_shp_alllayers_2013-00-30-07-00-47.ser 1 25 787.245 > 789.810 787.370 1.556 784.139 790.589 1.270 25 > dc_shp_alllayers_2013-00-30-07-00-47.ser 2 50 846.023 > 877.742 853.157 19.231 818.292 885.293 1.182 50 > dc_shp_alllayers_2013-00-30-07-00-47.ser 4 100 826.174 > 851.683 831.681 12.548 814.943 871.630 1.210 100 > > As you can see, the 8 to 10% difference corresponds to the ratio between > total times taken to render the map [877ms vs 780ms] (on the 95th > percentile) that happens with 2 or 4 concurrent threads. > > > I guess: > 1. threads are rendering many shapes in parallel (135k) so there is a > high pressure on the underlying ConcurrentLinkedQueue to add/remove the > RendererContext instances. > I tried also with a simple synchronized(ArrayDeque) that is slightly > slower than CLQ (high concurrency) as all threads are busy. > > 2. More garbage is produced as every CLQ.offer() will create a CLQ.Node > as seen with jmap -histo : > > num #instances #bytes class name > ---------------------------------------------- > 1: 3370992 107871744 java.awt.geom.Path2D$Float$CopyIterator > * 2: 3370996 80903904 > java.util.concurrent.ConcurrentLinkedQueue$Node > * 3: 135226 9736272 java.awt.geom.AffineTransform > 4: 858 8430464 [I > 5: 135309 7770696 [F > 6: 135213 6490224 it.geosolutions.java2d.DrawingCommand > 7: 135213 4326816 java.awt.geom.GeneralPath > 8: 99122 3964880 java.awt.BasicStroke > 9: 99118 3964720 > it.geosolutions.java2d.SerializableBasicStroke > 10: 135391 3500728 [B > 11: 135213 3245112 > it.geosolutions.java2d.SerializableAlphaComposite > > PS: the most important garbage corresponds to the > Path2D.Float.CopyIterator instances : that could also be improved ... > > Regards, > Laurent