100% CPU usage in "VM Thread" for Hotspot 10/11 on x64 platform within data processing application

Tue Feb 24 22:00:03 PST 2009

David,

Can you also send a GC log from a run where there
is not a problem?  As I understand it, that would
be a 32bit run.

Jon

David Sitsky wrote On 02/24/09 16:04,:

>Jon Masamitsu wrote:
>  
>
>>Jon Masamitsu wrote On 02/23/09 17:20,:
>>
>>    
>>
>>>...
>>>
>>>Increase the heap by 30%.  Also increase the the perm gen size
>>>(-XX:MaxPermSize=<nn>).
>>>
>>>Please use -XX:+PrintGCDetails -XX:+PrintGCTimeStamps when gathering the
>>>GC  logs.
>>>If you've already gathering some, send those but in future runs, use the
>>>above.
>>>      
>>>
>
>Here is a sample of output from a stuck process.  You can see its doing 
>a full GC about every 3 seconds, and it seems as if there is little 
>progress..
>
>Please let me know if you need more information.
>
>Cheers,
>David
>
>62402.320: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8063178 secs] [Times: user=2.81 sys=0.00, 
>real=2.81 secs]
>62405.128: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8029997 secs] [Times: user=2.79 sys=0.00, 
>real=2.81 secs]
>62407.932: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7917325 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62410.725: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7891387 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62413.515: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7649110 secs] [Times: user=2.76 sys=0.00, 
>real=2.76 secs]
>62416.281: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7803983 secs] [Times: user=2.78 sys=0.00, 
>real=2.78 secs]
>62419.063: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7643979 secs] [Times: user=2.76 sys=0.00, 
>real=2.76 secs]
>62421.828: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8114336 secs] [Times: user=2.81 sys=0.00, 
>real=2.81 secs]
>62424.640: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7964912 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62427.438: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8107278 secs] [Times: user=2.81 sys=0.00, 
>real=2.81 secs]
>62430.249: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 3.2345212 secs] [Times: user=2.84 sys=0.00, 
>real=3.24 secs]
>62433.484: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8341520 secs] [Times: user=2.82 sys=0.00, 
>real=2.82 secs]
>62436.319: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8698768 secs] [Times: user=2.87 sys=0.00, 
>real=2.87 secs]
>62439.190: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9323230 secs] [Times: user=2.90 sys=0.00, 
>real=2.92 secs]
>62442.124: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9644960 secs] [Times: user=2.96 sys=0.00, 
>real=2.96 secs]
>62445.089: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 3.0059221 secs] [Times: user=3.00 sys=0.00, 
>real=3.00 secs]
>62448.095: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9832815 secs] [Times: user=3.00 sys=0.00, 
>real=2.99 secs]
>62451.079: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9587156 secs] [Times: user=2.93 sys=0.00, 
>real=2.95 secs]
>62454.039: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9488345 secs] [Times: user=2.92 sys=0.00, 
>real=2.95 secs]
>62456.988: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8969788 secs] [Times: user=2.90 sys=0.00, 
>real=2.90 secs]
>62459.886: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8794991 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62462.766: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683675K->683675K(699072K)] 781915K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8842411 secs] [Times: user=2.87 sys=0.00, 
>real=2.89 secs]
>62465.651: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683675K(699072K)] 781916K->781915K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8669173 secs] [Times: user=2.85 sys=0.00, 
>real=2.85 secs]
>62468.519: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8664429 secs] [Times: user=2.85 sys=0.00, 
>real=2.86 secs]
>62471.386: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8844494 secs] [Times: user=2.87 sys=0.00, 
>real=2.89 secs]
>62474.271: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8648398 secs] [Times: user=2.87 sys=0.00, 
>real=2.87 secs]
>62477.137: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8971068 secs] [Times: user=2.87 sys=0.00, 
>real=2.90 secs]
>62480.034: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8655618 secs] [Times: user=2.86 sys=0.00, 
>real=2.86 secs]
>62482.901: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 3.0366140 secs] [Times: user=2.78 sys=0.00, 
>real=3.04 secs]
>62485.939: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8541753 secs] [Times: user=2.85 sys=0.00, 
>real=2.85 secs]
>62488.794: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8582816 secs] [Times: user=2.86 sys=0.00, 
>real=2.86 secs]
>62491.653: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8673218 secs] [Times: user=2.85 sys=0.00, 
>real=2.86 secs]
>62494.521: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9014120 secs] [Times: user=2.87 sys=0.00, 
>real=2.90 secs]
>62497.424: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8805843 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62500.305: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8905128 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62503.196: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9052007 secs] [Times: user=2.90 sys=0.00, 
>real=2.92 secs]
>62506.102: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9004575 secs] [Times: user=2.89 sys=0.00, 
>real=2.90 secs]
>62509.003: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9160655 secs] [Times: user=2.90 sys=0.00, 
>real=2.92 secs]
>62511.920: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9013277 secs] [Times: user=2.90 sys=0.00, 
>real=2.90 secs]
>62514.822: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8982061 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62517.721: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8922437 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62520.614: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8873520 secs] [Times: user=2.87 sys=0.00, 
>real=2.89 secs]
>62523.502: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8805296 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62526.383: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8958714 secs] [Times: user=2.85 sys=0.00, 
>real=2.89 secs]
>62529.279: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8735384 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62532.154: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8705676 secs] [Times: user=2.87 sys=0.00, 
>real=2.87 secs]
>62535.025: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8723947 secs] [Times: user=2.85 sys=0.00, 
>real=2.87 secs]
>62537.898: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8624400 secs] [Times: user=2.86 sys=0.00, 
>real=2.86 secs]
>62540.761: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8245748 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>62543.587: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8432269 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>62546.432: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8394157 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>62549.272: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8471951 secs] [Times: user=2.85 sys=0.00, 
>real=2.85 secs]
>62552.121: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8584107 secs] [Times: user=2.85 sys=0.00, 
>real=2.86 secs]
>62554.981: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683676K->683676K(699072K)] 781916K->781916K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8376807 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>62557.820: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8402486 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>62560.661: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8482704 secs] [Times: user=2.86 sys=0.00, 
>real=2.85 secs]
>62563.511: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8115973 secs] [Times: user=2.81 sys=0.00, 
>real=2.81 secs]
>62566.324: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8523278 secs] [Times: user=2.85 sys=0.00, 
>real=2.86 secs]
>62569.177: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8128563 secs] [Times: user=2.81 sys=0.00, 
>real=2.81 secs]
>62571.990: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7830644 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62574.774: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8065106 secs] [Times: user=2.79 sys=0.00, 
>real=2.81 secs]
>62577.582: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7892171 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62580.372: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8059306 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62583.179: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8641470 secs] [Times: user=2.82 sys=0.00, 
>real=2.86 secs]
>62586.044: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8421364 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>62588.887: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8852699 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62591.773: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9164279 secs] [Times: user=2.90 sys=0.00, 
>real=2.92 secs]
>62594.690: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9450010 secs] [Times: user=2.95 sys=0.00, 
>real=2.95 secs]
>62597.636: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9744636 secs] [Times: user=2.98 sys=0.00, 
>real=2.98 secs]
>62600.611: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9900849 secs] [Times: user=2.99 sys=0.00, 
>real=3.00 secs]
>62603.602: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.6332370 secs] [Times: user=2.62 sys=0.00, 
>real=2.62 secs]
>62606.236: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9801260 secs] [Times: user=2.95 sys=0.00, 
>real=2.98 secs]
>62609.226: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9166374 secs] [Times: user=2.89 sys=0.00, 
>real=2.92 secs]
>62612.150: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9475729 secs] [Times: user=2.95 sys=0.00, 
>real=2.95 secs]
>62615.098: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9328670 secs] [Times: user=2.90 sys=0.00, 
>real=2.93 secs]
>62618.040: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8963825 secs] [Times: user=2.89 sys=0.00, 
>real=2.90 secs]
>62620.937: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8834715 secs] [Times: user=2.89 sys=0.00, 
>real=2.89 secs]
>62623.821: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8800691 secs] [Times: user=2.86 sys=0.00, 
>real=2.87 secs]
>62626.701: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683677K(699072K)] 781918K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8642587 secs] [Times: user=2.87 sys=0.00, 
>real=2.87 secs]
>62629.566: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683677K->683677K(699072K)] 781917K->781917K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8574615 secs] [Times: user=2.84 sys=0.00, 
>real=2.86 secs]
>62632.424: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8383412 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>62635.264: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8409891 secs] [Times: user=2.82 sys=0.00, 
>real=2.84 secs]
>62638.106: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7906216 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62640.898: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7891730 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62643.688: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7892940 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62646.479: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7766807 secs] [Times: user=2.78 sys=0.00, 
>real=2.78 secs]
>62649.257: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7796531 secs] [Times: user=2.78 sys=0.00, 
>real=2.78 secs]
>62652.037: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7687240 secs] [Times: user=2.76 sys=0.00, 
>real=2.76 secs]
>62654.807: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7613769 secs] [Times: user=2.76 sys=0.00, 
>real=2.76 secs]
>62657.570: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7712254 secs] [Times: user=2.78 sys=0.00, 
>real=2.78 secs]
>62660.342: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7968108 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62663.139: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.7924173 secs] [Times: user=2.79 sys=0.00, 
>real=2.79 secs]
>62665.933: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8002912 secs] [Times: user=2.79 sys=0.00, 
>real=2.81 secs]
>62668.736: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8291434 secs] [Times: user=2.82 sys=0.00, 
>real=2.82 secs]
>62671.566: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8527186 secs] [Times: user=2.86 sys=0.00, 
>real=2.85 secs]
>62674.419: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8982825 secs] [Times: user=2.90 sys=0.00, 
>real=2.90 secs]
>62677.318: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9254483 secs] [Times: user=2.93 sys=0.00, 
>real=2.93 secs]
>62680.244: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9707015 secs] [Times: user=2.95 sys=0.00, 
>real=2.96 secs]
>62683.216: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9894145 secs] [Times: user=3.00 sys=0.00, 
>real=3.00 secs]
>62686.206: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9870305 secs] [Times: user=2.98 sys=0.00, 
>real=2.98 secs]
>62689.193: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9884647 secs] [Times: user=2.98 sys=0.00, 
>real=3.00 secs]
>62692.183: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9635276 secs] [Times: user=2.96 sys=0.00, 
>real=2.96 secs]
>62695.147: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9407559 secs] [Times: user=2.93 sys=0.00, 
>real=2.93 secs]
>62698.088: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9299386 secs] [Times: user=2.93 sys=0.00, 
>real=2.93 secs]
>62701.019: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8994903 secs] [Times: user=2.90 sys=0.00, 
>real=2.90 secs]
>62703.919: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9163417 secs] [Times: user=2.92 sys=0.00, 
>real=2.92 secs]
>62706.836: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9216473 secs] [Times: user=2.93 sys=0.00, 
>real=2.93 secs]
>62709.758: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.9052547 secs] [Times: user=2.89 sys=0.00, 
>real=2.90 secs]
>62712.664: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8902824 secs] [Times: user=2.85 sys=0.00, 
>real=2.89 secs]
>62715.555: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8865932 secs] [Times: user=2.87 sys=0.00, 
>real=2.89 secs]
>62718.442: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8605445 secs] [Times: user=2.87 sys=0.00, 
>real=2.87 secs]
>62721.304: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683678K->683678K(699072K)] 781918K->781918K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8662771 secs] [Times: user=2.84 sys=0.00, 
>real=2.86 secs]
>62724.171: [Full GC [PSYoungGen: 98240K->98240K(214720K)] [PSOldGen: 
>683679K->683679K(699072K)] 781919K->781919K(913792K) [PSPermGen: 
>49694K->49694K(49984K)], 2.8369076 secs] [Times: user=2.84 sys=0.00, 
>real=2.84 secs]
>
>  
>