Timer death

[Ariel Weisberg]

Hi Pawel,

One last thing I forgot to mention. I don't consider OOME to be
recoverable (hence it extends Error) although some do. The best you can
hope to do is log that it occurred and possibly limit the corruption to
persistent or foreign state that can occur before the JVM exits. If an
OOM occurs in one thread it might have occurred in another (one not
necessarily managed by your application) at a point that is not
exception safe. 

http://stackoverflow.com/questions/2679330/catching-java-lang-outofmemoryerror

Regards,
Ariel Weisberg 

On Fri, 09 Jul 2010 10:32 -0700, "Pawel Veselov"
<pawel.veselov at gmail.com> wrote:
> Greetings,
> 
> After debugging an issue in one of my projects, I've realized that the
> problem was that a timer task simply died because VM was out of
> memory.
> The fact that I catch any Throwable around the code that threw the OOM
> error didn't particularly help. The error was logged, but the timer
> thread still died.
> Well, may be it didn't die at this point, I do see the exception
> handling code executed (it logged something else from the catch{}
> block as well), and I really hope (don't believe) that a thread would
> die when normal exception handling was executed and the exception
> wasn't re-thrown upstream.
> 
> But, I'm not here to ramble about particulates about my specific
> situations. When I started thinking about how do I detect this
> somehow, I also realized that it's not normally possible to determine
> if any specific timer is alive or not. I see two ways to determine
> that:
> 
> a) Schedule a single empty task on a timer. If you get back an
> IllegalStateException, then the timer thread is gone (the code
> suggests if the thread leaves the main loop for any reason, any
> scheduling will cause IllegalStateException).
> b) When the timer is created, schedule a single task, and determine
> the timer thread during the task execution (current thread), and then
> check whether the thread is running or not.
> 
> Now, both of those are hacky, "a" schedules a task for no reason, and
> "b" will stop working if timer implementation becomes capable of
> scheduling tasks on multiple threads for whatever load-balancing
> reasons.
> 
> I would suggest adding a method to either retrieve a timer thread
> (sort of bad, because this will make the timer implementation to
> always use a single thread), or simply a method that tells whether the
> timer is alive or not (whether this would now translate into whether
> the thread is alive or not), so the program can attempt to recover (or
> take any other action, like shooting itself) if the timer stopped
> working.
> 
> My overall issue with this (I know, kind of late to realize this now),
> is that it's really hard to recover from OOMs, or even detect that
> they are happening, and all of the sudden your application threads
> start to simply disappear, and you need yet another thread that would
> go check they are alive and take some action (and who is to say that
> thread won't die as well, and you won't notice that either).
> 
> java.lang.OutOfMemoryError: Java heap space
> 	at sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown Source)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> 	at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
> 	at com.mysql.jdbc.ResultSetImpl.getInstance(ResultSetImpl.java:382)
> 	at com.mysql.jdbc.MysqlIO.buildResultSetWithRows(MysqlIO.java:2604)
> 	at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:487)
> 	at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2582)
> 	at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1758)
> 	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2172)
> 	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2690)
> 	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619)
> 	at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1465)
>         <classified>
> 	at java.util.TimerThread.mainLoop(Timer.java:512)
> 	at java.util.TimerThread.run(Timer.java:462)
> 
> 
> Thanks,
>   Pawel.
>