Multiple threads stuck on Tomcat behind Apache mod_proxy
- by Eddy
we just took a break at butting our collective heads against this maddening problem we're having. Basically this brand new deployment of Tomcat 6.0.36 crawls down to a halt every couple minutes with many of the worker threads stuck as in the example snippet; only after a while the server gets "unstuck" for another couple minutes.
The previous Tomcat works a charm though, but keeping it is not really an option...
On netstat, we also see a lot of FIN_WAIT and FIN2_WAIT.
"catalina-exec-25" daemon prio=10 tid=0x000000004f9d4000 nid=0x7459 runnable [0x0000000044567000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:756)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:448)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:363)
at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:780)
at org.apache.coyote.http11.filters.IdentityOutputFilter.doWrite(IdentityOutputFilter.java:118)
at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:593)
at org.apache.coyote.Response.doWrite(Response.java:560)
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:364)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:448)
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:318)
at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:274)
at org.apache.catalina.connector.Response.finishResponse(Response.java:493)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:317)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:396)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Any idea?
Eddy