Our production admin server (WebLogic 10.3.5 running on Solaris 10) crashes from time to time. Logs show tons of these errors (several each minute):
<1-Jun-2012 2:28:34 o'clock AM EDT> <Critical> <EmbeddedLDAP> <BEA-000000> <java.lang.NullPointerException
at weblogic.socket.DevPollSocketMuxer.cleanupSocket(DevPollSocketMuxer.java:150)
at weblogic.socket.DevPollSocketMuxer.cancelIo(DevPollSocketMuxer.java:166)
at weblogic.socket.SocketMuxer.deliverExceptionAndCleanup(SocketMuxer.java:836)
at weblogic.socket.SocketMuxer.deliverEndOfStream(SocketMuxer.java:760)
at weblogic.ldap.MuxableSocketLDAP$LDAPSocket.close(MuxableSocketLDAP.java:128)
at com.octetstring.vde.Connection.close(Connection.java:166)
at com.octetstring.vde.WorkThread.executeWorkQueueItem(WorkThread.java:89)
at weblogic.ldap.LDAPExecuteRequest.run(LDAPExecuteRequest.java:50)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
Eventually, the admin server runs out of memory:
<1-Jun-2012 12:29:59 o'clock PM EDT> <Error> <Kernel> <BEA-000802> <ExecuteRequest failed
java.lang.OutOfMemoryError: GC overhead limit exceeded.
One does not necessarily cause the other, but it seems like a pretty good fit.
When inspecting the WebLogic code, we see this:
void cleanupSocket(MuxableSocket paramMuxableSocket, SocketInfo paramSocketInfo) {
this.sockRecords[paramSocketInfo.getFD()] = null; // DevPollSocketMuxer.java:150
super.cleanupSocket(paramMuxableSocket, paramSocketInfo);
}
protected void cancelIo(MuxableSocket paramMuxableSocket)
{
super.cancelIo(paramMuxableSocket);
cleanupSocket(paramMuxableSocket, paramMuxableSocket.getSocketInfo()); // DevPollSocketMuxer.java:166
}
So paramMuxableSocket.getSocketInfo() would be null. I'm at a loss for explaining this... Anyone have an idea?
Thanks!
