I am a very newbie for AIX and system monitoring. Actually our application currently run production on jboss 5.1 in AIX 5.3. Please check below configuration & system settings.

1. AIX system configuration
   • OS Level 5.3.9.0 (oslevel -g)
   • Physical Memory size 24GB (svmon -G)
   • Page space 4GB (lsps -s)
   • processors 3 cores, Processor Type: PowerPC_POWER6, Processor Clock Speed: 4704 MHz (prtconf | grep Processor)
2. Java version
   • JRE 1.6.0 IBM AIX build pap6460sr10fp1-20120321_01 (SR10 FP1) (java -fullversion)
3. JBoss configuration
   • JBoss 5.1/JBoss ESB 4.11
   • Hornetq messaging with consumer flow control
   • java opts : -d64 -Xms2g -Xmx4g -XX:MaxPermSize=1024m

Sometime we observe very strange behavior in the JBoss that freeze without any error logs. Also server log stop without any further trace. We also not able to get thread dump (kill -3) and its not generate at that point. (kill -3 xxxxx works in normal circumstances) Only option available for us was restart the jboss server and its seem all messages that were in queues during the freeze time process after restarting.

We try tweak some of setting in JBoss hornetq, we though issue was there. Hornetq Stuck By Default. But we haven't any luck and also unable to isolate the issue in any point. We looking at tool like nmon for monitoring this but no clue is that good enough to do so.

Please provide some point to investigate this issue.

Thanks