We're running Apache Tomcat 7.0.41 on CentOS 6 with java version "1.7.0_21". We were getting a lot of too many open files errors so I did some research. The consensus was that it was to to with the number of open files. So I did the following:

• Increased max files in /etc/security/limits.conf

  soft     nofile         100000
  hard     nofile         100000
  

• Rebooted the server

• Checked the limits were valid for the user which was to run the process

  [app_admin@xxx ~]$ ulimit -Hn
  100000
  [app_admin@xxx ~]$ ulimit -Sn
  100000
  

• Monitored open files on the server using the lsof command

What I observed was when the total open files reached circa 13000 and tomcat had around 4500 open files the error reappeared.

I am confused. I thought it would have resolved the problem but clearly I don't fully understand the root cause and also how to set the parameter correctly.

To (maybe) help I have not modified the server.xml file for Tomcat (although I'm tempted). I don't want to start fiddling with that and make things worse.

I'm more than happy to share any more information if someone can give me some hints on where to start looking.