How to track down a file descriptor leak?

Posted by cclark on Server Fault See other posts from Server Fault or by cclark
Published on 2010-04-26T05:22:08Z Indexed on 2010/04/26 5:23 UTC
Read the original article Hit count: 443

Filed under:

max-file-descriptors

|

lsof

I have a java process (Glassfish) which is leaking file descriptors. I know this because I get the helpful java.io.IOException: Too many open files exception. I can look in /proc/PID#/fd and see all the open file descriptors. When I use lsof I get a very large number of entries like this:

java 18510 root 8811u sock 0,4 1576079 can't identify protocol
java 18510 root 8812u sock 0,4 1576111 can't identify protocol
java 18510 root 8813u sock 0,4 1576150 can't identify protocol

I see 12 new ones created per minute. What options can I use on lsof or what other tools are available to me to help track down socket file descriptors where the protocol can't be identified?

thanks, chuck

© Server Fault or respective owner

Related posts about max-file-descriptors

Increasing file descriptor limit on Debian does not work! Help!

as seen on Server Fault - Search for 'Server Fault'
I am running Debian 6 and I am trying to increase the file descriptor limit but it does not want to work. This is what I have done: I edited /etc/sysctl.conf by adding fs.file-max = 64000 at the end and applied the changes using sysctl -p. I then edited /etc/security/limits.conf and added the following… >>> More
How to track down a file descriptor leak?

as seen on Server Fault - Search for 'Server Fault'
I have a java process (Glassfish) which is leaking file descriptors. I know this because I get the helpful java.io.IOException: Too many open files exception. I can look in /proc/PID#/fd and see all the open file descriptors. When I use lsof I get a very large number of entries like this: java… >>> More
Is there a maximum of open files per process in Linux?

as seen on Server Fault - Search for 'Server Fault'
My question is pretty simple and is actually stated in the title. One of my applications throws errors regarding "too many open files" at me, even tho the limit for the user the application runs with is higher than the default of 1024 (lsof -u $USER reports 3000 open fds). Because I cannot imagine… >>> More
How to increase max FD limit for a daemon process running under a headless user?

as seen on Server Fault - Search for 'Server Fault'
To increase the FD limit for a daemon process running under a headless user on a Ubuntu Linux machine we did following changes in /etc/security/limits.conf soft nofile 10000 hard nofile 10000 We also added session required pam_limits.so in /etc/pam.d/login. The changes got reflected for all the… >>> More
How do ulimit -n and /proc/sys/fs/file-max differ?

as seen on Server Fault - Search for 'Server Fault'
I notice that on a new CentOS image that I just booted up off of EC2 that the ulimit default is 1024 open files, but /proc/sys/fs/file-max is set at 761,408 and I'm wondering how these two limits work together. I'm guessing that ulimit -n is a per-user limit of number of file descriptors while /proc/sys/fs/file-max… >>> More

Related posts about lsof

lsof not showing what port a proc is listening on

as seen on Server Fault - Search for 'Server Fault'
I have many processes on a box listening on several ports. I am trying to map ports to pids. The problem is that lsof is not telling me what ports belong to which process. Given an apache listening on port 80, I can see it listening via netstat: user@host% netstat -an|grep LISTEN|grep 80 *.80 … >>> More
txt file descriptor in lsof

as seen on Server Fault - Search for 'Server Fault'
In my experience, files that have the file descriptor of txt in lsof output are the executable file itself and shared objects. The lsof man page says that it means "program text (code and data)". While debugging a problem, I found a large number of data files (specifically, ElasticSearch database… >>> More
Why doesn't lsof show localhost TCP connections in 14.04?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I have two servers running 12.04.4 and 14.04.1 respectively. Both have nginx (port 80) and a Java process (port 8080). As expected, the lsof output for the Java process on the 12.04 machine shows a couple of established connections for port 8080 (e.g. TCP 127.0.0.1:8080->127.0.0.1:58067 (ESTABLISHED)) The… >>> More
How can I unbind a UDP port that has no entry in lsof?

as seen on Super User - Search for 'Super User'
On my Mac, I have a UDP port that is "already in use", but doesn't have an associated process: sudo netstat -na | grep "udp.*\.500\>" shows udp4 0 0 192.168.50.181.500 *.* udp4 0 0 192.168.29.166.500 *.* … >>> More
What's the fastest way using lsof to find a single open file?

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to test a single file if it is open using lsof. Is there a faster way than this? $result = exec('lsof | grep filename | wc -l'); if($result > 0) { //file is open } I'm thinking their must be a way to just test for one file if you know the filename. All I really need is a true… >>> More