File size limit exceeded in bash

Posted by yboren on Server Fault See other posts from Server Fault or by yboren
Published on 2012-09-18T05:09:07Z Indexed on 2012/09/18 9:40 UTC
Read the original article Hit count: 214

Filed under:

shell

|

awk

|

sort

|

cat

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem

the script has problem like this:

cat file | awk '{print $1" "$2" "$3}' | sort -n > result

the file's size is about 3.2G, and I get such error message: File size limit exceeded

in this shell, ulimit -f is unlimited

after I change script into this

cat file | awk '{print $1" "$2" "$3}' >tmp
sort -n tmp > result

the problem is gone.

I don't know why, can anyone help me with an explanation?

© Server Fault or respective owner

Related posts about shell

How to restrict the users' shell allowing to execute shell programs

as seen on Server Fault - Search for 'Server Fault'
Is it possible to prevent any user to not use commands like ls, rm and other system commands which could harm the system. But the users should be able to execute shell programs. >>> More
Shell extension installation not recognized by Windows 7 64-bit shell

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a Copy Hook Handler shell extension that I'm trying to install on Windows 7 64-bit. The shell extension DLL is compiled in two separate versions for 32-bit and 64-bit Windows. The DLL implements DLLRegisterServer which adds the necessary registry entries. After adding the registry entries… >>> More
Running shell commands without a shell window

as seen on Stack Overflow - Search for 'Stack Overflow'
With either subprocess.call or subprocess.Popen, executing a shell command makes a shell window quicky appear and disappear. How can I run the shell command without the shell window? >>> More
Why can't I reinstall MySQL?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've been looking all around the Internet for an answer but didn't find anything. I hope you can help me now. I have a server with MySQL. From one day to another, MySQL didn't let me enter with my root password anymore (accsess denied for user 'root'@'localhost' using password: 'YES'). So I tried… >>> More
Bash/shell script - shell output redirection inside a function

as seen on Stack Overflow - Search for 'Stack Overflow'
function grabSourceFile { cd /tmp/lmpsource wget $1 > $LOG baseName=$(basename $1) tar -xvf $baseName > $LOG cd $baseName } When I call this function The captured output is not going to the log file. The output redirection works fine until I call the… >>> More

Related posts about awk

problem with awk script

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, when I call my awk script, I keep getting an error : sam@sam-laptop:~/shell/td4$ awk -f agenda.awk -- -n Robert agenda.txt awk: agenda.awk:6: printf "Hello" awk: agenda.awk:6: ^ syntax error the script contains this : #!/usr/bin/awk BEGIN { } printf "Hello" END { } Thank you >>> More
Awk to grab colo(u)r codes from CSS files aka School me in Awk

as seen on Stack Overflow - Search for 'Stack Overflow'
Nice and (hopefully) easy. I am trying to work out how to grab the variable #XXX from a text file (css file) containing strings like hr { margin: 18px 0 17px; border-color: #ccc; } h1 a:hover, h2 a:hover, h3 a:hover { color: #001100; } Which I would like to return as ccc 777 The… >>> More
Parsing the output of "uptime" with bash

as seen on Super User - Search for 'Super User'
I would like to save the output of the uptime command into a csv file in a Bash script. Since the uptime command has different output formats based on the time since the last reboot I came up with a pretty heavy solution based on case, but there is surely a more elegant way of doing this. uptime… >>> More
AWK scripting :How to remove Field separator using awk

as seen on Stack Overflow - Search for 'Stack Overflow'
Need the following output ONGC044 ONGC043 ONGC042 ONGC041 ONGC046 ONGC047 from this input Medium Label Medium ID Free Blocks =============================================================================== [ONGC044] ECCPRDDB_FS_43 ac100076:4aed9b39:44f0:0001… >>> More
sort associative array in awk - help?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi all, I have an associative array in awk that gets populated like this... chr_count[$3]++ When I try to print my chr_counts I use this: for (i in chr_count) { print i,":",chr_count[i]; } But not surprisingly, the order of i is not sorted in any way. Is there an easy way to iterate over… >>> More