GUNZIP / Extract file "portion by portion"

Posted by Dave on Stack Overflow See other posts from Stack Overflow or by Dave
Published on 2010-05-07T03:56:47Z Indexed on 2010/05/07 3:58 UTC
Read the original article Hit count: 230

Filed under:

gzip

|

gunzip

|

compression

|

shell

|

shell-scripting

Hi. I'm on a shared server with restricted disk space and i've got a gz file that super expands into a HUGE file, more than what i've got. How can I extract it "portion" by "portion (lets say 10 MB at a time), and process each portion, without extracting the whole thing even temporarily!

No, this is just ONE super huge compressed file, not a set of files please...

© Stack Overflow or respective owner

Related posts about gzip

Twitterbot/0.1,gzip(gfe),gzip(gfe)

as seen on Server Fault - Search for 'Server Fault'
I see a lot of this on my google app engine error log: 05-18 06:44AM 00.897 /NTp9 405 17ms 0cpu_ms 0kb Twitterbot/0.1,gzip(gfe),gzip(gfe) 128.242.241.133 - - [18/May/2010:06:44:00 -0700] "HEAD /NTp9 HTTP/1.1" 405 124 - "Twitterbot/0.1,gzip(gfe),gzip(gfe)" Should I do something for it?? >>> More
ASP.NET GZip Encoding Caveats

as seen on West-Wind - Search for 'West-Wind'
GZip encoding in ASP.NET is pretty easy to accomplish using the built-in GZipStream and DeflateStream classes and applying them to the Response.Filter property. While applying GZip and Deflate behavior is pretty easy there are a few caveats that you have watch out for as I found out today for… >>> More
How do I turn on gzip in Apache2?

as seen on Server Fault - Search for 'Server Fault'
I simply did : sudo aptitude install apache2 And that's how my server is set up. No extra modules, no nothing. I'm on ubuntu jaunty. >>> More
I can't update my system properly, "no package header" error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
Every time I try to run sudo apt-get update or try running updates from the GUI interface I run into the following problem or something similar: Reading package lists... Error! E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_precise_restricted_binary-i386_Packages E:… >>> More
Twitterbot/0.1,gzip(gfe),gzip(gfe)

as seen on Stack Overflow - Search for 'Stack Overflow'
I see a lot of this on my google app engine error log: 05-18 06:44AM 00.897 /NTp9 405 17ms 0cpu_ms 0kb Twitterbot/0.1,gzip(gfe),gzip(gfe) 128.242.241.133 - - [18/May/2010:06:44:00 -0700] "HEAD /NTp9 HTTP/1.1" 405 124 - "Twitterbot/0.1,gzip(gfe),gzip(gfe)" May I know what is this? >>> More

Related posts about gunzip

how to gunzip to a different destination directory?

as seen on Super User - Search for 'Super User'
how do I gunzip to a destination directory other than the current one? this did not work: gunzip *.gz /putthemhere/ >>> More
GUNZIP / Extract file "portion by portion"

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi. I'm on a shared server with restricted disk space and i've got a gz file that super expands into a HUGE file, more than what i've got. How can I extract it "portion" by "portion (lets say 10 MB at a time), and process each portion, without extracting the whole thing even temporarily! No, this… >>> More
gunzip: invalid compressed data--format violated

as seen on Server Fault - Search for 'Server Fault'
Problem definition: I transferred a tar.gz file from a Linux machine to a Windows partition.The Windows partition has mounted with the Linux server as cifs. OS : Red Hat Enterprise Linux Server release 5 Symptom: After the copy process is successful, doing an integrity check with gunzip -t and… >>> More
gunzip unexpected end of file

as seen on Server Fault - Search for 'Server Fault'
I like to write high volume data directly to a zip file via perl. Works like a champ! However, on some rare occasions I've found that gunzip can't uncompress them declaring an unexpected end-of-file. The thing is I can uncompress it with a simple perl script, which probably misses the corrupted… >>> More
pipe multiple files (gz) into c program,

as seen on Stack Overflow - Search for 'Stack Overflow'
Ive written a cprogram that works when i pipe data into my program using stdin like gunzip -c IN.gz|./a.out If I want to run my program on a list of files I can do something like for i `cat list.txt` do gunzip -c $i |./a.out done But this will start my program 'number of files' times. I'm… >>> More