Edit very large sql dump/text file (on linux)

Posted by geo on Stack Overflow See other posts from Stack Overflow or by geo
Published on 2009-03-31T02:19:49Z Indexed on 2010/03/19 19:11 UTC
Read the original article Hit count: 131

Filed under:

large-files

|

linux

|

mysqldump

I have to import a large mysql dump (up to 10G). However the sql dump already predefined with a database structure with index definition. I want to speed up the db insert by removing the index and table definition.

That means I have to remove/edit the first few lines of a 10G text file. What is the most efficient way to do this on linux?

Programs that require loading the entire file into RAM will be an overkill to me.

© Stack Overflow or respective owner

Related posts about large-files

AS3 Working With Arbitrarily Large Files

as seen on Stack Overflow - Search for 'Stack Overflow'
I am trying to read a very large file in AS3 and am having problems with the runtime just crashing on me. I'm currently using a FileStream to open the file asynchronously. This does not work(crashes without an Exception) for files bigger than about 300MB. _fileStream = new FileStream(); _fileStream… >>> More
Use php to zip large files

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I have a php form that has a bunch of checkboxes that all contain links to files. Once a user clicks on which checkboxes (files) they want, it then zips up the files and forces a download. I got a simple php zip force download to work, but when one of the files is huge or if someone lets… >>> More
Opening first lines of large files in VIM

as seen on Super User - Search for 'Super User'
I run Windows XP and gvim7.2. I want to inspect a large (130 MB) XML file with VIM. Is it possible to simply open the first 1000 lines of the file? >>> More
Uploading Large Files

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I'm using a windows service on my server to receive large files (1 GB) from desktop clients when I use the TCPClient class it creates astream to send the file which bloks my memory, What is the well designe for this problem >>> More
RichFaces rich:insert takes a long time to output large files

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello I'm using a RichFaces <rich:insert like this: <rich:panel header="my head"> <a4j:outputPanel ajaxRendered="true"> <rich:insert src="#{MyBacking.myPath}" highlight="groovy" /> </a4j:outputPanel> </rich:panel> If I have a 60k file to output… >>> More

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More