Count the number of lines in a file with Ruby, without reading entire file into memory

Posted by smnirven on Stack Overflow See other posts from Stack Overflow or by smnirven
Published on 2010-04-16T04:02:31Z Indexed on 2010/04/16 4:13 UTC
Read the original article Hit count: 232

Filed under:

ruby

I'm processing huge data files (millions of lines each). Before I start processing I'd like to get a count of the number of lines in the file, so I can then indicate how far along the processing is. I am using Ruby, and because of the size of the files, it would not be practical to read the entire file into memory just to count how many lines there are. Does anyone have a good suggestion on how to do this?

Related posts about ruby

Setting up Rails to work with sqlserver

as seen on Stack Overflow - Search for 'Stack Overflow'
Ok I followed the steps for setting up ruby and rails on my Vista machine and I am having a problem connecting to the database. Contents of database.yml development: adapter: sqlserver database: APPS_SETUP Host: WindowsVT06\SQLEXPRESS Username: se Password: paswd Run rake db:migrate… >>> More
marshal data too short!!!

as seen on Stack Overflow - Search for 'Stack Overflow'
My application requires to keep large data objects in session. There are like 3-4 data objects each created by parsing a csv containing 150 X 20 cells having strings of 3-4 characters. My application shows this error- "marshal data too short". I tried this- Deleting the old session table. Deleting… >>> More
Sinatra and XML POST request

as seen on Stack Overflow - Search for 'Stack Overflow'
I don't know is it my mistake or no. So i have that code: <code> post '/singin/get_token' do content_type :xml puts request.body.read puts xmlRequest xmlRequest = REXML::Document.new(request.body.read) ... </code> And when i post something like that: <code> <?xml… >>> More
how to change ruby path from /usr/bin/ruby to /usr/local/bin/ruby

as seen on Stack Overflow - Search for 'Stack Overflow'
reading around the various ruby install tutorials it's required to change path from /usr/bin/ruby to /usr/local/bin/ruby but i cant seem to be able to do it. Ultimately i want to install Ruby 1.9.2, should i uninstall 1.8.7 or what? i tried to install Ruby 1.9.2 with macports, the installation seemed… >>> More
strange bundler error: tar_input.rb:49:in `initialize': not in gzip format (Zlib::GzipFile::Error) o

as seen on Stack Overflow - Search for 'Stack Overflow'
i am getting a strange bundler error when running bundle pack with bundler 0.9.12 any ideas? (see pastie for a better formatted code: http://pastie.org/881328 ) /opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/site_ruby/1.8/rubygems/package/tar_input.rb:49:in `initialize': not in gzip format (Zlib::GzipFile::Error) … >>> More

Developer IT

Count the number of lines in a file with Ruby, without reading entire file into memory - Developer IT

Count the number of lines in a file with Ruby, without reading entire file into memory

ruby

Related posts about ruby

Setting up Rails to work with sqlserver

marshal data too short!!!

Sinatra and XML POST request

how to change ruby path from /usr/bin/ruby to /usr/local/bin/ruby

strange bundler error: tar_input.rb:49:in `initialize': not in gzip format (Zlib::GzipFile::Error) o

Categories cloud