How can I speed up line by line reading of an ASCII file? (C++)

Posted by Jon on Stack Overflow See other posts from Stack Overflow or by Jon
Published on 2011-03-02T07:21:49Z Indexed on 2011/03/02 7:24 UTC
Read the original article Hit count: 197

Filed under:
|
|
|

Here's a bit of code that is a considerable bottleneck after doing some measuring:

//-----------------------------------------------------------------------------
//  Construct dictionary hash set from dictionary file
//-----------------------------------------------------------------------------
void constructDictionary(unordered_set<string> &dict)
{
    ifstream wordListFile;
    wordListFile.open("dictionary.txt");

    string word;
    while( wordListFile >> word )
    {
        if( !word.empty() )
        {
            dict.insert(word);
        }
    }

    wordListFile.close();
}

I'm reading in ~200,000 words and this takes about 240 ms on my machine. Is the use of ifstream here efficient? Can I do better? I'm reading about mmap() implementations but I'm not understanding them 100%. The input file is simply text strings with *nix line terminations.

© Stack Overflow or respective owner

Related posts about c++

Related posts about optimization