How can I speed up line by line reading of an ASCII file? (C++)
- by Jon
Here's a bit of code that is a considerable bottleneck after doing some measuring:
//-----------------------------------------------------------------------------
// Construct dictionary hash set from dictionary file
//-----------------------------------------------------------------------------
void constructDictionary(unordered_set<string> &dict)
{
ifstream wordListFile;
wordListFile.open("dictionary.txt");
string word;
while( wordListFile >> word )
{
if( !word.empty() )
{
dict.insert(word);
}
}
wordListFile.close();
}
I'm reading in ~200,000 words and this takes about 240 ms on my machine. Is the use of ifstream here efficient? Can I do better? I'm reading about mmap() implementations but I'm not understanding them 100%. The input file is simply text strings with *nix line terminations.
