Recently I had to process a text file with a certain format, around 1 million lines, and for each line, some segments are separated by blank space (one/two/three, not always single blank space). At the beginning, I used the most natural method, go through each line, and split with a regular expression: QRegularExpression reg_ex_empty = QRegularExpression ( "\\s+" ); Lines[i].split(reg_ex_empty) Unfortunately, the performance is very poor, for a file of merely 70k lines, it took almost 14s. I tried different kind of profiling, the bottleneck is not the initialization of regex. In fact, even if I just split(“ “), and then remove the empty element from the QStringList, it still took 14s. In the end, it seems the performance bottle neck is in the removal of empty elements in list(with StringList.removeAll() or regex). So I just leave the empty elements in the list, the whole time is reduced to 5 seconds. Maybe this comes with a memory cost, but in my case, it’s n...
Comments
Post a Comment