replace() incredible slow?

[highend]

Content of the variable $selectedFiles:

R:\1-800x600-110.jpg
R:\1-1024x768-110.jpg
R:\1-1920x1024-110.jpg
R:\2-800x600-110.jpg
R:\2-1024x768-110.jpg
R:\2-1920x1024-110.jpg
...
R:\30000-1920x1024-110.jpg

Over all: 30.000 files in a 8GB ram disk.

Written to disk = 253 KB.

Code: Select all

$resolutions = formatlist(replace(regexmatches($selectedFiles, "-\d+x\d+-"), "-"), "dents");

This takes 191 seconds...

Code: Select all

$resolutions = formatlist(regexreplace(regexmatches($selectedFiles, "-\d+x\d+-"), "-"), "dents");

Whereas this needs only 0,25 seconds...

So replace() can be about ~750 times slower than regexreplace()...

It's just a comparison between replace / regexreplace in the end I switched to a slightly modified command:

Code: Select all

$resolutions = formatlist(regexreplace($selectedFiles, "^.*?-(\d+x\d+)-.*?(\r?\n|$)", "$1|"), "dents");

[PeterH]

Maybe, just to be on the save side, you should add the 3rd operand for replace: the 'replacement string', here: "". As the replace-function specifies *3* must-operands. At least this *could* help.

[highend]

Thx PeterH,

but it doesn't make any difference if you leave the third parameter out or use "" instead.

[SkyFrontier]

Me and tiago had several discussions on this matter.
I found this thread in which he posted some notes on this
http://www.xyplorer.com/xyfc/viewtopic. ... =45#p66014
but I am not finding our private conversations. We made some clever improvements on codes we worked together, but replace() remains a speed breaker.

[highend]

It's the same for replacelist().

Code: Select all

   $selectedFiles = listpane(, "*.jpg", 1, "|") ;
    $metaCharacters = "\|*|^|$|.|+|(|)|[|{";
    $escapedCharacters = "\\|\*|\^|\$|\.|\+|\(|\)|\[|\{";
    $excludedFiles = replacelist($selectedFiles, $metaCharacters, $escapedCharacters, "|");

On 2000 files it takes 18 seconds.

Doing the same with regexreplace() takes 25 msecs.

Factor: 720 *g*

[admin]

SC replacelist is slow. But I made it a little faster just now.

SC replace is fast. I see no problems here.

[highend]

191 vs 0,25 seconds is fast (replace vs regexreplace)? oO

[admin]

OK, confirmed under Win8. Interesting. Looks like Windows changed something concerning string handling. What's your Windows?

[highend]

Server 2012 R2

[admin]

My string performance tests have all been made under XP. Well... ... so that's a good discovery! Next version will be much faster with SC replace, and also even generally a bit snappier, especially on start up.

[highend]

Any chances you could tune up selectitems as well? It's also rather slow if it has to process a variable that contains a few thousand items

[admin]

Any chances you could tune up selectitems as well? It's also rather slow if it has to process a variable that contains a few thousand items

Nope, that's a different story. Each item in the list has to be compared with each item in the variable. Takes time.

[highend]

Maybe a binary compare could speed it up (like you're doing it for the tag database)? File selecting via scripting in (large) branch views is a pain in the...

tag.dat:

Capitalization matters. You must care for the right capitalization of the file names. Reason: For better performance, the comparisons are done by byte, not by character.

[admin]

I'll check.

[highend]

v14.40.0302:
replace is now as fast as regexreplace (tested with 30.000 files)!

regexreplace is still far superior in comparison to replacelist().
50 msecs vs 21 secs for 5000 files (and 10 characters to replace)
But I prefer regexreplace anyway so I don't care too much *g*