Page 1 of 2
replace() incredible slow?
Posted: 03 Sep 2014 08:08
by highend
Content of the variable $selectedFiles:
R:\1-800x600-110.jpg
R:\1-1024x768-110.jpg
R:\1-1920x1024-110.jpg
R:\2-800x600-110.jpg
R:\2-1024x768-110.jpg
R:\2-1920x1024-110.jpg
...
R:\30000-1920x1024-110.jpg
Over all: 30.000 files in a 8GB ram disk.
Written to disk = 253 KB.
Code: Select all
$resolutions = formatlist(replace(regexmatches($selectedFiles, "-\d+x\d+-"), "-"), "dents");
This takes
191 seconds...
Code: Select all
$resolutions = formatlist(regexreplace(regexmatches($selectedFiles, "-\d+x\d+-"), "-"), "dents");
Whereas this needs only 0,25 seconds...
So replace() can be about ~750 times slower than regexreplace()...
It's just a comparison between replace / regexreplace in the end I switched to a slightly modified command:
Code: Select all
$resolutions = formatlist(regexreplace($selectedFiles, "^.*?-(\d+x\d+)-.*?(\r?\n|$)", "$1|"), "dents");
Re: replace() incredible slow?
Posted: 03 Sep 2014 12:06
by PeterH
Maybe, just to be on the save side, you should add the 3rd operand for replace: the 'replacement string', here: "". As the replace-function specifies *3* must-operands. At least this *could* help.
Re: replace() incredible slow?
Posted: 03 Sep 2014 13:19
by highend
Thx PeterH,
but it doesn't make any difference if you leave the third parameter out or use "" instead.
Re: replace() incredible slow?
Posted: 03 Sep 2014 15:57
by SkyFrontier
Me and tiago had several discussions on this matter.
I found this thread in which he posted some notes on this
http://www.xyplorer.com/xyfc/viewtopic. ... =45#p66014
but I am not finding our private conversations. We made some clever improvements on codes we worked together, but replace() remains a speed breaker.
Re: replace() incredible slow?
Posted: 03 Sep 2014 16:35
by highend
It's the same for replacelist().
Code: Select all
$selectedFiles = listpane(, "*.jpg", 1, "|") ;
$metaCharacters = "\|*|^|$|.|+|(|)|[|{";
$escapedCharacters = "\\|\*|\^|\$|\.|\+|\(|\)|\[|\{";
$excludedFiles = replacelist($selectedFiles, $metaCharacters, $escapedCharacters, "|");
On 2000 files it takes 18 seconds.
Doing the same with regexreplace() takes 25 msecs.
Factor: 720 *g*
Re: replace() incredible slow?
Posted: 03 Sep 2014 18:55
by admin
SC replacelist is slow. But I made it a little faster just now.
SC replace is fast. I see no problems here.
Re: replace() incredible slow?
Posted: 03 Sep 2014 19:47
by highend
191 vs 0,25 seconds is fast (replace vs regexreplace)? oO
Re: replace() incredible slow?
Posted: 04 Sep 2014 09:24
by admin
OK, confirmed under Win8. Interesting. Looks like Windows changed something concerning string handling. What's your Windows?
Re: replace() incredible slow?
Posted: 04 Sep 2014 09:39
by highend
Server 2012 R2
Re: replace() incredible slow?
Posted: 04 Sep 2014 09:46
by admin
My string performance tests have all been made under XP. Well...
... so that's a good discovery! Next version will be much faster with SC replace, and also even generally a bit snappier, especially on start up.
Re: replace() incredible slow?
Posted: 04 Sep 2014 10:07
by highend
Any chances you could tune up selectitems as well? It's also rather slow if it has to process a variable that contains a few thousand items
Re: replace() incredible slow?
Posted: 04 Sep 2014 11:51
by admin
highend wrote:Any chances you could tune up selectitems as well? It's also rather slow if it has to process a variable that contains a few thousand items
Nope, that's a different story. Each item in the list has to be compared with each item in the variable. Takes time.
Re: replace() incredible slow?
Posted: 04 Sep 2014 11:56
by highend
Maybe a binary compare could speed it up (like you're doing it for the tag database)? File selecting via scripting in (large) branch views is a pain in the...
tag.dat:
Capitalization matters. You must care for the right capitalization of the file names. Reason: For better performance, the comparisons are done by byte, not by character.
Re: replace() incredible slow?
Posted: 04 Sep 2014 12:16
by admin
I'll check.
Re: replace() incredible slow?
Posted: 04 Sep 2014 16:07
by highend
v14.40.0302:
replace is now as fast as regexreplace (tested with 30.000 files)!
regexreplace is still far superior in comparison to replacelist().
50 msecs vs 21 secs for 5000 files (and 10 characters to replace)
But I prefer regexreplace anyway so I don't care too much *g*