SC: Faster speed of subs (by loading from memory) [Granted]

[FluxTorpedoe]

Hi’

Hard to keep up with the changes!

That being said, I have a request / problem:
Each “sub” in a script seems to be read from file, which leads to delays when the file is big.
 ⮚ eg with SessionManager on my computer, just going to a “sub” (even empty) takes ~100ms.
   To avoid duplicating code blocks, an action may go through 5 subs, which results in half a second “lost” without doing anything!

• How to replicate:
Add the following to the start of a huge script file (commenting out the rest if needs be).

Code: Select all

"_Initialize"
	status "";

"Speed Test - Inline"
	$clock_1 = now("ssfff");
	wait 1;
	wait 500;	
	status eval(now("ssfff") - $clock_1), "FF0000";

"Speed Test - 5 Subs"
	$clock_1 = now("ssfff");
	sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub";
	wait 500;
	status eval(now("ssfff") - $clock_1), "FF0000";

"_sub"
	wait 1;

⇒ On my computer, the “inline” test takes about 500ms, the “5 Subs” takes about 1000ms.

So here’s the question (since it seems that functions are read from memory):

 ⮞ Could the subs be much faster (by whatever means necessary)?

Thanks,

     Flux

[PeterH]

Strange. Tested an a *very* slow laptop. Result is 510 / 550 - seems ok to me.
Then tested functions instead of sub-scripts. (Since years I *only* use functions.) Also 550.

[FluxTorpedoe]

Thanks for helping!

Indeed, it seems to depend on what the rest of the script has to process:
If the rest of the file is just a comment, the speed is not so different!

I found a more conclusive way to replicate (which confirms the performance of functions).
• Results with a 300 kB file:
 – inline ⭢ ~508 ms
 – 5 functions ⭢ ~513 ms
 – 5 subs ⭢ ~1930 ms

• New way to replicate
Copy this in a new file, and replace the end with any block of text.
⚠ Note: If the text is very heavy, XYplorer may hang for a few seconds.

Code: Select all

"_Initialize"
	status "";

"Speed Test - Inline"
	$clock_1 = now("ssfff");
	wait 1;
	wait 500;	
	status eval(now("ssfff") - $clock_1), "FF0000";

"Speed Test - 5 Functions"
	$clock_1 = now("ssfff");
	fun(); fun(); fun(); fun(); fun();
	wait 500;
	status eval(now("ssfff") - $clock_1), "FF0000";

"Speed Test - 5 Subs"
	$clock_1 = now("ssfff");
	sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub";
	wait 500;
	status eval(now("ssfff") - $clock_1), "FF0000";

"_sub"
	wait 1;

function fun() {
	wait 1;
}

"_Other"
	$anything = <<<#>>>

-> REPLACE THIS WITH A HUGE BLOCK OF TEXT …
-> REPLACE THIS WITH A HUGE BLOCK OF TEXT …
-> REPLACE THIS WITH A HUGE BLOCK OF TEXT …
	
#>>>;

[eil]

Didn't know script file is not read as a whole, so i support any enhancement to at least have an option to load everything in memory = not really concerned about speed, mainly about less I/O operations.

[Horst]

I tested the script with about 5KB text added.
There are no big diffs.
547
578
578

Hardware and software

CPU Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz, Rev. 36364
RAM 16GB
Windows 11 Home x64 Version 22H2 (OS Build 22621.1555)

[PeterH]

Had a 370k text file at hand :-)

Load&start of the script: slooow. And CPU!
Inline & Function: normal
Subs: -55sec - there must have been an overrun on the 2 digits for seconds...
... and high CPU for this time.

Exchanged code for Sub and function, i.e. function above sub: no difference

Whatever: we talk about a sub/script that's *not* used. Why should it be loaded every time it's not used? Maybe for a "_Terminate"? (If I have the name correct.)

[FluxTorpedoe]

Thanks for trying guys!

@Horst
The extra 30ms for a 5KB file could make sense: a 200KB (like SessionManager) is 40 times that, which could lead to ~1200ms.
What’s weird is that the function is ~slow too…

@PeterH
Yes, the order of entries doesn’t seem to make a difference.
But good to know about the CPU use, I didn’t think of checking that!

[admin]

Scripts are read from file only once. But the processing of huge scripts takes time even in memory. Sorry, but I don't see way to speed this up at the moment.

[FluxTorpedoe]

Oh, OK for the “read from file”, and the processing time as a whole.
Still, functions are nearly instantaneous, even when placed at the end of a huge script…
(example above yields the same results)
Which means the whole is (presumably) processed only once…

Could subs be accessed the same way as functions?

[admin]

1) You could use functions instead of subs.

2) I had an idea and could speed it up like crazy, try the next beta...

[FluxTorpedoe]

Code: Select all

v24.30.0113 - 2023-04-22 11:08
    % Scripting: Loading large script files over and over is now much faster.
    % Scripting: Processing subs over and over in huge script resources has become much 
      faster. 

You said “much faster”…
I think you meant “TREMENDOUSLY faster”!!!

Thanks a lot for taking the time to try it!
And sorry it introduced a few …

So, just a few things to iron out (maybe you’ve seen them already):
 • I had an “Out of range…” error once (apparently related to a sub?), but don’t know how to reproduce
 • Functions work the first time a script is launched, but on subsequent runs, it returns either:
   – “myFunction() is not a valid script command”, or
   – “Call to undefined function”
   Note: If another script is loaded, this bug is “reset” so functions work again once

❖ EDIT: Ken already posted about it here (I missed it) With beta v24.30.0113 missing icons and calls to undefined functions

[FluxTorpedoe]

❖ v24.30.0115
   ✔ Subs loading blazingly fast—it’s amazing…
   ✔ Functions working again
   ✖ I still have intermittent Error 9 (testing with Session Manager right now)

Error: 9 (0x00000009)
Desc: Subscript out of range
Dll: 2
Proc: script_PopupMenu

I don’t know yet how to reproduce…
Sometimes it occurs, sometimes not, while doing the (seemingly) same chain of actions.
Weirdly enough, it seems to only occur after the script actions were processed correctly!

[klownboy]

I can confirm as well getting Error 9 running Session Manager with v24.30.0115. And as Flux has mentioned it seems to happen after the script's work is done (i.e., opening a new session).

[FluxTorpedoe]

❖ Possible lead:
   The Error 9 seems related to the fact that a sub is loading a dynamic script…
   (ie in Session Manager: load $layAct,,"s"; )

   I can’t reproduce the error in a basic script, but will keep you updated if I find more info.

NB: IIRC, I used subs because functions didn’t exist in the first place, so now I use functions only when I need a return. But yes ofc, as a last resort, I know I could replace all subs by functions… Still, I’m liking the new sub speed very much!

[admin]

I think it had to do with the "_Terminate" event. Check out the next beta.