SC: Faster speed of subs (by loading from memory) [Granted]

Features wanted...
FluxTorpedoe
Posts: 877
Joined: 05 Oct 2011 13:15

SC: Faster speed of subs (by loading from memory) [Granted]

Post by FluxTorpedoe »

Hi’

Hard to keep up with the changes! 😉

That being said, I have a request / problem:
Each “sub” in a script seems to be read from file, which leads to delays when the file is big.
 ⮚ eg with SessionManager on my computer, just going to a “sub” (even empty) takes ~100ms.
   To avoid duplicating code blocks, an action may go through 5 subs, which results in half a second “lost” without doing anything!

• How to replicate:
Add the following to the start of a huge script file (commenting out the rest if needs be).

Code: Select all

"_Initialize"
	status "";

"Speed Test - Inline"
	$clock_1 = now("ssfff");
	wait 1;
	wait 500;	
	status eval(now("ssfff") - $clock_1), "FF0000";

"Speed Test - 5 Subs"
	$clock_1 = now("ssfff");
	sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub";
	wait 500;
	status eval(now("ssfff") - $clock_1), "FF0000";

"_sub"
	wait 1;
⇒ On my computer, the “inline” test takes about 500ms, the “5 Subs” takes about 1000ms.

So here’s the question (since it seems that functions are read from memory):

 ⮞ Could the subs be much faster (by whatever means necessary)?

Thanks,

     Flux
Last edited by FluxTorpedoe on 23 Apr 2023 16:35, edited 1 time in total.

PeterH
Posts: 2792
Joined: 21 Nov 2005 20:39
Location: Germany

Re: SC: Faster speed of subs (by loading from memory)

Post by PeterH »

Strange. Tested an a *very* slow laptop. Result is 510 / 550 - seems ok to me.
Then tested functions instead of sub-scripts. (Since years I *only* use functions.) Also 550.
Win11 Pro 223H2 Gerrman

FluxTorpedoe
Posts: 877
Joined: 05 Oct 2011 13:15

Re: SC: Faster speed of subs (by loading from memory)

Post by FluxTorpedoe »

Thanks for helping!

Indeed, it seems to depend on what the rest of the script has to process:
If the rest of the file is just a comment, the speed is not so different!

I found a more conclusive way to replicate (which confirms the performance of functions).
• Results with a 300 kB file:
 – inline ⭢ ~508 ms
 – 5 functions ⭢ ~513 ms
 – 5 subs ⭢ ~1930 ms

• New way to replicate
Copy this in a new file, and replace the end with any block of text.
⚠ Note: If the text is very heavy, XYplorer may hang for a few seconds.

Code: Select all

"_Initialize"
	status "";

"Speed Test - Inline"
	$clock_1 = now("ssfff");
	wait 1;
	wait 500;	
	status eval(now("ssfff") - $clock_1), "FF0000";

"Speed Test - 5 Functions"
	$clock_1 = now("ssfff");
	fun(); fun(); fun(); fun(); fun();
	wait 500;
	status eval(now("ssfff") - $clock_1), "FF0000";

"Speed Test - 5 Subs"
	$clock_1 = now("ssfff");
	sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub"; sub "_sub";
	wait 500;
	status eval(now("ssfff") - $clock_1), "FF0000";

"_sub"
	wait 1;

function fun() {
	wait 1;
}

"_Other"
	$anything = <<<#>>>

-> REPLACE THIS WITH A HUGE BLOCK OF TEXT …
-> REPLACE THIS WITH A HUGE BLOCK OF TEXT …
-> REPLACE THIS WITH A HUGE BLOCK OF TEXT …
	
#>>>;

eil
Posts: 1711
Joined: 13 Jan 2011 19:44

Re: SC: Faster speed of subs (by loading from memory)

Post by eil »

Didn't know script file is not read as a whole, so i support any enhancement to at least have an option to load everything in memory = not really concerned about speed, mainly about less I/O operations.
Win 7 SP1 x64 100% 1366x768|1900x1080

Horst
Posts: 1155
Joined: 24 Jan 2021 12:27
Location: Germany

Re: SC: Faster speed of subs (by loading from memory)

Post by Horst »

I tested the script with about 5KB text added.
There are no big diffs.
547
578
578

Hardware and software

CPU Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz, Rev. 36364
RAM 16GB
Windows 11 Home x64 Version 22H2 (OS Build 22621.1555)
Windows 11 Home, Version 24H2 (OS Build 26100.2033)
Portable XYplorer (Actual version, including betas)
Everything 1.5.0.1383a (x64), Everything Toolbar 1.5.1, Listary Pro 6.3.1.81

PeterH
Posts: 2792
Joined: 21 Nov 2005 20:39
Location: Germany

Re: SC: Faster speed of subs (by loading from memory)

Post by PeterH »

Had a 370k text file at hand :-)

Load&start of the script: slooow. And CPU!
Inline & Function: normal
Subs: -55sec - there must have been an overrun on the 2 digits for seconds...
... and high CPU for this time.

Exchanged code for Sub and function, i.e. function above sub: no difference

Whatever: we talk about a sub/script that's *not* used. Why should it be loaded every time it's not used? Maybe for a "_Terminate"? (If I have the name correct.)
Win11 Pro 223H2 Gerrman

FluxTorpedoe
Posts: 877
Joined: 05 Oct 2011 13:15

Re: SC: Faster speed of subs (by loading from memory)

Post by FluxTorpedoe »

Thanks for trying guys!

@Horst
The extra 30ms for a 5KB file could make sense: a 200KB (like SessionManager) is 40 times that, which could lead to ~1200ms.
What’s weird is that the function is ~slow too… 🤔

@PeterH
Yes, the order of entries doesn’t seem to make a difference.
But good to know about the CPU use, I didn’t think of checking that!

admin
Site Admin
Posts: 62188
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: SC: Faster speed of subs (by loading from memory)

Post by admin »

Scripts are read from file only once. But the processing of huge scripts takes time even in memory. Sorry, but I don't see way to speed this up at the moment.

FluxTorpedoe
Posts: 877
Joined: 05 Oct 2011 13:15

Re: SC: Faster speed of subs (by loading from memory)

Post by FluxTorpedoe »

Oh, OK for the “read from file”, and the processing time as a whole.
Still, functions are nearly instantaneous, even when placed at the end of a huge script… 🤔
(example above yields the same results)
Which means the whole is (presumably) processed only once…

Could subs be accessed the same way as functions?

admin
Site Admin
Posts: 62188
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: SC: Faster speed of subs (by loading from memory)

Post by admin »

1) You could use functions instead of subs.

2) I had an idea and could speed it up like crazy, try the next beta... :cup:

FluxTorpedoe
Posts: 877
Joined: 05 Oct 2011 13:15

Re: SC: Faster speed of subs (by loading from memory)

Post by FluxTorpedoe »

Code: Select all

v24.30.0113 - 2023-04-22 11:08
    % Scripting: Loading large script files over and over is now much faster.
    % Scripting: Processing subs over and over in huge script resources has become much 
      faster. 
You said “much faster”…
I think you meant “TREMENDOUSLY faster”!!! :biggrin: :appl:

Thanks a lot for taking the time to try it!
And sorry it introduced a few :bug:🙈

So, just a few things to iron out (maybe you’ve seen them already):
 • I had an “Out of range…” error once (apparently related to a sub?), but don’t know how to reproduce
 • Functions work the first time a script is launched, but on subsequent runs, it returns either:
   – “myFunction() is not a valid script command”, or
   – “Call to undefined function”
   Note: If another script is loaded, this bug is “reset” so functions work again once

❖ EDIT: Ken already posted about it here (I missed it) With beta v24.30.0113 missing icons and calls to undefined functions

FluxTorpedoe
Posts: 877
Joined: 05 Oct 2011 13:15

Re: SC: Faster speed of subs (by loading from memory)

Post by FluxTorpedoe »

❖ v24.30.0115
   ✔ Subs loading blazingly fast—it’s amazing… :biggrin:
   ✔ Functions working again :tup:
   ✖ I still have intermittent Error 9 (testing with Session Manager right now)
Error: 9 (0x00000009)
Desc: Subscript out of range
Dll: 2
Proc: script_PopupMenu
I don’t know yet how to reproduce…
Sometimes it occurs, sometimes not, while doing the (seemingly) same chain of actions.
Weirdly enough, it seems to only occur after the script actions were processed correctly!

klownboy
Posts: 4236
Joined: 28 Feb 2012 19:27

Re: SC: Faster speed of subs (by loading from memory)

Post by klownboy »

I can confirm as well getting Error 9 running Session Manager with v24.30.0115. And as Flux has mentioned it seems to happen after the script's work is done (i.e., opening a new session).
Windows 11, 23H2 Build 22631.3447 at 100% 2560x1440

FluxTorpedoe
Posts: 877
Joined: 05 Oct 2011 13:15

Re: SC: Faster speed of subs (by loading from memory)

Post by FluxTorpedoe »

❖ Possible lead:
   The Error 9 seems related to the fact that a sub is loading a dynamic script…
   (ie in Session Manager: load $layAct,,"s"; )

   I can’t reproduce the error in a basic script, but will keep you updated if I find more info.

NB: IIRC, I used subs because functions didn’t exist in the first place, so now I use functions only when I need a return. But yes ofc, as a last resort, I know I could replace all subs by functions… Still, I’m liking the new sub speed very much! ;)

admin
Site Admin
Posts: 62188
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: SC: Faster speed of subs (by loading from memory)

Post by admin »

I think it had to do with the "_Terminate" event. Check out the next beta.

Post Reply