Page 2 of 3

Re: Scripting Memory-Leak / Speed Degrading

Posted: 14 Sep 2019 17:24
by fishgod
I appreaciate the time you allready invested, but I don't think it has something to do with mediainfo.exe, because the workaround above does the same calls to mediainfo.exe without the speed-degradation.
As far as I can say from observation, the thing is the repeated use of load causes something.
The slowdown is not permanent, but only only happens while a script is running. When the script has finished and is started again, it starts fast and degrades again.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 14 Sep 2019 17:34
by admin
Yes, it's thrilling. I also tried this:

Code: Select all

"Execute MediaInfo"
  parseFolder(<curpath>);
  parseFolder(<curpath>);
  parseFolder(<curpath>);
  parseFolder(<curpath>);
and it gets slower and slower....

Re: Scripting Memory-Leak / Speed Degrading

Posted: 15 Sep 2019 11:43
by admin
Got it. The user functions were loaded and added to the functions pool again and again for each scanned file in the file list (= for each load 'xy-slowdown2.xys', '_callFunction';). Fixed by checking if the function has already been loaded before. Numbers look better now:

Code: Select all

31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 31 31 31 31 31 32 32 31 
Good bug! Thanks for the good analysis and detailed description! :beer: :tup:

Re: Scripting Memory-Leak / Speed Degrading

Posted: 15 Sep 2019 14:21
by fishgod
Thank you for fixing it.
That raises a new question that come in my mind in the last days:
Does redefining a function with the same name is:
  • ignored
  • overwrites the existing one
  • raises a warning/error (obviously not till now, maybe this would be a good thing)
Of cause, that info should be added to the help-file for future reference.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 15 Sep 2019 14:27
by admin
ignored.

It is assumed that a same-named function from a same-named resource is identical. IOW, it is assumed that the resource file is not changed during the script execution.

And this is also the old behavior. All those added functions were just ignored.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 15 Sep 2019 20:51
by fishgod
admin wrote:
15 Sep 2019 14:27
It is assumed that a same-named function from a same-named resource is identical.
Very reasonably. Your formulation indicate, that this might be different for different files. In a rational design nobody would do this on purpose, but it might happen by accident when multiple scripts are combined.
Would it be reasonable to raise a warning, when a function redefinition from a location different that first definition comes along?

Re: Scripting Memory-Leak / Speed Degrading

Posted: 16 Sep 2019 10:38
by admin
fishgod wrote:
15 Sep 2019 20:51
Would it be reasonable to raise a warning, when a function redefinition from a location different that first definition comes along?
Yes, good idea. It's a crass design error by the scriptor that should not be silently handled in whatever way.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 16 Sep 2019 11:27
by admin
admin wrote:
16 Sep 2019 10:38
fishgod wrote:
15 Sep 2019 20:51
Would it be reasonable to raise a warning, when a function redefinition from a location different that first definition comes along?
Yes, good idea. It's a crass design error by the scriptor that should not be silently handled in whatever way.
I could do it (raise a warning and terminate the script) for same-named functions loaded (SC load) from different resources. But not when they are included from different resources. Included stuff is integrated in the resource which includes it, so in that case the duplicate function is just ignored without a warning.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 16 Sep 2019 20:35
by fishgod
There is at least one issue with the new warning (refering to the discussion going on here: viewtopic.php?f=2&t=20928&p=172636)

Setup a custom column with a script like this:

Code: Select all

include_once 'lib.inc.xys';

return f1();
The lib.inc.xys-file:

Code: Select all

function f1() {
  return 'f1';
}
Run the following script-file:test.xys (e.g. by passing ::load 'test.xys' in the address-bar)
Suprisingly copying this code into Scripting > Run Script does not trigger the warning :veryconfused:

Code: Select all

include_once "lib.inc.xys";

"Test"
  f1();
  #1001; //refresh
The Column as such is working as expected, but when the list gets refreshed by the #1001; it looks like the column script is run in the context of test.xys-file, complaining about the allready defined function.
The Warning pops multiple times (the same amount as files are in the list):
xy-function-warning.png
xy-function-warning.png (3.94 KiB) Viewed 294 times
It would really help debugging when there is some kind of stack-trace/include-tree in the warning to locate the redefinition and source-definition. (took me some time to figure out that the warning was raised from a column-script)


There may be a really simple solution to decide when to raise the warning that covers all include/load scenarios:
Compare the function-definition and silently ignore later definitions if they are equal, as honoring them would make no difference either.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 17 Sep 2019 13:38
by admin
Fixed in next beta.

Your ideas are good, but not that easy to do. Maybe later.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 18 Sep 2019 20:16
by Filehero
I have isolated my issue which still occurs with 20.40.008.

Put this into <xyscripts>:
testLib.xys
(95 Bytes) Downloaded 7 times
Put the following code into <xyscripts> as whatever.xys and run it.

Code: Select all

include_once "testLib.xys";

// call lib function
  callTest();

// build in-the-fly script code
  $mscript = <<<SCRIPT
include_once "testLib.xys";
SCRIPT;

  // NOW THE ERROR GETS THROWN HERE - this logic has been working for years
  load($mscript, , "s");

Once the script code build is loaded the error is thrown just by including a function set included once once again. To me the new check is still colliding with include_once under at least one more condition.

I'm still confused since my issue here is that it breaks a lot of code which was running perfectly fine for years over here.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 18 Sep 2019 20:50
by admin
OK, I'll upload a fix.

Re: Scripting Memory-Leak / Speed Degrading

Posted: 18 Sep 2019 21:35
by Filehero
Hi Don,

first of all, thanks a lot! The issue with my example is fixed. :tup:

But there's always a "but". That's why I got a "bad feeling" yesterday once I have gone through this thread. And that's also why I had to open a bottle Doppio Passo Salento Primitivo right now. :mrgreen:

After I saw my example was working without any error I ran my menu code again - and :blackstorm:: my error is still there!

Hmmmm. Checking my "production" code, shifting code around. Bamm. :idea:

Indenting the include_once statement in the Heredoc script ist the one to blame. To make the test code better I've included the critical call statement.

Code: Select all

include_once "testLib.xys";

// call lib function
  callTest();

// build in-the-fly script code
  $mscript = <<<SCRIPT
  include_once "testLib.xys"; // <-- INDENTING THIS STATEMENT EVOKES ERROR AGAIN -->
  
  callTest(); // <-- THIS IS WHAT SHOULD WORK AFTER ALL -->
SCRIPT;

  // NOW THE ERROR GETS THROWN HERE - this logic has been working for years
  load($mscript, , "s");
Two "Viertel" later I'm happy for this issue because it made re-realize how much I like Doppio Passo Salento Primitivo. :tup: :tup: :beer: :whistle:

Thanks for your patience,
Filehero

Re: Scripting Memory-Leak / Speed Degrading

Posted: 19 Sep 2019 12:02
by admin
OK, in case you still can read this: Should be fixed in next beta. Prost! :beer:

Re: Scripting Memory-Leak / Speed Degrading

Posted: 19 Sep 2019 18:48
by Filehero
Yes, still can read. :wink:

Thanks for polishing. :-)

This drives me nuts. With my regular scripts, the problem is still there.
With my example script the error again has been cured by your fix. However, the loaded script doesn't seem to call the referenced function. So there some more quirks to be uncovered. :blackstorm:

I'll continue my analysis tomorrow (hopefully).

Citing Arnie: I'll be back! :wink: