Page 1 of 1
Problem reading large files.
Posted: 25 Nov 2011 12:34
by tiago
I cannot transfer an amount of data (~500 MB) for XY to deal with using clipboard as source, setting it as a variable
I can't make readfile() to read a file containing such data.
What are the limits? Documentation doesn't mention any.
It should be limited by RAM available amount only, isn't it?
Re: Problem reading large files.
Posted: 29 Nov 2011 08:59
by admin
What happens when you try?
Re: Problem reading large files.
Posted: 29 Nov 2011 14:23
by tiago
Error: 7 (<several zeroes>7)
Desc: Out of Memory
Dll: 0
Proc: script_Process
But there is memory, in one machine I had 4 GB and got the same for ~500/~700 MB files, one per script execution.
The error pops after a minute or so of script execution at the readfile() execution.
Same goes with a v.9.90 version (current: 10.40.0101).
Re: Problem reading large files.
Posted: 29 Nov 2011 14:36
by admin
What you do with scripting is
scary once again.
An out-of-memory error is usually not something I can fix in my code. It's a limit that's bigger than me.
Nevertheless, theoretically ReadFile should be able to handle "4GB minus 1" bytes. Without seeing your code it's not possible to say what's going on.
Re: Problem reading large files.
Posted: 29 Nov 2011 16:19
by tiago
Not a big deal.
I thought the problem was related to variables not accepting such a large input.
This will do the same anyway and prove that the problem is limited to readfile():
Same goes if you happen to have a large .avi file as <curitem>. Save your precious time.
Yet to test at which (fixed?) size of file the error begins to happen.
Code to generate a dummy file:
Code: Select all
//default values will generate a 64 MB file
//watch out as it is memory hungry.
$trgt = "25"; // '25' creates a 64 MB file...
$data = "10";
writefile("Dummy.txt", "$data", o); // ensure it's clean
$ctrl = 1;
WHILE ($ctrl <= $trgt) {
status "Op $ctrl/$trgt done...";
writefile("Dummy.txt", "$data", a);
$data = readfile("Dummy.txt");
$ctrl++;
}
echo "task finished";