Page 1 of 1

Join Files fails (for large files?)

Posted: 28 Jul 2020 08:39
by iycgtptyarvg
I have the following script (created by someone here on this forum) to join files:

Code: Select all

//Combine selected files in order
"Concatenate Files"
	$itemCount = get("CountSelected");
	if ($itemCount < 2){
		msg "Please select two or more files to concatenate.", 0 + 48;
	}
	else {
		//No modifier keys pressed - use defaults
		$outputFileExt = getpathcomponent("<selitem>", "ext");
		
		//Initialise variables used in loop
		$outputFilePath = <curpath> . "\" . <date yyyy-mm-dd-hh-mm-ss> . ".$outputFileExt";
		$index = 0;

		//Process files
		foreach($token, <get selecteditemspathnames |>) {
			$index = $index + 1;
			$fileContent = readfile("$token");
			writefile("$outputFilePath", $fileContent, "a");
		}
		status "Concatenated file generation complete - $outputFilePath";
	}
Unfortunately, when I join large files, it fails because of a bug.
In the example archive I put two text files I want to join (a.txt and b.txt). After I join them the joined file is SMALLER than the largest file! Also, when I compare them I see it goes wrong after a while (not the whole large file is joined).

Code: Select all

a.txt    = 120594130 bytes
b.txt    =  14565900 bytes
join.txt = 119423500 bytes
Is this a bug in Xyplorer, or in the script?

PS
I doubted where to post this bug/question. Should I have posted this in the questions forum? Or in the bugs forum?

Re: Join Files fails (for large files?)

Posted: 28 Jul 2020 09:05
by highend
writefile() doesn't process more than 100 MB, that's the reason.

Re: Join Files fails (for large files?)

Posted: 28 Jul 2020 09:07
by iycgtptyarvg
highend wrote: 28 Jul 2020 09:05 writefile() doesn't process more than 100 MB, that's the reason.
Oh no! Why not?!?
Isn't this simply using a 64KB buffer to copy blocks one at a time?

How can I fix this?

Re: Join Files fails (for large files?)

Posted: 28 Jul 2020 09:29
by highend
E.g by appending the files via the dos command type

Re: Join Files fails (for large files?)

Posted: 28 Jul 2020 10:06
by iycgtptyarvg
The dos command 'Type' doesn't work for binary files I think.

I'll program my own Join command-line tool in C++ and will use that to join files using Xyplorer.
Anyways, thanks for the info.

Re: Join Files fails (for large files?)

Posted: 28 Jul 2020 10:36
by highend
type should join binary files as well, otherwise there is copy /b (which can join text / binary)...

Re: Join Files fails (for large files?)

Posted: 28 Jul 2020 21:15
by iycgtptyarvg
Can you tell me how to create the following string which is then executed?

Copy /b "<file1>"+"<file2>"+"<file3>"+"<file4>" <outputfile>

Something like this?!?

Code: Select all

$outputFile = getpathcomponent("<selitem>", "ext");

$copycmd = "Copy /b ";
foreach($token, <get selecteditemspathnames |>)
{
  $index = $index + 1;
  $copycmd = $copycmd + "+" + "$token";
}
$copycmd = $copycmd + " " + $outputFile

execute $copycmd;

Re: Join Files fails (for large files?)

Posted: 28 Jul 2020 21:25
by highend
The unprofessional way

Code: Select all

    $itemCount = get("CountSelected");
    $outputFileExt = getpathcomponent("<selitem>", "ext");
    $outputFilePath = <curpath> . "\" . <date yyyy-mm-dd-hh-mm-ss> . ".$outputFileExt";
    $copycmd = "";
    foreach($token, <get selecteditemspathnames |>) {
        $copycmd = ($copycmd) ? $copycmd . " + " . quote($token) : quote($token);
    }
    $copycmd .= " " . quote($outputFilePath);
    run """cmd"" /c copy /b $copycmd", , , 0;