Join Files fails (for large files?)

Discuss and share scripts and script files...
Post Reply
iycgtptyarvg
Posts: 222
Joined: 10 Jun 2008 15:40
Location: Netherlands

Join Files fails (for large files?)

Post by iycgtptyarvg »

I have the following script (created by someone here on this forum) to join files:

Code: Select all

//Combine selected files in order
"Concatenate Files"
	$itemCount = get("CountSelected");
	if ($itemCount < 2){
		msg "Please select two or more files to concatenate.", 0 + 48;
	}
	else {
		//No modifier keys pressed - use defaults
		$outputFileExt = getpathcomponent("<selitem>", "ext");
		
		//Initialise variables used in loop
		$outputFilePath = <curpath> . "\" . <date yyyy-mm-dd-hh-mm-ss> . ".$outputFileExt";
		$index = 0;

		//Process files
		foreach($token, <get selecteditemspathnames |>) {
			$index = $index + 1;
			$fileContent = readfile("$token");
			writefile("$outputFilePath", $fileContent, "a");
		}
		status "Concatenated file generation complete - $outputFilePath";
	}
Unfortunately, when I join large files, it fails because of a bug.
In the example archive I put two text files I want to join (a.txt and b.txt). After I join them the joined file is SMALLER than the largest file! Also, when I compare them I see it goes wrong after a while (not the whole large file is joined).

Code: Select all

a.txt    = 120594130 bytes
b.txt    =  14565900 bytes
join.txt = 119423500 bytes
Is this a bug in Xyplorer, or in the script?

PS
I doubted where to post this bug/question. Should I have posted this in the questions forum? Or in the bugs forum?
(\__/)
(='.'=) This is Bunny. Copy and paste bunny into
(")_(") your signature to help him gain world domination.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Join Files fails (for large files?)

Post by highend »

writefile() doesn't process more than 100 MB, that's the reason.
One of my scripts helped you out? Please donate via Paypal

iycgtptyarvg
Posts: 222
Joined: 10 Jun 2008 15:40
Location: Netherlands

Re: Join Files fails (for large files?)

Post by iycgtptyarvg »

highend wrote: 28 Jul 2020 09:05 writefile() doesn't process more than 100 MB, that's the reason.
Oh no! Why not?!?
Isn't this simply using a 64KB buffer to copy blocks one at a time?

How can I fix this?
(\__/)
(='.'=) This is Bunny. Copy and paste bunny into
(")_(") your signature to help him gain world domination.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Join Files fails (for large files?)

Post by highend »

E.g by appending the files via the dos command type
One of my scripts helped you out? Please donate via Paypal

iycgtptyarvg
Posts: 222
Joined: 10 Jun 2008 15:40
Location: Netherlands

Re: Join Files fails (for large files?)

Post by iycgtptyarvg »

The dos command 'Type' doesn't work for binary files I think.

I'll program my own Join command-line tool in C++ and will use that to join files using Xyplorer.
Anyways, thanks for the info.
(\__/)
(='.'=) This is Bunny. Copy and paste bunny into
(")_(") your signature to help him gain world domination.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Join Files fails (for large files?)

Post by highend »

type should join binary files as well, otherwise there is copy /b (which can join text / binary)...
One of my scripts helped you out? Please donate via Paypal

iycgtptyarvg
Posts: 222
Joined: 10 Jun 2008 15:40
Location: Netherlands

Re: Join Files fails (for large files?)

Post by iycgtptyarvg »

Can you tell me how to create the following string which is then executed?

Copy /b "<file1>"+"<file2>"+"<file3>"+"<file4>" <outputfile>

Something like this?!?

Code: Select all

$outputFile = getpathcomponent("<selitem>", "ext");

$copycmd = "Copy /b ";
foreach($token, <get selecteditemspathnames |>)
{
  $index = $index + 1;
  $copycmd = $copycmd + "+" + "$token";
}
$copycmd = $copycmd + " " + $outputFile

execute $copycmd;
(\__/)
(='.'=) This is Bunny. Copy and paste bunny into
(")_(") your signature to help him gain world domination.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Join Files fails (for large files?)

Post by highend »

The unprofessional way

Code: Select all

    $itemCount = get("CountSelected");
    $outputFileExt = getpathcomponent("<selitem>", "ext");
    $outputFilePath = <curpath> . "\" . <date yyyy-mm-dd-hh-mm-ss> . ".$outputFileExt";
    $copycmd = "";
    foreach($token, <get selecteditemspathnames |>) {
        $copycmd = ($copycmd) ? $copycmd . " + " . quote($token) : quote($token);
    }
    $copycmd .= " " . quote($outputFilePath);
    run """cmd"" /c copy /b $copycmd", , , 0;
One of my scripts helped you out? Please donate via Paypal

Post Reply