Generating/Verifying checksums

[clauderenaud]

Hi,


Could it be possible to add a function to create/verify file checksums (total commander did that and it was practical...)

[admin]

Could it be possible to add a function to create/verify file checksums (total commander did that and it was practical...)

Tell me more about it. I never needed it in my life.

[j_c_hallgren]

I've used it maybe 3 times, but it's been quite a while...Had a freeware utility somewhere to do it...maybe it was similar to this one that I found via Google: http://www.slavasoft.com/fsum/

My use was to verify that a download was matching what vendor said it was supposed to be...

There's very probably some source code floating around on Net to do it so as to not fully "reinvent the wheel"...

Update: Found this spec about MD5 : http://www.ietf.org/rfc/rfc1321.txt via this http://www.protect-folder.com/visualmd5.html that I also found...

Then, of course, there's also : http://en.wikipedia.org/wiki/MD5 where it seems that its use may be not as much as before, as a modified file that results in the same check value can be easily created as I read it.

[admin]

There's very probably some source code floating around on Net to do it so as to not fully "reinvent the wheel"...

MD5 is built into XY anyway. I use it to generate the names of the thumbnail caches.

[jacky]

Could it be possible to add a function to create/verify file checksums (total commander did that and it was practical...)

Tell me more about it. I never needed it in my life.

A few sites gives the MD5 of download files to make sure everything goes allright. Also, CRC32 is used a lot when downloading from FTP servers, usually though SFV files (containing the list of filenames along with the CRCs) to make sure, again, all files were correctly downloaded.

[zer0]

I think after almost 2 years in hibernation, this thread deserves a bump. Especially because recently more and more often I find myself having to use MD5/SFV files to check files' integrity. For this I had to resort to a 3rd-party tool that integrates into context menu and associates itself with several checksum files. Like clauderenaud pointed out, this is possible within Total Commander and, as far as power management of files is concerned, being able to verify that files are safe and sound is an excellent feature to have.

[TheQwerty]

Well Don gave us an MD5 scripting function a while back but I don't believe he ever incorporated it into the GUI or CKS.

I also think someone else posted a script to do this already, but here's something I just whipped up.

MD5 Compare.xys

(1.17 KiB) Downloaded 192 times

Usage:
1) Select the files (it uses the focused item if no file is selected).
2) Copy to the clipboard any MD5 hashes you'd like the calculated results compared with (they can be within other text).
3) Execute the script.
4) It then presents a (tab-separated) list of the following which you can copy into Excel and sort/manipulate:
Yes/No/- : Was the calculated hash found on the clipboard?
Hash/[Skipped] : The calculated hash or "[Skipped]" is displayed if the item is not a file or could not be accessed.*
File : Full Path\Filename of the item.

*I've tried to prevent it from attempting to read directories or drives, but if run without a selection it will use the focused item and cannot easily detect if that is file or not prior to reading it. Also, unfortunately the way ReadFile() works is that if there is a problem it pops up the Script Stepping dialog. I have no control over that, so tough.

Code: Select all

"MD5 Calculator"
	//Version Check
	$MIN_VERSION = "7.90.0257";
	End(Compare("$MIN_VERSION", "<xyver>", "v") > 0, "You need at least version $MIN_VERSION of XYplorer for this script.");

	//Get Selected (or just Focused) Item(s).
	$itemList = Report("{Dir |{FullName};|}", 1);
	if ("$itemList" Like "") {
		$itemList = "<focitem>;";
	}

	//Store the Clipboard for comparisons.
	RegExReplace($cb, "<clipboard>", "\r?\n", "¶");

	//Loop through each item and generate results.
	$results = "?	MD5	File";
	$i = 1;
	$token = GetToken("$itemList", $i, ";");

	while (true) {
		if ("$token" Like "") {
			break;
		} else {
			$string = ReadFile("$token", "b");
			if ("$string" Like "") {
				//There was an error reading the file, or you somehow managed to give it a non-file.
				$hash = "[Skipped]";
				$match = "-";
			} else {
				//Calculate the hash & see if it exists anywhere on the clipped text.
				$hash = MD5("$string");
				StrPos($pos, "$cb", "$hash");
				$match = ($pos < 0) ? "No" : "Yes";
			}

			$results = "$results<br>$match	$hash	$token";

			//NEXT!
			$i++;
			$token = GetToken("$itemList", $i, ";");
		}
	}
	Text("$results",,,"MD5");

[admin]

*I've tried to prevent it from attempting to read directories or drives, but if run without a selection it will use the focused item and cannot easily detect if that is file or not prior to reading it. Also, unfortunately the way ReadFile() works is that if there is a problem it pops up the Script Stepping dialog. I have no control over that, so tough.

A little function like ExistItem(item) is on my list, with returns:
0 = no exist
1 = file
2 = folder
3 = drive

or so...

[TheQwerty]

A little function like ExistItem(item) is on my list, with returns:
0 = no exist
1 = file
2 = folder
3 = drive

or so...

Not sure I like those returns (might be better if it was 1,2,4 and geared towards binary comparisons), but I figured it would come eventually.

I could have manipulated the selection to select the focused item and repeat the report if it was that big of an issue.

The real problem there is if the file is locked and/or cannot be read by XY it displays the stepping dialog and there's no way for us to know in the script whether or not it was successful or just an empty file, or to prevent the dialog. Also, it's not exactly easy for a user unfamiliar with scripting to know what to click there (the first few times at least). I don't particularly like the idea of a new <ERROR_LEVEL> XY variable, which leaves me thinking the only way to handle this would be adding a new function to test if the item is accessible before reading, and that's not a particularly nice solution either. :shrugs:

That said, I don't particularly have those needs in my usage, just something I ran into coming up with this script, so maybe someone who is truly bothered by it should comment.

[zer0]

Well Don gave us an MD5 scripting function a while back but I don't believe he ever incorporated it into the GUI or CKS.

Herein lies a difference. XY's MD5 scripting function doesn't actually calculate the md5 string of a file. It calculates a md5 hash of a string. What string though? I used HashCheck (http://code.kliu.org/hashcheck/) to create a md5 checksum of a file and then asked XYplorer to do the same using this code:

Code: Select all

$a = md5(<focitem>);
 writefile("<curpath>\md5.txt", $a);

When I compared the strings, they weren't the same. Even when applying the XY function to the same file twice the resulting strings are different. And this doesn't happen with HashCheck.

[TheQwerty]

Well Don gave us an MD5 scripting function a while back but I don't believe he ever incorporated it into the GUI or CKS.

Herein lies a difference. XY's MD5 scripting function doesn't actually calculate the md5 string of a file. It calculates a md5 hash of a string. What string though? I used HashCheck (http://code.kliu.org/hashcheck/) to create a md5 checksum of a file and then asked XYplorer to do the same using this code:

Code: Select all

$a = md5(<focitem>);
 writefile("<curpath>\md5.txt", $a);

When I compared the strings, they weren't the same. Even when applying the XY function to the same file twice the resulting strings are different. And this doesn't happen with HashCheck.

The MD5 function accepts a string, not a file path, so that was creating the MD5 for path to <focitem> not the actually focused file.

Try comparing HashCheck to the result of this:

Code: Select all

$fileContents = ReadFile("<focitem>", "b"); //Read file as binary input.
 $md5Hash = MD5("$fileContents"); //Calculate MD5 of file's contents.
 WriteFile("md5.txt", "$md5Hash"); //Write MD5 Hash to file.

[Muroph]

Had a freeware utility somewhere to do it...maybe it was similar to this one that I found via Google: http://www.slavasoft.com/fsum/

i do a lot of crc32 and md5 checks, and i love this tool.
it's really easy to use it via scripting because it works with command line parameters.

A little function like ExistItem(item) is on my list

i can't wait for that!

[admin]

A little function like ExistItem(item) is on my list, with returns:
0 = no exist
1 = file
2 = folder
3 = drive

or so...

Not sure I like those returns (might be better if it was 1,2,4 and geared towards binary comparisons), but I figured it would come eventually.

Which reminds me that binary operators are still missing...
Unless you say a drive is a special sort of folder, I don't see the need for a binary field here because the results are mutually exclusive. But 1,2,4 would not hurt either...

[zer0]

Try comparing HashCheck to the result of this:

Code: Select all

$fileContents = ReadFile("<focitem>", "b"); //Read file as binary input.
 $md5Hash = MD5("$fileContents"); //Calculate MD5 of file's contents.
 WriteFile("md5.txt", "$md5Hash"); //Write MD5 Hash to file.

Yep, that worked. In fact, I adopted your large MD5 script and came up with one of my own:

Code: Select all

"MD5 Checksum Creator"
    //Version Check
    $MIN_VERSION = "7.90.0260";
    End(Compare("$MIN_VERSION", "<xyver>", "v") > 0, "You need at least version $MIN_VERSION of XYplorer for this script.");

    //Get Selected (or just Focused) Item(s).
    $itemList = Report("{Dir |{FullName};|}", 1);
    if ("$itemList" Like "") {
        $itemList = "<focitem>;";
    }

    //Loop through each item and generate results.
    $results = "MD5 *File";
    $i = 1;
    $token = GetToken("$itemList", $i, ";");

    while (true) {
        if ("$token" Like "") {
            break;
    } else {
        $fileContents = ReadFile("$token", "b");
        if ("$string" Like "") {
            //There was an error reading the file, or you somehow managed to give it a non-file.
            $hash = "[Skipped]";
            $match = "-";
         } else {

         $md5Hash = MD5("$fileContents"); //Calculate MD5 of file's contents.
         $a = "$token";
         }

         $b = " *";
         $c = "$c<crlf>$md5Hash $b$a";

         $i++;
         $token = GetToken("$itemList", $i, ";");
        }
   }
  $d = compare("$itemList", "1", "n");
  if $d == 0 {
    msg $d;
    WriteFile("$a.md5", $c); //Write MD5 Hash to file.   
  } else {
      msg $d;
      $e = <curfolder>;
      WriteFile("$e.md5", $c); //Write MD5 Hash to file.        
  }

There is a hiccup, however. In that last piece of code (the last "if" loop), I'm trying to determine whether the number of files that I've created md5 hashes for is greater than one. If it is then the resulting .md5 file will be called the same as the current folder and if it is not then it will be called the same as the only file that was hashed. The problem is that it seems that whether I select just one or several files the script evaluates $d to be "-1", which suggests that it is empty but it is not. I've tried trial and error, but ended up digging myself into a deeper hole.

[TheQwerty]

There is a hiccup, however. In that last piece of code (the last "if" loop), I'm trying to determine whether the number of files that I've created md5 hashes for is greater than one. If it is then the resulting .md5 file will be called the same as the current folder and if it is not then it will be called the same as the only file that was hashed. The problem is that it seems that whether I select just one or several files the script evaluates $d to be "-1", which suggests that it is empty but it is not. I've tried trial and error, but ended up digging myself into a deeper hole.

The problem is that compare("$itemList", "1", "n"); is comparing the contents of $itemList to the value 1. It's not comparing the number of lines or number of items. You could set a variable at the beginning using $itemCount = GetInfo("CountSelected"); (and also set this to 1 if using the focused item), and then use this instead of $d.

You could also give this new version I've cooked up this morning a shot, it now is a bit complex but pretty flexible. Just read the comments in the "Calculate MD5s" method and adjust the variables as desired. (I haven't given it a lot of testing, but it seems okay thus far.)

Since it's getting long and we don't have a scrolling code box in the forums I'm not going to inline the code this time.

With this version I've broken it into a calling script and a main script, so you can easily run it with different settings by just creating several calling scripts.

From the script you posted I would change the variables in _callScript as follows:

Code: Select all

$HEADER = "";
$TEMPLATE = "{HASH} *{FILE}<crlf>";
$CHECK_CLIPBOARD = FALSE;
$SHOW_NONFILES = FALSE;
$SHOW_ERRS = FALSE;
$SHOW_RESULTS = FALSE;
$OUTPUT_RESULTS = TRUE;
$OUTPUT_ONEXIST = "o";