Purging XY Thumbnails script

[klownboy]

Hi,
The script attached has been very helpful in ridding my system of old unneeded thumbnails, mostly thumbs on flash drives or fixed drive folders in which I no longer need thumbnails or thumbnails in sizes that I no longer use. It will provide you with a list (obtained from XYthumbs.txt file) of the folders for which you have thumbsnails and their sizes and the date the thumbsfile was last modified. With that information, you can make a hopefully smart decision on which thumbnails you can, and probably should, get rid of in your XY thumbsnails folder. I had one computer that had over a gig and a half of thumbs in the thumbnails folder and 2 others that were well over a gig. I cut them in half and I still have quite a lot of photos/thumbs. See thread...http://www.xyplorer.com/xyfc/viewtopic.php?f=3&t=10048

At some point Don may incorporate built-in thumbnail maintenance in XY, but until then. I did this more as a learning experience than anything else. Thank you very much Marco for the regex. If you inadvertantly check off a folder which you wanted to keep the thumbnails and it gets deleted, it's not the end of the world anyway, just go to the folder and regenerate them. If you're not trusting initially, you can check it's operation first by going into some folders in which you wouldn't normally have thumbs generated, or to a folder which you have thumbs, but make them in a different size, and then use the script to delete them. Yes, you can pluck through your thumbnail folder and delete those thumbs by date and even look at the ".dat2" files to see the thumbnail folder and sizes, but this script should make slimming down the XY thumbnails folder much easier.

EDIT 1: Before trying this script, I'd recommend doing the full backup and I'd also save a copy of the XYthumbs.txt file separately right in the thumbnails folder. Save it as XYthumbs_orig.txt just in case. I wanted to point out that on one computer, the data which came back on the inputselect box was missing or not all there so to speak and it turns out it's due to the age old issues with Unicode vs Ansi. I think I might have messed with the XYthumbs.txt using good old Textpad and somehow screwed it up. I just ran it on a Windows 8 machine where I hadn't touched XYthumbs.txt in anyway and the script ran just fine. I may have to dump TextPad. It doesn't seem to be very good at maintaining code format. Unless I save "XYplorer.ini" in ANSI it never works and I have to get a backup.
EDIT 2: I updated the post to provide an indication in the inputselect box of those folders which no longer exists. These old folders will display with a check mark so if for some reason you don't want the associated thumbs deleted you'll have to uncheck the box. To test: (1) create a temp folder, and (2) copy some pictures into the folder, (3) make sure you view the pictures in a thumbnail view (try a couple different thumbnail sizes), (5) run the script (I call it "ThumbChaser.xys"), (6) you should see entries for each thumbsize that you made in the new folder, (7) select at least one thumb size/folder combination and execute the script, (8) view the "XYthumb.txt file" and confirm that the folder/thumbnail size was deleted, (9) now delete the temp folder you made and (10) run the script again (11) you should see lines pre-checked for each of the thumbsizes in that folder that you didn't delete previously.

Code: Select all

   $thumbsmod = "";
   $thumbstxt = readfile("<xythumbs>\XYthumbs.txt");
  foreach($fileline, $thumbstxt, "<crlf>") {
      if (gettoken($fileline, "count", "|") == "3") {    //use only lines containing: folder|thumb size|hash
        $thumbsfolder = gettoken($fileline, -3, "|");     //pull-out the thumb folder
        $thumbssize = gettoken($fileline, -2, "|");       //pull-out thumb size
        $filehash = gettoken($fileline, -1, "|");         //pull-out the hash
        $filedate = property("#3", "<xythumbs>\$filehash.dat2");    //obtain filedate
        if !(exists($thumbsfolder)) {                     //adds a pre-checked "+" for "inputselect" for folders that do not exists.
          $thumbsmod = $thumbsmod . "+" . $thumbsfolder . "|" . $thumbssize . "|" . $filedate . "|" . $filehash . "<crlf>";}
         else {$thumbsmod = $thumbsmod . $thumbsfolder . "|" . $thumbssize . "|" . $filedate . "|" . $filehash . "<crlf>";}
      }
   }
       $thumbsmod = formatlist($thumbsmod, "se", "<crlf>");
  $sel_hash = inputselect("Select Thumbnails files to delete<crlf>based on folder location, thumbnail size and date.", $thumbsmod, "<crlf>", 2, , 800, 800, "Thumbnail File Selection");
    foreach($hash, $sel_hash, "<crlf>")
    {
     $hash = gettoken($hash, -1, "|");             //pull-out the hash only portion of line
     delete 0, 0, "<xythumbs>\". $hash .".*";      //change 2nd parameter to "1" to prompt prior to deleting
     $thumbstxt = regexreplace($thumbstxt, ".*".$hash.".*$\r?\n?");
    }
   writefile("<xythumbs>\XYthumbs.txt", $thumbstxt, "o", "tu");
   status "XY thumbnail clean-up completed.";

I hope you can get some use out of it. If anyone can see ways of improving it or has any problems, please pass it on. Thanks,
Ken

[klownboy]

Hi,
In the back of my mind when thinking about this script, I always thought it would be best to extract the thumbnail folder and size data directly from the actual ".dat2" files within the XY's thumbnails folder, not from the "XYthumbs.txt" file. I wasn't sure if I could do that since I'm still a newb, but I made another version which does just that. I'm surprised at the power and speed of XY scripting in that, I'm extracting information from what could be hundreds of files and it accomplishes the task in the blink of an eye. This should be quite a bit more trustworthy or fool proof since it's not relying on the data in the XTthumbs.txt file. It only accesses and updates the "XYthumb.txt" file after deleting the selected thumb files. I ran it on 3 different computers, Windows 7 and 8 with no issues though all 3 are English based and I'm not sure if the ".dat2" files would be affected by language. The starting point for the XY thumb data in the ".dat2" files may be different.

Code: Select all

    $thumbsmod = "";
    $thumbstxt = readfile("<xythumbs>\XYthumbs.txt");
    $dat2 = formatlist(folderreport("files", "r", "<xythumbs>", , , "<crlf>"), "f", "<crlf>", "*.dat2");
   foreach($dat2file, $dat2, "<crlf>") {
     $filehash = getpathcomponent($dat2file, "base");
     $dat2info = readfile("$dat2file", "b", 70, , 17);
     $thumbsfolder = gettoken($dat2info, 1, "*", "t");
     $thumbssize = gettoken($dat2info, "2", "*", "t");
     $filedate = property("#3", "$dat2file");
     $thumbsmod = $thumbsmod . $thumbsfolder . "|" . $thumbssize . "|" . $filedate . "|" . $filehash . "<crlf>";
     }
     $thumbsmod = formatlist($thumbsmod, "se", "<crlf>"); 
  $sel_hash = inputselect("Select Thumbnails files to delete<crlf>based on folder location, thumbnail size and date.", $thumbsmod, "<crlf>", 2, , 800, 800, "Thumbnail File Selection");
    foreach($hash, $sel_hash, "<crlf>")
    {
     $hash = gettoken($hash, -1, "|");           //pull-out the hash only portion of line
     delete 0, 0, "<xythumbs>\". $hash .".*";    //change 2nd parameter to "1" to prompt prior to deleting
     $thumbstxt = regexreplace($thumbstxt, ".*".$hash.".*$\r?\n?");
    }
   writefile("<xythumbs>\XYthumbs.txt", $thumbstxt); 
   status "Thumbnail clean-up completed.";

Please let me know if you have any issues. Thanks,
Ken

[nerdweed]

I tried the script in the first post and it works great.
I would prefer using the script from first post instead of second, due to the reduced disk activity (might resort to it, if this does stop working).

Can I suggest to add something, optionally check if the paths are existing and delete thumbs file if the path isn't existing.

[klownboy]

Hi nerdweed,

I'm glad the first version is working for you. I want to make sure I understand your suggestion. Are you asking to check that the path of each hash file, as listed in the "XYthumbs.txt" file, exists? And, if doesn't delete the corresponding thumbnail files in the thumbnail folder (e.g., *.dat3 and *.dbit files). It would certainly have to be an option as you said. Or, the other way around, check each thumb file itself and see if there's a corresponding line in the "XYthumbs.txt" file... that one could result in adding to lines to the XYthumbs.txt file (when the file was deficient). I'd be reluctant to do that based on below.

That's one of the reasons dealing directly with the thumb files themselves (version 2) is a better option. Then, it really doesn't matter what's in the XYthumbs.txt file. That file from what I can tell is only updated when you update or change an actual folder in which you display thumbnails. I see it as nothing more than a log of the changes. XY doesn't use it as a pointer or index to display thumbs in any way. If you delete "XYthumbs.txt", the thumbs still display just fine (i.e., it doesn't cause a regeneration of thumbs). The file is simply regenerated from scratch starting with the first folder you access, but the thumbs themselves are still fine and valid. Try deleting it after making a copy of course.

Thanks,
Ken

[nerdweed]

A sample line from XYthumbs.txt

D:\Portables\Documents\|128x72|ea9a6a3b4b8d3fc4b8aa17b6bf21dc0b

If D:\Portables\Documents doesn't exist, list all such paths that can be deleted.

A corresponding line check in XYthumbs.txt is not required - as the purpose of this script would be to ensure that the thumbs folder doesn't get too huge.

The only reason, I would prefer version 1 is that it prevents me from checking the hash of all folders and there might be multiple files for each folder if the thumbnail modes are frequently changed (thumbnails are also created for tiles view). So there might 5 different thumb files for each folder.

[klownboy]

Hey again nerdweed,
We may be closer to being in sync. You said,

D:\Portables\Documents\|128x72|ea9a6a3b4b8d3fc4b8aa17b6bf21dc0b
If D:\Portables\Documents doesn't exist, list all such paths that can be deleted.

You say "paths that can be deleted". Are you saying, for any "thumbfiles" (i.e., .dat2 and .dbit files) that have that path (D:\Portables\Documents\) that no longer exists, list those folders and optionally delete the associated thumb files? I could build a list of folders that no longer exist and allow the user to select from them, but don't you already get that opportunity in the current inputselect check box? If you see either a folder that doesn't exist or a folder that has thumbs in a size you know no longer want, check it off and the associated thumbfiles get deleted. It's logical, but at the same time I don't want to be redundant.

...So there might 5 different thumb files for each folder.

In reality, like you said, each folder has 5 possible thumb sizes, but that means there could be 10 thumbfiles since each combination (each hash) has both a hash.dat2 and hash.dbit.
Thanks,
Ken

[nerdweed]

True. That can be done through input select and I am doing it that way only. Just thought it might be good to have, but it does become redundant as well.

[klownboy]

Hi nerdweed,
It would be nice to have the option to delete those thumb files when the associated folder is non-existent, but I like to do it in conjunction with the normal selection if I could find a way. The inputselect command allows us to have selections pre-selected or pre-checked which is possibly what I could do with those non-existent folders. I'll have to see how or if I can do it when the list is contained in a variable full of items.

By the way, a few times now I've come across 2 different hash files for the same folder and same thumb size. They showed up in the input select box using the 2nd version. I checked the actual hash files and they were indeed present. I'm not sure you'd see this using the first version - maybe. That doesn't seem right to me. I don't think XYplorer should be allowing that situation to arise. Have you seen it?

I just finished another version which builds on version 2 (i.e., using the actual ".dat2" files to obtain the data). It goes even further by allowing the user to select whether they want to simply update the XYthumbs.txt file or regenerate the file completely using the new updated data. This option can be very useful if the XYthumb.txt file is way off or incorrect or has been restarted by XY from scratch. I'll post it after some more testing on a few different computers.

EDIT: Updated the first post (see EDIT 2) to provide an indication in the "inputselect" box of which folders no longer exist (i.e., they're pre-checked). It turns out nerdweed, it can be very helpful in that we may have changed the name of a folder or deleted it long ago, and we just don't recall doing it when viewing the folders in a very long list. I'll be updating the 2nd version which obtains its data directly from the thumbnail files in lieu of XYthumbs.txt with the same capability.

[Nighted]

Check your processes after running this script. I get several instances of xycopy.exe stuck in memory.

[klownboy]

Very strange, to check it out I make up some temporary folders and viewed the jpgs in various thumb sizes and ran both versions and then looked at the running processes in both "Process Explorer" and Windows "Task Manager", and I saw no instances of XYCopy stuck in memory. I'm not sure when XY invokes the XYcopy command. I assumed it's just to copy and backup files. I "read" files and "write" to files, but neither version of the scripts copies anything so I can't explain why they'd be there. I assume you checked before you ran the script(s) and saw no instances of xycopy. You didn't mention what version you were using.

[Nighted]

I'm not sure, but I think it happens when you are removing many defunct folders and you try to move/copy/delete during the process (slow enough on my netbook, not my I7 2600k though).

I'm sure this is a bug within XY and not the script itself.

[admin]

I assumed it's just to copy and backup files.

True. This script nowhere uses XYcopy.

[Nighted]

Well, no other time since using this script have I seen any instance of xycopy.exe sitting in memory, much less 20+ instances setting in memory. I have a screen shot somewhere, I will post it.

[admin]

Well, no other time since using this script have I seen any instance of xycopy.exe sitting in memory, much less 20+ instances setting in memory. I have a screen shot somewhere, I will post it.

No need for proof, I believe you. The question is: what's causing it. Surely not this script.

[Nighted]

I've been listening to a lot of Coast to Coast AM podcasts...maybe my computer is haunted.