Script: Thumbnail Maintenance

Discuss and share scripts and script files...
CFTwFSgJaLsDUrLv
Posts: 19
Joined: 28 Jul 2020 13:07

Re: Script: Thumbnail Maintenance

Post by CFTwFSgJaLsDUrLv »

Hi klownboy

Thanks for your interest, and I hope my insights help you.

That goto takes me to E:\OneDrive\Pictures\2014 which is shown in the XYplorer list as
THumbnail - location of E.jpg
THumbnail - location of E.jpg (6.18 KiB) Viewed 2806 times
THumbnail - properties of E.jpg
THumbnail - properties of E.jpg (37.88 KiB) Viewed 2806 times

My laptop is set to store all files and folders on the actual physical hard disk, as they were all on there before I bought into the OneDrive system, and I don't want to be without them if the internet connection fails. I use the OneDrive to mirror them to my other more portable laptop, which also stores them on its physical hard disk. Also backed up to a stand-alone physical hard disk. Hopefully belt and braces for backup.

I do have some photos in a backup folder on my C drive, "C:\Users\Jim\Documents OLD\Backups", which has nested sub folders. I ran option 6 on that - it contains 635 Files, 66 Folders, 1.12 GB. It got stuck after a few minutes on the 8th subfolder - which contains only 12 jpg files. After 20 minutes it was still asking me to wait there.
THumbnail - script stuck.jpg
THumbnail - script stuck.jpg (9.94 KiB) Viewed 2806 times
I'm afraid I mistakenly then asked XYp to run a report on the current folder which stopped your script. However, despite it having shown in the information field that it was looking in each subfolder up to that one, it had only created a cache for one of them - which is actually the 4th subfolder with just one image inside.
THumbnail - script stuck - this is all it did.jpg
THumbnail - script stuck - this is all it did.jpg (34.67 KiB) Viewed 2806 times

To see if it had anything to do with the file location, I copied that whole folder structure to a new one in the root of C, removed non-image files, renamed it to "C:\TESTFOLDER for Pictures" and loaded the script, choosing option 6. It ran, completed and showed no images in any folder.
THumbnail - no images in C_TESTFOLDER for Pictures.jpg
THumbnail - no images in C_TESTFOLDER for Pictures.jpg (94.97 KiB) Viewed 2806 times
The only entry shown when I use option 1 is
C:\TESTFOLDER for Pictures\ ● 300x200 ● 22/05/2022 12:26 ● cc9e70c874db92b5585f1f11c7287d6b ● 1.19 MB
The main TESTFOLDER itself has 11 jpg images totalling 2.31MB.

As I was typing the above, I found that what I was typing was not reaching the screen. Your script had restarted itself on the previous folder that had been interrupted. Every time the information box changed, it took focus from my typing. I'm afraid I couldn't live with it doing that any time I ran it.
Unfortunately, the information box that popped up also showed no images.


TO speed up testing, I deleted the images in the main folder, and most of the subfolders, leaving 5, one of which has a subfolder. Running option 7 to remove the cache takes no time, then I ran option 6 which again shows no images. But it still shows there is a cache for the main folder which now has no images of its own.
Folder [C:\TESTFOLDER for Pictures\]
[300x200]
Before: 1,224 KB After: NA no images in folder.

I'm flummoxed. I'm using XYplorer 23.00.0300, and Windows 10 which is different to yours.

Best of luck sorting it out. I'm off to do some family history research!

Jim
Windows 10 v21H2. Dell XPS17 L702, i7 2720QM, 16GB RAM, 1920*1080 on external display.

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

Thanks again Jim for the additional comments. I'm going to need some serious alone time to review and do an update. While reviewing your comments, I ran across an issue where I refer to a sub routine that I hadn't written yet. D'oh I guess it really was a work in progress and I forgot where I was in the process. It doesn't help being away from something for a long time, forgetting key details on the script, and then trying to relearn it at my age. :roll:
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

Don issued v23.50.0012 which transitions thumbnail cache "dat2" files to "ini" files which in the end is a good thing. This script will work initially since you still have the dat2 files, but since it gathers thumbnail cache information from the dat2 files, once the transition is complete for any particular image folder, the script will fail. I would recommend using the first option to refresh the thumbnail database cache files to ease the transition from "dat2" to "ini" files. I've got some work to do in updating the script.
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Script: Thumbnail Maintenance

Post by admin »

Don't work too soon. I would wait some days to see of the new way works well or needs polishing...

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

admin wrote: 14 Aug 2022 17:27 Don't work too soon. I would wait some days to see of the new way works well or needs polishing...
Yes, I will wait until we know it's good-to-go. :) Thanks.
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

Don, it's too bad we couldn't have a command/function that could have eased the transition by doing a conversion dat2 to ini and avoided having to rebuild all the dbit files and also saved on some SSD wear as well.
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Script: Thumbnail Maintenance

Post by admin »

Sorry, already overworked :)

And what SSD wear. The dat2 files work fine and when you update your cache the dbits file is written. The same would have happened before the change. No difference in SSD wear.

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

admin wrote: 14 Aug 2022 18:17 Sorry, already overworked
No problem Don, my caches are all rebuilt. I figured I'd go ahead and put the script to work to convert them all while I was away. Now the script is useless until updated, but as you said I'll hold off awhile.
admin wrote: 14 Aug 2022 18:17 And what SSD wear. The dat2 files work fine and when you update your cache the dbits file is written. The same would have happened before the change. No difference in SSD wear.
I understand, but it is rewriting the dbits files so those cache files are being rewritten to the SSD and since the files can be large and many of them, is contributing to SSD writes. Yes, it is as before, but now we are re-writing them all to support this change.

EDIT: As mentioned, all my dat2 to ini file conversions are complete and I haven't seen any ill effects...yet. :)
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Script: Thumbnail Maintenance

Post by admin »

Cool! :tup:

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

One major issue I'm going to face though is that I noticed your new "ini" files have no [General] key for the folder itself. The old "dat2 files I could extract the folder and if needed I could extract both the folder and thumbnail size to check or obtain the hash. I'm not sure if I can even obtain the folder from the hash. In any case I wouldn't want to try.
If you were doing everything locally that may not be a problem since you are in the folder where the thumbnail exists, but for most it is.

One example, I build thumbnail cache database for inputselect which relies on seeing the folder along with thumbnail size and other info such as this:
G:\Wallpaper\1920x1080\ ● 240x240 ● 7/2/2022 6:46 PM ● 31f99e73faec17e371f538c4856c34e3 ● 8.30 MB

I know we can get the folder from XYthumbs.txt when we know the hash, but I look at that as more of a log. I don't think we should have to retrieve key information like the folder from there.
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Script: Thumbnail Maintenance

Post by admin »

Yes, I already saw this. 0014 will have it.

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

Thanks Don. Since silly me already went ahead and converted all my dat2 files by rebuilding my images, I'll have to do it again or maybe I could write a small script that would simply obtain the current folder and add it to the [General] however you have it and loop through all the subfolders.

Will the folder path have the final backslash as it did or does in the dat2 files?

Thanks again.
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Script: Thumbnail Maintenance

Post by admin »

Yes, final backslash.

I told you to wait. And I still do... :)

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

admin wrote: 15 Aug 2022 12:11I told you to wait. And I still do...
Yeah, I know. Not a big deal. On a high note, I still haven't seen any problems or performance issues with the new ini files. :tup:
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Script: Thumbnail Maintenance

Post by klownboy »

Thumbnail Maintenance version 1.20
Thumbnail Maintenance_v1.20.jpg
Thumbnail Maintenance_v1.20.jpg (112.57 KiB) Viewed 1379 times
New in this version...
▪ updated the script to recognize the XYplorer new thumbnail cache "ini" files which replaced the old "dat2" files. The thumbnail "dbits" files remained the same.
▪ the script is not setup to transition the cache files from the old "dat2" to the new "ini" files. XYplorer will accomplish that on its own, but that should be done "before" you use this script.
▪ recognizes the new allowances in XYplorer for thumbnail cache path "Resolve cache path from current folder". There are 3 main possibilities for XYplorer cache paths
▪ First: is the most common where the user specifies a single folder location for their thumbnail cache files. This was the only option for years up until recently.
▪ Second: is the user has "Resolve cache path from current folder" in thumbnail configuration and has the cache path is blank. In this case the thumbnail cache files are located in each and every folder containing the cacheable images.
▪ Third: is the user has "Resolve cache path from current folder" in thumbnail configuration and specifies a drive with a "?" and a folder on the drives for the thumbnail caches (e.g., "?:\XYthumbs\"). So in this case, each drive will have all the thumbnail cache files for that drive on that path. So the thumnails for a image folder, "F:\pictures" or "F:\Wallpaper" will be located in "F:\XYthumb".
▪ I've eliminated background thumbnail generation (i.e., in a second instance). My hope is eventually Don will institute background thumbnail building within XYplorer. He has this on his to-do list.
▪ Hopefully I've picked up on other issues on this revision along the way. I do not have a network setup so it's unlikely this will work across network paths. I don't know.
▪ I've been testing now for over a month squashing bugs. That's not to say there won't be more unfortunately. I established another copy of XY where I could test in the 3 different configurations mentioned above. It should be noted that you can't have left over cache files from one of the above (3) configurations and then use the script on a different configuration. It has to read "ini" files to do its work. Left-over cache files from using or testing in another cache file configuration will result in failures. This shouldn't be a problem since in real life use since you can only you one cache file configuration.
▪ The script will detect if "Resolve cache path from current folder" is checked and will require the user to select a drive to accomplish thumbnail maintence. With that option checked there are way too many than avenues to travel across an entire computer, so the limitation to a single drive. If you have more to do more then go back to the menu afterwards and select another drive.
Thanks for trying.
ThumbnailMaintenance_v1.20.zip
(166.86 KiB) Downloaded 126 times
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

Post Reply