Thumbnail [dat2 & dbits] basefile name same for 2 folders

[klownboy]

Hi, I came across something I never scene before in all my dealings with thumbnail files and the hash file name generated. I have two folders in which I've generated thumbs and there is only one pair of thumbnail files present in the thumbs folder (instead of two separate pairs of dat2 and dbits files for each folder as there normally are or should be) . Though there is only one pair of thumb files, this is how the thumbnail txt file looks for the 2 folders in question.

Code: Select all

    E:\3\|240x180|01d4b4cc01e46791d66be7940c78e831
E:\3\ken\|240x180|01d4b4cc01e46791d66be7940c78e831     //I moved these into alignment to quickly see they are the same

It showing the 2 folders with the same hash file name which should never happen ...from what I understand. The hex preview for the dat2 file shows "E:\3\". I was wondering why my Thumbnail script wan't working for both folders. I suppose this is why. I performed a thumbnail rebuild in both folders and still have the same result - one pair of thumb files and the same entries in the thumb txt file. Strange wouldn't you say? I may put this in bugs when Don returns. Thanks.

[highend]

If both files are identical, they have the same hash.

The question is: Is there a reason why they must be different (maybe they were used for storing the exact same pictures)?

[klownboy]

Good point highend, these 2 folders were made for troubleshooting my thumbnail script (the folders were made and images loaded in them just this morning). The folders are different in that "E:\3\" contains 44 image files and "E:\3\ken\" has 42 image files. Yes, most of the files within the 2 folders are identical, but "E:\3\" has at least 3 totally different files that are not in "E:\3\ken\". Could XY be generating or using the same hash base name in this case or should it be?

[highend]

Regardless of how many pictures are in these folders, are the contents of both files identical or not? The chance that two hashes for two non identical files is the same is... veeeery minor (but ofc not impossible). I guess your files are the same

[Marco]

That md5 hash is not of the files, but the string

Code: Select all

E:\3\*240x180

(as per docs, http://www.xyplorer.com/xyfc/viewtopic. ... 590#p90590)
klownboy, can you give the exact steps to repro?

[klownboy]

Hi highend, no the contents were not the same. As I said in my last post, one folder had "3" images that were not in the other folder. So yes it is odd that that would generate the same hash.

Hi Marco, if the hash is simply based on that string as you pointed out, how could I end up with entries in the thumbnails.txt file as I pointed out above (2 different folder names with the same hash beside it). Obviously that's the case since Don knows how the hash is generated. From what I recall I had made the folders "E:\3\" and "E:\3\ken" and dumped files into them. There were differences in the 2 folders but they did have many of the same images (see 1st para). I then generated thumbs by going into each folder and clicking on the large thumbnail size. This was to set myself up to test my thumbnail script in the "Refresh" mode which refreshes based on the "existing" thumbnail sizes in each folder. That's when I noticed in stepping through the script that one of the 2 folders was not being refreshed and then I looked through the thumbnail folder and the txt file and reported the above.

I since deleted the dat2/dbits files for the subject folder(s), regenerated the thumbs and now have different thumb files for each folder. So who knows what happened...I'll keep an eye on it.

Edit: My thumbnails script also regenerates the XYthumbs.txt file so I'll have to look at that as well. It simply uses the information from the dat2 files present in the thumbnails folder. Though in this particular case, I was using the "Refresh" option which doesn't touch the thumbs.txt file because it doesn't need to.

[Stef123]

That md5 hash is not of the files, but the string

Code: Select all

E:\3\*240x180

(as per docs, http://www.xyplorer.com/xyfc/viewtopic. ... 590#p90590)

Thanks for this link, it sheds some light on the cancerous outgrowth of my cache folders before Ken's script came along. XY kept losing thumbs that I knew for sure I had created, it felt like I was continually starting over with caching while browsing folders, a tediously slow process, the thumbs folder got ever more bloated ...

With the current design, aren't you bound to run into conflicts when you hook up your notebook to different domains or when you switch backup drives? They plug in with identical drive letters and paths, but their folder content is not synced yet. To avoid messing up the cache I switch it OFF, but that doesn't keep XY from writing into the very same thumbs folder anyway. On exiting XY I wonder: Will it delete "real" and wanted cache files along with its temporary "non-real" cache files? Not to mention the orphan leftovers that don't get deleted. No way to distinguish between ~temp files and the real stuff, both carry the same file endings, the same hash codes, and both get chiseled in stone into the same location.

[klownboy]

I think all is well when you work as I do and probably most users who have caching either on or off - one or the other. It's when you work in situation as you've discussed turning caching on and off depending on the situation that you are more likely to encounter issues. I issued another update to the ThumbnailMaintenance script [v1.14] which will now take care of both stray or orphaned dat2 files as well as dbits files. It's the stray matched pairs that are tough to correct based on the hash name or whatever. Maybe if immediately after using a computer with the cache "off" we could filter the files in the <xythumbs> folder look at the files generated since they will have the new modified or created dates.

I wonder (until Don changes how it XY caches with caching off) for your specialized use if you might be better off with a script that on-the-fly actually changes the designated <xythumbs> folder (e.g., to another temp folder) and turns the caching off. When your done with the situation requiring caching off, then you'd hit the CTB again and it would turn caching back "on" and change the designated <xythumbs> folder back to the normal folder. Of course XY would need some saves to make the changed setting stick. At least then youcould see what really happening with the cache files in the temp folder. Maybe you should try that manually before we attempt to automate.

[Stef123]

Very smart idea. Will do that from now on. Thanks.

[klownboy]

Hi Stef, I made a script which will take you in and out of a normal thumbnail cache mode ON using the normal XY thumbnails folder to the caching OFF mode using a temporary folder of your choosing. I've tested and it seems to be working fine. I would recommend that you do a backup first of course as I did before testing.

With Caching OFF, I noticed in my designated temp thumb folder a bunch of "dbits" files remaining even after going back to caching ON. The most important thing is from what I see, there were "NO" changes made to the "normal" thumbnail folder when I had caching OFF. That's a real good thing. There were only files (dbits) deposited in the temp folder. The script has to set keys in XYplorer.ini and therefore has to exit without saving and then restart for the new setting to take affect.

Code: Select all

   $ThumbsCache = getkey("Cache", "Thumbs");
   $TempCacheFolder = "D:\Temp\";                        // change to whatever temp folder you'd like, use a final backslash
   
   if($ThumbsCache == "1") {                             // if cache is currently "On" turn it "Off"
     perm $ThumbsFolder = getkey("CacheDir", "Thumbs");  // msg "Cache folder: $ThumbsFolder<crlf>Cache On or Off: $ThumbsCache";
     writepv;
     setkey("$TempCacheFolder", "CacheDir", "Thumbs");
     setkey("0", "Cache", "Thumbs");                     //set cache off
     msg "Thumbnail caching is now off and thumbs cache<crlf>folder is set to $TempCacheFolder";
     #190;                                               //restart without saving
	 }                                             
   else {                                                //if cache is currently "Off" turn it "On"
	  setkey("$ThumbsFolder", "CacheDir", "Thumbs");
     setkey("1", "Cache", "Thumbs");
 	 msg "Caching is now ON and thumbs cache<crlf>folder is set to $ThumbsFolder";
     unset $ThumbsFolder; writepv;
     #190;
    }

I can refine further if you'd like. Right now it simply cycles on and off (i.e., cache OFF use temporary folder; Cache ON use normal folder). It does provide a message telling you what it's doing. I could have a confirmation dialog to accomplish the cycling. Try it, but like I said, first back up.

Edit: I removed the delete line which deleted the temp dat2 and dbits files since it would cause XY "file in use" messages. We can always delete these at any time when back in the normal mode.

[Stef123]

Thank you, Ken,
this comes at just the right moment, I made some stupid mistake and messed up an existing cache that took a whole night to create. I generally fare better relying on scripted routines that protect me from my own disastrous clicks.