Page 1 of 2

PIH

Posted: 30 Jan 2020 18:37
by Marco
Do you have any other algorithms to try?

Re: PIH

Posted: 30 Jan 2020 20:11
by admin
Hihi LOL.

Yes, I'm working on it... :biggrin:

Re: PIH

Posted: 03 Feb 2020 00:13
by eil

Code: Select all

Set it to a value larger than 0 to define the tolerated distance for two 
      images to be counted as dupes.
      Already tolerance 1 will ignore scale, brightness, contrast, saturation, 
      and hue. Higher tolerance will ignore JPEG artifacts and minor differences 
      in detail.
just a thought, but wouldn't it be better to make tolerance be "more flexible" by putting each "ignore" on different percentage?..
smt like:
1 = ignore scale
2 = ignore brightness
4 = ignore contrast
8 = ignore saturation
or at least 1,2,3,4, and only on 5 "Higher tolerance will ignore JPEG artifacts"

Re: PIH

Posted: 03 Feb 2020 11:29
by admin
I see what you mean, but this (partly) sounds more easy than it's done. At the moment I'm quite satisfied with the current state as "phase one" and would like to publish it to the masses and see the kind of feedback I'll get.

But what about the feedback here in the forum? Does it work for you as good as for me? (in my personal photo collection it does a perfect job at the moment)

Re: PIH

Posted: 03 Feb 2020 19:48
by eil
that's what i can show as feedback. ~79Gb scanned, quite fast, especially as it's done on rather old laptop.
default SHA-1, 9 tolerance
2020-02-03_220005.png
just for test SHA-256, 9 tolerance.
2020-02-03_220707.png
kinda surprised it found 2 extras. :kidding:

Re: PIH

Posted: 03 Feb 2020 20:02
by admin
Thanks! And did the results look good? Not too many false positives are false negatives?

Now the interesting part: You cannot set the hash method here, this dropdown belongs to the "Content" dupe search above. So you have run the same search two times and got different results! Hmm, there is no learning built in... :) But I once saw the same here during my tests. I'll watch it...

PS: Your second search was much slower??? That cannot be. Or was it actually the first search? (There is heavy caching here, any non-first search will be MUCH faster).

Re: PIH

Posted: 04 Feb 2020 20:46
by eil
Thanks! And did the results look good? Not too many false positives are false negatives?
visual presentation is good, groups are easily determined, text info is well informative. didn't check all dupes, don't really want to sort them right now, but checked about 10-15 groups = no false at all.
You cannot set the hash method here, this dropdown belongs to the "Content" dupe search above. So you have run the same search two times and got different results!
though you say that makes no difference, still probably speed really depends on CPU business, 'cause here i made 9 searches with so to say "different methods" and there are differences in time. though i must mention that 1-4 was while doing some other stuff, and 5-9 was while doing nothing waiting for results. plus can't say that "next runs" were actually "much faster", though again maybe it's due to old laptop.
2020-02-04_121627.png
2020-02-04_121849.png
2020-02-04_122055.png
2020-02-04_122307.png
2020-02-04_122556.png
2020-02-04_122803.png
2020-02-04_122900.png
2020-02-04_123017.png
2020-02-04_123144.png

Re: PIH

Posted: 04 Feb 2020 22:28
by admin
I see you have Content and Image ticked. That does not make much sense since Content dupes are logically always also Image dupes. :)

Thanks for that insight which it allows me to make my code faster: if Content is ticked then any check for Image dupes can be skipped for logical reasons.
a) if content differs we don't have to check for image since all Dupes filters are AND-ed
b) if content is same then image is same as well

Re: PIH

Posted: 05 Feb 2020 15:31
by admin
FYI, you can now search images by similarity to a given image. ATM it's a two-step process:

1. Retrieve image hash of the given image:

Code: Select all

text "<prop #image.hash>";
2. Go to your search location and use this search term (1b116dc5c3c5311b is an example for the hash returned above):

Code: Select all

prop:#image.hash:1b116dc5c3c5311b
Future enhancements could be
- a way to pass a tolerance value to the search (at the moment the default here is tolerance = 1)
- a fully wrapped comfort feature where you point to an image and order "find similar images!"

Re: PIH

Posted: 05 Feb 2020 21:01
by admin
PS: Of course, the same syntax also lets you define color filters for similar images, how cool is that?

Code: Select all

prop:#image.hash:1b116dc5c3c5311b

Re: PIH

Posted: 05 Feb 2020 21:52
by Enternal
Well, not very cool. I'm supposed to be managing files and not playing with neat new features. What are you doing to me?

(And yes, wow, this really is a neat feature! I can finally get rid of this other ancient tool I have been using for the longest time.)

Re: PIH

Posted: 05 Feb 2020 21:57
by admin
It might even get uncooler. So many things are imaginable now. What about sorting photos by their dominant color hue (like some folks do with their books on the shelf)?

Re: PIH

Posted: 05 Feb 2020 22:22
by Enternal
Yes, please. Uhh, I meant no. :biggrin:

Re: PIH

Posted: 06 Feb 2020 21:59
by eil
admin wrote: 05 Feb 2020 15:31 FYI, you can now search images by similarity to a given image.
sounds as neat feature for huge photo archives. :biggrin:
i wonder if it may allow in future to hash an image with lots of green(photo in forest), and with some # tolerance find not similar, but same "lots-of-green"-images?. it's just a question of curiosity. :roll:

Re: PIH

Posted: 07 Feb 2020 12:04
by admin
Sure, that's possible.