Replacing duplicates with hard links

Features wanted...
zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Replacing duplicates with hard links

Post by zakoul »

Is it possible to add the ability to replace duplicates with hard links?
When searching for duplicates in the search results in the "Duplicates" column, add the item "Remove the non-first items in each group and replace them with hard links to the first items in the group"
In this case, the name of the deleted file replaced by the hard link must remain the same.
Sorry google translator.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Replacing duplicates with hard links

Post by highend »

At least you can achieve that via scripting...

Requirements:
1
Change the variable $dupeColumnName = "Dupes"; inside the script
if you're not using the english language

2
Your search result must have at least these columns in the given order:
1: # (line numbers)
2: Dupes (groups)
3: Path (shows only the relative path)
4: Name

Any other columns are optional

Do not try to use the script on multiple joined paths (<path 1>;<path 2> under "Name & Location" - "Location:")
I haven't tested that scenario^^

Example:
Initial dupe search
1.png
1.png (10.45 KiB) Viewed 900 times
Show fsutil output after the inital search and after executing the script
2.png
2.png (4.49 KiB) Viewed 900 times
Script:
One of my scripts helped you out? Please donate via Paypal

zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Re: Replacing duplicates with hard links

Post by zakoul »

Thank you. I checked it on a small number of takes, it works. But it works only if English is selected in XYplorer, the script does not work with other languages.
With a large number of dulicats (1526) XYplorer freezes and does not respond.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Replacing duplicates with hard links

Post by highend »

Try v0.2

and replace the content of the
$dupeColumnName = "Dupes";
line with the name of the duplicate column in your language first...
One of my scripts helped you out? Please donate via Paypal

zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Re: Replacing duplicates with hard links

Post by zakoul »

Line Processing Group appears and counts to infinity.
Screen
In my case, 1526 duplicates were found, 716 groups were created, only 810 files need to be deleted and replaced with hard links, and the script reads endlessly.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Replacing duplicates with hard links

Post by highend »

v0.3

For whatever reason it doesn't remove the groups from the $view variable on your system
so here is a different approach...

Run it once, check the output of the window (that hopefully opens). If it does look correct,
make this change:

Code: Select all

            run "$batFile", "%TEMP%";
            // text $batContent;
One of my scripts helped you out? Please donate via Paypal

zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Re: Replacing duplicates with hard links

Post by zakoul »

It works, but for some reason not all groups are processed. Of the 716 groups, only 526 were processed.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Replacing duplicates with hard links

Post by highend »

Sorry, that's not any information to help me tracking down a problem...

$i just counts up so if $cntAllGroups found 716 groups (from your status bar)
than it executes the loop 716 times...

Where does the $batContent begins to differ from what you see in your dupe search pane?...

Here is a v0.4 that will add the group that is being process to the bat content...

Code: Select all

REM Group: 1
DEL /F /Q "T:\!tmp\dup2\dhcp.png"
mklink /H "T:\!tmp\dup2\dhcp.png" "T:\!tmp\dup1\dhcp.png"

DEL /F /Q "T:\!tmp\src\dhcp.png"
mklink /H "T:\!tmp\src\dhcp.png" "T:\!tmp\dup1\dhcp.png"


REM Group: 2
DEL /F /Q "T:\!tmp\src\ImgBurn methods.txt"
mklink /H "T:\!tmp\src\ImgBurn methods.txt" "T:\!tmp\dup1\ImgBurn methods.txt"


REM Group: 3
DEL /F /Q "T:\!tmp\dup2\Italienische Vokabeln.txt"
mklink /H "T:\!tmp\dup2\Italienische Vokabeln.txt" "T:\!tmp\dup1\Italienische Vokabeln.txt"


REM Group: 4
DEL /F /Q "T:\!tmp\dup2\Netflix.txt"
mklink /H "T:\!tmp\dup2\Netflix.txt" "T:\!tmp\dup1\Netflix.txt"

DEL /F /Q "T:\!tmp\src\Netflix.txt"
mklink /H "T:\!tmp\src\Netflix.txt" "T:\!tmp\dup1\Netflix.txt"
One of my scripts helped you out? Please donate via Paypal

zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Re: Replacing duplicates with hard links

Post by zakoul »

Perhaps the reason is that there are more than 2 duplicates in groups, maybe 3 and 5 and 10 ...
The attachment contains the contents of batFile. A total of 526 groups were reversed instead of 716.

Try to create more than 2 identical files and test it. So that there are more than 2 duplicates in the group.
Attachments
batFile.zip
(16.02 KiB) Downloaded 34 times

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Replacing duplicates with hard links

Post by highend »

That's not the problem.

Using a modified version (only with a slightly enhanced output):

Code: Select all

REM Group: 1 => 2 dupe(s)
DEL /F /Q "T:\!tmp\dup2\dhcp.png"
mklink /H "T:\!tmp\dup2\dhcp.png" "T:\!tmp\dup1\dhcp.png"

DEL /F /Q "T:\!tmp\src\dhcp.png"
mklink /H "T:\!tmp\src\dhcp.png" "T:\!tmp\dup1\dhcp.png"


REM Group: 2 => 4 dupe(s)
DEL /F /Q "T:\!tmp\dup3\ImgBurn methods.txt"
mklink /H "T:\!tmp\dup3\ImgBurn methods.txt" "T:\!tmp\dup1\ImgBurn methods.txt"

DEL /F /Q "T:\!tmp\dup4\ImgBurn methods.txt"
mklink /H "T:\!tmp\dup4\ImgBurn methods.txt" "T:\!tmp\dup1\ImgBurn methods.txt"

DEL /F /Q "T:\!tmp\dup5\ImgBurn methods.txt"
mklink /H "T:\!tmp\dup5\ImgBurn methods.txt" "T:\!tmp\dup1\ImgBurn methods.txt"

DEL /F /Q "T:\!tmp\src\ImgBurn methods.txt"
mklink /H "T:\!tmp\src\ImgBurn methods.txt" "T:\!tmp\dup1\ImgBurn methods.txt"


REM Group: 3 => 1 dupe(s)
DEL /F /Q "T:\!tmp\dup2\Italienische Vokabeln.txt"
mklink /H "T:\!tmp\dup2\Italienische Vokabeln.txt" "T:\!tmp\dup1\Italienische Vokabeln.txt"


REM Group: 4 => 2 dupe(s)
DEL /F /Q "T:\!tmp\dup2\Netflix.txt"
mklink /H "T:\!tmp\dup2\Netflix.txt" "T:\!tmp\dup1\Netflix.txt"

DEL /F /Q "T:\!tmp\src\Netflix.txt"
mklink /H "T:\!tmp\src\Netflix.txt" "T:\!tmp\dup1\Netflix.txt"
which 100% matches this search result:
1.png
1.png (19.04 KiB) Viewed 834 times
One of my scripts helped you out? Please donate via Paypal

zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Re: Replacing duplicates with hard links

Post by zakoul »

I changed the script a little, added

Code: Select all

$batContent      .= <crlf> . "REM AllGroup: $cntAllGroups" . <crlf>;
to see the value of $ cntAllGroups and it does not correspond to the real number of groups.
In fact, there are 716 groups and the $ cntAllGroups script has a value of 526

Batfile:

Code: Select all

REM Group: 1

REM AllGroup: 526
DEL /F /Q "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\LaniBeats Trap Toronto Nexus Expansion\Paino.fxp"
mklink /H "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\LaniBeats Trap Toronto Nexus Expansion\Paino.fxp" "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Dance Orchestra\A Piano Concert.fxp"

DEL /F /Q "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\TRACK$TAR XP\PN Concert Piano.fxp"
mklink /H "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\TRACK$TAR XP\PN Concert Piano.fxp" "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Dance Orchestra\A Piano Concert.fxp"


REM Group: 2

REM AllGroup: 526
DEL /F /Q "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Minimal House\Gehts Noch.fxp"
mklink /H "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Minimal House\Gehts Noch.fxp" "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\LaniBeats Trap Toronto Nexus Expansion\Anaconda.fxp"


REM Group: 3

REM AllGroup: 526
DEL /F /Q "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Psytrance\AR Solid Arpeggio.fxp"
mklink /H "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Psytrance\AR Solid Arpeggio.fxp" "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Virus TI\AR Solid Arpeggio Indigo.fxp"


REM Group: 4

REM AllGroup: 526
DEL /F /Q "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Psytrance\AR Solid Bass.fxp"
mklink /H "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Psytrance\AR Solid Bass.fxp" "D:\Soft\Sound\ReFX\ReFX Nexus\Presets\Presets Original\Presets\XP Virus TI\AR Solid Bass Indigo.fxp"

zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Re: Replacing duplicates with hard links

Post by zakoul »

This is where 526 comes from.
Attachments
2021-07-19_153244.jpg
2021-07-19_153244.jpg (121.44 KiB) Viewed 826 times

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Replacing duplicates with hard links

Post by highend »

I see. Look at your status bar:

found: 1 526 dupes in 716 groups

Switch to chinese or whatever language you're normally using and execute this from the address bar and show the ouput (in the dupe search tab of course):
text get("Status", 1)
One of my scripts helped you out? Please donate via Paypal

zakoul
Posts: 130
Joined: 11 Dec 2019 10:44

Re: Replacing duplicates with hard links

Post by zakoul »

Code: Select all

найдено: 1 526 дублей в 716 группах

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Replacing duplicates with hard links

Post by highend »

Ok, than v0.5 should finally work...
One of my scripts helped you out? Please donate via Paypal

Post Reply