XYcopy wishes

[nas8e9]

I can't find a wish list for XYcopy (currently under development), so here goes:

1. Verify after copy/move: MD5 or SHA1-hashing after cache flush or something? Possible life saver for when using wireless network links and USB sticks.

2. "Intelligent" queue:
* Moving files on the same partition (no file reading/writing, just MFT-updating) jumping to the top of the queue.
* File operations writing to different physical devices (HD1 to HD2, HD1 to USB1, etc.) run in parallel, whereas operations involving the same device are queued.
* To expand on the second point, when writing to a physical device and then starting a further copy/move operation to the same device, to have the additional operation(s) queued so as not to overload a device with multiple concurrent data streams (think USB sticks, although conventional HD's don't do multitasking that well either).

3. Network throttling: perhaps more for XYcopy 2.0, but very useful when limited to wireless network speed while still needing to use the internet connection.

[Mesh]

I can't find a wish list for XYcopy (currently under development), so here goes:

1. Verify after copy/move: MD5 or SHA1-hashing after cache flush or something? Possible life saver for when using wireless network links and USB sticks.

2. "Intelligent" queue:
* Moving files on the same partition (no file reading/writing, just MFT-updating) jumping to the top of the queue.
* File operations writing to different physical devices (HD1 to HD2, HD1 to USB1, etc.) run in parallel, whereas operations involving the same device are queued.
* To expand on the second point, when writing to a physical device and then starting a further copy/move operation to the same device, to have the additional operation(s) queued so as not to overload a device with multiple concurrent data streams (think USB sticks, although conventional HD's don't do multitasking that well either).

3. Network throttling: perhaps more for XYcopy 2.0, but very useful when limited to wireless network speed while still needing to use the internet connection.

Personally, I don't have a need for item 3, but I'll definitely second items 1 and 2!

In addition to all the visual bells and whistles Don mentioned (such as a total progress bar, in addition to individual file progress bars, estimated time to completion, etc...)

[admin]

Let me ask about point 1): MD5 or SHA1-hashing. The idea is to compare source and copy to verify it's identical? I would do no hashing then, far too slow. I would just compare the bytes. Or?

[nas8e9]

Let me ask about point 1): MD5 or SHA1-hashing. The idea is to compare source and copy to verify it's identical? I would do no hashing then, far too slow. I would just compare the bytes. Or?

The aim would be to verify that source and (physical, i.e. not Window's file cache) target are identical. I see TeraCopy using a hash function; I'm not familiar with the pros and cons of hashing versus direct comparison. As hashing would need to read the entire target file anyway, adding a hash operation doesn't sound faster. In short: (some) others use hashing, but I don't know why .

[admin]

Let me ask about point 1): MD5 or SHA1-hashing. The idea is to compare source and copy to verify it's identical? I would do no hashing then, far too slow. I would just compare the bytes. Or?

The aim would be to verify that source and (physical, i.e. not Window's file cache) target are identical. I see TeraCopy using a hash function; I'm not familiar with the pros and cons of hashing versus direct comparison. As hashing would need to read the entire target file anyway, adding a hash operation doesn't sound faster. In short: (some) others use hashing, but I don't know why .

OK, still this verification will take its time. A long time if many files were copied.

Do you really have reasons to distrust the shell copy routines? I never ever experienced an imperfect copy, at least not without being notified about it in a very clear way (like: black screen because no more electricity)

[Stefan]

I think optional "comparing the bytes" is an good, reasonable idea, e.g. for large copy sessions over the net.

But i think i have also seen sometime two files with same size... but still damaged.
So an optional " ,1" flag to do an hash compare could be nifty... just to be on the sure side.
No matter how long it takes! Providing intact files to an costumer is mandatory.
(For me, time isn't an issue, - i just would start the copy - and go on with other work - maybe even on an other pc)

(I am, for example, have got two or three cd-rom from an company with broken files, one after another, till they have proofed there cds themselves and send me an valid one )

As an workaround, If one want to do an hash compare, maybe just for this-or-that issue,
he could perhaps use an third party tool before and after the copy?
Like DVDsig from http://members.ozemail.com.au/~nulifetv ... /freeware/

DVDsig v1.01 [10kB]
DVDsig is a files verification software for DVD-ROM, DVD-Video, CD-ROM and other removable media.
It will scan all files and directories of a new disk compilation and create a list of reliable MD5 file signatures.
DVDsig is small enough to be conveniently included on any disk compilation and offers immediate,
independent verification of the files after burning is completed.
The inclusion of DVDsig along with the signature list it generates is a simple, quick
and effective way of validating your data at any time and anywhere

[nas8e9]

Let me ask about point 1): MD5 or SHA1-hashing. The idea is to compare source and copy to verify it's identical? I would do no hashing then, far too slow. I would just compare the bytes. Or?

The aim would be to verify that source and (physical, i.e. not Window's file cache) target are identical. I see TeraCopy using a hash function; I'm not familiar with the pros and cons of hashing versus direct comparison. As hashing would need to read the entire target file anyway, adding a hash operation doesn't sound faster. In short: (some) others use hashing, but I don't know why .

OK, still this verification will take its time. A long time if many files were copied.

Do you really have reasons to distrust the shell copy routines? I never ever experienced an imperfect copy, at least not without being notified about it in a very clear way (like: black screen because no more electricity)

I do need it for verifying network copies over a wireless network. I absolutely agree that the OS/Shell should make sure this doesn't happen, but network operations by wireless base stations and so on are not fully under its control, so accidents have happened to me in the past.

In terms of implementation, I'd like it not unlike AutoQueue:
1. settable per operation via TB button;
2. perhaps subject to the same kind of exclusion rules for AutoQueue: by default not for HD to HD file operations, but enabled for network and external storage (USB, etc) file operations. If these rules would be user-configurable, it would be very powerful indeed.

[admin]

Let me ask about point 1): MD5 or SHA1-hashing. The idea is to compare source and copy to verify it's identical? I would do no hashing then, far too slow. I would just compare the bytes. Or?

The aim would be to verify that source and (physical, i.e. not Window's file cache) target are identical. I see TeraCopy using a hash function; I'm not familiar with the pros and cons of hashing versus direct comparison. As hashing would need to read the entire target file anyway, adding a hash operation doesn't sound faster. In short: (some) others use hashing, but I don't know why .

OK, still this verification will take its time. A long time if many files were copied.

Do you really have reasons to distrust the shell copy routines? I never ever experienced an imperfect copy, at least not without being notified about it in a very clear way (like: black screen because no more electricity)

I do need it for verifying network copies over a wireless network. I absolutely agree that the OS/Shell should make sure this doesn't happen, but network operations by wireless base stations and so on are not fully under its control, so accidents have happened to me in the past.

In terms of implementation, I'd like it not unlike AutoQueue:
1. settable per operation via TB button;
2. perhaps subject to the same kind of exclusion rules for AutoQueue: by default not for HD to HD file operations, but enabled for network and external storage (USB, etc) file operations. If these rules would be user-configurable, it would be very powerful indeed.

From my POV it would be more practical to add this validation service to the Backup operation. Later, it will be added to XYcopy anyway (but under the name "XYcopy", and it will have an option to delete the sources when done (so it will be a fully configurable Move named "XYmove").

@Stefan Note that I was not thinking of merely comparing the byte count, but the bytes themselves. Generating a Hash is not necessary and would needlessly slow down the job.

[nas8e9]

From my POV it would be more practical to add this validation service to the Backup operation. Later, it will be added to XYcopy anyway (but under the name "XYcopy", and it will have an option to delete the sources when done (so it will be a fully configurable Move named "XYmove").

I'm not entirely sure what you mean, but:
1. Backup operations should have verify, so in terms of prioritizing I get that.
2. Having said that, the verify option is at this point the sole reason for me to use TeraCopy for network file operations. To me, it's a big deal to have it available for copy/move as well.

Is this also to do with you envisioning XYcopy 1.0 to rely on the Windows Shell functions, while verification will require your own hand-rolled kernel functions which you don't envision creating until XYcopy 2.0?

Finally, do you see any general merit in my suggestions for enabling verification only for certain kinds of operations (network, external storage) and with the same UI (TB button to enable/disable it for the next command issued) as XYcopy currently has?

[admin]

From my POV it would be more practical to add this validation service to the Backup operation. Later, it will be added to XYcopy anyway (but under the name "XYcopy", and it will have an option to delete the sources when done (so it will be a fully configurable Move named "XYmove").

I'm not entirely sure what you mean, but:
1. Backup operations should have verify, so in terms of prioritizing I get that.
2. Having said that, the verify option is at this point the sole reason for me to use TeraCopy for network file operations. To me, it's a big deal to have it available for copy/move as well.

Is this also to do with you envisioning XYcopy 1.0 to rely on the Windows Shell functions, while verification will require your own hand-rolled kernel functions which you don't envision creating until XYcopy 2.0?

Finally, do you see any general merit in my suggestions for enabling verification only for certain kinds of operations (network, external storage) and with the same UI (TB button to enable/disable it for the next command issued) as XYcopy currently has?

Actually, thinking about it in today's spring time sun I think verification would make a very cool additional feature for version 9.00. So, yes, I'll add it to XYcopy 1.0.

@all: (hello jacky!) Thanks for all the other input to XYcopy -- very exciting stuff, ain't it?! More tomorrow...

[nas8e9]

Actually, thinking about it in today's spring time sun I think verification would make a very cool additional feature for version 9.00. So, yes, I'll add it to XYcopy 1.0.

@all: (hello jacky!) Thanks for all the other input to XYcopy -- very exciting stuff, ain't it?! More tomorrow...

Thank you for including verification in your XYcopy 1.0 plans. Based on trying out some of the competition, the whole of XYcopy (non-blocking, optional queuing and verification) would really give XYplorer a leg up regarding its core competency, i.e. file management. Very interested to see what pops up tomorrow!

[admin]

Actually, thinking about it in today's spring time sun I think verification would make a very cool additional feature for version 9.00. So, yes, I'll add it to XYcopy 1.0.

@all: (hello jacky!) Thanks for all the other input to XYcopy -- very exciting stuff, ain't it?! More tomorrow...

Thank you for including verification in your XYcopy 1.0 plans. Based on trying out some of the competition, the whole of XYcopy (non-blocking, optional queuing and verification) would really give XYplorer a leg up regarding its core competency, i.e. file management. Very interested to see what pops up tomorrow!

I have to add that while working with the shell API only (= XYcopy 1.0) I can add a verification only for COPY jobs, because after a MOVE there's nothing left to compare.

A remaining question is: How and when would I present the results of a negative verification. This could be a very long list...

[admin]

Conc. the Queue toolbar icon. If you got a better one...

[Mesh]

I have to add that while working with the shell API only (= XYcopy 1.0) I can add a verification only for COPY jobs, because after a MOVE there's nothing left to compare.

A remaining question is: How and when would I present the results of a negative verification. This could be a very long list...

I'm not sure what you mean by a negative verification. Do you mean when problems have been detected?

In general, this works best when file ops get their own command center window.


However, for the time being, I think it would be sufficient to do the following:


Verification Successful - pop a message in XY's status bar saying "Verification Successful".


Verification Failed - pop a dialog box, indicating that verification failed, showing a log of failed files/directories (along with the reason why they failed), and having the following buttons:

1. Retry failed files

2. Save log of failed files

3. Accept current state

[Mesh]

Conc. the Queue toolbar icon. If you got a better one...

Honestly, if you picked a nice font and just made the icon a blue letter "Q", I'd like that better than what it currently is.