XYplorer Beta Club

admin wrote:...

If background processing is enabled:
- Copy and move commands are queued IF queuing is enabled, else they are spawned to separate background processes.
- Delete operations are always spawned to separate background processes.

Powerful stuff! At some point, some intelligence about when to go parallel (HD1 > HD2, HD1 > USB1) vs. queued would be even better, but I should leave something for the XYcopy 2.0 wish thread, of course...

nas8e9 wrote:

admin wrote:...

If background processing is enabled:
- Copy and move commands are queued IF queuing is enabled, else they are spawned to separate background processes.
- Delete operations are always spawned to separate background processes.

Powerful stuff! At some point, some intelligence about when to go parallel (HD1 > HD2, HD1 > USB1) vs. queued would be even better, but I should leave something for the XYcopy 2.0 wish thread, of course...

Yep, cool. This will be, BTW, very easy to do. The harder stuff is the live progress GUI which is one of the next things I'll do. I'll just might first squeeze in the TXT preview just to get into the coding habit again after 2 weeks of full English breakfast...

admin wrote:...The harder stuff is the live progress GUI which is one of the next things I'll do...

If I may cheekily squeeze in a wish there -- an ability to move queued jobs up/down the order. As someone who moves gigabytes of data between partitions/HDDs on a daily basis, the order in which files (or batches of files) arrive to their destination is pretty important to me.

admin wrote:If background processing is enabled:
- Copy and move commands are queued IF queuing is enabled, else they are spawned to separate background processes.
- Delete operations are always spawned to separate background processes.

I forgot to ask about intra-partition moves: I assume that, like deletes, they are always spawned to separate background processes and are never queued?

Also, do XYcopy.exe instances handle more than one queue item? By way of an example, when there are two copy jobs in the queue, are they handled (sequentially, of course) by one and the same XYcopy.exe instance or does each queue item always spawns a new XYcopy.exe process?

nas8e9 wrote:Also, do XYcopy.exe instances handle more than one queue item? By way of an example, when there are two copy jobs in the queue, are they handled (sequentially, of course) by one and the same XYcopy.exe instance or does each queue item always spawns a new XYcopy.exe process?

That's easy to test. Disable queuing, fire up some inter-partition copies and check out Task Manager's process list

zer0 wrote:

nas8e9 wrote:Also, do XYcopy.exe instances handle more than one queue item? By way of an example, when there are two copy jobs in the queue, are they handled (sequentially, of course) by one and the same XYcopy.exe instance or does each queue item always spawns a new XYcopy.exe process?

That's easy to test. Disable queuing, fire up some inter-partition copies and check out Task Manager's process list

*Disable* queueing?

For the record, I only see one XYcopy.exe process being spawned to process a 10-item queue, so one XYcopy.exe instance apparently handles more than one queue item.

zer0 wrote:

admin wrote:...The harder stuff is the live progress GUI which is one of the next things I'll do...

If I may cheekily squeeze in a wish there -- an ability to move queued jobs up/down the order. As someone who moves gigabytes of data between partitions/HDDs on a daily basis, the order in which files (or batches of files) arrive to their destination is pretty important to me.

What about triggering the jobs in the order you want them right away?

nas8e9 wrote:

admin wrote:If background processing is enabled:
- Copy and move commands are queued IF queuing is enabled, else they are spawned to separate background processes.
- Delete operations are always spawned to separate background processes.

I forgot to ask about intra-partition moves: I assume that, like deletes, they are always spawned to separate background processes and are never queued?

Also, do XYcopy.exe instances handle more than one queue item? By way of an example, when there are two copy jobs in the queue, are they handled (sequentially, of course) by one and the same XYcopy.exe instance or does each queue item always spawns a new XYcopy.exe process?

1. intra-partition moves are never handled by XYcopy but just done by the XYplorer process.
2. Yes, by one and the same XYcopy.exe. When a job is finished XYcopy asks XYplorer if there are more jobs pending and if yes it gets fed the next job right away without terminating/restarting a new XYcopy process.

admin wrote:1. intra-partition moves are never handled by XYcopy but just done by the XYplorer process.

This was the original design for deletes as well, which you changed to accommodate much larger than average deletes. By the same logic, would it be an idea to farm out intra-partition moves to background processing as well?

nas8e9 wrote:

admin wrote:1. intra-partition moves are never handled by XYcopy but just done by the XYplorer process.

This was the original design for deletes as well, which you changed to accommodate much larger than average deletes. By the same logic, would it be an idea to farm out intra-partition moves to background processing as well?

I don't see a reason for this. These are just modifications of the file tables, no bytes are moved (and no slow DB is updated as is the case with recycle-deletes). They must be fast.

admin wrote:

nas8e9 wrote:

admin wrote:1. intra-partition moves are never handled by XYcopy but just done by the XYplorer process.

This was the original design for deletes as well, which you changed to accommodate much larger than average deletes. By the same logic, would it be an idea to farm out intra-partition moves to background processing as well?

I don't see a reason for this. These are just modifications of the file tables, no bytes are moved (and no slow DB is updated as is the case with recycle-deletes). They must be fast.

I wasn't aware that recycling files has a much larger overhead. Fine with me.

admin wrote:

nas8e9 wrote:

admin wrote:1. intra-partition moves are never handled by XYcopy but just done by the XYplorer process.

This was the original design for deletes as well, which you changed to accommodate much larger than average deletes. By the same logic, would it be an idea to farm out intra-partition moves to background processing as well?

I don't see a reason for this. These are just modifications of the file tables, no bytes are moved (and no slow DB is updated as is the case with recycle-deletes). They must be fast.

I see a reason for it - you've obviously never had to work with Sophos Antivirus installed It makes even intra-partition moves slow. I know it shouldn't, but I think it check every single file you're "moving" and hence stuff like that can take an age.

I hate Sophos, but have no choice (at work) in having it removed.

lukescammell wrote:

admin wrote:

nas8e9 wrote:

admin wrote:1. intra-partition moves are never handled by XYcopy but just done by the XYplorer process.

This was the original design for deletes as well, which you changed to accommodate much larger than average deletes. By the same logic, would it be an idea to farm out intra-partition moves to background processing as well?

I don't see a reason for this. These are just modifications of the file tables, no bytes are moved (and no slow DB is updated as is the case with recycle-deletes). They must be fast.

I see a reason for it - you've obviously never had to work with Sophos Antivirus installed It makes even intra-partition moves slow. I know it shouldn't, but I think it check every single file you're "moving" and hence stuff like that can take an age.

I hate Sophos, but have no choice (at work) in having it removed.

I saw an additional reason: Isn't it a bit strange/confusing that intra-volume moves are missing from the (Background) Status Log? I could easily generate Huh?-moments with users who are not fully aware of the backgrounding conditions.

Another possibility would be to make it optional:

Background the following operations:
[ ] Copy
[ ] Cross-volume move
[ ] Intra-volume move
[ ] Delete
[ ] Backup

admin wrote:I saw an additional reason: Isn't it a bit strange/confusing that intra-volume moves are missing from the (Background) Status Log? I could easily generate Huh?-moments with users who are not fully aware of the backgrounding conditions.

Another possibility would be to make it optional:

Background the following operations:
[ ] Copy
[ ] Cross-volume move
[ ] Intra-volume move
[ ] Delete
[ ] Backup

I'm a bit confused about the function of the Status Log vs. the Action Log. All file operations are logged by the Action Log, but only background operations are logged in the Status Log. When all file operations are backgrounded, both logs will log all. Other than for diagnostic purposes, what does the Status Log add? Will the Status Log become the queue status window?

I do like the backgrounding of intra-partition moves, but I'm not sure why people would want to disable backgrounding.

nas8e9 wrote:

admin wrote:I saw an additional reason: Isn't it a bit strange/confusing that intra-volume moves are missing from the (Background) Status Log? I could easily generate Huh?-moments with users who are not fully aware of the backgrounding conditions.

Another possibility would be to make it optional:

Background the following operations:
[ ] Copy
[ ] Cross-volume move
[ ] Intra-volume move
[ ] Delete
[ ] Backup

I'm a bit confused about the function of the Status Log vs. the Action Log. All file operations are logged by the Action Log, but only background operations are logged in the Status Log. When all file operations are backgrounded, both logs will log all. Other than for diagnostic purposes, what does the Status Log add? Will the Status Log become the queue status window?

I do like the backgrounding of intra-partition moves, but I'm not sure why people would want to disable backgrounding.

Action Log and Status Log both can be turned off so they have to be kept independent. Sure they overlap in parts, but internally they are different things for different purposes. "Status Log" might not be a good name for it and add to the confusion.

The Status Log is your diagnostic interface to the job status of XYcopy. I might add some interaction here later (pausing, deleting, reordering jobs) -- not sure yet.

The live progress dialog will be a different window that's directly owned by XYcopy.