Background processing doesn't always work

[yogi]

I'm on Win7 64-bit and I'm using XYplorer v9.00.0000.
I have set the two appropriate flags in the settings for 'background processing' and 'queue file operations' and now my long copies and moves do queue up.
Also, I am able to continue using XYplorer during copies without starting a new instance.

But it is still possible to start a copy/move that doesn't get executed in the background as I would have expected.
If I select-and-RIGHTMOUSE-drag my files the resulting action does not get executed in the background.

Overall I am very happy with this feature and I am curious how it will turn out in the future!

[admin]

What's the target of your drag operation? If it is another app or the Desktop then the target application will handle the file operation, not XYplorer.

[yogi]

The target of the drag operation is any directory in the XYplorer tree.

I have to admit that by now I have gotten attached to this loophole,
and I am actually using it to force a copy or move to execute immediately.

If (several) long copy-tasks are queued up, and you quickly need to copy
a small file you are working on, it is not convenient to wait for the rest
of the queue to complete before your small file gets copied.

So I hope that once XYcopy matures it will get the ability to pause/hold the
long queue while your small task gets executed immediately.

[nas8e9]

If I select-and-RIGHTMOUSE-drag my files the resulting action does not get executed in the background.

I'm afraid I can't reproduce this with 9.10.0017: when I right-click and drag a file to a directory in the tree, I get the pop-up menu. After selecting Copy, I see the XYcopy.exe-icon popping up; giving a second copy command does nothing until the first copy job is finished indicating backgrounding and queueing work as expected. Do you use a different right mouse action? Also, could you post your Windows and XYplorer version and if it's not the latest beta, could you try that one? (9.10.0017 at the moment.)

Edited to add:

I have to admit that by now I have gotten attached to this loophole,
and I am actually using it to force a copy or move to execute immediately.

If (several) long copy-tasks are queued up, and you quickly need to copy
a small file you are working on, it is not convenient to wait for the rest
of the queue to complete before your small file gets copied.

So I hope that once XYcopy matures it will get the ability to pause/hold the
long queue while your small task gets executed immediately.

It's possible to control whether a job gets queued by adding the Queue File Operations button to the toolbar. By depressing it, jobs started after that are handled immediately by a separate instance of XYcopy instead of waiting to be queued. Pressing the button again makes jobs started after that become queued again. Also, deletes are never queued and are always immediately executed (by a separate instance of XYcopy.exe if backgrounding is enabled).

[yogi]

I'm afraid I can't reproduce this with 9.10.0017: when I right-click and drag a file to a directory in the tree, I get the pop-up menu. After selecting Copy, I see the XYcopy.exe-icon popping up; giving a second copy command does nothing until the first copy job is finished indicating backgrounding and queueing work as expected. Do you use a different right mouse action? Also, could you post your Windows and XYplorer version and if it's not the latest beta, could you try that one? (9.10.0017 at the moment.)

Surprising, I did not expect this to be different for other users. I'll provide more details.
First of all, I'm a new user since V9 coming from DirectoryOpus and I am still evaluating.
Being a unregistered user I don't have access to the 9.00.0017 latest beta.
I have updated to 9.00.0010 and this did not change the issue.
The Windows version I am using is Win 7 64-bit Professional, the 7600 build with available
patches and updates.

When I right-click and drag a file to a directory I also get the popup. Selecting 'Copy here'
there is *NO* child-process XYcopy. The copy-progress-dialog that pops up in this situation
blocks me from using XYplorer until the copy has finished.

A normal copy does create the XYcopy process you mentioned. I checked both situations for the XYcopy.exe
process in Process Explorer, the right-click-drag does not show it while a normal copy does.

It's possible to control whether a job gets queued by adding the Queue File Operations button to the toolbar. By depressing it, jobs started after that are handled immediately by a separate instance of XYcopy instead of waiting to be queued.

True (and I had not discovered that yet), but still this requires two actions AND
it requires you to remember that a long copy is running in the background.
Ideally (imho!) XYplorer should allow the user to define a threshold for small tasks
that pause the running queue and get top priority. Other solutions might achieve
the same goal obviously.

Also, deletes are never queued and are always immediately executed (by a separate instance of XYcopy.exe if backgrounding is enabled).

Off topic, but immediately executing deletes is somewhat dangerous.
This way it is possible to delete files that are still queued for copying,
which I discovered myself recently
Ofcourse this is my own mistake but once I queue a task in my mind the
task has been executed and I just continue doing other things.

[nas8e9]

The Windows version I am using is Win 7 64-bit Professional, the 7600 build with available
patches and updates.

Same here.

I have updated to 9.00.0010 and this did not change the issue.

I've tried both 9.00.0010 as well as 9.00.0017 beta; both versions don't show your problem here.

When I right-click and drag a file to a directory I also get the popup. Selecting 'Copy here'
there is *NO* child-process XYcopy. The copy-progress-dialog that pops up in this situation
blocks me from using XYplorer until the copy has finished.

A normal copy does create the XYcopy process you mentioned. I checked both situations for the XYcopy.exe
process in Process Explorer, the right-click-drag does not show it while a normal copy does.

Again, can't reproduce, unfortunately.

It's possible to control whether a job gets queued by adding the Queue File Operations button to the toolbar. By depressing it, jobs started after that are handled immediately by a separate instance of XYcopy instead of waiting to be queued.

True (and I had not discovered that yet), but still this requires two actions AND
it requires you to remember that a long copy is running in the background.
Ideally (imho!) XYplorer should allow the user to define a treshold for small tasks
that pause the running queue and get top priority. Other solutions might achieve
the same goal obviously.

Don is planning to add intelligence with regard to when jobs are queued and when they can run in parallel. This is mostly to allow jobs having different targets (and therefore not impeding on each other's performance), to run simultaneously. I guess adding a size threshold could be made part of that. Also, there have been requests for real-time job reordering and broader, real-time job control, although apparently that's not that easy to implement. In summary, *something* should crop up to help automate and/or better control this.

Other solutions might achieve the same goal obviously.

So far Don has implemented out-of-order operations by starting an additional XYcopy.exe-instance. This obviates the need for the queue-instance of XYcopy.exe to stop what it is doing.

Also, deletes are never queued and are always immediately executed (by a separate instance of XYcopy.exe if backgrounding is enabled).

Off topic, but immediately executing deletes is somewhat dangerous.
This way it is possible to delete files that are still queued for copying,
which I discovered myself recently
Ofcourse this is my own mistake but once I queue a task in my mind the
task has been executed and I just continue doing other things.

I agree with you in principle. OTOH, *hopefully* you delete something that should indeed be deleted (and if not, there's always the recycle bin). When the queue encounters one or more items that are no longer there, it could either give you a harmless error message or silently continue.

Edited to add: I can however see potential concurrency issues given that parallel file operations of all types are now possible. I suspect Don wouldn't like to have to build a consistency/conflict-resolution engine to keep parallel operations from conflicting, but on paper...

[admin]

I agree fully with nas8e9's last post. I checked "Copy Here" and it works as expected. I don't see a possible way how this could not work. Strange!

...
Edited to add: I can however see potential concurrency issues given that parallel file operations of all types are now possible. I suspect Don wouldn't like to have to build a consistency/conflict-resolution engine to keep parallel operations from conflicting, but on paper...

Right, I would rather leave this responsibility to the user. I also cannot really see why you'd add jobs to the queue and later decide to reorder the jobs. I would not like to support this kind of confused user behavior...
One should also keep in mind that the Status Log is not live. It just gives you a snapshot of the situation in the moment you opened the log. So looking at the log and then reordering jobs could be already too late...

[nas8e9]

...
Edited to add: I can however see potential concurrency issues given that parallel file operations of all types are now possible. I suspect Don wouldn't like to have to build a consistency/conflict-resolution engine to keep parallel operations from conflicting, but on paper...

Right, I would rather leave this responsibility to the user. I also cannot really see why you'd add jobs to the queue and later decide to reorder the jobs. I would not like to support this kind of confused user behavior...

I can do confused as well as anyone . Basically, the Windows Shell file functions will error out when jobs "compete" for the same item(s), so yeah...

One should also keep in mind that the Status Log is not live. It just gives you a snapshot of the situation in the moment you opened the log. So looking at the log and then reordering jobs could be already too late...

I think when combining zer0's request (mentioning huge, long-running jobs) with yogi's (small jobs to be handled immediately), I do think there is a case to be made for either automatically and/or manually changing the order of file operations in addition to switching auto-queue on/off through the toolbar button. I gather that manual control (especially pausing) of already running jobs is difficult, but would the following two things be possible?

1. Re-ordering of not yet started queue jobs.
2. Automatically handling copy/move operations below a certain count/size threshold in parallel to already running operations. (Possibly tying in with the planned target intelligence.)

[admin]

...
Edited to add: I can however see potential concurrency issues given that parallel file operations of all types are now possible. I suspect Don wouldn't like to have to build a consistency/conflict-resolution engine to keep parallel operations from conflicting, but on paper...

Right, I would rather leave this responsibility to the user. I also cannot really see why you'd add jobs to the queue and later decide to reorder the jobs. I would not like to support this kind of confused user behavior...

I can do confused as well as anyone . Basically, the Windows Shell file functions will error out when jobs "compete" for the same item(s), so yeah...

One should also keep in mind that the Status Log is not live. It just gives you a snapshot of the situation in the moment you opened the log. So looking at the log and then reordering jobs could be already too late...

I think when combining zer0's request (mentioning huge, long-running jobs) with yogi's (small jobs to be handled immediately), I do think there is a case to be made for either automatically and/or manually changing the order of file operations in addition to switching auto-queue on/off through the toolbar button. I gather that manual control (especially pausing) of already running jobs is difficult, but would the following two things be possible?

1. Re-ordering of not yet started queue jobs.
2. Automatically handling copy/move operations below a certain count/size threshold in parallel to already running operations. (Possibly tying in with the planned target intelligence.)

1. principally possible of course, but to be as good as possibile it would need the status log to be live which is a bit of work for me...
2. also possible, but calculating the size of a job can take a lot of time with deeply nested folders... so one must find a way to define a threshold that avoids getting into a huge calculation effort...

[nas8e9]

1. principally possible of course, but to be as good as possibile it would need the status log to be live which is a bit of work for me...

Currently it's a static list. Would adding a refresh button with the re-order buttons be a pragmatic stop-gap solution?

2. also possible, but calculating the size of a job can take a lot of time with deeply nested folders... so one must find a way to define a threshold that avoids getting into a huge calculation effort...

Perhaps limiting it to (a certain count/size of) files only, with any number of folders included disqualifying it for immediate execution?

Edited to add: I do appreciate that the above suggestions are stop-gap at best and do still entail work for you. In terms of priorities, I'd guess that the first item would be the most appreciated by many (well, several ).

[admin]

1. principally possible of course, but to be as good as possibile it would need the status log to be live which is a bit of work for me...

Currently it's a static list. Would adding a refresh button with the re-order buttons be a pragmatic stop-gap solution?

2. also possible, but calculating the size of a job can take a lot of time with deeply nested folders... so one must find a way to define a threshold that avoids getting into a huge calculation effort...

Perhaps limiting it to (a certain count/size of) files only, with any number of folders included disqualifying it for immediate execution?

Edited to add: I do appreciate that the above suggestions are stop-gap at best and do still entail work for you. In terms of priorities, I'd guess that the first item would be the most appreciated by many (well, several ).

I'll think about it. There are some other things before XYcopy 2.0...