File xfer (multiple or single) over subnet

[stephr1]

First off, XYplorer is great. I've gotten to the point where I almost never (well, mostly almost use Win File Explorer anymore.

I have noticed something that I'm not sure is a bug, a design "consideration" or an option I have yet to set. I'm on Win 10 Pro, latest version/updates installed (but this has been going for a much longer time back).

When I xfer files around and within my Win 10 system, it seems to work fine. Good xfer speed, etc. However, if I try to transfer across my subnet (in-home network), I occasionally see issues. The prime example is that I have a RPI 3-based Kodi HTPC system set up. When I try to xfer files from my Win 10 to the Kodi sys, if it's a single file, or a file that's not too large, it mostly does OK. Multiple files and files larger than, say, 1GB, cause the xfer to stall (I can watch the speed slow down) and most of the time XYplorer shows "Not responding". It can also happen if I'm working with other programs at the same time I'm xfer'ing files.

Many times, tho not always, when this happens, I have to go downstairs and reset the Kodi system and start things back up. Since this has been going on for quite some time, I just figured the glitches were being caused by the Kodi system (I did have a not-so-reliable PS powering the RPI 3)...which is why I hadn't written before today. Yesterday, after having yet another problem with file xfers, I decided to experiment, so to speak. I used Win File Explorer to xfer the same files that had just failed to xfer with XYplorer.

The results kind of surprised me for 2 reasons. Firstly, I xfered 4 large (2GB+) files and they xfered fast and without any problems. Secondly, I noticed that the average speed using Win File Explorer was higher than that of XYplorer by a bit over 1MB/sec (~11.6 MB/S for File Explorer, ~10.3 MB/S for XYplorer). Both my network (which uses CAT 5E cable) and my Win 10 sys are good for 1 GB LAN connectivity. Alas, the RPI 3 is only good for 100 MB/S...and maybe that's on good days I may assume the speed difference between the 2 programs could be due to how network connections and/or protocols might be handled in the coding. However, for XYplorer to go non-responsive is problematic.

Any ideas or suggestions? Is there an option I need to select to fix this?

Thanks in advance....

[stephr1]

Bump!!

There is no reason to fullquote yourself -> Removed

[stephr1]

Apologies for that. I'll mind my manners better. Thx.

There is no reason to fullquote yourself -> Removed[/quote]

[admin]

Are you using Custom Copy? (Configuration | File Operations | File Operations | Custom Copy Operations | Use Custom Copy)

If yes, you might check out this recent tweak (you need at least v19.10.0201 for it):

Code: Select all

v19.10.0201 - 2018-08-17 11:29
    + Custom Copy: Added tweak CustomCopyNoBufferingFromBytes to allow 
      unbuffered file copying for large files. It avoids the file system cache 
      overhead and prevents the file system cache from being effectively flushed 
      by large file data.
      Unbuffered file copying is recommended by Microsoft for "very large file 
      transfers" without getting more specific. So it's up to you to define the 
      threshold when it should kick in.
      For example, to invoke unbuffered copying for files of 1 GB or larger:
        CustomCopyNoBufferingFromBytes=1073741824
      Notes:
      - The underlying technology is supported only from Windows Vista / Server 
        2008 onwards. Windows XP / Server 2003 are out.
      - Move operations don't support unbuffered processing. For unknown reasons 
        the Windows API doesn't offer it. It's only there for Copy operations.
      - Unbuffered copying is also supported in XYcopy (= background processing).
      - Quite likely the tweak will later be lifted to the Configuration dialog.

How to tweak. https://www.xyplorer.com/faq-topic.php?id=tweak

[stephr1]

Appreciate the response.

I checked and Custom Copy is enabled (tho, I don't specifically remember doing so).

I followed the FAQ about tweaking and, sure enuf, that tweak doesn't exist in my version (19.10.0200). Can I add that tweak and have it work? Or do I need to download 19.10.0201 (sounds like a "nightly release") and how do I get access to it? Or do I just have to wait?

Thx and cheers....

Are you using Custom Copy? (Configuration | File Operations | File Operations | Custom Copy Operations | Use Custom Copy)

If yes, you might check out this recent tweak (you need at least v19.10.0201 for it):

Code: Select all

v19.10.0201 - 2018-08-17 11:29
    + Custom Copy: Added tweak CustomCopyNoBufferingFromBytes to allow 
      unbuffered file copying for large files. It avoids the file system cache 
      overhead and prevents the file system cache from being effectively flushed 
      by large file data.
      Unbuffered file copying is recommended by Microsoft for "very large file 
      transfers" without getting more specific. So it's up to you to define the 
      threshold when it should kick in.
      For example, to invoke unbuffered copying for files of 1 GB or larger:
        CustomCopyNoBufferingFromBytes=1073741824
      Notes:
      - The underlying technology is supported only from Windows Vista / Server 
        2008 onwards. Windows XP / Server 2003 are out.
      - Move operations don't support unbuffered processing. For unknown reasons 
        the Windows API doesn't offer it. It's only there for Copy operations.
      - Unbuffered copying is also supported in XYcopy (= background processing).
      - Quite likely the tweak will later be lifted to the Configuration dialog.

How to tweak. https://www.xyplorer.com/faq-topic.php?id=tweak

[jupe]

You would need to download a newer beta, from here:

viewtopic.php?f=1&t=4&sd=d

Or hold Ctrl button down and click on this menu item:

Help | Online Support | Check for Updates

PS: you will also have to save you settings once after upgrading, for the ini key to become visible.

[stephr1]

Great and thx.

I d/l'd and installed (V19.10.0209 beta). Since I used the noinstall version, that did not affect the XYplorer.ini file so I added the tweak as suggested (under the "[Custom Copy]" header). If for some reason I decide to go back to 19.10.0200 and I leave the tweak, will that cause problems, or will XYplorer just ignore it?

I did a xfer of 6 files (1 @ 1GB+ and 5 @ 500MB-870MB) both to and from my Kodi/RPI 3 over my subnet. Results are definitely better and no crash on the Kodi system (yay!!! What I did notice is that from my Win 10 sys to the Kodi sys xfer was ~10.3 MB/S and in the other direction it was ~11.2 MB/S. Not quite as speedy as using MS File Explorer (@~11.6 MB/S) but fast enuf for me, and the reliability is now acceptable.

As I get the chance, I'll play around with this version and let you know if I experience something you should know about.

Thx again for the help and support. Cheers.....

You would need to download a newer beta, from here:

viewtopic.php?f=1&t=4&sd=d

Or hold Ctrl button down and click on this menu item:

Help | Online Support | Check for Updates

PS: you will also have to save you settings once after upgrading, for the ini key to become visible.

[admin]

Thanks for the report! Actually that was the first report I got on that new feature.

...will that cause problems, or will XYplorer just ignore it?

It will ignore it and forget it on next save.

[stephr1]

You're quite welcome and glad to help!!

I wanted to clarify about my file xfer test....I didn't xfer the files 1 @ a time (click/copy/paste) and that typically wasn't the problem if the file wasn't too big (i.e. <1 GB?). I clicked on all 6 then did a Copy/Paste so that all 6 files were queued up for xfer. That was when things typically went "non-responsive". On my test I even did some other xfers at the same time from the web and local to my Win 10 sys (which also caused problems) and speed remained good and no glitches. However, I'll keep an eye on things and let you know if something comes up.

Again, truly appreciate the help and fast response!!

Thanks for the report! Actually that was the first report I got on that new feature.

...will that cause problems, or will XYplorer just ignore it?

It will ignore it and forget it on next save.