It turned out that my MaxiTreePaths pointing to my network drive contributed to the freezing. I think it wasn't any path in particular, but the number of paths. The more MaxiTreePaths were pointing to my network drive, the longer the freeze would be. Experimenting with the number of paths, I got ~0.08s per network path.
Here are all the settings (so far) that need to exist in XYplorer.ini to causes freezes after moving/deleting files/folders:
RestoreMaxiTree=1
MaxiTreePaths="paths pointing to a network drive; ~0.08s freeze per path"
EnableCustomFileIcons=1
*\ /e><xypath>\XYicon_FolderEmpty.ico
must be enabledCBFINetwork=1