Timeline for Saving blend-files straight to server nets 0.5% speed of a windows file copy, how to fix?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 29, 2019 at 13:34 | comment | added | Wouter Vandenneucker | While other protocols would be a lot faster; my goal is to enable way faster transfers without the need to change infrastructure. This is something that a developer can do without needing support from a sysadmin. | |
| Nov 29, 2019 at 12:18 | comment | added | Moog | I have always found Windows copy over network is extremely slow for large files in general. It seems to do a lot of background stuff that slows it down. You might find that some other file transfer protocol is much faster. Make sure that the target drive is either RAID or completely defragmented. Don't rely on python for the transfer, it just uses windows under the hood. You might get more performancefrom a NAS with SCP/FTP support for upload that you can access as a share for later access. | |
| Nov 29, 2019 at 9:25 | vote | accept | Wouter Vandenneucker | ||
| Nov 29, 2019 at 9:24 | answer | added | Wouter Vandenneucker | timeline score: 3 | |
| Sep 5, 2019 at 15:45 | comment | added | Wouter Vandenneucker | I actually tried that in the meantime! for the 2GB file it took 4 minutes and 12 seconds and resulted in a file of 1.1GB (35Mbps) which is already a lot better, but still only 4% of our available network speed... | |
| Sep 5, 2019 at 15:29 | comment | added | Robert Gützkow | Might be worth trying the Compress File option. This will have an overhead for the compression, but perhaps it writes bigger chunks of data at once. It's just a guess, I haven't looked at the source for these operations. | |
| Sep 5, 2019 at 15:19 | history | edited | Wouter Vandenneucker | CC BY-SA 4.0 |
added my progress so far
|
| Aug 27, 2019 at 12:09 | comment | added | Wouter Vandenneucker | I couldn't find it back then; decided I'd wait for 2.8 before trying again... Might try it again, but that would be next week at its earliest. | |
| Aug 27, 2019 at 12:07 | comment | added | Duarte Farrajota Ramos♦ | I guess it would have to be done in a separate processing thread to release Blender, so it is executed as a background tasks and Blender isn't waiting for the tasks to complete. This is a wild speculation on my part though, not sure this is possible, I'm not a coder myself. Maybe it would have to use a separate python installation | |
| Aug 27, 2019 at 12:00 | history | tweeted | twitter.com/StackBlender/status/1166319586381312000 | ||
| Aug 27, 2019 at 12:00 | comment | added | Wouter Vandenneucker | Hey Duarte, thanks for your input! Funny thing is that I already have a script that does that (somewhat). Since a couple of months we offload final renders to a render server, we wrote our own plugin for that who also tweaks the file (force append all, zip the file afterwards, then upload it to the server using a http file transfer). It still hangs Blender for several minutes. That script uses requests.post(). I thought it was slow because of the recieving server, but I'm not sure at this point. | |
| Aug 27, 2019 at 11:49 | comment | added | Duarte Farrajota Ramos♦ |
Blind guess: saving files is somehow "progressive" and saves in several "writing stages"; I noticed saving to a slow USB thumb drive is equally painful. Not sure there is any realistic workaround. Wild idea: create some sort of addon that saves the file locally first (like some sort of local caching to temp for example), then background copies to the final network destination
|
|
| Aug 27, 2019 at 10:08 | comment | added | Wouter Vandenneucker | Crossposted this as a bug on: developer.blender.org/T69206 | |
| Aug 27, 2019 at 8:43 | history | asked | Wouter Vandenneucker | CC BY-SA 4.0 |