3
$\begingroup$

I've run into a rather irritating issue when dealing with a somewhat computationally complicated scene (attached below) that is essentially just a pile of volumetrics and lighting.

Whenever I set my project to GPU Compute mode (and set the tile settings to optimal for that), I run the risk of crashing my computer the moment I hit Render. I've Googled around for this issue, only retrieving responses for "Display Driver has stopped responding" messages and black image outputs during rendering. In my case, my displays just turn off after displaying "No Signal". It does not happen EVERY time, though, and I'm having trouble pinning down the trigger.

I've given it 5 minutes of black screen in hopes that it would come back after the scene finishes rendering (in the attached, the current settings usually wrap up after about 2 minutes on my GPU). I have also checked every optimization setting I could find in the project in hopes of alleviating this to no avail.

This has not happened yet in a simple scene. I am exceedingly new to Blender so I can only assume that I've done something terrible to my setup here. However, I would also not be averse to any sort of GPU version of Memtest that I have not tried, in case it's something hardware-related.

CPU Compute mode is not really an option for me, my aging i7 takes almost 30 hours to complete rendering at 1920x1080 (16x16 tiles, path tracing, 7 threads so I can still use the computer, 576 samples after hitting "Square Samples").

Specs:
- Blender Version: 2.77a
- OS: Windows 7 Ultimate x64
- CPU: Intel i7-2600 @ 3.4Ghz
- GPU: NVidia GeForce GTX 780 (3GB)
- RAM: 16GB DDR3 1600

Project Files: https://dl.dropboxusercontent.com/u/39096583/Blender.zip

P.S. Thank goodness for RAID1: every time this issue happens, I get notified that the array needs to be verified and repaired. Feeling good about my choice to go with that over RAID0 or a JBOD...

Improve this question
$\endgroup$
14
  • 1
    $\begingroup$ Since smaller scenes work, could your GPU have not enough memory for that complicated scene? I have no GPU but I know all scene has to be stored in faster, but smaller GPU memory, or it cannot be rendered there... You could try somewhat splitting the work in smaller parts and rendering them separately, then compositing them back later. $\endgroup$
    – m.ardito
    May 23, 2016 at 7:27
  • $\begingroup$ I thought about that, but the scene data never indicates that it's exceeding 50 MB of RAM usage. Is there a way to directly check how much the raw scene data really takes up, if the "Mem" space is inaccurate? Also, I'm not sure splitting up the image would work all that well. Eventually this is supposed to be made into a skybox object (reflective cube, baked textures) so the compositor won't be of use to me (unless I'm missing something on that), and I'm trying to get cloud self-shadowing to play a part in the scene dynamics, so I'm not sure how to split without breaking shadows. $\endgroup$
    – Steve Green
    May 23, 2016 at 7:37
  • $\begingroup$ Update: Ran GPU-Z while rendering (this time successfully, still can't find the trigger) and the scene data pushes the GPU memory load up by almost exactly 1 GB. I'll continue to run tests to see if I can get this to black-screen and what the memory load gets to at that point... $\endgroup$
    – Steve Green
    May 23, 2016 at 8:10
  • $\begingroup$ The scene renders fine on my system, even older i7, same RAM and dual GTX 780 on Windows 10. $\endgroup$
    – user1853
    May 23, 2016 at 14:55
  • $\begingroup$ I think the dual 780s might be the cinch here, that or Win10 handling errors better than whatever the heck my computer is doing. $\endgroup$
    – Steve Green
    May 23, 2016 at 15:21

3 Answers 3

Reset to default
2
$\begingroup$

It's common for this problem to be caused by running out of VRAM on your card.

One way to alleviate the problem is to reduce the tile size. Some of the things in a Blender scene can be swapped to and from the GPU on a per-tile basis. In those cases, a smaller tile size means less data on the GPU.

Other things will be the same size on the GPU regardless of which tile is being rendered, or how big the tile is. Hopefully, that overhead is small enough that reducing the tile size will help.

Cetagon's suggestions are also worth checking out.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I will definitely give your and Cetagon's advice a try. However, running GPU-Z as mentioned in the other post shows that VRAM usage is not changing between tile sizes, whatever that bit of information is worth. Whatever the case may be, it definitely seems to be instantiation based as it happens just after the data is sent to the GPU, never mid-render. $\endgroup$
    – Steve Green
    May 23, 2016 at 15:35
  • $\begingroup$ If Blender is requesting more VRAM than is available, that RAM will never be allocated. E.g. if Blender is using 50% of your VRAM, and then for another stage of the process Blender asks for a chunk that's 60% of total VRAM, then the driver will deny the request, Cycles will crash, the driver will crash, and you'll never see the VRAM increase. I agree with your suspicion that it's NOT vram related, but I'd keep that in the back of your mind, because it's not totally out of the question. $\endgroup$
    – Matt
    May 23, 2016 at 15:39
  • $\begingroup$ Yeah, but what's really strange is that the only info I can find on this is that Blender gives an "Out of Memory" error on the info bar, or Windows displays a "Display Driver was Unresponsive" message. Also my scene is (currently) a 2-pass render, one for the scene and one for the star alone for post processing needs. While watching GPU-Z during a successful preview-level render, the RAM usage did not change until the render completed. $\endgroup$
    – Steve Green
    May 23, 2016 at 15:53
0
$\begingroup$

Download an app to monitor GPU usage (I use MSI afterburner) to see what is happening with your vRAM

If this is the only GPU on the system close down all other apps.

Make sure the power supply is adequate for your GPU.

While rendering, GPUs can get quite hot. Ensure you have adequate cooling on your system.

On the display section of the render settings set it to Keep UI (so that the image being rendered doesn't display as it is being rendered)

Looking at your file I think there are a few things you can do to optimize your scene:

Reduce the number of subdivisions for subsurf. Read this post: https://blender.stackexchange.com/a/15667/1853

In the performance section disable Use Spatial Splits and Persistent Images

Improve this answer
$\endgroup$
5
  • $\begingroup$ I ran GPU-Z until I could get the issue to occur. VRAM on a successful run peaked out at 1.6 / 3 GB usage (baseline 400mb), and stayed at 1.5-ish between runs. Interestingly, my RAM usage dropped moments before the issue occurs, alluding to maybe the program failing to clear up resources properly. Temperature never exceeds 65°C. I'll definitely try out those optimizations as well, though the subsurf is giving my clouds most of the smooth detailing, not sure what that'll do at a lower setting. $\endgroup$
    – Steve Green
    May 23, 2016 at 15:22
  • $\begingroup$ Also my system is currently running a mid-tier 1200W power supply, which should be enough to run tri-SLI at full blast on this particular set of cards if my math is correct. $\endgroup$
    – Steve Green
    May 23, 2016 at 15:29
  • $\begingroup$ Unfortunately, wattage might not be enough :-/ On my current build, I had serious problems because the MAX amperage on my EATX rail was sufficient, but the continuous amperage wasn't high enough, to make matters worse, the continuous amperage rating wasn't reported anywhere :-( If you haven't been having any other problems with the card, then this probably isn't the problem, but it's worth at least giving a passing thought. $\endgroup$
    – Matt
    May 23, 2016 at 15:36
  • $\begingroup$ Sadly, the issue still occurs after these changes. I pushed subsurf down, but only 1 level on the particle clouds as anything less just removes all detail and my clouds look flat. @Matt - It would be kinda hard for me to test that, I can't afford to get a newer processor/GPU, let alone an electrician's hardware kit :P. And yeah, I have not noticed any other issues in the gaming that I do, and the worst that GPU-Z outputs is PerfCap: Pwr which just means that it reached the target power level for that clock. $\endgroup$
    – Steve Green
    May 23, 2016 at 15:56
  • $\begingroup$ So here's a bit of information that may or may not be the ACTUAL cause: I've noticed that this is a lot more common when I've changed a render setting, be it max bounces, tile size, sample rate, or what have you. Anything in your guys' knowledge base that might fit the bill? $\endgroup$
    – Steve Green
    May 23, 2016 at 16:07
0
$\begingroup$

During last two weeks I spent hours to solve crash problem with cycles rendering on my pc. Whatever I had been trying - nothing helped! I had usual 'cuSynchronize' and 'cuMemFree' or something like that which are well discussed on blender support forums but nothing helped. I played with nodes and even removed some shaders, I played with scene render triggers, with render setup and a lot of other things including Windows 7 regedit TrdDelay and Nvidia driver setup.

During two weeks each second render in 5-15 minutes I had crash and I wrote special script to continue baking from mesh where it had crashed on. So it was a disaster.

After all I decided to add radeon card to split display and rendering facilities. Just after that I got couple of hours of rendering before next crash. And that was a good achievement meaning all before. Then I paid an attention on GPU temperatures - I use multi GPUs so they obstruct each other from fresh air as stay close in pci-e ports and to install another card I used raiser and moved one card away opening fresh air for others. In addition I played with nvidia overclocking switches and reduced GPUs nominal working temperature.

I am not sure that was the real solution in my case whether display and render separation helps or temperature - I didn't want to repeat all of that but could say that just by adding separate card for display and reducing temperature of GPUs from 80-87 to 75-77 solved the case I had with Blender rendering on GPUs. Also I mentioned that TdrLimit doesn't work well and do not prevent driver reloading. Finally I put all DWORDS I have found on Microsoft site putting up to 1800 seconds to TdrLimit - I anyway had crashes. Finally I changed tiles size from 2048 to 1024 and then to 512 to ensure that any heavy samples will be rendered in 10-15 seconds. Hope it will also add stability.

Improve this answer
$\endgroup$
1
  • $\begingroup$ My issue had nothing to do with heating, as the crash happens seconds after hitting "Render". Load temps on my card struggle to breach 65°C even on Furmark (I got it up to 71°C only after an hour of running). Also, I still have yet to find a solution to this issue, in case anyone is wondering. I've just kinda given up on the project until I can get myself a shiny new NVidia 1080. $\endgroup$
    – Steve Green
    Sep 15, 2016 at 2:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .