Author Topic: Animation noise  (Read 836 times)

Offline rolland1013

  • Member
  • *
  • Posts: 28
Animation noise
« on: June 19, 2017, 09:41:19 PM »
Hi all,

This is my first post here, so bare with me as I'm 1000% sure this topic has been covered before.  I'm still in the process of learning Terragen (version 4) and have started doing some animations.  I've created a desert scene with a basic camera move on it.  The foreground terrain and color map was created in World Machine, the rest in Terragen.  As you can see there is popping going on in the clouds, shadows and several other places.  My question is; how do I eliminate this?  I've written out a GI cache file for a previous animation, but it took 2 days to write out 10 frames.  Then it took 4 hours a frame to render at 960x540 resolution.  This will make it difficult to use in my work in television vfx.  The turn-around times wouldn't allow for revisions in many cases.  Typically I need to render my work in HD or 4K resolution for many of the shows I work on.

Also any other comments/suggestions on my work would be appreciated.

Thanks,
Niel

Offline dorianvan

  • Member
  • *
  • Posts: 374
    • Vandegrift Renderings
Re: Animation noise
« Reply #1 on: June 20, 2017, 01:54:48 AM »
Nice animation. Are you trying to do animation without caching this time? Rendering animation has to be done with cache-ing, as far as I know. One way to eliminate the render time of clouds is to use 3d camera tracking in post and add the clouds in that way. Or the background could be just a still plate rendered in TG, then use a 3d program like max/vray to render the imported geometry/map (no trees). Also, I generally render the cache at half my final output size and it seems to work good. For the render times, maybe share your settings to see what more you can do with the scene as it is.
-Dorian

Offline Dune

  • Member
  • *
  • Posts: 12729
  • Corkscrew Bird
    • www.ulco-art.nl
Re: Animation noise
« Reply #2 on: June 20, 2017, 06:19:32 AM »
Also, in a terrain like this, skip the compute terrain (check what it does to your landscape first); that often saves quite some time.
In case you still haven't seen enough of my work: www.ulco-art.nl

Offline Oshyan

  • Planetside Staff
  • *
  • Posts: 11745
  • Holy snagging ducks!
Re: Animation noise
« Reply #3 on: June 20, 2017, 11:28:50 PM »
You are seeing two sources of popping/flicker here.

In the clouds (more minor, in my view) you are seeing a need for higher cloud GI settings (GI in Clouds tab of GI Settings). We have some improvements in the upcoming 4.1 release which may also help.

On the ground, and more severe (again, in my view), you are seeing some shadow "popping". This will not be fixed by GI settings or caching, it is due to performance-related reduction of detail for geometry behind the camera which is casting shadows onto your terrain. The solution is to use Ray Detail Region Padding (on the Advanced tab of the Render node). In this case since the terrain casting shadows is directly behind the camera you may need to use "360 degree detail", but you can try with Detail in Crop Region or In Camera set at 0.25 or 0.5 to see if that helps with lesser render time than the full 360 Degree Detail.

As Dorian says, for GI caching you can often render the cache at a lower resolution than the final frames. Also the best strategy is to render a "sparse" cache, which means rendering every x frame. Usually rendering every 5th frame to a cache and then rendering with a blend of 3-5 caches works well.

More generally there may be other ways to optimize the render times you're seeing, although use of multiple v3 cloud layers simultaneously will definitely increase render times. Again in version 4.1 we will have some optimizations for this. If you are able to share your TGD files, or at least some of your render and cloud/atmosphere settings, we may be able to help reduce render times without losing quality. If you need to share a file confidentially you can do so to support AT planetside.co.uk.

- Oshyan

Offline rolland1013

  • Member
  • *
  • Posts: 28
Re: Animation noise
« Reply #4 on: June 23, 2017, 10:30:48 PM »
Thanks Dorian, Dune and Oshyan.

I've attached the TGD (there's no confidentiality issues with it).  But the World Machine generated .ter file and color map are over 500,000KB, and exceed your attachment limit.

Thx

Offline dorianvan

  • Member
  • *
  • Posts: 374
    • Vandegrift Renderings
Re: Animation noise
« Reply #5 on: June 24, 2017, 01:54:03 AM »
Try this one Rolland. Your AA needed to go up to 8 or so. In the GI Settings write a new cache. Go to Sequence/Output tab and hit Render Sequence. Then go back and read the GI Cache. Try blending every 3 frames, 5 or 6 if needed. Then render final frames. If you want you can render out just a crop to test how it looks.
Good luck.
-Dorian

Offline rolland1013

  • Member
  • *
  • Posts: 28
Re: Animation noise
« Reply #6 on: July 27, 2017, 10:51:46 PM »
Thanks again Oshyan, dorianvan and Dune. 

Your solutions worked.  I set Ray detail region to 360 degree detail, set the AA to 8, and created a sparse GI cache.  Sorry for the slow response, but wasn't able to implement all the suggestions until this week.

Niel

Offline Oshyan

  • Planetside Staff
  • *
  • Posts: 11745
  • Holy snagging ducks!
Re: Animation noise
« Reply #7 on: July 28, 2017, 07:40:48 AM »
Glad to hear that solved your issues. :)

- Oshyan

Offline rolland1013

  • Member
  • *
  • Posts: 28
Re: Animation noise
« Reply #8 on: November 06, 2017, 04:41:02 PM »
It appears I've run into a noise issue again.  I've attached a .mov and scene file that shows the problem.  As you can see in the Terragen scene file, I am creating a GI cache with detail and quality set at 4 and blur radius at 16.  Another interesting problem is render times, on average the frames took about 50 minutes to render.  But when I changed the lighting (made it look more like sunset) the render times skyrocketed to 8-12 hours a frame, and the noise issue was much more extreme.

Does anyone have any suggestions?

Thanks,
Niel

Offline Dune

  • Member
  • *
  • Posts: 12729
  • Corkscrew Bird
    • www.ulco-art.nl
Re: Animation noise
« Reply #9 on: November 07, 2017, 06:31:18 AM »
One suggestion I'll repeat (see a few posts back): turn off the compute terrain, and you'll render faster. For this kind of terrrain it won't really be necessary. And I wouldn't use such high detail settings, as most is overgrown anyway (and moving).
In case you still haven't seen enough of my work: www.ulco-art.nl

Offline rolland1013

  • Member
  • *
  • Posts: 28
Re: Animation noise
« Reply #10 on: November 07, 2017, 06:27:47 PM »
Thanks Dune,

I'll try disabling the compute terrain.  Are you saying to lower the Micropoly detail?  What would you suggest lowering it to?

Offline Oshyan

  • Planetside Staff
  • *
  • Posts: 11745
  • Holy snagging ducks!
Re: Animation noise
« Reply #11 on: November 07, 2017, 08:55:05 PM »
Be aware that disabling the Compute Terrain in an already working setup is likely to have some impact (i.e. it will create changes). Designing a scene from the beginning without one *can* net some performance gains, but it complicates workflow and requires some tricks and workarounds. We don't just put it in there to slow things down for no good reason. ;) So there's a reason Ulco (Dune) is one of the few people using that workflow successfully. It is definitely not the first place I would suggest for optimization. I'll get to my suggestions in a moment, I've been doing some testing to verify so I can give best advice, but having looked at your TGD I can say there is definitely *major* room for optimization without noticeable quality loss.

- Oshyan

Offline Dune

  • Member
  • *
  • Posts: 12729
  • Corkscrew Bird
    • www.ulco-art.nl
Re: Animation noise
« Reply #12 on: November 08, 2017, 06:24:47 AM »
In advance of Oshyan's reply, I'd say detail of 0.5-0.6 would do. And a few tests with or without compute terrain would tell you soon enough if it changes the landscape (too) much, or none at all.
In case you still haven't seen enough of my work: www.ulco-art.nl

Offline rolland1013

  • Member
  • *
  • Posts: 28
Re: Animation noise
« Reply #13 on: November 08, 2017, 04:52:02 PM »
Thanks Dune,

It sounds like your suggestions are addressing render times issue, yes?  Or does reducing the detail also effect the noise in the clouds?

Offline Oshyan

  • Planetside Staff
  • *
  • Posts: 11745
  • Holy snagging ducks!
Re: Animation noise
« Reply #14 on: November 08, 2017, 09:57:50 PM »
The noise in that animation clip is only in the clouds, as far as I can tell, and they are probably v3 clouds. v3 cloud GI quality (and thus likelihood of flickering) is controlled separately from primary GI (in the Cloud GI tab), and is not fully cacheable at this time. You need to use higher Cloud GI quality settings to avoid this. Reducing detail won't help with that issue, unfortunately.

As for why noise issues get worse at sunset, it's because you're dealing with a much higher proportion of indirect light, which is more diffuse, so you need more samples to reach an equivalent level of quality/noise. This is virtually unavoidable in a realistic light simulation. However, looking at your .tgd, I think there are some good ways to optimize your settings.

First and foremost, I strongly recommend reducing Micropoly Detail below 0.9 (also Ulco also noted). Micropoly Detail is only going to affect the detail of your terrain, not your clouds or atmosphere, and not your objects (trees, etc.). Since your terrain is almost entirely covered by trees here, you will see virtually no difference between say 0.6 or 0.7 Micropoly Detail and 0.9 as you have it set here. So I would recommend trying 0.6 first and do a short frame sequence and if it's not quite enough detail *in the exposed terrain parts*, then try 0.7. You really should not need more than that though. And this will save you some render time.

I also recommend not using 3D Motion Blur here. You'll see minimal blur at this altitude and speed, but some additional samples may still be calculated for it, to minimal beneficial effect. If necessary use 2D motion blur, either in Terragen, or in another package.

Since you are using AA8 and are ray tracing the atmosphere ("defer"), I would also suggest reducing the atmosphere samples quite a lot. 16 ought to be fine with AA8, or at most 24, even for the dusk shot. If you are using higher atmosphere samples for the daytime shot you should definitely be able to use only around 8-12 samples with an AA of 8. AA8 is quite high and should be sampling the atmosphere very well. You should also strongly consider whether you need the "Receive shadows from surfaces" setting. Only keep it enabled if you see shadows *in your atmosphere* (*not* onto clouds) that you want to keep. I would suggest trying a test frame of the sunset scene with it disabled and see if it makes a difference. I do see there is an area with somewhat subtle rays coming off a terrain element right in the middle of frame, so it may be worth it to you to keep it.

Also consider whether your higher Cumulus clouds (Layer 01) actually need Receive Shadows enabled. This should only be for shadows *from* the terrain, and they seem too high up to receive any shadows from the terrain here. With the setting disabled they'll still get shadows from other cloud layers. It may change the lighting slightly, but should not have a dramatic effect and will save render time.

For all cloud layers you can consider/test using "Conservative Acceleration" in the Acceleration Cache (on Optimization tab). It may take you some hours to test all this, but if you can reduce render times per frame by 50% or more (likely, if you test all the above), then you will have saved yourself hundreds of hours of render time.

I also see you have 360 degree detail enabled here, and set to "Highest" rather than "Optimal". This may be your biggest potential source of render time savings. I'm not sure if you are using this just because of the problem you had previously in a different (though somewhat similar) scene, or if you actually tested in this scene to make sure it's necessary, but if you have not tested you definitely should. Test first with Detail in Crop, and then try some amount of non-360 padding or "Optimal" 360 padding if you see any issues.

The reason this may be a very different case than your other scene is that you are facing into the sun here (at least later in the animation), and so shadows are being cast in most cases from *visible terrain geometry*, i.e. terrain that is already being handled by the main render process for the primary view. The need for Ray Detail Region Padding depends almost entirely on any shadows cast from *off-camera* geometry. Such shadows will vary tremendously, even for the same sun altitudes, depending on camera angle relative to sun angle. So again, if you are seeing the terrain from which shadows are cast, the shadows will be correct, it's only when shadows are coming from off-camera geometry that you need the region padding. Turning this setting down or off can save you a lot of render time.

I would also note that turning off Ray Detail entirely ("No Detail") saves a huge amount of render time, at the expense of most of those rays in the atmosphere coming off that center-frame area of terrain. If that subtle detail is not vital to your quality of output, then you can get frame render times down to less than a 10th of what it was, however it does produce differences in the final output so do test it first to see if it's acceptable.

Attached is an optimized TGD that cuts render time in half.

- Oshyan