Render Layers/Render Elements

Started by j meyer, April 12, 2014, 01:14:54 pm

Previous topic - Next topic

j meyer

Took me a while to get to test the Render layers/elments,but finally found
some time to do so.
And although I didn't test this feature thoroughly yet I already ran into a few
things I don't understand.
The one that bugs me most is the lack of a separate Shadows Render Element,
which is a pretty crucial thing in compositing.At least it is a standard pass in
other renderers and in every workflow I am familiar with.
Why is that? Where's the advantage,I don't get it?
In case I'm just too dense to find it,please help me out.

Another thing is the "exr chauvinism" ,personally I'd like to be able to decide for
myself,if I want to use exr or tif or whatever.Especially since I don't have any
image editor that can deal with 16 or 32bit hdr images.
Fortunately the exr plug-in for Photoshop,provided by ILM,worked for PShopElements
also and thus saved my day.
Connected to this,but minor are the punctuation marks (dots,periods whatever
you call them) in the titles,which can cause problems on loading the pics,because
sometimes the first dot in the title makes a program think the format name starts
there and so results in an unknown format message for example.
Or,if you don't change the default setting for the extra output images to exr,just
because you don't know about that initially,you end up with .bmp.exr and of course
it won't open correctly.
And in case of making different passes from one render layer accidentally,instead
of using 2 or more layers the extra images get overwritten without warning,so you
loose any extra output you might have rendered before.Once you know that it's
of course no problem anymore.

Shouldn't cast shadows and other rays be treated as separate features,too?

There are other things in conjunction with transparency,but those are most likely
a case for the support section.

What are your experiences?







j meyer

In the meantime I came to the conclusion that the aforementioned transparency
related things are not for the support section.Apparently I just had a misconception
in regard to how things should work,again.
For testing purposes I chose one of my Landy files,thinking it's as good a file as any
other with models in it.
So I wanted to try to separate the landscape from the model to be able to alter
the terrain surface without affecting the car and driver.Put the Landy and driver
in one group and deactivated the windows.The separate windows were made a
second group.In the Render layer I set the first group to holdout with Cast shadows
and other rays checked,the second group to invisible and All other objects to visible
with Cast sh.... checked.

[attachimg=1]

Then group 1 (car+driver) to visible with Cast sh... checked,group 2 still invisible
and All other objects to invisible with Cast sh.. checked.

[attachimg=2]

So far so good.
Then I wanted to get the windows alone for reflections etc.
Group 1 to holdout with Cast sh... checked,group 2 to visible (separate windows
have cast shadows unchecked) with Cast sh...checked(tried unchecked aswell)
and All other objects invisible with Cast sh... checked.
I expected to see just the reflections and specular of the windows and some black
silhouette from seats and driver,because they were on holdout.
What I got was this,though.

[attachimg=3]

Imagine my surprise.
Needed some time to dig that.(or to grok it ;)) Was really confusing at first.
My guess now is that transparency is treated as other rays.

I know it's not something most people will need in their TG work,but it might
be worth to take into consideration when rendering water,especially when there
is something to see beneath the surface.



dorianvan

I need to experiment with rendering layers too. It would be nice if there were more video tutorials on not only exporting, but using render elements and rendering layers. Here's one on Render Elements from digitalguru, I like it, but more would be better: https://www.youtube.com/watch?v=ihY_kp2rzxg&feature=youtu.be
-Dorian

j meyer

dorian - there is a lot of compositing tuts to find on the web,not TG specific,
            but generally.The principle goes for TG too.Of course it'd be nice to
            have some explanations for the more exotic elements,like the 2D motion
            vector and normal and position stuff TG offers.



j meyer

And of course I would like to get some answers to my questions from
the opening post.
Let me assure you those are serious questions not just rhetorical.
So,please Planetside,if you could kindly respond it would be very much
appreciated.


bobbystahr

Hang in there J meyer...we're all waiting to see the reply....
something borrowed,
something Blue.
Ring out the Old.
Bring in the New
Bobby Stahr, Paracosmologist

PabloMack

Quote from: j meyer on April 12, 2014, 01:14:54 pm...you end up with .bmp.exr and of course it won't open correctly.


Shouldn't that be easy to fix with a quick file rename by removing the ".bmp"? The file can only be one or the other.

j meyer

 ;D Thanks Bobby.

PabloMack - I'm afraid you missed the correlation.

Matt

Quote from: j meyer on April 12, 2014, 01:14:54 pm
The one that bugs me most is the lack of a separate Shadows Render Element,
which is a pretty crucial thing in compositing.At least it is a standard pass in
other renderers and in every workflow I am familiar with.
Why is that? Where's the advantage,I don't get it?
In case I'm just too dense to find it,please help me out.


I haven't added a shadow element because I'm not clear how it should work in Terragen. Some questions that need to answered are: should atmosphere shadows be part of the shadow element? Atmosphere casts a shadow in Terragen. When is it correct to include it and when is it not? Should it only be shadows from direct lights (e.g. Sunlight) or does ambient occlusion get included? How do we handle shadows from multiple light sources? How do you weight contribution of each light's shadow? Different people (and renderers) have different ideas about what a shadow pass should be.

There are other workflows that avoid this problem. Terragen outputs direct lighting as a separate element from indirect lighting. This means that if you want to adjust the colour or brightness of your shadows, you simply adjust the indirect lighting element. This is a more realistic way to control your lighting.

If you have a specific use for the shadow pass, can you explain what it is so I can understand what you want the shadow element to look like.

Quote
Another thing is the "exr chauvinism" ,personally I'd like to be able to decide for
myself,if I want to use exr or tif or whatever.Especially since I don't have any
image editor that can deal with 16 or 32bit hdr images.
Fortunately the exr plug-in for Photoshop,provided by ILM,worked for PShopElements
also and thus saved my day.


EXR allows me to output motion vectors, normals and position elements with negative values. It is generally safe to assume that EXRs contain values in a linear colour space which is essential for working with additive lighting elements, whereas with other formats you usually don't have that security. I had to choose whether to compromise on some of the ideals I had for these elements by fitting them into file formats that don't have the same latitude as EXR, or allow other formats to be used but clip the data, or output the data differently depending on the format chosen. These all add complexity to the software, increase our development time and costs for support and documentation. EXR is well supported and avoids these problems. It's a format chosen by most visual effects productions for good reasons. Perhaps this is EXR chauvinism; OK, I hold up my hand to that one ;)

I will think about relaxing the restrictions on the file format - but I don't know if that will make things more or less complicated yet.

Quote
Connected to this,but minor are the punctuation marks (dots,periods whatever
you call them) in the titles,which can cause problems on loading the pics,because
sometimes the first dot in the title makes a program think the format name starts
there and so results in an unknown format message for example.
Or,if you don't change the default setting for the extra output images to exr,just
because you don't know about that initially,you end up with .bmp.exr and of course
it won't open correctly.


I didn't know that some programs misinterpret the first dot this way. In any case, I think it's better that if we save an EXR that Terragen enforces the file extension to be .exr, and I don't think it's always right to guess which part of the filename to replace without asking the user, so we just add .exr.

I realise this isn't ideal and there might be ways we can improve this.

In case you're talking about the ".IMAGETYPE" part of te filename, that can be changed from the default to use an underscore or anything else.

Quote
And in case of making different passes from one render layer accidentally,instead
of using 2 or more layers the extra images get overwritten without warning,so you
loose any extra output you might have rendered before.Once you know that it's
of course no problem anymore.


I see this as necessary behaviour so that files go exactly where you tell them to. But maybe we should perform a pre check when you hit the Render button.

Quote
Shouldn't cast shadows and other rays be treated as separate features,too?


This might be possible in future. I believed it would be acceptable in 90% of situations to couple these together and simplify the interface a little bit. It sort of encourages a certain workflow which I like, and which has benefits once you settle into it, too, but maybe that's not a good reason.

I'll answer your other questions later.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

digitalguru

for my two cents worth...

shadows passes are like insurance - you may never need them, but if you do - you'll be glad you rendered them...

a suggestion might be to have terrain shadows - i.e shadows from direct lights and terrain and atmospheric shadows as separate render elements - I could definitely think of situations where reducing shadows from the atmosphere onto the terrain might be useful.

Also if occlusion is calculated internally and could be exposed as a render element that might useful also - it might not be technically correct, but I've often thought that a slight amount of occlusion might add some "definition" to a Terragen render.

Some comp tricks like the re-lighting technique using world, normal and diffuse elements aren't really complete until you have shadows to add in (so long and you don't deviate too much from you original lighting)


j meyer

 :) Hi Matt,thanks for replying with that much valuable info.

Shadows:I'd say atmospheric shadows,direct light(s) shadows and ambient occlusion
             should be separate elements.
             Multiple light sources shadows could be handled via the grouping feature.
             Multiple layers are required anyway,probably,when working with multiple lights.
             As for weighting each light's shadows isn't that in relation with the light's
             strength anyhow?

             Adjusting color or brightness would have been one reason,so thanks for the
             tip,will have to experiment with that.
             Having used render passes and compositing almost exclusively for hero objects,
             as opposed to landscapes,so far.Thus another use would be changing/altering
             the diffuse texture without affecting the shadows.And when combining again
             there is no need to remove anything in order to avoid applying things two or
             more times.
             Or using shadows or parts thereof as a mask.
             Or blurring shadows or parts thereof manually.
             Stuff like that mainly.Maybe there are other uses I can't think of at the moment.
             Don't know how much sense that makes in pure landscape comping,though.

EXR:      Didn't know about the technical aspects and the resulting problems.
            Still I'd like to have tiff to choose,but if it's complicating things unnecessarily
            it'd be better to go with exr alone of course.
            Anyone working with 8bit images has to live with clipped data so or so,I guess.

            As for the dots it's fortunately only the dot before bmp nowadays.When I noticed
            that behaviour the first time some years ago it would have been the first dot
            at all.
            When the files get saved as exr it would make sense to remove the bmp when
            exr is added.
            A pre check  when hitting the render button seems to be a really good idea!
            In particular since I came across another thing,that is somehow related.
            When using a tgd that was saved with TG2.x the extra output images is still
            checked (of course) and the path is still the old one,too.
            So I tried it to see if the path would automatically update.Unfortunately it
            didn't and after the render had finished a message popped up informing me
            about that invalid path and no extra images were saved.So,yes on hitting
            the render button would be great.

           
Cast shadows and other rays:
            Agreed,but those remaining 10% or even less can be problematic sometimes.
            Looking forward to what you may come up with eventually.


Thank you very much for taking the time to discuss this,very appreciated,
J.