Planetside Software Forums

Support => Terragen Support => Topic started by: N-drju on November 29, 2016, 04:55:41 AM

Title: Reducing RAM usage
Post by: N-drju on November 29, 2016, 04:55:41 AM
Is there any way to reduce the amount of RAM that is used by the objects?

Let's be more precise: I usually use DAZ Studio to construct and export human models for use in Terragen. It is not unusual for me to see RAM usage jumping up by 500MB upon loading the object into the scene. I can still manage my renders with ease, but I get nervous whenever RAM usage climbs up too rapidly.

So yes - is there any way to reduce this effect?

P.S.: My human objects usually have about 55 parts that are distributed among around 16 shaders (skin + clothing).
Title: Re: Reducing RAM usage
Post by: Dune on November 29, 2016, 05:14:34 AM
One thing I always do is rearrange the surfaces/parts, so they group into one for each surface, in Poseray. If you have the mouth closed; delete teeth, innermouth, tongue and one more.
If you use (a lot) more objects using the same surfaces, you can load the surfaces once (as a default shader set), then link that to each part in each object. More work (a lot more), but saves loading the same surfaces/textures with every object.
For more distant use I have a set of reduced texture maps.
Title: Re: Reducing RAM usage
Post by: N-drju on November 29, 2016, 05:31:09 AM
Quote from: Dune on November 29, 2016, 05:14:34 AM
One thing I always do is rearrange the surfaces/parts, so they group into one for each surface, in Poseray.

Yes, I already do that in Terragen (connecting lips, face, ears etc. to the "Face" shader for instance) and I still end up with 500MB RAM increases.

Hm, and can I have an object consisting of just one part? :) Something like "baking" the shader?

DAZ Studio in theory has an option to bake the shaders! However any time I try this option out it just returns me a huge black square and nothing more. :-\

Oh, and one more thing... Would saving the texture files in some other, lighter formats help?
Title: Re: Reducing RAM usage
Post by: Dune on November 29, 2016, 05:48:26 AM
I just use Poseray, copy materials to groups, delete checked/invisible groups (like teeth...), merge groups that use the same material (give it the name of that common material), after having left maybe 5-10 groups (so, each using one material), go to materials tab again, copy groups to materials (you'll loose materials it says, and they'll all turn white), copy the name of the material (and possibly path) into the box, apply... and it should be back. Or relink to that material. Then export.
What I also do is save the nodes needed for a certain object/model as tgc. In that case relinking isn't even needed, just open in TG and relink the imported tgc nodes to the necessary parts' nodes. If I make myself clear....
Oh and no, if you save as jpg or LZW tiff, it'll still be expanded in TG. Only thing is to reduce size from 4k to 1-2k or so, if possible for distant views.

I don't think it can be one part, as the UV's of those parts usually overlap. But in some cases that may work.
Title: Re: Reducing RAM usage
Post by: N-drju on November 29, 2016, 06:37:58 AM
Dune, one more thing - once you are done pre-processing the object in PoseRay, do you still use standard part-to-texture method of putting textures at the object, or do you load the MTL file which I can see might be helpful? Not really sure what the MTL actually is though...
Title: Re: Reducing RAM usage
Post by: Dune on November 29, 2016, 07:07:11 AM
If you save an object the mtl file points to the texture files, so if you have saved from Poseray with textures loaded, all you have to do is import in TG. TG will read the mtl file. If you haven't an mtl file, I guess you can (have to)  load textures within TG by hand. Does that answer your question?
Title: Re: Reducing RAM usage
Post by: N-drju on November 29, 2016, 07:53:51 AM
Yep, it does.

So let's sum it up - by using Poseray I can for instance remove character's legs (;D) leaving only the trousers on (fine enough). This way, the smaller in parts and textures an object will be - the less RAM space it will use when loaded?

This makes sense as XFrog plants, which usually have 3 to 4 parts, never cause a spikes that high in RAM use.

Sounds like it makes sense?
Title: Re: Reducing RAM usage
Post by: Dune on November 29, 2016, 08:22:48 AM
Yes. I sometimes cut legs off too  8)  But on the whole, these characters aren't really that big are they? 15MB or so? Cut a leg off and it's maybe 13. Perhaps most RAM is taken up by loading (big) textures. How much RAM have you got?
Title: Re: Reducing RAM usage
Post by: N-drju on November 29, 2016, 08:36:20 AM
Hm, looks promising. I'll post an update after I try it. :) Unfortunately, that's an additional program that I need to learn to operate... Hopefully it will be worth it.

I have 8GB RAM on my current computer. So I imagine that means that, calculating it roughly, I can add around ten fully detailed human figures to a complex scene without seeing any adverse effects.
Title: Re: Reducing RAM usage
Post by: Dune on November 29, 2016, 09:44:22 AM
Yeah, 8 is not much. TG needs around 4 I think, with all the new fancy stuff. Can't you put some extra in? Won't cost a leg  ;D
Title: Re: Reducing RAM usage
Post by: N-drju on November 29, 2016, 12:11:04 PM
Hahah. ;D I guess you're right. But still, I'm quite satisfied with what I have now. With 8GB I managed to create many things I could not do before with 4GB. But of course - it can always be better. Maybe I'll get myself an extra "cube" underneath a Christmas Tree?

I just made a simple test with Poseray and deleted some groups as you suggested. Some I could not delete - they proved to be just too big. I managed to reduce RAM usage on this particular object by 30%. Not too bad I guess!

To be honest, I don't mind removing legs, teeth and cutting tongues. ;D After all... home is where the hurt is. ;D
Title: Re: Reducing RAM usage
Post by: Dune on November 29, 2016, 12:32:03 PM
 ;D
Title: Re: Reducing RAM usage
Post by: Oshyan on November 29, 2016, 05:37:51 PM
The main reason human figures probably take up more memory than your average Xfrog plant is the use of many unique image map textures. XfrogPlants can be very detailed in geometry, which can sometimes take up a lot of memory, but they use very few textures in most cases, and not crazy high resolution ones (2k at most). An entire tree might use 1 or at most 3 different leaf textures, and a single bark texture usually as well. So for a whole tree you might have 4 textures, which really does not take up a lot of memory. Poser figures that are fully clothed will have high resolution skin, eye, hair, and clothing textures. Often, as you said, upwards of 50 different parts each with their own textures. So that's what tends to take up a lot of memory for each.

- Oshyan
Title: Re: Reducing RAM usage
Post by: N-drju on November 30, 2016, 02:59:49 AM
Yes, that's true Oshyan. The model I will use in my next picture is from the newest Genesis 3 generation (in case you are familiar) and they went downright crazy - textures for limbs, face and even eyes are 4000x4000 pixels each! ;D

Fortunately enough, what Dune suggests seems to cut down the memory usage. Also, some difference can be felt with Photoshop "intervention" when resaving textures into JPEG formats of quality 9 (down from 12). Though this step gives you only tiny reduction in RAM (20-70MB per texture). I might try to resize these textures yet down to 2K.
Title: Re: Reducing RAM usage
Post by: Dune on November 30, 2016, 06:00:44 AM
The jpg quality doesn't reduce RAM, afaik, as it will be unpacked in TG again to full size (I think that is so anyway). Reducing real size to 2k or even 1k is a better option, especially for parts that are not in close-up view (like eyes). But do save as name_small.jpg or you'll loose the original, but I don't have to tell you that  ::)
Title: Re: Reducing RAM usage
Post by: N-drju on November 30, 2016, 08:47:59 AM
Well, afaik (;)) overwriting the "maximum" quality JPEG file into "high" quality file did reduce RAM usage by about 100MB total, with three textures overwritten. Terragen has no chance of loading full size because after overwriting there isn't any full size anymore! :) This step however is not quite as effective as I expected it to be. And yes, I have the size reduction experiment scheduled for today's afternoon.

Besides, DAZ Studio has an option to collect all textures and save their copies in the "Maps" folder within the folder the object is saved to. So you can easily edit the textures any way you want and the original will not be affected even with the wildest adjustments. Also, I have my original DAZ library stored on an external drive.
Title: Re: Reducing RAM usage
Post by: Dune on November 30, 2016, 10:33:53 AM
You're right, they're copies anyway.
Title: Re: Reducing RAM usage
Post by: Oshyan on November 30, 2016, 07:56:45 PM
The reason JPG compression shouldn't have an effect on *in memory use* is that once the texture is loaded, it is loaded into memory *decompressed*. That way we don't have to do decompression on the image every time the texture is accessed during rendering (which would create needless rendering overhead). Instead we just have direct access to the per-pixel color, with no decode step. That's my understanding anyway. In other words, no matter how compressed an image is (even if you use max JPG compression), the *in memory* size should be basically the same.

- Oshyan
Title: Re: Reducing RAM usage
Post by: N-drju on December 01, 2016, 08:36:40 AM
But I keep telling you it does have an effect and I checked it five times... :P Though it needs at least four textures resaved to see 100MB reduction in RAM use. I mean, at least this is what I can see on my machine.




Update:
I made a "full" (so to say) object preparation consisting of:

1. Passing the object through Poseray, cutting unnecessary object parts (like nipples, genitals, I mean who needs them anyway? :P)
2. Attaching all parts that use the same texture to only one default shader with the correspondent texture (i.e. forearms, arms, hands, fingernails ---> limbs.jpeg)
3. Reducing the quality of the JPEG images used for textures (from 12 to 8, sometimes 9)
4. Reducing the size of the JPEG images used for the model (from 2000px to around 1250px)

As a whole I managed to reduce RAM usage for the tested object by 50%! Great. :) However it seems that removing invisible parts from the object (works very well with internal walls in case of buildings) is the most effective "gain" during this kind of pre-processing.