Planetside Software Forums

News => Announcements => Topic started by: Oshyan on October 02, 2014, 08:24:11 PM

Title: Terragen 3 Benchmark released
Post by: Oshyan on October 02, 2014, 08:24:11 PM
After helpful feedback from the community on an earlier draft, we're ready to release the final version of our new Terragen 3 Benchmark. This scene has been specifically developed to test render times in Terragen 3 and is compromised in large part of presets that ship with TG3 itself. Thanks go to Volker Haroon, Ulco Glimmerveen, and Jack Marsh for their contributions.

The new benchmark has been designed to test a wide variety of TG3 functions, including GI, volumetric cloud shading, water, displaceable objects, populations, water, and more. It has also been tuned to provide a good range of render times across a variety of modern systems, with newer, faster hardware turning in times of 5 minutes or below, while older machines may take 45 minutes or more. This allows for statistically reliable results on faster systems, without making render times overly long on slower machines. We hope it will prove to be a useful tool for the community!

[attach=1]

Hop on over to our Terragen 3 Benchmark page to download the scene and submit your render time, then check out the full list of results:
http://planetside.co.uk/products/tg3-benchmark (http://planetside.co.uk/products/tg3-benchmark)

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on October 05, 2014, 09:53:43 AM
I managed to upload wrong data >:(
Is there a way to correct it? :)
Title: Re: Terragen 3 Benchmark released
Post by: jaf on October 05, 2014, 01:17:33 PM
I was able to edit my data when I realized I made a couple of mistakes, but the data entry form was still "active" (after submitting my data, I went to the view results spreadsheet and saw my mistakes.)  Somewhere there was an "edit" link, but now I can't find it.
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 05, 2014, 06:51:33 PM
Yes, you can edit your submissions.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on October 06, 2014, 08:27:10 AM
Can't see any way to do it... :(
Title: Re: Terragen 3 Benchmark released
Post by: jaf on October 06, 2014, 09:22:53 PM
Don't see it either.  I can view my data in the spreadsheet and I can see the little black triangles where I was able to edit it yesterday, but can't find a way to edit it today (maybe going through the form again using the same identification data?)  I'm logged in because I see my name near the upper-right hand corner of the screen. ???
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 07, 2014, 01:08:59 AM
It looks like you can only edit it from the Confirmation screen, or if you requested to have a copy of your submission emailed to you (there's a link to edit in the email). So just email us at support AT planetside.co.uk with the changes you need and we'll take care of it.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: TheBadger on October 08, 2014, 01:43:49 AM
added mine.

Curious if anyone here is using one of the new macs? Would like to see what the times are from the various configurations.
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 08, 2014, 02:00:07 AM
No test results from a new MacPro yet, but they're standard PC hardware at this point, just a single 6 core CPU. So should be about 6 minutes, similar to JBT27 on the current list.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: TheBadger on October 08, 2014, 07:12:44 PM
Still want to see.

Hey, can you explain how PS uses this data you are collecting? In general its pretty interesting, but how do you, and perhaps even other soft makers  use this data?
Does it just show you how different set ups work with TG, or are you able to use it to make improvements to TG?
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 08, 2014, 07:36:46 PM
We don't make a ton of direct use of it, but it *is* fairly helpful to know how the render engine performs across a range of hardware, and especially how it "scales" (efficiency as you add more resources, especially processing cores/threads). There is still room for improvement in our render threading with higher numbers of threads (above 12 I'd say), although we've made big improvements over the years already. Other than that it's more a tool for the community, to get an idea of how their hardware should be performing (by comparing it to other similar hardware), or to get ideas for what hardware to buy if they're thinking of a new machine that will be used a lot for Terragen (simple: buy the best-performing CPU on the chart :D).

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: TheBadger on October 08, 2014, 08:25:51 PM
Thanks,

Would you also like to propose an explanation for me to why my time could be 2 seconds better than Jo's (for example) who posted two times, one of which was a comparable system to what I have? Two seconds is nothing, So I am guessing that that has to do with perhaps he had some other soft running when rendering? Just curious why People running the same system as me would have any variation... I am guessing that is completely normal between any product?

I was glad to see all this really. If I don't render water, I think my time is good enough to not have to worry for a little while that my desktop needs to be replaced.
Though who wouldn't want one of those 5 min times! That would be nice  8)
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 08, 2014, 08:59:13 PM
There is always a small amount of variation, even on subsequent tests on the exact same machine. A 2 second difference is extremely small, statistically speaking. But this is indeed one of the reasons why you don't want a benchmark that is *too quick* to render. Variations of a few seconds are to be expected, even on the same machine, let alone on different machines of similar hardware, and of course on different hardware entirely. But imagine if the total benchmark render time was only 10 seconds. In that case a 2-3 second difference is 20-30%! Which, statistically speaking, is a huge difference. The problem in that scenario is that 2-3 second variations in render times are normal, so these variations have a bigger impact on shorter render times, making them less accurate for comparative purposes.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: pokoy on October 13, 2014, 04:40:54 AM
This one went even slightly faster than the original preliminary benchmark file from 2 weeks ago. Judging by the submissions, it seems I have the fastest machine here  :o

I'll do another test on my older render machines and will try to get both of my co-workers to run the benchmark on their new Mac Pro machines (they have different ones), wonder how they'll perform.
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 13, 2014, 05:43:25 PM
Indeed pokoy, the benchmark was tweaked a little to render slightly faster and also to be within the capabilities of the Free version. It still shows a nice spread of results so far, and I'm pleased to see that nobody has had to crack the 1hr time yet. ;) Your result definitely shows that having lots of threads/cores is still quite beneficial, even over faster (in raw clock speed) machines with fewer cores. So that's quite good to know. Unfortunately those dual machines are quite expensive, but for maximum space performance in the smallest space and power profile, a dual CPU machine is clearly best. I'd love to see what the new 16 core E5-2697 v3 would do in a dual config. Of course each CPU is $2800, hehe. And just wait until the 16 and 18 core models coming later this year...

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: jaf on October 13, 2014, 10:37:31 PM
.... also, if you load the benchmark , render and then render again without re-loading the benchmark you will save time (ten seconds on my system.)  I believe it's because the instances are not recalculated. 

I was able to save 20 seconds by exiting/killing many non-needed processes and switching to a single monitor (verses dual) mode.  However, to me the benchmark makes the most sense by running it the way you normally work -- not to get the best possible time.  So I would urge booting up as normal, loading TG and the benchmark and hitting Ctrl-R and simply reporting that time.
Title: Re: Terragen 3 Benchmark released
Post by: TheBadger on October 13, 2014, 10:51:51 PM
Quote from: pokoy on October 13, 2014, 04:40:54 AM
This one went even slightly faster than the original preliminary benchmark file from 2 weeks ago. Judging by the submissions, it seems I have the fastest machine here  :o

I'll do another test on my older render machines and will try to get both of my co-workers to run the benchmark on their new Mac Pro machines (they have different ones), wonder how they'll perform.

I will look forward to that Pokoy! I am very curious my self about the new macs. I have slowly been working to increase my general knowledge of hardware. And stuff like this benchmark is kinda a fun way for me to get some info. Really its got to be one of the easiest ways to see what the hardware can do in a context that I have some familiarity with. So please do it!  :)

I do wish that there were also times posted without water though. I felt like that was the only thing that slowed my time down. Not really by all that much, about a min maybe.
And I am not saying anything bad about water here. Just I don't often render a scene with any water, so I also am curious how much faster all the other systems would be if they did not render water?

@ JAf
Thats good to hear. when I am rendering something I am taking seriously, I always make sure everything else is turned off (everything that I know how to turn off). I even turn off wifi.

I wondered about having two monitors and rendering. So thanks for that info!!
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 14, 2014, 04:39:18 AM
Please do not report times that don't include the population phase. It is there intentionally, to test multithreading of the populator.

Water is not a significant contributor to the render times here actually, and it was designed that way intentionally. Try disabling it in the scene and you'll see. We included it because it *is* a common element of scenes though, in fact it's surprising to say you don't often see scenes with it. Of the most recent 10 images submitted in Image Sharing, 8 have water, so... ;)

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: RogueNZ on October 14, 2014, 05:07:26 AM
Tried to edit my submission but ended up submitting twice - apologies  :-[
Title: Re: Terragen 3 Benchmark released
Post by: TheBadger on October 14, 2014, 05:47:19 AM
QuoteOf the most recent 10 images submitted in Image Sharing, 8 have water, so..
I said *I* don't often render water.
But I will believe you on all the rest :)
Title: Re: Terragen 3 Benchmark released
Post by: jaf on October 14, 2014, 11:51:45 AM
Quote from: Oshyan on October 14, 2014, 04:39:18 AM
Please do not report times that don't include the population phase. It is there intentionally, to test multithreading of the populator.

Water is not a significant contributor to the render times here actually, and it was designed that way intentionally. Try disabling it in the scene and you'll see. We included it because it *is* a common element of scenes though, in fact it's surprising to say you don't often see scenes with it. Of the most recent 10 images submitted in Image Sharing, 8 have water, so... ;)

- Oshyan

Just to be clear, I didn't report mine that way.  I was attempting to make the point we should use the benchmark the way we normally use Terragen and don't try to get the best score possible.   That said, don't run other processes or applications that cut into your cpu usage either because that won't give a good "picture" of what your system is capable of compared to others.

As far as dual monitor or not, I didn't test each process individually so it likely didn't change the benchmark time -- just killed everything I could (and I'm sure I missed some) to see how much difference it would make. [edit]  That was clear as mud!  I meant to say, I didn't test to see how much a dual monitor setup hurt the benchmark time, and I don't think it makes a difference.  But many of the other processes do, over the app. ten minutes my system ran the benchmark, cut a few seconds off the time.
Title: Re: Terragen 3 Benchmark released
Post by: lat 64 on October 14, 2014, 02:04:52 PM
Very telling test. Nineteen minutes for my old mac!
I sorted the results by column "c"(time), and I see that I was not the worst. ;D
I could not list my proccessor model no. because I couldn't figure out which item that would be on my Hardware Preview.

Here is that data if anyone else can figure it out for me:
Hardware Overview:
  Model Name:   Mac Pro
  Model Identifier:   MacPro1,1
  Processor Name:   Dual-Core Intel Xeon
  Processor Speed:   2.66 GHz
  Number of Processors:   2
  Total Number of Cores:   4
  L2 Cache (per Processor):   4 MB
  Memory:   6 GB
  Bus Speed:   1.33 GHz
  Boot ROM Version:   MP11.005D.B00
  SMC Version (system):   1.7f10
  Serial Number (system):   G87493LA0GP
  Hardware UUID:   00000000-0000-1000-8000-0017F20FEF9A
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on October 14, 2014, 02:21:02 PM
That CPU is the Intel Xeon 5150. The orig MACs were shipped with that :) We got 2 towers around here collecting dust... :(
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 14, 2014, 05:44:08 PM
Archonforest is correct. Lat 64, you can ignore my email asking about this, I've updated your result with the correct CPU.

RogeNZ, I've deleted your first (incorrect) submission. Thanks!

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: pokoy on October 15, 2014, 07:27:26 AM
I've added a new MacPro to the benchmark list, 5m 11 sec on a 6-core 3.5 GHz single CPU - not bad.
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 15, 2014, 03:36:27 PM
Quote from: pokoy on October 15, 2014, 07:27:26 AM
I've added a new MacPro to the benchmark list, 5m 11 sec on a 6-core 3.5 GHz single CPU - not bad.

Not bad perhaps, but for $5200 (with 64GB RAM) also not very compelling. For $5700 I can get a Boxx workstation with 2x 6 core at 2.4Ghz. Or the same for $4670 at Puget. :D

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: pokoy on October 15, 2014, 03:47:45 PM
I meant the CPU speed compared to my Dual CPU workstation. I agree, spending money on Mac hardware is like burning money AND throwing it out of the window at the same time ;)
Title: Re: Terragen 3 Benchmark released
Post by: TheBadger on October 15, 2014, 06:07:02 PM
Care to start a thread in open on the top performing system in the benchmark?
Parts, names, options, places to buy at the best price?

A real breakdown for shopping.

Oshyan, its not exactly fair to add 64 GB of mac mem. Who buys mac mem at their prices anymore? Anyway, I am learning about linux now. I hope that I will find it as good as efflux says.  ;D He is surly convinced.
Title: Re: Terragen 3 Benchmark released
Post by: lat 64 on October 16, 2014, 01:00:19 PM
Yea, I like Bager's idea.
I get it that a fancy GPU does not really benefit this software, but other than that, I don't know much. I am(like Badger I think) just studying the benchmark results to get a feel for what a person would focus on to improve render times and not waste efforts on unnecessary configuration "toys".
Since we can divide up the project so handily into several threads for rendering, I'm getting the impression that the smart money should be spent on getting the maximum number of processor cores—speed comes next, and then ram.
I have four cores and 6GB ram and watching the activity monitor during renders, I don't see it using up my ram.

Do I have any of this right? :-\
Russ
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on October 16, 2014, 01:24:07 PM
Cores are pretty important in TG :)
Then u are right about the magic trio TG needs the most. Cores/Ghz/RAM. The good news is that u do not have to spend a fortune to get something decent. I bought this year a dual quad xeon workstation with 8Gb ram for about 800 bucks and if u look up the spread of the timings it came out pretty okay. 8 min for the test render. I think it is good for the price I paid.
RAM is pretty cheap these days so u can have 8 Gb for nothing and that is also a good start. More is better when u have a more complex scene with lots of populations...etc. Also when u buy a cpu for rendering make sure it is hyperthread capable as that can give some nice boost also :)
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on October 16, 2014, 01:40:32 PM
Just found something interesting.

A bit slower but 99 percent identical Xeon CPU finished the benchmark almost 2 min earlier than mine. Difference? I got only 8 gig ram he got 24. Hmmm... Interesting...does the ram can make such a big difference?
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 16, 2014, 02:24:46 PM
RAM should not make a significant difference in render time. When you say his CPU is "slower", do you mean purely in terms of clock speed (Ghz)? What is the model number of his CPU vs. yours? Or is one just overclocked and the other not?

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on October 16, 2014, 02:50:27 PM
Here it is:

mine: E5530 Xeon on 2.4Ghz   - no overclock - 8Gb ddr3 - Win 7 64 bit
his:    E5520 Xeon on 2.26Ghz - no overclock - 24Gb ddr3 - Mac OS X 64 bit

Hmm... hope is not the Mac :P
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on October 16, 2014, 02:59:05 PM
Well, operating system or other software config details *can* make a notable difference. In the past for example Macs were less efficient with a larger number of threads in Terragen, though we have since improved this quite a bit and it's about even now for similar hardware I believe. So I don't have an immediate explanation. What I would do though is freshly reboot both machines and make sure any background tasks are stopped or at least that there is minimal other stuff active on the machines, then run the benchmark again on both.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: Dune on October 17, 2014, 03:33:41 AM
Speaking of background tasks; that would have an effect on render time, wouldn't it? My machine is offline and has no other tasks running, such as firewall, or antivirus... hard to compare to machines playing music, games and stopping vermin.
Title: Re: Terragen 3 Benchmark released
Post by: jaf on October 17, 2014, 12:56:06 PM
Quote from: Dune on October 17, 2014, 03:33:41 AM
Speaking of background tasks; that would have an effect on render time, wouldn't it? My machine is offline and has no other tasks running, such as firewall, or antivirus... hard to compare to machines playing music, games and stopping vermin.

Sure would.  I was able to save 20 seconds on a 9 minutes 16 second render by simply killing a bunch of unneeded processes (and I'm sure I could have killed more -- I didn't go offline or kill Windows Defender.)  So using a savings of 15 seconds, which if my math is correct is .02697%, I would save ~16 minutes on a 10 hour render.  Of course that percentage probably wouldn't be linear since there's so many factors that determine render time.

However, I work a 24/7 job where I depend on my computer a lot for contact information, research, etc.  If only I could save a partial TG render to disk and resume later, then I could do a quick reboot when needed.  So for me, the time savings isn't worth it for that amount of time I could save.
Title: Re: Terragen 3 Benchmark released
Post by: TheBadger on October 17, 2014, 10:20:49 PM
In my specific case, TG drives my Graphics card bonkers. So I cant have TG open and do to much of anything else without getting "crazy screen".

So I learned from that to turn things off. I don't really know how much time it shaves off by having other programs stopped. But I know it saves a lot of frustration. And whatever time it saves, its at least clear that on my system some time is being saved, since you can visually see TG slow to a crawl when Im doing other thing while rendering.

Although, I don't reduce the amount of resources TG can use even for browsing. I figure I cant changes things mid render so there is no reason to reduce power for TG just to look on line for an hour during a long render. (threads, cores)
Title: Re: Terragen 3 Benchmark released
Post by: mundus on November 22, 2014, 05:20:07 AM
10m:28s on an old i7-960 OC to 3.6Ghz on win7 64, 16GB.
I'm really happy to see i7-5820K OC to 4.2 Ghz cutting my time in half.

Maybe time for an upgrade next year  8)
Title: Re: Terragen 3 Benchmark released
Post by: terracr8or on January 02, 2015, 04:48:46 PM
x
Title: Re: Terragen 3 Benchmark released
Post by: masonspappy on June 27, 2015, 05:19:54 AM
Would have been nice if dates and timestamps were split into separate columns.
Title: Re: Terragen 3 Benchmark released
Post by: masonspappy on June 27, 2015, 05:39:49 AM
Quote from: masonspappy on June 27, 2015, 05:19:54 AM
Would have been nice if dates and timestamps were split into separate columns.
Never mind, looked in wrong column...
Title: Re: Terragen 3 Benchmark released
Post by: Dune on August 09, 2015, 11:59:48 AM
Just had my first run of the new i7 6-core, which will be used as rendering machine, and one of the first things I did was run the benchmark scene; 6'.10". My i7 2600k ran it at 10'.21", so I am pleased. So now getting all libraries and stuff over to that machine (I hardly hear it run, excellent!)
Title: Re: Terragen 3 Benchmark released
Post by: kaedorg on August 19, 2015, 03:32:04 AM
Did anyone try to test the Benchmark with Windows 10 and check if there is a difference in render time with his previous version ?

David
Title: Re: Terragen 3 Benchmark released
Post by: bobbystahr on December 03, 2016, 01:05:49 AM
Submitted my benchmark for the new AMD Corsair
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on December 03, 2016, 02:04:25 PM
We're due for a *new* benchmark soon that willl be more reflective of Terragen 4 performance...

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: bobbystahr on December 03, 2016, 02:31:57 PM
Quote from: Oshyan on December 03, 2016, 02:04:25 PM
We're due for a *new* benchmark soon that willl be more reflective of Terragen 4 performance...

- Oshyan

noted...I learned a bunch of stuff just exploring the first one and anticipate a similar reacation....
Title: Re: Terragen 3 Benchmark released
Post by: luvsmuzik on December 04, 2016, 10:41:09 AM
I was just rereading this to see and yep there was the answer. Perhaps cost of upgrading system and increase use of laptops and other gadgets has also dwindled appearance of users of this forum. I think I am on the newsletter list somewhere, are you still doing that?
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on December 04, 2016, 02:43:40 PM
Someone can explain the below to me?

a 4 core I7 on 3.4Ghz that gives 13.6Ghz finish the test render in 08:15 sec

an 8 core Xeon on 2.4Ghz that gives 38.4Ghz finish the test render in 08:58 sec???? >:(

WTH? ???  How is this possible? The Xeon is older for sure but Ghz is Ghz..... :(
Title: Re: Terragen 3 Benchmark released
Post by: bobbystahr on December 04, 2016, 03:51:10 PM
Quote from: archonforest on December 04, 2016, 02:43:40 PM
Someone can explain the below to me?

a 4 core I7 on 3.4Ghz that gives 13.6Ghz finish the test render in 08:15 sec

an 8 core Xeon on 2.4Ghz that gives 38.4Ghz finish the test render in 08:58 sec???? >:(

WTH? ???  How is this possible? The Xeon is older for sure but Ghz is Ghz..... :(


not sure as I'm always bit confused always but I thought cores were sharing the Ghz not multiplying it
Title: Re: Terragen 3 Benchmark released
Post by: Kadri on December 04, 2016, 06:30:01 PM
Quote from: archonforest on December 04, 2016, 02:43:40 PM
...  How is this possible? The Xeon is older for sure but Ghz is Ghz..... :(

Nope. That was in the past when the Ghz race was still on with AMD and Intel.
Now it is not so easy by just looking at the Ghz which one will be faster in which program (depends).
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on December 04, 2016, 08:44:43 PM
luvsmuzik, yes we're still doing newsletters. Another one coming up soon. And as far as cost of computer hardware, it's all actually cheaper than it ever has been, and performance has not increased much in the past few years so you can have even a relatively fast system that might still be several years old. This is helpful both for buying a "new" system (you can purchase some older hardware that still performs well but is cheaper), and for keeping our existing systems around for longer. My i7-2700k is 4-5 years old now, but it still gives decent performance.

The Xeon that got that render time is quite old, I believe it's the E5530. You can see its performance relative to the i7-2700k that Dune benchmarked with here:
https://www.cpubenchmark.net/compare.php?cmp%5B%5D=1244&cmp%5B%5D=868
It's about 1/2 the speed (on the Passmark CPU test, at least), and there were 2 of those CPUs in the machine, so indeed the result is as you would expect.
The short conclusion is that no, indeed, Ghz is not Ghz, you cannot determine raw performance just by looking at that number. That is *only* true when you are considering CPUs *in the same generation*. The i7-2700k was released 2 full years after the E5530. The 2700k is the Sandy Bridge architecture while the E5330 is the Nehalem architecture. You can see some more benchmarks on how performance varies even at similar clock speeds across generations of CPU architectures:
http://www.anandtech.com/show/7003/the-haswell-review-intel-core-i74770k-i54560k-tested/6
And all that is not to mention the big differences between clock speed, cores, and performance between AMD and Intel!

- Oshyan

Title: Re: Terragen 3 Benchmark released
Post by: archonforest on December 05, 2016, 03:49:52 AM
Wow that is insane. Guess the i7 doing more calculation per second than my Xeon. Well I have to live with that :D
...or just get 2 6 cores xeon in the box :D
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on December 05, 2016, 09:37:38 PM
This is what I'm thinking of getting... Lots of great deals on eBay for used workstations and servers!

http://www.ebay.com/itm/HP-Z820-Workstation-8C-Xeon-E5-2680-2-7GHz-32GB-2TB-Quadro-4000-w-Win-10-Pro-CD/172381735185?_trksid=p2047675.c100009.m1982&_trkparms=aid%3D888007%26algo%3DDISC.MBE%26ao%3D1%26asc%3D38530%26meid%3Da7f71989227e41d9b6273b2ea5c659a0%26pid%3D100009%26rk%3D1%26rkt%3D2%26sd%3D391641888981&autorefresh=true

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: KyL on December 05, 2016, 10:46:48 PM
To my taste an i7 is a much better compromise as a workstation. Even if in the long run a Xeon will render faster because it has more thread, it is nowhere as fast an i7 for every day use. An i7 just works faster and you can clearly feel it on everyday use, it feels more responsive.
I have a workstation very similar to this at work and even though it has 4 times more threads that my home computer and more Ghz power, it is just a merely 30% faster.... But as a rendering machine it is probably worth it!

But personally I rather like to spend on a fast CPU/fast RAM/ fast hard drives! ::)
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on December 06, 2016, 04:46:19 AM
Quote from: Oshyan on December 05, 2016, 09:37:38 PM
This is what I'm thinking of getting... Lots of great deals on eBay for used workstations and servers!

http://www.ebay.com/itm/HP-Z820-Workstation-8C-Xeon-E5-2680-2-7GHz-32GB-2TB-Quadro-4000-w-Win-10-Pro-CD/172381735185?_trksid=p2047675.c100009.m1982&_trkparms=aid%3D888007%26algo%3DDISC.MBE%26ao%3D1%26asc%3D38530%26meid%3Da7f71989227e41d9b6273b2ea5c659a0%26pid%3D100009%26rk%3D1%26rkt%3D2%26sd%3D391641888981&autorefresh=true

- Oshyan

Interesting. Why did u choose this rig? The CPU only slightly faster than my older Xeon(per CPUBoss). Or u will have 2 of this cpu?
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on December 08, 2016, 01:20:30 AM
2 of those CPUs, yes. Look on the current Terragen 3 benchmark list, you'll see variations of this CPU right at the top... 2670, 2680, 2690 (what I ended up getting was dual 2690s, 300Mhz per-core faster, and 128GB of RAM :D ).

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on December 08, 2016, 03:13:35 AM
Quote from: Oshyan on December 08, 2016, 01:20:30 AM
2 of those CPUs, yes. Look on the current Terragen 3 benchmark list, you'll see variations of this CPU right at the top... 2670, 2680, 2690 (what I ended up getting was dual 2690s, 300Mhz per-core faster, and 128GB of RAM :D ).

- Oshyan

Wow that is cool.
Title: Re: Terragen 3 Benchmark released
Post by: nito80 on March 07, 2017, 06:59:37 AM
Hi and hello,

I'm relatively new to Terragen, but already in love with it. But recently I stumbled upon some strange behaviour I'd like to ask about. I'm using a HP Z820 with dual 2690s, Quadro6000 and 128GB RAM (the same as user Oshyan mentioned in this thread) but have some issues with render speed and cpu/core usage.

In short, with any given scene, the machine renders faster, if I lower the core detection override in the preferences. So if I'm going with all 32 cores of the machine, a simple scene would render in HD at about 12 minutes (process in task manager at 100%). If I lower the cores to 8 (process in task manager at 25%) the same scene strangely renders within 3 - 4 minutes (seems to be the sweet spot). Also, the terrain preview (not RTP) is significantly faster with lower core usage.

I have no real explanation for this other than that threading in Terragen may be confused with 32 cores. Since user Oshyan mentioned the same workstation, I'd be very much interested if this could be an issue with my Z820 or something else.

Oh, I forgot: using Terragen4, Win7 Pro (fresh install)

Thanks,
Norman
Title: Re: Terragen 3 Benchmark released
Post by: Matt on March 07, 2017, 08:37:27 PM
Oshyan had a similar issue and he told me that he needed to change one of the NVIDIA Quadro settings. In particular, "Threaded Optimization" needs to be turned OFF. After making changes you should restart Terragen.

Here is some advice given by another software vendor which might be helpful:

https://artecgroup.zendesk.com/hc/en-us/articles/203385381-How-to-optimize-the-performance-of-your-NVIDIA-Quadro

Matt
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on March 08, 2017, 07:38:09 PM
I actually found, eventually, that the setting(s) you need to change are *not* exposed for individual adjustment in the drivers. What you need to do is switch your overall graphics driver profile to "3D App - Game Development" and that sets the correct options *internally*. That should give you full performance with all threads.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: nito80 on March 09, 2017, 02:50:23 PM
Thanks for your advice! I just tried the 3D App Game Development setting (threaded optimization is turned off) in the Quadro driver and the rendering speed seems to be improved, but only slightly.

To look at numbers: (terrain with simple Perlin noise, local erosion, some cumulus clouds, 500x237px, Micropoly detail 0.85, AA6)
Rendering with 8 cores with threaded opt. off: 13:22 minutes
Rendering with 32 cores with threaded opt. off: 10:12 minutes
Rendering with 32 cores with 3D App Game Dev. setting: 9:38 minutes

So there seems to be an improvement, but not as much as expected (~4 minutes, comparing the 8/32 core times). I'm also wondering what impact the Quadro on rendering performance has, since Terragen is purely (?) CPU-based? While rendering, the Quadro idles at about 11% usage.

Thanks,
Norman
Title: Re: Terragen 3 Benchmark released
Post by: masonspappy on March 09, 2017, 03:53:19 PM

I recall an example from a few years ago (and I hope I'm recalling correctly, since I couldn't find the original thread) were someone was trying to explain why a render did not finish any faster than it did. I think the example given was an image being rendered on a 12-thread machine, and the image was broken down into 16 parts.  Each image part is assigned to a single thread. My assumption was that as each thread completed it's original image part, it would start rendering out one of the remaining 4 parts, and that all 12 threads would subdivide the remaining 4 parts among themselves.  But what really happens is that only 1 thread gets assigned to one image part, so there will be at least  8 threads sitting idle while the other 4 threads completed their image part. The apparent effect would be that the rendering process had slowed down.

Hope I got that right. If not there are people here who are sure to jump in and correct me.  ;)
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on March 09, 2017, 07:04:54 PM
Norman, that's strange because I have that same setting on my Quadro driver and it fixes the performance issues. You can test some of the other "3D App" presets, I recall one or two others also having a positive effect, but 3D App - Game Dev seemed to work best for me. Since your machine is quite similar to mine I would suggest you test either with the default scene or the Terragen 3 benchmark so that you can know more precisely whether your render times match what is expected. With the benchmark your machine should finish in 4 minutes or under.

We're not sure why this Quadro setting affects render time, it's very odd. You're right of course that Terragen does not use GPU to render, so it's a bit of a mystery. Something to do with the way we use OpenGL or the way the Quadro drivers implement a particular function. It is hard to figure out for certain but I have found some evidence to suggest there are some broken/inefficient "auto-optimize" settings that Nvidia tries to use by default. So for example it may be polling the application many times per second to figure out whether it should use some particular optimization or not and the process of polling in itself slows the application down. That's not necessarily exactly what's happening but an example of something that might be related to the problem.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: nito80 on March 10, 2017, 05:15:09 AM
Thank you for your explanation and suggesting the benchmark, Oshyan. I've just rendered the Terragen 3 benchmark in 3:37 minutes. So I think the driver settings seem to be totally fine now since this time matches your score in the benchmark results. I've also had a closer look at thread usage while rendering. Masonspappy pointed out an interesting idea about thread allocation and I think this may contribute to the divergence in render time/cores usage. With my little test scene, the render starts with 100% usage of cores, but shortly after slows down to 70% and gradually lower, though there are still unrendered parts of the image. Strangely, I would expect some "idle" cores then but the lower usage spreads over all cores equally. But my expectation may be incorrect, depending on how threading is implemented.

Ok, now I'm sratching my head. I just disabled "Defer atmo/cloud" and rendered my test scene again, with all cores. Whoosh - 3:33 minutes (was 9:38 minutes with option on). Rendered again with only 8 cores - 5:20 minutes. So the overall render is much faster, but the timing difference between 8/32 cores equals my previous measures (factor of ~1,5).

The TG3 benchmark renders as follows:
32 cores - 3:37 minutes
8 cores - 7:03 minutes (factor ~2,0)
So either the 32 cores are much slower than one would expect due to the guts of the Z820 or the 8 cores render incredibly fast. :) But I think my inital issue is solved - the more cores used, the faster.

Since disabling "Defer atmo/cloud" offers a great improvement in speed, are there any drawbacks when not using this option?

Thanks again,
Norman
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on March 10, 2017, 04:24:40 PM
It sounds like your machine is rendering as expected now, so that's good to hear. Multithreaded performance does not scale strictly linearly (twice the threads does not equal twice the performance) due to various factors that we tend to refer to as "overhead" for multithreading. In the case of your 8 thread vs 32 thread test, however, I would say the likely biggest contributor is the fact that you have 16 physical cores, thus when rendering with 8, render calculation can be distributed entirely on the physical cores, which are much faster than hyperthreading resources. To compare fairly, I would suggest 8 vs. 16 threads, where you should see close to that 2x performance difference. Adding an additional 16 hyperthreading threads should only increase performance by a *maximum* of 20% due to known limitations of hyperthreading itself. The actual typical hyperthreading performance increase is usually more like 10%, and this can easily be lost to multithreading overhead when dealing with large numbers of threads (>16).

Defer Atmo/Cloud renders the atmosphere and clouds differently, with higher quality vs. when disabled. You can generally get equivalent performance with adjustment of antialiasing settings (which control the number of samples taken when rendering the atmosphere elements with Defer Atmo enabled). If you want very fast rendering, reduce AA to low levels (e.g. 2). You will generally get higher quality results with Defer Atmo, but you do need to get familiar with how to best tune the AA settings, including customizing the "First Sampling Level" and "Pixel Noise Threshold". There are some extensive discussions of all of that elsewhere on the forums.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: DannyG on May 09, 2017, 07:11:47 AM
Is there going to be a benchmark for Version 4 with the new clouds? Going by the emails I recieve from customers I think this is something that is very important. New clouds are great but with they require long rendertimes. This needs to be trouble shot and a standard benchmark scene can help everyone understand them better, by comparing results etc. I know things are busy back there. Perhaps in the near future? 
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on May 09, 2017, 10:48:29 AM
Quote from: Danny on May 09, 2017, 07:11:47 AM
Is there going to be a benchmark for Version 4 with the new clouds? Going by the emails I recieve from customers I think this is something that is very important. New clouds are great but with they require long rendertimes. This needs to be trouble shot and a standard benchmark scene can help everyone understand them better, by comparing results etc. I know things are busy back there. Perhaps in the near future?

I think we do not have to wait for PS to do this. Anybody can mock up a sky with the new cloud(s) and share it so people can render it and see the numbers. No?
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on May 10, 2017, 08:28:43 PM
We do intend to do a new benchmark specifically for TG4. It will likely include at least one free object (not necessarily new) to properly test the object rendering aspect, as well as v3 clouds. We don't have a timeline for delivery of the benchmark, but I would guess some time in the next couple months.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: jaf on August 07, 2017, 07:22:03 PM
Any word on the a new benchmark?
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on August 08, 2017, 05:50:51 PM
We'll be releasing a couple of new things in the next month or two that go hand-in-hand (resources, benchmark). :)

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: jaf on August 08, 2017, 06:09:10 PM
Thanks Oshyan!
Title: Re: Terragen 3 Benchmark released
Post by: silentgod on August 10, 2017, 03:36:56 PM
Terragen got a couple of mentions with regards to Threadripper.   :D

https://www.youtube.com/watch?v=LrG30TZMQS8

Benchmarked:
Skip to 10:25 ~ 2:38sec
Skip to 13:54 ~ 2:46sec

Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on August 11, 2017, 03:07:16 PM
Wow, those are some incredibly impressive render times. They'd be 2nd on our list of the older TG3 benchmark times, bested only by a dual CPU, 14 core-per-CPU Xeon E5-2690 v4 machine, which of course is far, far more expensive than Threadripper. That's an absolutely amazing result for a single CPU machine. I'm looking forward to seeing how Threadripper does on the upcoming Terragen 4 benchmark too.

It's unfortunate that 2p (dual CPU) motherboards seem to be reserved for Epyc at the moment because it runs at significantly lower clock speeds, even in the 16 core model. Imagine a dual Threadripper machine...

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: silentgod on August 12, 2017, 08:08:34 AM
Quote from: Oshyan on August 11, 2017, 03:07:16 PM
Wow, those are some incredibly impressive render times. They'd be 2nd on our list of the older TG3 benchmark times, bested only by a dual CPU, 14 core-per-CPU Xeon E5-2690 v4 machine, which of course is far, far more expensive than Threadripper. That's an absolutely amazing result for a single CPU machine. I'm looking forward to seeing how Threadripper does on the upcoming Terragen 4 benchmark too.

It's unfortunate that 2p (dual CPU) motherboards seem to be reserved for Epyc at the moment because it runs at significantly lower clock speeds, even in the 16 core model. Imagine a dual Threadripper machine...

- Oshyan


Dual Threadripper....?

(http://static.rcgroups.net/forums/attachments/3/4/0/8/6/7/a5333082-82-drooling-homer-simpson.jpg.png)

If only the cost of my liver, kidney & limbs would cover the cost of these.

Title: Re: Terragen 3 Benchmark released
Post by: D.A. Bentley (SuddenPlanet) on November 11, 2017, 04:41:27 PM
Not that it's all that important, but I noticed on the Terragen 3 Benchmark spreadsheet there is a minor error on the 14th line.  Someone named "viewport" listed their CPU as an Intel Xeon E5-2697v2, which is the same CPU I have in my workstation.  I happen to know it's a 12-core single cpu (not a Dual 6-core).  Just a minor distinction.  https://ark.intel.com/products/75283/Intel-Xeon-Processor-E5-2697-v2-30M-Cache-2_70-GHz (https://ark.intel.com/products/75283/Intel-Xeon-Processor-E5-2697-v2-30M-Cache-2_70-GHz)

Interestingly enough, I am on Windows 10 64-bit, and my score is slower than the same CPU running under Mac OS 10.9.  I got a time or 4:22 (TG3 v3.4.00) but my RAM is only running at 1333MHz.  I would be curious to see how much difference it would make to OC my RAM to 1600MHz.  My rendering time in TG4 4.1.17 was 4:28.
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on November 11, 2017, 04:47:24 PM
Thanks for the correction, I have updated that info.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: D.A. Bentley (SuddenPlanet) on November 15, 2017, 04:39:45 PM
Just FYI, I did get my RAM running at 1600MHz, and it took my TG3 time down to 4:16 from 4:22, so 6 seconds gain in performance (not bad).  I submitted my result to the form, and noticed it was there at the bottom.  ;)
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on November 15, 2017, 08:46:08 PM
Yeah, we have to manually sort that spreadsheet. We are notified whenever new submissions are made, but sometimes it takes a little while for us to re-sort it. I'll do that shortly.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: jaf on February 24, 2018, 11:57:08 AM
Is the new benchmark on the horizon?  Why not "throw it out" to the group here and let us build, tear apart, and build again.  I'm pretty sure most of us want to see Planetside work on bugs and new features, so letting the community do the majority of the benchmark prep ideal.

Make some guidelines.... TG4 1.18.1 the base -- a person should be able to purchase TG4 and load and render the benchmark without error.  Try to include the features added after TG3.  Not sure if the free version of TG4 should be included or not -- probably should so a new buyer can have a good idea how his/her computer will respond.  New benchmark gets final approval from Planetside.

Just a thought and I think it would be educational -- different ideas, pros and cons, etc.
Title: Re: Terragen 3 Benchmark released
Post by: jaf on February 26, 2018, 10:48:29 AM
Well, I guess no one cares.

I use the benchmark to audit my system once-in-a-while to see if there's any slowdown (especially now with the Spectre and Meltdown chip flaws.)   Though I have an AMD processor, I still see it could have problems (some say AMD's are clean, others, they aren't.)  TG is a major part of my owning a desktop computer.

I did update my bios a few days ago so that got me thinking about the promised benchmark (when I purchased TG4.)
Title: Re: Terragen 3 Benchmark released
Post by: D.A. Bentley (SuddenPlanet) on February 26, 2018, 04:13:46 PM
I'm just replying to show I care.  I read your last post, and was waiting for some replies as well.  Forum seems less active lately.  Maybe everyone is busy being employed.  heh.

Anyway, maybe you should post once more.  Your current post number might offend someone.  ;)
Title: Re: Terragen 3 Benchmark released
Post by: Dune on February 27, 2018, 02:14:39 AM
A lot of us here can make a benchmark scene, no doubt. Shouldn't be hard, should it? Some basic river scene maybe, with some simple cliff hanging over, a few fake stones, a distorted cube again, some (free) shrub, grass and/or trees and a v3 sky?
Title: Re: Terragen 3 Benchmark released
Post by: archonforest on February 27, 2018, 05:02:15 AM
Quote from: Dune on February 27, 2018, 02:14:39 AM
A lot of us here can make a benchmark scene, no doubt. Shouldn't be hard, should it? Some basic river scene maybe, with some simple cliff hanging over, a few fake stones, a distorted cube again, some (free) shrub, grass and/or trees and a v3 sky?
Exactly and just send it to PS for a fast approval and we have it.... I would be interested to do one but I am not very good on making nice cliffs or rocks...
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on February 27, 2018, 06:56:41 PM
Creating a good benchmark scene that actually does a good job of testing a variety of Terragen features and giving a representative value for a broad range of computer hardware is more challenging than you might think. Not so much in the sense of actually building the scene, but as far as determining how to build it, what to put it in it, what render settings, etc. Combine that with other constraints or desires, such as including object files (size of download then becomes a concern, usage rights, compressibility, etc.). Basically there are a lot of factors that are not so obvious and are particular to benchmarking, as opposed to just making a "nice looking scene".

That being said, we could potentially open it up to submissions, scenes that people were happy to share and did not mind being redistributed for the purposes of benchmarking. But we would have to take any submitted scenes and evaluate them for suitability, likely adjusting settings (especially render settings), and adding or removing things as-needed. It might be less work, it might not be, hard to say. But we might get a prettier end-result scene. ;)

Anyway, we have not spent a ton of time on the benchmark of late, but it's on the list of things to tackle in the next month or two. Once I do dive into it, it will not take weeks, so you needn't worry it's taking a lot of time away from other things. If there is genuine interest from multiple people in submitting scenes for possible use as a benchmark, I'll discuss with Matt and see if he's amenable to that, and if so what general criteria we'd want submissions to adhere to. But we'd want to see at least a couple of people throwing their hat in on submissions to take that route I think. If not we'll just do the scene internally, it will be simple, but will showcase a reasonable variety of TG functions, and will accomplish the other more technical goals.

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: jaf on March 01, 2018, 06:38:33 PM
In my previous life (seems like previous because it was close to 40 years ago) I had the job of writing ATP's for a major military flight simulation contract.  As Oshyan wrote, it's not easy.  The benchmark should feature the changes from TG3 to 4 and the database would be an asset to potential customers as to the capability of their system (for using TG4 -- Edit.)

Just thought it would offload the task from the Planetside staff and be an interesting project.
Title: Re: Terragen 3 Benchmark released
Post by: Dune on March 02, 2018, 02:03:06 AM
I can understand this. So I won't burn myself on it.
Title: Re: Terragen 3 Benchmark released
Post by: D.A. Bentley (SuddenPlanet) on March 02, 2018, 02:11:46 PM
This might be a silly idea, but what about having a contest/challenge to build the TG4 Benchmark scene.  Maybe wait for the Cliff Challenge to finish up, or maybe not, but it might be a good exercise to see what many different TG users come up with.  It would be helpful to know what new features in TG4 should be tested in the benchmark scene.  For example, isn't Cloud Layer v3 new in TG4?

Just thinking out loud...

Derek 
Title: Re: Terragen 3 Benchmark released
Post by: jaf on March 02, 2018, 02:49:59 PM
That might work ... have to stay away from imported objects/terrains and have a resolution and memory size limits.  Should it run on the free version?  If it did, a person not owning TG4 could still evaluate it.

I really think a collaborative effort, where everyone could run the same scene as it developed, would work better  So many diverse talents like nodes/terrain texturing, clouds, lighting, water, etc. that could contribute.

I do have my own system of auditing my system (for TG rendering) by simply using the Project Settings.  If there's a major change to my hardware or software, I can load a TG4 scene, render it, and compare the results.  I keep a text file (Signature) so I can know what hardware configuration I originally rendered it on.
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on March 02, 2018, 05:20:50 PM
Yes, we'd want to test v3 clouds, and we actually do want to include at least 1 actual external object this time as it is of course something commonly used in TG. It should be a relatively small (file size) object though.

Being able to run in the Free version is a definite requirement, and that sets up specific render resolution limits too, of course.

Anyway, after discussion with Matt internally, I think we may want to use (or include elements from) one or more user's scenes. We'll figure out exactly how we want to tackle that soon. Probably not a contest or open collaboration though as once again we'll have very specific goals and that would probably be too chaotic and challenging to coordinate. But the feedback and ideas are definitely appreciated!

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: jaf on March 03, 2018, 10:42:04 AM
Sounds good.  :)
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on February 24, 2019, 04:18:22 PM
I'll post a separate thread about this, but I wanted to mention we need a couple volunteers with mid-range modern CPUs to test a draft of the benchmark and make sure it has a reasonable render time on their hardware. Jaf, your 1800x would be a good test. :D We're roughly looking at aiming for 10-15 mins on a Ryzen 2700x. Anyone interested in testing please email support AT planetside.co.uk

- Oshyan
Title: Re: Terragen 3 Benchmark released
Post by: geekatplay on February 26, 2019, 11:58:11 AM
I can run tests if you need (i9 9900k / RTX1070 / 64Gb RAM)
Title: Re: Terragen 3 Benchmark released
Post by: Oshyan on February 26, 2019, 09:21:54 PM
Thanks, I'll get back to those who volunteered directly. Still would be ideal to have a newer Ryzen, but we'll make do.

- Oshyan