Should global points on CPU/GPU get a tune up?

I think global points could use a tune up. Just opening up a discussion. I don't really care for the outcome... It's just easier to make suggestions for hwbot than it is to make your own hwbot.

Taking a quick look, at a glance the following is true for the current global point categories for CPU and GPU benchmarks...

Covering the 3d tests divided by GPU count:

Doesn't scale with anything beyond 1x GPUs (scores in 2x-4x just get worse): 3d01, 3d05, aquamark, 3d06

Doesn't scale with anything beyond 2x GPUs (scores in 3x-4x just get worse): 3d03

Doesn't scale with anything beyond 3x GPUs (scores in 4x just get worse): Unigine Heaven

Scales with 4x: Vantage, 3d11

Covering the 2d tests divided by core count:

Doesn't scale with anything beyond 4x cores (extra cores cannot be utilized): PCM05

Doesn't scale (really) with anything beyond 12x cores: wprime32m, wprime1024m

Scales on everything (but doesn't get any globals): UCBench2011

I'll only mention that I think UCBench should be eligible for Globals...

Ohh... And that I now agree with those who originally said that PCM05 was broken and needed to be removed. When tweaks alone will more than double a benchmark.. It's no longer a hardware benchmark. That's a discussion for another thread though.

I'm not sure. I'm ok with having clear and easy to follow rankings and leaderboards. You are benching XYZ, so it is ranked individually as XYZ, instead of users having to check if GHI is included as well, if you know what I mean?

Also, benching e.g.... 3-way GPU is partly for the challenge of benching 3-way GPU.

Most benchmarks scale more with CPU than GPU, so i'd be more interested in seeing e.g... AM3 reclassed as a CPU benchmark, which is "the other way" of dealing with your suggestion.

wprime1024m scales with more than 12 threads. I'm 99% sure of that, but if you can "prove" me wrong, feel free to do so I think 12 is also too low for 32m, but there is indeed a limitation because of the thread execution time (time needed to execute thread X is higher than time needed for thread 1 to finish it's part of the total computation). This also applies to 1024m, but the critical thread count will be much higher, as the computation pr thread is 32 time bigger. On older systems the optimal thread count for both of these will differ, as the computation time increases - so while there may not be a point in having a wprime32m ranking for modern 24 core systems, maybe the oldest rigs with this configuration will benefit from all 24 threads. THis also means that later, the optimized thread count will drop, so we'd have to remove points as tech gets better - which is indeed a problem.

PCMarks do in fact scale beyond the "official" thread count, but not by that much. You can verify this for pcmark04 for example, use a thuban set to 2,3,4,5,6 cores or whatever. Not that much, but there is a difference. S_A_V did some testing on this I believe. 05 does the same thing. Got no clue about 3D, so can't comment on those - but:

Note that what's the case today doesn't have to be the case in the future. We don't know if Heaven won't scale to 4x with the next generation of GPUs. I'm pretty sure that what you say about 01 is correct, can't imagine there being any 4-way scaling in that bench ever

I'm also thinking that the optimal thread count stuff applies to 3D, as in optimal GPU count. AM3 does scale with 2x GPU on older cards, but not any more (in other words we do at least have to keep the hardware rankings for 2x GPU forever). Perhaps all 3D benchmarks will go that route as time goes by, 1x is the best option to get the highest score. Then we again would have to revise these rankings every now and then, which is not a good thing IMO.

Points should be based on popularity factor (already in "what to add/fix in next revisions" list), not on scaling factor.

Scaling factor is not constant, it is based on current (latest) gen CPU/GPU performance, OS and drivers support, benchmark settings, etc.

What if today we "improve" points for current situation with scaling and tomorrow next gen CPU (IB-E for example) will make better scaling for 3/4-way configurations, then we will need another improvement?

Hwbot point system already a bit too complex. Adding another 1-2 variables in calculation algorithm may slow down database server and make point system more hard to understand.

IMO, the problem will only change if/when GPUs stop doubling in power every 12-18 months and CPUs start to scale in performance faster than ~10-15% every generation.

It's kinda weird.... a lot of people are happy if a new CPU adds 10% performance, but if a new GPU is less than 50% better, people complain or refuse to buy it.

Points should be based on popularity factor (already in "what to add/fix in next revisions" list), not on scaling factor.

 

Scaling factor is not constant, it is based on current (latest) gen CPU/GPU performance, OS and drivers support, benchmark settings, etc.

What if today we "improve" points for current situation with scaling and tomorrow next gen CPU (IB-E for example) will make better scaling for 3/4-way configurations, then we will need another improvement?

 

Hwbot point system already a bit too complex. Adding another 1-2 variables in calculation algorithm may slow down database server and make point system more hard to understand.

Agree, during Gulftown era 4x 5870 or 2x 5970 was the way to go in 3d03,3d05,06, aquamark or heaven

Points should be based on popularity factor (already in "what to add/fix in next revisions" list), not on scaling factor.

 

Scaling factor is not constant, it is based on current (latest) gen CPU/GPU performance, OS and drivers support, benchmark settings, etc.

What if today we "improve" points for current situation with scaling and tomorrow next gen CPU (IB-E for example) will make better scaling for 3/4-way configurations, then we will need another improvement?

 

Hwbot point system already a bit too complex. Adding another 1-2 variables in calculation algorithm may slow down database server and make point system more hard to understand.

Popularity already decides the amount of points given?

The time factor could be removed, though - atm single core wprime scores are worth 60p, although there is an insane amount of scores in those categories. Dropping because time passes by is not a good idea. There's no good reason to reward 1x and 4x differently because the 4x category has more recent subs.

Wow really it lets you vote for ALL 3 of them?

The only thing that needs a 'tune' is adjusting global points for low categories. E.g. 40 core PC05, only one score, gets 60pts. As per the suggested item in the bug database which has been there a long while now. http://bugs.hwbot.org/browse/HWBOT-694

Did the UCBench issue ever get sorted? Is it ok for Globals? Tbf PiFast has a worse bug and is still in the rankings.

I like the current system

I suggested the following set:

30-20-15-13-11-10-8-7-6-5-4-3-2-1-0.5 for 1st-15th for categories with less than or equal to... 30 subs would solve some if the issues... and then go for the regular distribution above that.

Top scores are top scores and should be rewarded to some extent regardless of the competition. 30 is still not much at all, you can easily get alot of low skill points in the other categories, too - take a random 3770k, go full pot, and you'll see what I mean.

Top scores are top scores and should be rewarded to some extent regardless of the competition. 30 is still not much at all, you can easily get alot of low skill points in the other categories, too - take a random 3770k, go full pot, and you'll see what I mean.

I agree with that. Any 3770K I've tested (just under ten or so) can do about 6.2GHz on LN2 - take it out for a spin on 3D and you've got a gold mine for hardware points. Similar was true for any SB at 5.5GHz+, except that was doable on DICE/SS. In the popular 3D hardware categories those are a formula for low skill automatic/easy points.

That is hardware points though mainly, and possibly a separate topic - there are lots of ways to grind hardware points depending on platform and cooling method... This topic isn't about hardware grinding. This topic is global points. The global points in question in this topic are almost exclusively exotic globals with relatively low competition.

You can't take any random 3770K full pot and dominate in 3d03, vantage, 3d11, or heaven in multi-GPU. Global points in those categories are hard in that they require hard clocking on both CPU/GPU.

If you look at multi-GPU benchmarks in categories other than those, there are a lot of scores from years ago where top spots are still held because no one has bothered running it again. 4x 7970 on IB would be easy to take those spots versus 4x 5870 ran just above stock, but pros/enthusiasts mostly don't even bother with those easy points because they'd rather compete in 3d11/vantage/heaven where the GPU scaling matters and people pay attention.

My goal with bringing this topic up is to raise awareness about "gaming" global points, and perhaps lead to the rankings being more "valid" in the eyes of the community.

By the way, one of the biggest complaints in our previous HWBoint systems was that it was very difficult for the 'regular' guys to compete for the top ranks. Creating a system that tolerates a wider range of categories to generate +/_ the same amount of points will allow more people to be able to compete, but will also reduce the direct competition aspect of it. Direct competition as in "I beat you, so I'm ranked higher".

It's hard to find a balance between making sure there's intense competition, which is an exclusive format as it requires lots of time and resource, and that everyone can have a shot at the top ranking.

I agree with Massman, if the point of globals is too create a bunch of guys running LN2 on the exact same, newest hardware (which if the hobby keeps getting more popular will increasingly be those with sponsors that are able to bin chips and GPUs) then start restricting the so called "no-skill" globals and you'll have mission accomplished. By putting good points in those odd categories you encourage people to compete in them. I would much rather see globals start to force someone to pour LN2 on a quad socket server rig than yet another 'clocker running LN2 on an Ivy 7970 setup.

It almost makes me want the globals to be expanded, perhaps to include globals for each socket platform and for each GPU platform. Why does the guy who is the very best with 775 or Nvidia 8 series unable to get globals just because he might not have the cash to outlay on the newest CPU or new GPUs. According to HWBOT rankings he is not as good of an overclocker. This would also create a way to get some good points for beginners/intermediates that might be discouraged by the current system of 0.1 pts for their main rig submissions.

Well...that's what global means.... the best scores for the benchmark.

I'm happy with the level of support HWB gives to older hardware.

If you are thinking of adding architecture or socket WR points...... silverware maybe, I don't know about points.