Go Back   HWBOT forum > HWBOT Information > HWBOT Development: bugs, features and suggestions > HWBOT Rev.4

HWBOT Rev.4 Everything about HWBOT Rev.4

 
 
Thread Tools Display Modes
  #1  
Old 11-11-2011
Massman's Avatar
Massman Massman is offline
In the distance
 
Join Date: Jul 2006
Location: Taipei
Posts: 22,579
Thanks: 808
Thanked 1,136 Times in 557 Posts
Massman has a spectacular aura aboutMassman has a spectacular aura aboutMassman has a spectacular aura about
Send a message via MSN to Massman
Default R4.2.4/R4.3: HWBoints finetuning

As mentioned earlier in this subforum, we have been looking at the feedback we received from the transition to HWBOT R4 and the changes it brought to the point algoritm. We're currently in the process of finetuning the boints and will introduce all changes in one big package which will be included in either R4.2.4 (due 19/11) or R4.3 (due 01/12).

FYI - next to the updates mentioned below, we will also implement a fix for the existing problems with the Global Team Powerpoints as described here. The fix is currently planned for R4.3, but the development is R4.2.4 is going so incredibly smooth that we are considering moving the fix to the current development sprint. If we can't include the GTP fix in R4.2.4, we will postpone the fixes described below and package them in R4.3. Just a matter of keeping things coherent.

Changes:

Currently, we have installed two updates to the member hwboints algoritm for global rankings:

1) participation weight based on 12-month period instead of 6 months
2) introduce min_participation constant

Background information:

The current hwboint algoritm used for calculating the points awarded to global member rankings is, just like the very first hwboint revision, mainly based around the amount of competition there is within the ranking. In Rev4, we introduced a time limitation to calculated the level of competition within one ranking. From Rev4, competitivity is determined by the amount of different users who have submitted to a certain global ranking over the past 6 months.

Based on feedback such as "I'm losing points in Wprime if I submit more results", we found out that a six month period is quite short. We also found that the drop in points is quite big if, for some reason, a benchmark is suddenly not so popular anymore. Extreme scenarios can be found in Wprime where the global points for #1 position dropped from 133p (without 6-month period) to 13,1p (with 6 month period).

To resolve this situation we've prolonged the period for calculating the level of competition in a specific ranking as well as introduced a minimum participation level for every global ranking.

Practical consequences:

Most importantly, practically this means that pretty much everyone is gaining points. Hooray!

On a more serious note, these two updates do have some interesting consequences. For one, in case a benchmark is no longer popular, the points will drop at a much lower speed than they did now. Secondly, this also means that every global ranking has a minimum amount of points. For every #1 global rank, you will now receive a minimum of 59,8 points. This is quite interesting for:

- people who frequently run 3/4-way setups, but not subzero
- for enthusiast league members who score top scores in overall benchmark rankings
- 1xCPU multi-threaded CPU applications (where competitivity drops due to lack of newly released single core cpus)
- unpopular but challenging benchmarks (eg: pcmark)

In general, you can just think about this update as a guarantee that a global #1 will always receive a minimum of 59.8p. I also want to stress again that these fixes only affect the points awarded in global member rankings and that the formula for calculating the league ranking is not affected. In other words: the game is completely the same, only a few more ways to be competitive.

Test environment:

This weekend, we'll updated our test server environment with a new dump from the result database and open it up for you to check it out and see your rank. For now, here are just some examples.

Some examples:

- Booj' 3xGPU Heaven DX11: 5.1p -> 59.8p
- Ananerbe's 1xCPU Wprime 1024M record: 11.0p -> 59.8p
- **DP**'s 3xGPU Vantage record: 8.1p -> 59.8p
- Nacho_arroyo's 6xCPU Wprime1024M record: 9.6p -> 59.8p
- Moonman's 3xGPU 3DMark01 record: 6.3p -> 59.8p

Example of the updated leagues. Note: this is using an outdated dump of the production database. It's very likely that your latest submissions are not taken into account yet.





__________________
Where courage, motivation and ignorance meet, a persistent idiot awakens.

For all HWBOT community related questions, contact Christian Ney or Websmile. For any other questions, contact me at pieter@hwbot.org.
  #2  
Old 11-12-2011
Hondacity's Avatar
Hondacity Hondacity is offline
Really Xtremely Doomed
 
Join Date: Jun 2008
Posts: 2,435
Thanks: 0
Thanked 0 Times in 0 Posts
Hondacity
Default

im confused x 10^2
  #3  
Old 11-12-2011
Mr.Scott's Avatar
Mr.Scott Mr.Scott is offline
Fully capable of playing games
 
Join Date: Jan 2009
Location: UNITED STATES
Posts: 2,197
Thanks: 848
Thanked 361 Times in 185 Posts
Mr.Scott is on a distinguished road
Default

Well....Faster should be happy now.
__________________

Warp9-systems, the Antichrist of competitive benchmarking.
Loved by thousands, hated by millions.
  #4  
Old 11-12-2011
richba5tard's Avatar
richba5tard richba5tard is offline
HWBOT Forum Administrators
 
Join Date: Mar 2006
Location: Belgium
Posts: 6,199
Thanks: 6
Thanked 93 Times in 29 Posts
richba5tard has disabled reputation
Default

Quote:
Originally Posted by Hondacity View Post
im confused x 10^2
What's there to be confused about? To sum up massman's lengthy explenation in one sentence: #1 in global points will always get you about at least +-60 points, no matter how unpopular the ranking may be.

Another change scheduled for the near future is a better way to handle team power points, but that has not been implemented yet.
__________________
  #5  
Old 11-12-2011
TaPaKaH TaPaKaH is offline
robot overlord
 
Join Date: Dec 2006
Location: The Netherlands
Posts: 2,808
Thanks: 34
Thanked 268 Times in 102 Posts
TaPaKaH
Send a message via ICQ to TaPaKaH Send a message via Skype™ to TaPaKaH
Default

so now there's an even bigger disbalance between TPPG and TPPH as you started awarding more of the prior ?
  #6  
Old 11-12-2011
Massman's Avatar
Massman Massman is offline
In the distance
 
Join Date: Jul 2006
Location: Taipei
Posts: 22,579
Thanks: 808
Thanked 1,136 Times in 557 Posts
Massman has a spectacular aura aboutMassman has a spectacular aura aboutMassman has a spectacular aura about
Send a message via MSN to Massman
Default

Quote:
Originally Posted by Sam OCX View Post
so now there's an even bigger disbalance between TPPG and TPPH as you started awarding more of the prior ?
No, because this change only affects MEMBER points.
__________________
Where courage, motivation and ignorance meet, a persistent idiot awakens.

For all HWBOT community related questions, contact Christian Ney or Websmile. For any other questions, contact me at pieter@hwbot.org.
  #7  
Old 11-12-2011
K404's Avatar
K404 K404 is offline
nVidia FTW!
 
Join Date: Jan 1970
Location: Glasgowvic, Scotistan. Average Kingdom of Un-United Britain..
Posts: 3,855
Thanks: 16
Thanked 146 Times in 85 Posts
K404
Default

Well I guess if the unpopular crap gets awarded a lot more points, more people will be motivated to bench and fewer categories will be unpopular.

Good for keeping a wide selection of hardware under some form of spotlight, bad (open to interpretation) because again points are the motivator to bench. Points bring binning. Binning starts to undermine the "little guy"

I know not everyone feels the same way, that's cool



Weirdly, getting a good score with.... e.g.... an AMD tri-core is worth more points because it has a global ranking than it does for being any kind of popular chip
__________________
Quote:
Originally Posted by Hondacity View Post
i believe in K404! :P
https://twitter.com/K404ExtremeOC

I do not believe in 1st place by any means, at any cost.

Quote:
Originally Posted by der8auer View Post
Knowing workarounds to get a benchmark to run is not a skill. It's a combination of knowledge and wasting time.

Last edited by K404; 11-12-2011 at 13:17.
  #8  
Old 11-12-2011
Massman's Avatar
Massman Massman is offline
In the distance
 
Join Date: Jul 2006
Location: Taipei
Posts: 22,579
Thanks: 808
Thanked 1,136 Times in 557 Posts
Massman has a spectacular aura aboutMassman has a spectacular aura aboutMassman has a spectacular aura about
Send a message via MSN to Massman
Default

FYI, the point distribution comparison between "uncompetitive Wprime32M 3xCPU" (global points) and "competitive 8800GTX 3DM01 1xGPU" (hardware points).

Beating lots of people in a hardware class yields more points (in general) than beating few people in a global class.



__________________
Where courage, motivation and ignorance meet, a persistent idiot awakens.

For all HWBOT community related questions, contact Christian Ney or Websmile. For any other questions, contact me at pieter@hwbot.org.
  #9  
Old 11-12-2011
K404's Avatar
K404 K404 is offline
nVidia FTW!
 
Join Date: Jan 1970
Location: Glasgowvic, Scotistan. Average Kingdom of Un-United Britain..
Posts: 3,855
Thanks: 16
Thanked 146 Times in 85 Posts
K404
Default

Personally, I think the exponent is too sharp. I know we've talked about that before, but the global tail off is even worse
__________________
Quote:
Originally Posted by Hondacity View Post
i believe in K404! :P
https://twitter.com/K404ExtremeOC

I do not believe in 1st place by any means, at any cost.

Quote:
Originally Posted by der8auer View Post
Knowing workarounds to get a benchmark to run is not a skill. It's a combination of knowledge and wasting time.
  #10  
Old 11-12-2011
richba5tard's Avatar
richba5tard richba5tard is offline
HWBOT Forum Administrators
 
Join Date: Mar 2006
Location: Belgium
Posts: 6,199
Thanks: 6
Thanked 93 Times in 29 Posts
richba5tard has disabled reputation
Default

Please explain exponent. : ) Do you mean the curve is too sharp? Too many points for #1, too few for #10?
__________________
 


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:04.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright HWBOT 2004 - 2015