View Full Version : R4.2.4/R4.3: HWBoints finetuning
As mentioned earlier in this subforum, we have been looking at the feedback we received from the transition to HWBOT R4 and the changes it brought to the point algoritm. We're currently in the process of finetuning the boints and will introduce all changes in one big package which will be included in either R4.2.4 (due 19/11) or R4.3 (due 01/12).
FYI - next to the updates mentioned below, we will also implement a fix for the existing problems with the Global Team Powerpoints as described here (http://hwbot.org/forum/showthread.php?t=27487). The fix is currently planned for R4.3, but the development is R4.2.4 is going so incredibly smooth that we are considering moving the fix to the current development sprint. If we can't include the GTP fix in R4.2.4, we will postpone the fixes described below and package them in R4.3. Just a matter of keeping things coherent.
Changes:
Currently, we have installed two updates to the member hwboints algoritm for global rankings:
1) participation weight based on 12-month period instead of 6 months
2) introduce min_participation constant
Background information:
The current hwboint algoritm used for calculating the points awarded to global member rankings is, just like the very first hwboint revision, mainly based around the amount of competition there is within the ranking. In Rev4, we introduced a time limitation to calculated the level of competition within one ranking. From Rev4, competitivity is determined by the amount of different users who have submitted to a certain global ranking over the past 6 months.
Based on feedback such as "I'm losing points in Wprime if I submit more results", we found out that a six month period is quite short. We also found that the drop in points is quite big if, for some reason, a benchmark is suddenly not so popular anymore. Extreme scenarios can be found in Wprime where the global points for #1 position dropped from 133p (without 6-month period) to 13,1p (with 6 month period).
To resolve this situation we've prolonged the period for calculating the level of competition in a specific ranking as well as introduced a minimum participation level for every global ranking.
Practical consequences:
Most importantly, practically this means that pretty much everyone is gaining points. Hooray!
On a more serious note, these two updates do have some interesting consequences. For one, in case a benchmark is no longer popular, the points will drop at a much lower speed than they did now. Secondly, this also means that every global ranking has a minimum amount of points. For every #1 global rank, you will now receive a minimum of 59,8 points. This is quite interesting for:
- people who frequently run 3/4-way setups, but not subzero
- for enthusiast league members who score top scores in overall benchmark rankings
- 1xCPU multi-threaded CPU applications (where competitivity drops due to lack of newly released single core cpus)
- unpopular but challenging benchmarks (eg: pcmark)
In general, you can just think about this update as a guarantee that a global #1 will always receive a minimum of 59.8p. I also want to stress again that these fixes only affect the points awarded in global member rankings and that the formula for calculating the league ranking is not affected. In other words: the game is completely the same, only a few more ways to be competitive.
Test environment:
This weekend, we'll updated our test server environment with a new dump from the result database and open it up for you to check it out and see your rank. For now, here are just some examples.
Some examples:
- Booj' 3xGPU Heaven DX11 (http://hwbot.org/submission/2210893_booj_unigine_heaven___xtreme_preset_dx11_3 x_radeon_hd_6970_4740.25_dx11_marks): 5.1p -> 59.8p
- Ananerbe's 1xCPU Wprime 1024M record (http://hwbot.org/submission/2215151_ananerbe_wprime_1024m_sempron_150_755sec_6 25ms): 11.0p -> 59.8p
- **DP**'s 3xGPU Vantage record (http://hwbot.org/submission/2149355_dp_3dmark_vantage___performance_3x_geforce _gtx_580_66417_marks): 8.1p -> 59.8p
- Nacho_arroyo's 6xCPU Wprime1024M record (http://hwbot.org/submission/2179555_nacho_arroyo_wprime_1024m_core_i7_extreme_ 980x_75sec_437ms): 9.6p -> 59.8p
- Moonman's 3xGPU 3DMark01 record (http://hwbot.org/submission/2218030_moonman_3dmark2001_se_3x_geforce_gtx_275_9 2957_marks): 6.3p -> 59.8p
Example of the updated leagues. Note: this is using an outdated dump of the production database. It's very likely that your latest submissions are not taken into account yet.
http://hwbot.org/blog/wp-content//pro1.jpg
http://hwbot.org/blog/wp-content//ranking.jpg
http://hwbot.org/blog/wp-content//enthusiast.jpg
Hondacity
11-12-2011, 00:14
im confused x 10^2
Mr.Scott
11-12-2011, 01:16
Well....Faster should be happy now.
richba5tard
11-12-2011, 02:39
im confused x 10^2
What's there to be confused about? To sum up massman's lengthy explenation in one sentence: #1 in global points will always get you about at least +-60 points, no matter how unpopular the ranking may be.
Another change scheduled for the near future is a better way to handle team power points, but that has not been implemented yet.
so now there's an even bigger disbalance between TPPG and TPPH as you started awarding more of the prior ?
so now there's an even bigger disbalance between TPPG and TPPH as you started awarding more of the prior ?
No, because this change only affects MEMBER points.
Well I guess if the unpopular crap gets awarded a lot more points, more people will be motivated to bench and fewer categories will be unpopular.
Good for keeping a wide selection of hardware under some form of spotlight, bad (open to interpretation) because again points are the motivator to bench. Points bring binning. Binning starts to undermine the "little guy"
I know not everyone feels the same way, that's cool :)
Weirdly, getting a good score with.... e.g.... an AMD tri-core is worth more points because it has a global ranking than it does for being any kind of popular chip
FYI, the point distribution comparison between "uncompetitive Wprime32M 3xCPU" (global points) and "competitive 8800GTX 3DM01 1xGPU" (hardware points).
Beating lots of people in a hardware class yields more points (in general) than beating few people in a global class.
http://hwbot.org/blog/wp-content//gl-distribution.jpg
http://hwbot.org/blog/wp-content//hw-distribution.jpg
Personally, I think the exponent is too sharp. :) I know we've talked about that before, but the global tail off is even worse :p
richba5tard
11-12-2011, 13:42
Please explain exponent. : ) Do you mean the curve is too sharp? Too many points for #1, too few for #10?
So, the competition points are still not implemented? The ideea is still on the "to do list"?
Ontopic: I don't think it's a good ideea to give so much global points to all the shitty overall's WR :D. Makes the WR's that really matters (single card 3D for example) less important...
Competition points are implemented, but not very transparent. R4.2.4 has some extra overview options for it.
FYI, the difference between "shitty overall wr" and a "single gpu wr" is over 100p: 59.8p (eg: 3x CPU wprime) versus 168p (eg: 1x GPU 3DM06).
@ RB.... Either too many points for #1 or not enough for #10 :D
Massman and me spoke about this once.... what if the points increase was smoother, but with "bumps" at.... eg.... 100, 50, 20, 10 and 5
http://i2.photobucket.com/albums/y38/K404/pureawesome.jpg
@PJ.... at the moment, there is a difference. If there is enough of a reason "for the masses" to bench the category then the points gap will close, you know that :)
@PJ.... at the moment, there is a difference. If there is enough of a reason "for the masses" to bench the category then the points gap will close, you know that :)
?
It's 'unreasonable' that the points gap closes when more people compete in a ranking?
My profile says I got 12 points from competitions but I cannot find any details about that.
I'm reffering to MOA, GOOC, Country Cup, etc. Where can I see what points I have and how can I add the competitions I've won? :)
FYI, the difference between "shitty overall wr" and a "single gpu wr" is over 100p: 59.8p (eg: 3x CPU wprime) versus 168p (eg: 1x GPU 3DM06).
Yes, but 2 x "shitty overall wr" = "single gpu wr" and that's not exactly fair in my book. Single GPU WR takes much more work to obtain that 2 x 1-core WPrime WR's for example :)
?
It's 'unreasonable' that the points gap closes when more people compete in a ranking?
I honestly don't know. :)
Basing points purely on popularity....... if there were 10,000 results each of 3DM11 single card and 3xCPU wP32M.... are they equally impressive achievements that deserve equal points?
Should the least popular global leaderboard be worth 1/3-1/2 of the points of the most popular?
Not commenting, just giving ya food for thought :)
EDIT: Matose and me are thinking along the same lines. Quick, everyone take cover! :D
My profile says I got 12 points from competitions but I cannot find any details about that.
I'm reffering to MOA, GOOC, Country Cup, etc. Where can I see what points I have and how can I add the competitions I've won? :)
An overview is coming in R4.2.4 and will be visible under the points tab in your profile. Example:
http://hwbot.org/blog/wp-content//cp.jpg
For MOA and GOOC we need additional code to support competitions that are not based on the hwbot database. The code is planned for R4.3 release.
Yes, but 2 x "shitty overall wr" = "single gpu wr" and that's not exactly fair in my book. Single GPU WR takes much more work to obtain that 2 x 1-core WPrime WR's for example :)
Yes, but you only have 15 slots for Global points for your league ranking.
You're missing the point Pieter :).
If I have 6 x "shitty WR" and you have 3 x "single-card WR" we will have the same points, this makes me as good as you?!
richba5tard
11-12-2011, 15:35
6x WR in unpopular ranking is still < 3x single card WR. It's more like 8 to 3. And 3x or 4x SLI requires skill too.
We're not levelling the points, just give an incentive to bench more exotic ranking too. It's not fair that 3x CPU WR now gives you a mere 10 points.
Why? If there's only a few results in the category.......
Because of kinda random categories like the (AMD) 3x CPUs, it's *almost* the same as the hardware ranking. That's not popular, so it only gets a few hardware points. Why should a category with 292 results be worth 60 global points? The PheII 555 only gets 5.8 hardware points
I have the Unigine DX9 world record and that category has 626 results.... zero ponts. It's a world record isn't it?
(I don't want points added for Unigine DX9, but I hope you see my point)
so, now we're coming back to CPU binning, aren't we?
found a good GT => benched 03/05/06/AM3 with 2/3/4way on air, got much higher globals than before
The points don't define how good someone is 100% accurately; it's merely an estimation using some algoritm that weighs in the amount of competition and the position in a certain category. The idea of "good" is very subjective and everyone has a different opinion about it. For you, "good" may mean having a Single-GPU record, for others "good" might mean scoring top positions on various platforms and architectures.
Let's take some examples of those so-called "shitty" records:
- Single core Wprime 1024M record (sempron over 6.1GHz) (http://hwbot.org/submission/2215151_ananerbe_wprime_1024m_sempron_150_755sec_6 25ms)
- Kingpin's 12xCPU Wprime32M record (2 cpus at 5.6G+) (http://hwbot.org/submission/1090781_kingpin_wprime_32m_2x_xeon_x5680_1sec_547m s)
- SF3D's 3xGPU 3DMark11 score (3 GPUs over 1350MHz core) (http://hwbot.org/submission/2200890_sf3d_3dmark11___performance_3x_geforce_gtx _580_22646_marks)
- Mtech's 4xCPU PCMark05 record (overall record) (http://hwbot.org/submission/2218295_mtech_pcmark_2005_core_i7_2600k_50458_mark s)
Each of these scores have some level of difficulty here, but are awarded too little points because there's no competition in the ranking. If someone would try to beat any of these scores, it would take a lot of time and effort to get there. Now, if someone were to be the holder of these 4 records, it would give that person ~ 240p.
Do you honestly believe that, for instance, Single GPU 3DMark06 + Single GPU 3DMark05 (together ~320p) is actually underrated in comparison to those 4 other records?
How much importance you attributed to a certain ranking depends on how you personally feel about benchmarks, how the benchmarks behave (cpu-limited 3D benchmarks), etc. We all know how the Sandy Bridge has affected the benchmarking scene or how Gulftown affected the scene.
people won't aim for those categories if they want points ... they'll just go for the multiCPU server access and Gulftown binning
knopflerbruce
11-12-2011, 19:42
You're missing the point Pieter :).
If I have 6 x "shitty WR" and you have 3 x "single-card WR" we will have the same points, this makes me as good as you?!
Even the "shitty" WRs are hard to beat (unless we're talking about 24x wprime and stuff like that). It's not fair that you can buy some random gulftown and a gtx580, ln2 both, and get plenty of globals in all 3dmarks with no effort, whereas you get only a handful of boints when submitting a score that's actually really good - for example a wprime 3x gold cup. Being top 10 is quite easy in some of those rankings, but making it to top 3 requires alot of skill and effort, so it's a good thing that such submissions will receive an amount of boints that shows the quality of the result.
Edit: this shouldn't apply to ALL rankings, only where you can actually overclock - we don't have to give 60 boints to a server ran at stock, that's just silly. But anything up to and including 12x CPU - yes :)
Hondacity
11-12-2011, 19:49
question by example...for this thread.
e8600 wprime 2core 1st place, 60pts?
i5-670 wprime 2core 1st place, 60pts?
those are two different classes imo.
knopflerbruce
11-12-2011, 20:11
question by example...for this thread.
e8600 wprime 2core 1st place, 60pts?
i5-670 wprime 2core 1st place, 60pts?
those are two different classes imo.
I don't think this change is about the core/thread discussion, so I assume we still count cores, and nothing else => both are considered to be dual cores, and therefore 2x.
As we speak, we're re-calculating all the submissions on our test server with adjusted TPP algoritms. The updated algoritm has fixed the TPP Global bug and has some additional weight finetuning.
Hondacity
11-16-2011, 19:11
any sneak-peak/ preview of top 100 overclockers?
massman is #1 thru #18 ;) jk jk
Hondacity
11-16-2011, 20:16
ok #19 - #100 please lol
Christian Ney
11-16-2011, 21:39
multiple accounts => BAN!
bye bye Massman and Hondacity,
WIN! :D
Hondacity
11-16-2011, 21:54
LOL
bad bad bad!
Please don't use my username, last time massman lost his points. I don't want that to happen. I don't have backups hahahahaha
I was just inquiring about rank changes after this "finetuning".
Recalculation finished, here are the updated top-20 rankings.
http://hwbot.org/blog/wp-content//proleague1.jpg
http://hwbot.org/blog/wp-content//ocleague.jpg
http://hwbot.org/blog/wp-content//enthusiastleague.jpg
http://hwbot.org/blog/wp-content//hwmasters.jpg
http://hwbot.org/blog/wp-content//teamsleague.jpg
Update to TPP rankings:
0 - The concept of the Team League remains the same.
1 - We fixed the issue where uploading a new score could actually decrease your points, as described here (http://hwbot.org/forum/showthread.php?t=27487), by limiting the amount of scores that receive global TPP to 1 (best of team). This is a similar implementation like the hardware TPP.
2 - We've separated the ranking weight calculation for global TPP and hardware TPP. This allows us to finetune each TPP class specifically and, as such, provide more correct scoring.
3 - We removed the requirement of having at least 10 competing teams before any TPP ranking generates points. Now every hardware TPP ranking will generate at least 5 points for #1; every global TPP ranking will generate 50p.
4 - The maximum points for global TPP is ~ 500p with 800 competing teams. For hardware TPP, the maximum is ~ 150p with 300 competing teams.
5 - The algoritm remains the same: GTTP + HTTP + UP/10. Or: sum of "global teampowerpoints" + "hardware teampowerpoints" + "user points divided by 10".
In practical terms, this update should shift the Teams League more towards the hardware TPP rankings. Note, though, that scoring in the global TPP rankings is important with its minimum 50p for #1.
I'll post a couple of point distribution screenshots to show the effect.
Oh, right. Yes, we're really excited and happy with this finetuning.
I like it if #3 turns out to be true.
http://hwbot.org/blog/wp-content//ranking1.jpg
BIG changes so soon after the shakeup :(
I suppose I like that there's some more ephasis on global benching.... but....... :(
Ahhhh.... let's see how it looks after a few months.
Can't you just remove ALL the points and see how things work out then? :)
I don't like this update :)
I never like any update.
But after a while things always seem fine again.
Hondacity
11-19-2011, 07:22
still studying the results...interesting...
Updated this:
Now every hardware TPP ranking will generate at least 5 points for #1
Right before we installed this release on production, we had a look at the team rankings and made this small update. For all teams with a rank higher than 25, the balance between GTTP (global teampowperpoints) and HTTP (hardware teampowerpoints) is 1/6 or smaller, meaning both have a great influence on your result. For ranks lower than 25, things are a bit more unbalanced, meaning you need more hardware teampowerpoints (ie: strong in hardware rankings).
A couple of things to check out as well:
1. Competition Points Overview
Go to your user profile, then points tab. You'll see a competition points tab. Click it to get an overview of all your acquired competition points.
http://hwbot.org/blog/wp-content//Clipboard0126.jpg
2. New Hardware/Benchmark Score Overview
From now on, you will see how you and your team is ranking in a benchmark or hardware ranking just with one click. Go visit one of the benchmark or hardware overview pages (search in top search bar) and you'll see a nice overview of the benchmark records in that class (including your and your team's best score)
http://hwbot.org/blog/wp-content//Clipboard0216.jpg
3. Benchmark Ranking Target Scores
For every ranking you visit, the engine will give you two target scores: your best result and your team's best result. In other words, everytime you visit a ranking, you will see how you can improve your result in a second.
http://hwbot.org/blog/wp-content//Clipboard034.jpg
Have fun!
I think I will learn pcm05 amd 3x and 8x :) :)
overclockers
11-20-2011, 17:59
List of countries will be added in The current version or later versions?
tnx:)
It's not planned for R4.4 (since the features for that rev were planned a month a go), so probably R4.5.
FYI, the difference between "shitty overall wr" and a "single gpu wr" is over 100p: 59.8p (eg: 3x CPU wprime) versus 168p (eg: 1x GPU 3DM06).
This is one of the many problems that proves that I was right and 60 Global Point for every WR was a bad ideea: http://hwbot.org/forum/showthread.php?t=34300&page=2
95k 3DMark01 is a spectacular WR that deserves 60 Global Points?
Note that it has more points than no. 7 in single card ranking which is infinitely more difficult to achieve: http://hwbot.org/submission/2213728_splave_3dmark2001_se_geforce_gtx_580_13740 7_marks
Is it unfair or I'm crazy? :)
effort * difficulty =!= instant points
Part of the game is identifying where the points are then challenging that record. Not just 'I put 200hrs and $$$ into single card and get less points than someone who ran 4 cards'. If you want to run single card, it's up to you. If you've got access to 2x6990, then it's up to you as well.
Is reaching #7 infinitely more difficult to achieve? It depends on what hardware you have, right?
If I have NickShih's CPU, run 6GHz and get #7 in single GPU ... do I deserve 'infinitely' more points?
I thought the reason for giving points is to reward a good score, not a category is has absolutely no interest for benching. To bench 4-way in Win7 for 3DMark01 is like competing in the special olympics :)
L.E: You're changing a little the rules of the game. You want to give normal people reason to bench and I get that, but the distribution of points does not reflect the value of score. This is just my opinion
Playing devils advocate.... more or less...
If enough guys are tempted by an "easy" 60 global points, soon it will be worth more than 60 points and the level of skill required to get those 60+ points becomes greater.
Why do none of the Pro league (well... why do none of the guys usually associated with the best scores) care about e.g..... 3D01 4x GPU? If it scaled, would there be more interest? Is it because it gives worse scores than 1 & 2-way GPU setups?
There is a WR for it. It's no-ones fault that the bench doesn't scale, so why not work with it anyway? :)
3D03 doesn't scale properly with 3 & 4-way.... but people still bench it.... why? :)
I wonder.... if the 3D01 4x GPU World record was worth 169 points like a usual bench.... how would the leaderboard look in 3 weeks time?
For 4-way you have to use Win7 which gives much worse scores than WinXP and it don't scale with frequency past a certain point
3D03 scales with 3 & 4-Way if you have a high frequency CPU ;) (read GT over 6200-6300MHz)
Oh, I know about Win7 bro :) That's no-ones fault :)
Does it not scale with CPU MHz either? I have never ran a 3 or 4 way system of any kind..... We cannot complain about it not scaling with GPU MHz because that happens with single card too....
3D03... yea, it's CPU bottlenecked.... but no-one complains about the slow scaling once we are at CPU limit...
:)
Actually, in retrospect, Matose has a good point. Take 'Dead Things'
http://hwbot.org/user/dead_things/#Points
Check his global points. Lots of 60pts, because he has access to multi-CPU servers, so is taking crown in the 24x CPU space and so on, and is now top 5 in the enthusiast league.
Yeap, there are drawbacks to every system. The SR-2 score of KP with 6G Westmere's now gets the points it deserves as does **DP**'s 3Way LN2 score. Others get too much points.
I wonder.... if the 3D01 4x GPU World record was worth 169 points like a usual bench.... how would the leaderboard look in 3 weeks time?
I'm sure that within 3 weeks we'll see numerous scores of 3x/4x benches where scores don't scale. 05, 06, AM3 the WR for 1x and 4x is maybe 2-3k difference and most of the time it's the same or less.
I don't see how running 3x 580's on air merits WR points or deserves to be rewarded in any way. It's just not impressive. The same applies to any other bench where you can do the same. Right now we've only seen it in 01, the rest will follow suit shortly.
There are 5 results in this entire category.... and gold is worth 60 global points. If it was hardware-class, it'd be worth 2 points. Unbalanced much?
http://hwbot.org/benchmark/pcmark_2005/rankings?cores=12
Stromatolites
12-07-2011, 02:19
There are 5 results in this entire category.... and gold is worth 60 global points. If it was hardware-class, it'd be worth 2 points. Unbalanced much?
http://hwbot.org/benchmark/pcmark_2005/rankings?cores=12
In my own opinion it isn't unbalanced. You have a point allowance for each category, so allow it, so that people who bench un popular kit get rewarded.
On a side note, sorry about whats going on at the moment.
Personal opinion, unpopular kit shouldn't be rewarded heavily at all, no matter what it is or who benches it. I believe there should be a minimum number of results and a minimum number of pople/ teams / nationalities benching something before it gets points. I've spoken to PJ about it before, I hope one day he comes around to my way of thinking :D
If you are speaking of what I think you are, thanks for the words of support :)
I shared the same opinion when we discussed this latest revision internally. My suggestion was to only award a minimum of 60p if the category, at one point, was so competitive it naturally went over 60p.
In the end, we dismissed this idea because it made everything too complex. KISS-principle was applied. I think it's actually better this way.
Christian Ney
12-07-2011, 10:48
Those rankings are becoming popular now ;)
look at this the other way around, Kenny
you "only" need to buy two hexacores, an SR2, lots of other crap ... and 60 globals are yours :D
Dead Things
12-07-2011, 20:43
Actually, in retrospect, Matose has a good point. Take 'Dead Things'
http://hwbot.org/user/dead_things/#Points
Check his global points. Lots of 60pts, because he has access to multi-CPU servers, so is taking crown in the 24x CPU space and so on, and is now top 5 in the enthusiast league.
w00t! Who knew folding boxes could make for such good benchers too! lol
Seriously, though. I will say that due to my lack of cooling, I have been forced to target more obscure hardware and less popular benchmarks if I had any hopes of posting a competitive score. Now that those benches are worth something, guys with more skill and time on their hands will be taking those boints away from me right quick, and I'll be back in my rightful spot in the rankings eventually.
For now, I'm enjoying my time in the Enthusiast League sun while I still can, taking lots of screencaps so I can show my grandkids one day!
Hondacity
12-11-2011, 03:11
biggest change imo
http://i24.photobucket.com/albums/c5/hondacity25/735fccc0.jpg
cheers!
Hondacity
01-24-2012, 09:35
I thought the reason for giving points is to reward a good score, not a category is has absolutely no interest for benching. To bench 4-way in Win7 for 3DMark01 is like competing in the special olympics :)
L.E: You're changing a little the rules of the game. You want to give normal people reason to bench and I get that, but the distribution of points does not reflect the value of score. This is just my opinion
look at deadthings points lol
60 global points for stock speed and the category (20x CPU) has ONE result.
Please can someone fix this retarded rule?
(Dead things..... I don't blame you for this, I have nothing against you.. but at the same time, I am not gonna congratulate you for stock speed.)
You should fix your language too ...
Fix:
10pts for a global with 10 or less submissions
20pts for 10-20 subs
30pts for 20-30 subs
40pts for 30-40
50pts for 40-50
60pts for 51+
The joy of that global in obscure hw combo should be the cup rather than the points.
I agree, Borandi. The minimum points rule should have been variable since the beginning ...
http://bugs.hwbot.org/browse/HWBOT-694
knopflerbruce
01-24-2012, 13:27
IMO the top spot should have a high minimum. 1st place is (nearly) always a PITA to get (no easy ones below 20 cores). Then we can let the points drop faster instead, if there are just like 10 people fighting, then #1 still gets 50p or so, but 5th place won't get too much (maybe 50-35-25-15-10 is a good way to distribute points).
I've been battling these type of low-popularity rankings for a few years... the top scores are top scores for a reason, not just because someone bothered benching his CPU while watching a sitcom. Top 5 is usually quite easy, but that's it. Then again, getting 15-20 points in the more popular rankings is also a walk in the park these days.
Dead Things
01-24-2012, 14:51
I think the case of my E7-8860 rig, though, proves that there should be variable awards for top spots, as suggested. Having a rare machine like this and benching it should be worth something, but it should not be what is essentially a bye to first place in the EL.
I.M.O.G.
01-24-2012, 17:12
I agree, Borandi. The minimum points rule should have been variable since the beginning ...
http://bugs.hwbot.org/browse/HWBOT-694
I don't mean to complicate things, but this seems pretty similar to how Rev4 handles TPP. As individuals can dominate the enthusiast league with obscure hardware, so can teams dominate the teams league on obscure hardware.
The easiest way to get 50 TPP? Request a new CPU be added, run the 10 2D benches which award HTPP at stock.
Or you could take 3x pcmark05 global 1st and get 80 TPP. One way is very expensive and challenging enough that few people could do it - the other way...
I chose the pcmark05 route because it was the cooler way. However, any smart team would recognize there are a lot of quick, easy, and valuable points to be had by benching rare stuff or low competition stuff.
In spirit, the way HTPP are awarded should encourage more competition in those categories since they are valuable (as I understood it, that was part of the idea behind the Rev4 TPP scheme) - I get the intention, however in 95% of oddball categories it is not creating competition, its just giving easy points.
If I had DB access to hwbot, I'd be curious to run a query looking at the top 20 teams, to see how many 1st place HTPP each team has in categories where there is no competition.
By the way, just a minor gripe - I think overall the scoring system is working pretty good.
knopflerbruce
01-24-2012, 18:47
I don't mean to complicate things, but this seems pretty similar to how Rev4 handles TPP. As individuals can dominate the enthusiast league with obscure hardware, so can teams dominate the teams league on obscure hardware.
The easiest way to get 50 TPP? Request a new CPU be added, run the 10 2D benches which award HTPP at stock.
Or you could take 3x pcmark05 global 1st and get 80 TPP. One way is very expensive and challenging enough that few people could do it - the other way...
I chose the pcmark05 route because it was the cooler way. However, any smart team would recognize there are a lot of quick, easy, and valuable points to be had by benching rare stuff or low competition stuff.
In spirit, the way HTPP are awarded should encourage more competition in those categories since they are valuable (as I understood it, that was part of the idea behind the Rev4 TPP scheme) - I get the intention, however in 95% of oddball categories it is not creating competition, its just giving easy points.
If I had DB access to hwbot, I'd be curious to run a query looking at the top 20 teams, to see how many 1st place HTPP each team has in categories where there is no competition.
By the way, just a minor gripe - I think overall the scoring system is working pretty good.
This kinda makes sense. I wonder if massman can post the "score formula" for this one... 5 points could be a tad too high if there's no competition, but it's also interesting to see what it looks like when more teams are competing. I checked my Q6600 PiFast score, and it gets about 95 HTPP. That's 19 times higher than the no competition reward. To compare: the user points for hardware is 2 for 1-20 competitors, and it maxes out at 50. The same ratio is 25 here. If we assume the max is 100 for HTPP, then a similar ratio means 4 HTPP for submissions with no competition from other teams.
Edit: We don't want 20 teams to compete before it starts to go above 4 boints :D just so no-one gets any ideas :p
If I had DB access to hwbot, I'd be curious to run a query looking at the top 20 teams, to see how many 1st place HTPP each team has in categories where there is no competition
I'd do it, but it's not so simple to extract that information from the database.
What you say is correct though, there are a couple of "backdoor" options to get points fast and easy. Most of those routes are function of the balance we seek between giving people incentive to submit a result and awarding points on a fair basis. The exact position of the balance is subjective, so hard to make it perfect to everyone.
I think the balance in the current revision is a little more to the "give incentive" than the "fair and exact rating" aspect.
Mr.Scott
01-24-2012, 20:08
I don't mean to complicate things, but this seems pretty similar to how Rev4 handles TPP. As individuals can dominate the enthusiast league with obscure hardware, so can teams dominate the teams league on obscure hardware.
The easiest way to get 50 TPP? Request a new CPU be added, run the 10 2D benches which award HTPP at stock.
Or you could take 3x pcmark05 global 1st and get 80 TPP. One way is very expensive and challenging enough that few people could do it - the other way...
I chose the pcmark05 route because it was the cooler way. However, any smart team would recognize there are a lot of quick, easy, and valuable points to be had by benching rare stuff or low competition stuff.
In spirit, the way HTPP are awarded should encourage more competition in those categories since they are valuable (as I understood it, that was part of the idea behind the Rev4 TPP scheme) - I get the intention, however in 95% of oddball categories it is not creating competition, its just giving easy points.
If I had DB access to hwbot, I'd be curious to run a query looking at the top 20 teams, to see how many 1st place HTPP each team has in categories where there is no competition.
By the way, just a minor gripe - I think overall the scoring system is working pretty good.
Ya know, you do an awful lot of complaining about how many points people get and what they get them on. You have the opportunity to cash in on rare and low comp stuff too you know. Just let it go. Stop screwing with the point system, it's already hard enough to keep track of. Only mentioning it because every time one of these threads comes up, you're all over it. Give it a rest.
I.M.O.G.
01-24-2012, 20:26
I think Frederik and Pieter understand I sometimes offer my input. I'm not complaining really, just offering something to consider with respect to those who make the decisions. I'm just a participant here, because I enjoy it either way. I pretty much fully agree with the post just above yours.
You on the other hand, with all due respect, *nevermind*. I appreciate your interjection, and live forever in your debt.
vBulletin® v3.7.1, Copyright ©2000-2013, Jelsoft Enterprises Ltd.