Whats going on with Team Crazy-OC Germany?

[Massman]

Originally Posted by komadyret

I tell you, hwbot isn't fair. It will never be.

Great, you have my full attention!

Originally Posted by komadyret

The whole design is too simplified to take into account every hardware configuration. Benchers with 2 cards, totaling 3 gpu's are forced to enter their scores as if they used 4 gpus, instead of competing against similar setups. Even the points distribution logaritms are unfair from some viewpoint.

Of course it's simplified! It's absolutely necessary to design an algorythm that represents the actual difference in skill, still is understandible for crew/users and is not that complext so that calculating the points won't cause the server to crash each hour. You are referring to the 4870X2 + 4870 issue as well: it's not that easy to add these extra categories, because the entire hwbot algorythm would have to be re-written then. In it's current format, it can only calculate points based on setups that have one type of videocard. CF-X has been released long after the first revision of the hwboints saw the daylight, we couldn't forsee this to happen.

At this moment, the current rules regarding setups with different types of videocards is the most fair and most correct way to treat these scores.

Originally Posted by komadyret

But to limit the total of hardware points one can be ranked with is like saying that one guys effort is inferior to anothers, just because the first guy spends days refining and benching old hardware (which undoubtly takes much longer time. Have you ever tried to run the long pis on a Pentium or Pentium 2 or a K6-2 only to have it crash after 3+ hours?) and the second guy spends a few hours fiddling with insulations and rips out some impossible clocks in just a few hours thereafter. (phew, that was a long one)

Have you had the guts to spend over 2000$ on a setup and bench it with extreme cooling? It's a lot harder then you'd imagine. Basicly, this discussion will never stop, because the only arguments you can use are personal opinions, which are always biased by the situation that person is in. People who spend hours and hours of benching an K6/2 will say that's the most difficult, people who have spend hours and hours of benching a QX9650 will say that's the most difficult.

We decided to go the most logical way and award more points to the scores done with high-end systems being overclocked with extreme cooling. The Hardware Masters league was introduced because there is indeed a lot of skill and determination needed to bench a lot of hardware.

Originally Posted by komadyret

I must agree that it is totaly unwanted to raise the limit for hardware points, the limit should be removed. It will not pose any threat to anyone in the top 10, 20 or probably even top 50 anyway, and it will increase the prestige of the hardware-junkies lists.

If you really believe that, you don't fully understand how the hwboints algorythm works. If we removed the cap of 300 hardware points, the ranking would be as follows:

2094,9 - Bwanasoft
2008,4 - hipro
1996,1 - SAV
1814,5 - kingpin
1799 - Matti_oc
1787,9 - Gradus
1623,9 - Turrican
1502,7 - Alibabar
1333,4 - Sard
1314,7 - Demiurg
1212,1 - no_name
1211,1 - gprhellas
1199,8 - joe_cool
1191,8 - MickeyMouse
1162,8 - AndreyKV
1151,9 - Icecube
1109,9 - tsan
1077,3 - gautam
976,7 - sf3D
949,5 - Stummerwinter

In bold, you see all the people in the top10 who weren't there before.

By the way, we do know that we still can improve hwbot A LOT.

Originally Posted by komadyret

I don't say I have to solution to hwbots problems. If I had, I'd launch my own site, but there should be room for discussion and brainstorming, and a fair bit of common sense. There are always room for improvements.

Common sense is what you ask, it's mostly not what you get in return. We ask people to use common sense and not cheat, apparently people do. There is room for discussion and brainstorming, hence that's what the forums are for and what the crew is doing in each PM/mail they recieve, but what rests is many ideas but no time to develop those ideas. Please understand that this site is fully working on the spare time of volunteers.

[knopflerbruce]

Originally Posted by Massman

Great, you have my full attention!
It's absolutely necessary to design an algorythm that represents the actual difference in skill

That's true, but how do you define "skill"? It's not as simple as it may sound.

Have you had the guts to spend over 2000$ on a setup and bench it with extreme cooling? It's a lot harder then you'd imagine.

It's not harder to bench $2000-HW than to bench other stuff. I never benched that expensive HW, but it just doesnt make any sense - a QX has an unlocked multiplier, and like a Q6600 has NOT, so it MUST be harder to OC a Q6600 than the QX as you have to fight the FSB-wall as well, not just the limit of the CPU.

Besides, $2000 is just a number. If you can afford the chip, you buy it and OC the crap out of it. It doesnt make you skilled just because you dare. If so, I should be #1. I bet NO-ONE here except me spend so much cash on HW that they eat bread and water 2-3 weeks per month because the rest of the cash has been used on rent and HW. Period.

Basicly, this discussion will never stop, because the only arguments you can use are personal opinions, which are always biased by the situation that person is in. People who spend hours and hours of benching an K6/2 will say that's the most difficult, people who have spend hours and hours of benching a QX9650 will say that's the most difficult.

...and by having the limit HWBot says that old HW is less difficult to OC than new HW. Which is BS, it's equally hard. To max a CPU (or GPU) you need to do more or less the same stuff anyway, with a few minor adjustments.

We decided to go the most logical way and award more points to the scores done with high-end systems being overclocked with extreme cooling. The Hardware Masters league was introduced because there is indeed a lot of skill and determination needed to bench a lot of hardware.

...which is fair enough. The only problem is that most people have the (wrong?) impression that the skill is in the overclockers league only. I wouldn't be too shocked if the general opinion is that HW-masters are point-whores who just boot up and run their CPUs to get their 2.x points I don't hear about the top 10 of the HW league get alot of praise, if the two rankings were equal the same would be the situation with the OC league. Is it?

In bold, you see all the people in the top10 who weren't there before.

...and why don't they deserve to be there?

Just as it's possible to get alot of points just by benching some kind of old parts on near default settings it's possible to buying high-end memory, a QX9650/9770 + E8600, a good mobo, 3 GTX 280's, pots and a dewar of LN2 and get high on the ranks because of that.

PS: I asked about being a member of the "HWBot team" a few times in the past, but apparently there are enough mods here - at least I'm not in atm

[komadyret]

I think it is just fair to base the points given on the number of users that have submitted scores with the hardware in particular (but not on the number of submissions, that could make someone spam the system with results for a particular hardware to boost points).
What is not fair, is that you end up with "half grade" points for benching hardware that few are taking time to bench, be that newer low-power mobile chips, or older performance chips.

I could perhaps propose a solution that just dropped into my head (not like I don't think you have thought of it yourself):
Do a complete separation of global points and hardware points, and present 2 different top-lists.
How does this affect the precious "top overclockers" position? Global top scores will be duly rewarded, and it will show who is the current top dog, and hardware top scores will show who is the most dedicated and hardcore bencher. Team scores could remain as now.
(PS if you read this, you'll see it's genarally very similar to what is presented today, it just gives hardwarejunkies a separate list, that the extreme clockers won't score well on because they stick to a few top-notch components)

And about mods? I don't think there are enough mods. Score verification (and invalidation of faulty scores) are a tad slow. But you're doing a great job. Thank you very much

[Massman]

Originally Posted by knopflerbruce

That's true, but how do you define "skill"? It's not as simple as it may sound.

There are indeed different ways to approach the issue of defining the term "skill", but in this case, we choose for a definition that makes it possible to create an online competition between benchers based on points that was doable to code and could use the older data in our database, which was already filled with 10000 results when the hwboints were released. I still believe that the current definition of skill is the correct definition to use in this kind of competition, although we find many flaws using this definition.

To resolve those flaws, we would need to re-code the framework of the complete hwbot database, which would take approximately three to four months if you'd work full time on the project. This, however, would give us the possibility to fine-tune the way points are handed out (I currently have a new suggestion ready which distributes points on 6 different levels instead of the 2 we have now) and to fine-tune the design of the complete hwbot website in itself, so that we can make more interesting rankings and create more levels to compete against each other. Please note that at this moment, no one of the crew members have to option to invest that much time into HWbot and, more importantly, that RichBa5tard is at the moment the only person who truely knows the coding part of hwbot.

Originally Posted by knopflerbruce

It's not harder to bench $2000-HW than to bench other stuff. I never benched that expensive HW, but it just doesnt make any sense - a QX has an unlocked multiplier, and like a Q6600 has NOT, so it MUST be harder to OC a Q6600 than the QX as you have to fight the FSB-wall as well, not just the limit of the CPU.

Actually, it's very incorrect to judge about something that you have never experienced yourself. You assume that, because you have an unlocked multiplier, overclocking is easier in comparison to with a locked multiplier. In fact, the actual overclocking difficulty stays exactly the same, the approach and end result is different though.

Unlocked multiplier => Limited by MAX frequency
Locked multiplier => Limited by MAX FSB

Now, if you hit the MAX FSB with a locked multiplier, you have in fact hit the MAX frequency of that chip. Ergo, in both cases you will be limited by the chip itself, with that difference that an unlocked multiplier will be able to hit higher frequencies and has therefor a bigger potential in terms of performance.

I think you were rather pointing at the implications of this issue: in many cases, a higher multiplier leads towards higher raw frequencies, which lead to higher performance, which leads to better scores and eventually at hwbot at higher points. So yes, it's easier to get higher points with a more expensive processor, be it in the global rankings. The hardware points were added to the algorythm to even the playing field more, because it's not fair to only award points to the top scores, often produced on the most expensive setups.

(later on your post, you've said that both are equally difficult to overclock, by the way)

Originally Posted by knopflerbruce

Besides, $2000 is just a number. If you can afford the chip, you buy it and OC the crap out of it. It doesnt make you skilled just because you dare. If so, I should be #1. I bet NO-ONE here except me spend so much cash on HW that they eat bread and water 2-3 weeks per month because the rest of the cash has been used on rent and HW. Period.

It's not that normal to go out and buy $2000 of hardware and 'oc the crap out of it'. If you'd ask anyone, I bet 95+% would rather risk to wreck a low-end card than a high-end card. However, what people spend on hardware is something that hwbot is not responsible for. In fact, the entire problem of "more $ = more points" is induced by the hardware manufacturers, who ask more money for better hardware. If low-end hardware would cost more than high-end hardware, the rankings would still be the same, isn't it?

Money spent on hardware is never a good method to rank overclockers by, but as it's directly connected to the performance of hardware, it's inevitable that money will play a role in overclocking results, as it does in nearly every other aspect of life.

Originally Posted by knopflerbruce

...and by having the limit HWBot says that old HW is less difficult to OC than new HW. Which is BS, it's equally hard. To max a CPU (or GPU) you need to do more or less the same stuff anyway, with a few minor adjustments.

We limited the hardware points, because in the past we noticed that the so-called grinders would take over all top positions in the overclockers league in no time, which would be completely adverse to the definition we used of "skill". In our opinion, and hopefully in the opinion of many benchers, an overclockers league

- Based on points awarded to benchmark scores for which
- Higher scores mean higher points and
- Higher score means higher level of overclocking

should be representing the correct order of how good a person is in benchmarking. Now, this concept of overclocking can in fact only be applied when you compare identical systems, because a certain benchmark score is obviously limited by more than only the factor "skill". As there is NO way to completely exclude factors such as luck and money, we need to settle for a system that comes close to reflecting the level of overclocking (or skill). And no, the hwboint system is not thé best way to do so, but it's that way that makes it possible to keep the database somewhat clean and actually code a fully working league, which is a very important aspect of all this. Many of you tend to forget that everything has to be coded and balanced as well .

The second part of the sentence (difficulty of overclocking), I have covered in the alinea's above this one.

Originally Posted by knopflerbruce

...which is fair enough. The only problem is that most people have the (wrong?) impression that the skill is in the overclockers league only. I wouldn't be too shocked if the general opinion is that HW-masters are point-whores who just boot up and run their CPUs to get their 2.x points I don't hear about the top 10 of the HW league get alot of praise, if the two rankings were equal the same would be the situation with the OC league. Is it?

We are actually aware of this and we absolutely need to find a solution to 'praise' the people who don't overclock the expensive hardware, but bench 'the crap out of' a lot of hardware, sometimes having to put more effort in the overclock than people who just buy and bench. It's very difficult to actually rate people on results only as, in my opinion, it's not always the bare result that makes the score impressive or not. If I post in overclocking forums, I try not to congratulate based on scores only, but also on overclocking process itself. It's, by the way, completely inaccurate to only find the top scores to be excellent and, for instance, Athlon XP scores completely worthless.

Originally Posted by knopflerbruce

...and why don't they deserve to be there?

If you use the term 'deserve', you're talking about grading the effort people had to do to achieve a certain overclock, which is even less easy to define than the concept of "skill" and which leads in many cases to a discussion with arguments based on personal feelings and other emo-crap. Rephrase the question to "and why shouldn't they be there?" and the answer has already been given in the previous answers.

Originally Posted by knopflerbruce

Just as it's possible to get alot of points just by benching some kind of old parts on near default settings it's possible to buying high-end memory, a QX9650/9770 + E8600, a good mobo, 3 GTX 280's, pots and a dewar of LN2 and get high on the ranks because of that.

Yes it is. BUT, the points you recieve in the global rankings can drop very easily when new hardware is released (check out the 1M global ranking, for instance), which requires you to invest money in every new technology and high-end hardware to stay on top of the league. We've seen that phenomenon pop up when the ATI 3870X2 was released and we (correctly) decided to consider that as a single card.

In other words, it's easy as far as you know that the hardware you buy and bench will give you a lot of points in the beginning, but will rapidly drop when new hardware is released.

Originally Posted by knopflerbruce

PS: I asked about being a member of the "HWBot team" a few times in the past, but apparently there are enough mods here - at least I'm not in atm

I have you on MSN right? The main issue is coding personal, but it would take days to fully explain the hwbot source code as it's a bit messy .

[knopflerbruce]

Originally Posted by Massman

There are indeed different ways to approach the issue of defining the term "skill", but in this case, we choose for a definition that makes it possible to create an online competition between benchers based on points that was doable to code and could use the older data in our database, which was already filled with 10000 results when the hwboints were released. I still believe that the current definition of skill is the correct definition to use in this kind of competition, although we find many flaws using this definition.

I just didn't like the way you used the word "skill" As it kind of excluded the skill of getting a 3 point score, that sometimes may not be as easy to get a you would think (if s7e9h3n had uploaded his 4005.19Mhz Opteron 154 score to the bot, it would be #2. It was #1 once. I think you know how hard it is to break, as it WAS the fastest Opteron on earth - even if the score that's now above it only generates 2.2 HWBoints or something.).

To resolve those flaws, we would need to re-code the framework of the complete hwbot database, which would take approximately three to four months if you'd work full time on the project. This, however, would give us the possibility to fine-tune the way points are handed out (I currently have a new suggestion ready which distributes points on 6 different levels instead of the 2 we have now) and to fine-tune the design of the complete hwbot website in itself, so that we can make more interesting rankings and create more levels to compete against each other. Please note that at this moment, no one of the crew members have to option to invest that much time into HWbot and, more importantly, that RichBa5tard is at the moment the only person who truely knows the coding part of hwbot.

New ideas are always interesting! But I must say I don't think it's necessary to re-code the whole forum Tweaks should be enough, today's system isn't THAT bad - I just like to point out what could be done better.

Actually, it's very incorrect to judge about something that you have never experienced yourself. You assume that, because you have an unlocked multiplier, overclocking is easier in comparison to with a locked multiplier. In fact, the actual overclocking difficulty stays exactly the same, the approach and end result is different though.

Unlocked multiplier => Limited by MAX frequency
Locked multiplier => Limited by MAX FSB

Now, if you hit the MAX FSB with a locked multiplier, you have in fact hit the MAX frequency of that chip. Ergo, in both cases you will be limited by the chip itself, with that difference that an unlocked multiplier will be able to hit higher frequencies and has therefor a bigger potential in terms of performance.

I think you were rather pointing at the implications of this issue: in many cases, a higher multiplier leads towards higher raw frequencies, which lead to higher performance, which leads to better scores and eventually at hwbot at higher points. So yes, it's easier to get higher points with a more expensive processor, be it in the global rankings. The hardware points were added to the algorythm to even the playing field more, because it's not fair to only award points to the top scores, often produced on the most expensive setups.

I read your first post it was as if the people with expensive gear would sit at their desk with a QX9770 in their hands, putting in into the socket with great fear, legs shaking when the vcore exceeded 1.6v and nearly crying of fear of the expensive sucker's potential death under LN2 Scores makes you deserve points, not potentially wrecking $1000 chips. I probably read it the wrong way, I guess

(later on your post, you've said that both are equally difficult to overclock, by the way)

It was late, I blame that But equal is the most right, I think. There must be SOME FSB-wall tweaks (I know it's temperature dependant, and it's also important to tweak the MOBO right because of those limitations). I'm just guessing, so I'll leave it at that.

It's not that normal to go out and buy $2000 of hardware and 'oc the crap out of it'. If you'd ask anyone, I bet 95+% would rather risk to wreck a low-end card than a high-end card. However, what people spend on hardware is something that hwbot is not responsible for. In fact, the entire problem of "more $ = more points" is induced by the hardware manufacturers, who ask more money for better hardware. If low-end hardware would cost more than high-end hardware, the rankings would still be the same, isn't it?

The rankings would be different, as the people with less money would buy better HW, and get better scores Apart from that there wouldn't be much difference, I think.

We limited the hardware points, because in the past we noticed that the so-called grinders would take over all top positions in the overclockers league in no time, which would be completely adverse to the definition we used of "skill". In our opinion, and hopefully in the opinion of many benchers, an overclockers league

- Based on points awarded to benchmark scores for which
- Higher scores mean higher points and
- Higher score means higher level of overclocking

should be representing the correct order of how good a person is in benchmarking. Now, this concept of overclocking can in fact only be applied when you compare identical systems, because a certain benchmark score is obviously limited by more than only the factor "skill". As there is NO way to completely exclude factors such as luck and money, we need to settle for a system that comes close to reflecting the level of overclocking (or skill). And no, the hwboint system is not thé best way to do so, but it's that way that makes it possible to keep the database somewhat clean and actually code a fully working league, which is a very important aspect of all this. Many of you tend to forget that everything has to be coded and balanced as well .

The second part of the sentence (difficulty of overclocking), I have covered in the alinea's above this one.

The thought is quite nice, but in the beginning you could get a decent score just by having alot of HW points - even if there was a limit. I don't know the exact rank, but I'm pretty sure it was within top 50 at least, probably even a bit better. Now, with my 300 points, I'm at 171st place. That limit needs to be doubled! But for some reason, no-one agrees with me. I wonder, in 10 years maybe the #1 has 3000 points and I'm at 800th place with my 300 points - where's the logic and fairness in that?

We are actually aware of this and we absolutely need to find a solution to 'praise' the people who don't overclock the expensive hardware, but bench 'the crap out of' a lot of hardware, sometimes having to put more effort in the overclock than people who just buy and bench. It's very difficult to actually rate people on results only as, in my opinion, it's not always the bare result that makes the score impressive or not. If I post in overclocking forums, I try not to congratulate based on scores only, but also on overclocking process itself. It's, by the way, completely inaccurate to only find the top scores to be excellent and, for instance, Athlon XP scores completely worthless.

Good

Yes it is. BUT, the points you recieve in the global rankings can drop very easily when new hardware is released (check out the 1M global ranking, for instance), which requires you to invest money in every new technology and high-end hardware to stay on top of the league. We've seen that phenomenon pop up when the ATI 3870X2 was released and we (correctly) decided to consider that as a single card.

In other words, it's easy as far as you know that the hardware you buy and bench will give you a lot of points in the beginning, but will rapidly drop when new hardware is released.

This is true, but you'll stay close to the top for a while at least. Which was my point

I have you on MSN right? The main issue is coding personal, but it would take days to fully explain the hwbot source code as it's a bit messy .

You have me on MSN, yes I don't apply for a coding mod, but as a result or hw-mod I think (it's been a while, and I'm too lazy to check my sentbox)

[komadyret]

Originally Posted by jmke

you like this and this?

Yeah, close. But I can't follow the hardware-list further, to see how I rank in hardware points against team-mates on that list. It sort of stops at 300 points... and the top dog don't even have his points listed on that page

Also, there are a few pages (database queries etc.) I've discovered the last days, that are not accessible by looking at menus and links on the pages. I like some of them.
I guess a few of these will be fixed/made easily available in the future

Also, I must say I tend to agree with knopflerbruce on most points. It's interresting to follow this thread