Clarification in PCMark'05 about Tweaks // Tricks allowed

[Sweet]

This post will be updated periodically

Originally Posted by General Rules

3. Tweaks and cheats

A big part of the benchmarking process is setting up your operating system to be as fast as possible in the benchmark you want to run. Stripping down to the bare minimum, messing with the registry settings or changing the video card drivers from ‘quality’ to ‘performance’, these are all more than legit tweaks. A tweak is what can be considered as a manual change in system settings to increase the performance, which in the end affects your benchmark result positively. Not all tweaks are to be used in 24/7 setups as they might decrease performance for your daily applications or even limit you in what you can do.

However, where there’s competition, there are cheaters! HWBot will not allow people to cheat their way to higher scores leaving less honor and/or points for those who play fair. Therefor, we try to be as clear and strict as possible when it comes to cheaters: we block the score instantly. Depending on the cheat used and the motivation behind the cheat, we can also ban the member.

What do we consider as severe cheating (instant ban):
- Submitting scores that belong to other people
- Using software, performing hardware modifications or by human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster and thus producing a better result. (adjusted 31st of August 2010)
- Using a program to edit screenshots to have more points

What do we consider as illegal tweaks (instant block):
- PhysX enabled drivers
- Mipmap (either through software or registry)
- Using a program to launch your benchmark, unless officially approved by HWBOT; benchmarks must always be launched using the shortcuts found in the installation directory.
- Software based ramdrives

Update 17th March 2010: Any software or human interaction causing the benchmark application to fail to (properly) render the textures or making the rendered image to differ too much from the original due to other software tweaking

Update 20th July 2010: Any software or human interaction causing the benchmark to use different calculation algorithms as intended by the original software creator. E.g. replacing the codec or browser in PCMark2005.

Originally Posted by General rules

At HWBot, we are all for sharing information such as tweaks, voltage modifications and many more. However, we do understand that certain tweaks are rather kept silent because it gives a small advantage over the competition, which can come in handy when competing for that special golden cup. If questions are raised about a certain score or a series of scores, we strongly suggest trying to clear the air as fast as possible; not because of HWBot or any other member, but to prevent you from getting a bad name in the community (even if you don’t deserve it!). From past experience, we know that when serious questions are raised regarding the legitimacy of a score or overclocker, it’s always best to choose the honorable path by for instance explaining a few tricks or even removing the questioned score from HWBot.

If you are uncertain about the validity of a tweak/cheat, you can always contact one of the crew members. The tweak/cheat will be tested and discussed and you will receive an answer categorized as legit/not legit. If you don’t want the tweak to be leaked, no problem: we guarantee that the tweak will only be shared with those who took part in the evaluation.

24/10/2012 time to put things in order (Thanks Pro)

Originally Posted by Pro

okay guys let me weigh in on this,

my position

firstly im waiting for all my scores to be removed and i will not submit pcmark scores to hwbot anymore, most of my scores feature totally legitimate tweaks, some scores have things like transparent windows with resized windows that we have worked out is not allowed. the main reason i am removing my scores is so i can stay objective in decision making and no one thinks that i am using their tweaks against them... and just to clarify im not using any tweaks that are replacing benchmark files, so if your doing that, your doing it wrong

ill always bench pcm05 privately though as i love the benchmark


why keep pcm05?

for those that are saying "get rid of pcm05", we are basically at that point and i am sure if this doesnt work then that is what will happen, but there are lots of reasons to keep this benchmark, for example,

we have guys like Steve and Aristides that are always strong in this bench and are getting beaten by tweaks that they are not comfortable using and rightly so, this benchmark has a place and is a source of enjoyment for lots of people, so i believe we should have a proper effort to try and fix it up before abandoning it


please be patient

now guys i see lots of people saying, whys this not banned, why is mine banned and not his, etc, please be patient, this will takes weeks, months, and will be ongoing, but it is no different than any other benchmark, other than the fact it has gone unchecked for so long.. if this happened in 3dmark11 it would be immediately checked and in the future pcmark05 will be the same


what is allowed and whats not?

lets get something cleared up, a cheat is not a super-tweak, i WOULD call bobs TW a supertweak, i would call replacing codecs in a benchmark cheating, if it pleases you, then call it a super-cheat, whatever, but if its outside the rules its not a tweak, and i am just as much to blame for things like TW window resizing as anyone else benching PCM,

now lets get onto the juicy stuff, whats allowed and whats not, this is extremely simple, lets go back to two very basic hwbot rules,

1. "Using software, performing hardware modifications or by human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster and thus producing a better result. (adjusted 31st of August 2010)"

2. not modifying the benchmark itself

Yes d3d, video encoding via registry or powertoy and LOD go against these rules, BUT they have been used and accepted as officially OK, unless staff have specifically said something is allowed that goes against HWBOTs general rules then it is NOT allowed

if what you are doing in pcmark05 goes against one of those 2 rules, remove it now. staff will be contacting users with out of line submissions asking for proof on how to replicate them within the rules, if you are worried you cant provide that proof, remove your score now or it will be removed for you

use your brain guys, if you know something is not right, 2400MB/s virusscan on a weak AMD platform, then its probably not right, this is the ethical side of hwbot we can't enforce but hope is in place

Key areas we are targeting to start with

Some of the key areas we will be targeting, but not limited to:

Video encoding replacement, audio encoding
Irregularly large trans windows
Irregularly large memory latency
Massive browser scores
Anything out of line with file decoding
Crazy virusscans or hdd general


What is OK

I am not going to go through every tweak and say its ok or its not ok, thats what the general rules are for and they cover alot of the "super tweaks", but not all,

FD2 as a program is totally allowed, but using it to manipulate the window size or minimize windows is not allowed, similarly using a script to do this is not allowed as it breaks the "benchmark percieved speed rule"

BUT as with any benchmark, if you can do it via control panel such as personalization or registry then it SHOULD be allowed, but there are always exceptions to the rule such as MIPMAP and in the future there might be other exceptions, but this is the current position, there is no one rule fits all model

so take TW for example, if you can make the windows really small and change the text type in control panel then that is legal,

SCRIPTS ARE LEGAL, i use scripts in nearly every benchmark, but to do things like control cpu clock, change gpu clocks on the fly, and to change things like the drives for different harddrive during PCMARK, that is all OK

Using a script to do something that goes against the general rules is NOT ok. For example replacing a file that the benchmark uses to make TE go faster, NOT OK.

Things like codecs can be tricky, as microsoft may update them or another program installed for watching videos may influence the scores, this is not the case in PCM05,

if you are putting codecs specially in the pcm05 directory or replacing codecs that pcm05 installs, this is definatly NOT OK, i think that is common sense

also forcing pcm05 to look in a folder it doesnt normally so it uses your codec, not OK, again common sense,

guys you get the drift right? if it breaks hwbot general rules then it is NOT OK


Raid XPERT

Still up in the air is raid xpert, we are discussing the outcome of this one right now, there and positives and negatives to both sides of the fence and we need to make sure everything is worked out properly

Massman said it was allowed, but now that we are aware that their is some bugging going on, we are reviewing it.

We don't want peoples expensive raid setups obsoleted by a bugged software,



Fairness

This is not a witch hunt, everyone will be given the oppurtunity to prove their tweak is fair in a confidential environment. If you tweak is legitimate, it won't be shared, it will be approved and you can continue using it privately if you so wish.

If you choose to not share the tweak with staff and your score looks out of line then it will be removed.

Lets hope we can clean the benchmark up once and for all, remember guys, please use your brain, we dont want to remove scores, so please do it yourself if you know they go against rules

http://hwbot.org/forum/showpost.php?...&postcount=122

Originally Posted by Pro

oh and of course, the ie9 css is not allowed, there is really no difference between replacing the web files and doing that CSS,

the benchmark is no longer rendering the data that it was designed to render, hence skewing the results,

so just to clarify IE9 CSS is not allowed

http://hwbot.org/forum/showpost.php?...&postcount=167




synthesis and to take into account

This list will be updated periodically (thanks borandi)

Originally Posted by borandi

So Tweaks:

Bob80's - YES

Mouse Movement - YES

Resizing windows (F2D or Registry) - NO

IE Settings - YES

AMD RaidXpert - YES (for now, new decision pending, may be retroactively removed)

Changing Codecs in the directory - NO

Changing Codecs outside the directory - (Sweet: i think not by now )

Script to change files for Web Pages - NO

Ie9 CSS software - NO

Script to change files for Text Edit - NO

D3DOverrider - YES

Video Encoding via registry (d3d) - YES

LOD - YES

Other registry tweaks - TO BE DISCUSSED BY ADMINS

Scripts to change drive letters - YES

Scripts to change CPU clock/cores - YES

Forcing PC05 to look in other directories - NO

Audio Encoding via registry (powertoy) - YES

Remember: If it breaks hwbot general rules then it is NOT OK

To all benchers : If you have used this type of tweaks - trick -cheats o supertweak, please delete this submission, and do not use it anymore



Since October 24, 2012 - New (thanks SteveRo)

Until the wrapper are ready

1. Futuremark link -ORB-required for all top 20 submissions global

2. For all non-top 20 submissions either Futuremark link or Pcm05 Results.txt required via screen shot.

3. Include real pictures of your rig, specifically storage and cooling.

This is Mandatory for all submissions in PcMark'05

If you use ORB link, screenshot basic is equally mandatory

Example .txt (thanks zafiropo)



Example Screenshot with .txt (thanks to me )

[Moose83]

Ok but what about flash 2 Desktop? I need to know it for delete some scores

[Sweet]

Originally Posted by Moose83

Ok but what about flash 2 Desktop? I need to know it for delete some scores

Moose you and the others guys who have used these cheats know what is right and what is wrong in each submission made .

Flash 2 Desktop "by now" isnt allowed, in the staff no conclusions about this

[Moose83]

Ok than i leave flash 2 Desktop scores till decission.

[Sweet]

Originally Posted by Moose83

Ok than i leave flash 2 Desktop scores till decission.

So...we will block this

Originally Posted by Flash Desktop

Flash Desktops is the first REAL TIME "virtual" desktop manager

Here in Hwbot, we do overclocking right in a real storage system like Hdd, ssd, revodrives, and more, but this is real, like a processor or mainboard or a real windows install in the storage.

Overclocking is real not virtual

Then, Flash 2 Desktop is "not allowed"

[Sweet]

Originally Posted by Wikipedia

In computing, a virtual desktop is a term used with respect to user interfaces, usually within the WIMP paradigm, to describe ways in which the virtual space of a computer's desktop environment is expanded beyond the physical limits of the screen's real estate through the use of software

I know that all knows this, but is important remember

Originally Posted by Rules PcMark

* - any software or human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster

[Mikecdm]

In 3d benches, when the screen goes black and you get a boosted, we consider that a glitch and an invalid score. How is it any different when we use flash2desktops? The screen goes black for the duration of the Transparent windows test and we get a boosted score.

[Sweet]

When the screen goes blanck in 3D, its a bug or something else, not a software, instead it would be a cheat in 3D if anyone use a software

Like write before because you are using software to improve the score, and is not allowed. These rules were published in 2008 and known to all

Guys, its easy, enforce rules that are from 2008 and do not ruin a benchmark #no human interaction or software or change codecs or files in Benchmark#, and this will always be so.

Thats all

[Moose83]

Ok, i beginn to delete, but this will took some time
But i want to see, that Christian Ney, Pro and so on will all do this!

[Sam OCX]

If they don't, we will do it .. so please proceed