Lucid Virtu MVP: Revolution in benching?

[michaelrw]

Originally Posted by Christian Ney

because I am god, I know everything

lol

[FM_Jarnis]

Just a note on this; Looks like Lucidlogix has released an update for their Virtu MVP software - 2.1.113 (May 21) includes support for Futuremark SystemInfo to detect Lucid MVP settings so any results benchmarked with that will show up without ambiguity on 3dmark.com ("if you used HyperFormance, the result is flagged. If not, it's fine")

[Luconosa]

Hi all, my 1st post here.

Now about Lucid Virtu. I think it should be allowed.

I will say the most important reason, and that is the fact that charts are already messed up with LV. Today I tried to confirm my suspicions and I think that I am right.
You will be the judge.

Here is what I got, and if that is detectable (not suspicious but detectable and with proof that LV is used) I will cover myself with my ears and never post anything here.

The problem is I don't believe I am the 1st who managed how to do this. We can't know how many results are already Virtu results. Unles someone is ready to revise all. And even then it is all by feeling, no proof at all.
All that can be done is to check every single result with LV capable CPUs, or adapt and allow it. Other ways will always be suspicions by members that they are losing points by "illegal" means. Then make it legal and remove those suspicions.

OK, let’s see

1st bench - LV off, proof with verification link, all by the rules.



Ver. link: http://3dmark.com/3dm11/3568007

2nd bench - LV on, proof with verification link (detected by 3DM), easy to notice if posted and report



Ver. link: http://3dmark.com/3dm11/3567902


3rd bench - what do you think? I would say suspicious (that if I look at all the specs, not suspicious for quick look at GTX 680 scores), also with all verifications by the book it must be regular. I would never report score like this, even if it is not legit I can't prove anything. No one can. All left is just suspicion. I do not want to make fool of myself and risk raising false alarm. That is how most of us think, and that is why I think that charts are already having many results which are gone under the radar.



http://3dmark.com/3dm11/3568117

SS are not edited by any means.

If any of mods of HWB want to know (if already don't) how I've done this please let me know. Same is for people from 3DM, because they have big flaw in system which they are not aware of.
I think it is not clever to go public with guide how to "cheat". Wasn't even sure should I public this. I've decided to show what can be done so the people do not live in belief how everything is OK, when most probably isn't. As I said above, I do not believe that I am 1st person in the world who get idea how system works.
If I am the 1st, and this can be solved before damage is done I will be happy.

Now, my personal opinion about LV.

It is new technology, we all agree that. What ppl do not agree is, is this software tweak or hardware improvement. I say is it really matter?

What are we doing here on HWBOT? We are competing who can show SS of bigger or smaller number on the screen with machine he have.
With HW/SW evolving the numbers are bigger or smaller (depends on bench we are doing).
I think there is no big difference between LV and tweaking comp in BIOS. If you know how to change IDE to AHCI you will have better results.
The point is to use everything you can to get better numbers, no one even asking you to have everyday working rig, just be able to take SS and that is all.
Ironically LV is technology incorporated in fully functional (still to see how it will work in everyday life, but obviously it has effect on benchmarks), everyday machines and it is not allowed.
Especially if you look at PCMark05. My God it's a tweak fest. Whole hardware tweaks in PCM05 are enabling RAID.
Of course, to achieve crazy score you must tweak your OS to almost non working conditions, and do some very questionable steps during test (anyone stuck on 0.1 Pixel shader after big jump in TW with Flashdesktops )))))) ) . If you ask me, boints earned that way are better candidates to be erased.

Sorry for big post, and please excuse me my spelling and grammar, English is not my native language.

Enjoy

..........Luconosa is putting his antiflame coat and raiding into sunset )))))))))))))

p.s. sorry for the 2nd SS, now I noticed that all 3 CPU-Zs are opened at CPU tab, but it is all the same as other 2.

[FM_Jarnis]

3DMark.com Lucid MVP detection is not even designed to be 100% foolproof. It is designed to catch cases where the user simply is using settings that shouldn't be used for getting proper comparable scores. I can immediately think a couple of ways to fool it, mostly due to the fact that the detection is tied to SI scan which is done before the actual benchmark run starts. Now if you have figured out some novel way, feel free to send email to info [at] futuremark.com with the details and we'll take a look.

In a more general sense, the only way to truly prevent active cheating "on purpose" is either by installing some Punkbuster-grade background bit that snoops everything, or by doing estimate comparison ("score is too good to be true").

The third score stands out like a sore thumb on the "too good to be true" scale but at the moment we haven't implemented automated flagging based on that, mostly because CPU/GPU clock frequency detection isn't an exact science (especially with new hardware) and it is a key bit in making such determination.

On a sidenote, we are considering adding further checks during the actual benchmark for some specific bits for the upcoming 3DMark but in the end it all boils down to "if you are determined to cheat in a benchmark, there is no practical way to prevent it".

As for PCMark05, we've given up on trying to detect "tweaks" on it long time ago. PCMark benchmarks are designed to benchmark Windows functionality in action and Windows offers bazillion different ways to modify how the operating system works when it is used for common tasks, without any easy way to see what is being done.

As for Lucid MVP itself, I'm still baffled why people don't seem to get it; It only inflates the numbers, it doesn't actually give any real benefits beyond a slight improvement in responsiveness (time it takes for the visuals to react to user inputs - as the the frames shown are rendered "later" than normal) and the reason you see bigger FPS numbers and bigger 3DMark numbers is because MVP effectively skips rendering calls while responding instantly "okay, rendered that, give next frame" and those non-rendered (partial) frames are given full credit in FPS counter and benchmark scores.

[Christian Ney]

@ Luconosa

Okayyyy, let me double check all your scores

As for the ''there is no way to tell if LV was on or not'' we (Moderators at hwbot) are supergods (omniscient and full of powers)... we are like... let's say Chuck Norris in an Iron Man suit

(no kidding)

[Luconosa]

Originally Posted by FM_Jarnis

3DMark.com Lucid MVP detection is not even designed to be 100% foolproof. It is designed to catch cases where the user simply is using settings that shouldn't be used for getting proper comparable scores.

Of course it is not. But let me give you an advice as someone who has quite expirience in psyshology of marketing.
Never make claims like this in public about your company ( I assume you are working for 3DM, because you said " we'll take a look" ).
It is like you've put big banner on 3DM page : " Use/buy our software, but do not mind the results, we are not designed to be 100% foolproof. "
Your company is always the best, all problems are just small bugs, and of course you are already working on it.

Sure, the cheats can not be stopped, there are no perfect protection, but my point was that "legit results" from 3DM are not so legit, and 3DM link on result is proof here on HWB.

Originally Posted by FM_Jarnis

The third score stands out like a sore thumb on the "too good to be true" scale but at the moment we haven't implemented automated flagging based on that, mostly because CPU/GPU clock frequency detection isn't an exact science (especially with new hardware) and it is a key bit in making such determination.

Yes, it stands out, but just because I listed you all 3 results. It is not so unusual if you are looking just for GTX 680 and do not looking at CPU freq.
For a average user even then with all 3 SS it would not be suspicious unless he know what are average results for rigs similar to mine, but then he is not so average user .
3DMs target group is average customer, not maniacs from here )))))) Correct me if I am wrong.

Originally Posted by Christian Ney

@ Luconosa

Okayyyy, let me double check all your scores

Do that, but be aware of one thing, my parents live in St-Prex, from there it is easy to locate guy with LN2, and house which smells by burned components. ))))))))))

Originally Posted by Christian Ney

As for the ''there is no way to tell if LV was on or not'' we (Moderators at hwbot) are supergods (omniscient and full of powers)... we are like... let's say Chuck Norris in an Iron Man suit

(no kidding)

You can only be suspicious because you know which result is out of space, but you can not prove anything. Soner or latter that way without proof, wrong person is accused. If you do not mind about collateral damage that is OK.

Ask yourself, what would Chuck do in your place? )))))))))))))

Best regards,

Luconosa

[Christian Ney]

I would say: ''Respect my authoritah''

[Luconosa]

Originally Posted by Christian Ney

I would say: ''Respect my authoritah''

Something like that yes.

[FM_Jarnis]

We've investigated this "cheat" and duplicated it in-house. Effectively if you delete 3DMark 11 from the Lucid MVP application list, that causes the Lucid MVP API to just tell SystemInfo that "no, HyperFormance is not defined to be on for this executable" (which is technically true - it isn't listed) but at that point the global switch for HyperFormance takes priority. This is different from "it is on the list and set as HyperFormance off" in which case it overrides the global switch for that executable.

We'll fix this later this week on 3dmark.com. Practical effect will be that no result that has HyperFormance global switch "on" will be accepted - even if App-specific switch for 3DMark 11 would be effectively disabling it. We'll have to discuss with Lucidlogix if we can further refine the detection later (would require update to SI and probably update to Lucidlogix software as well)

As for the "3DMark 100% accurate" thing, we strive for a simple goal on this; 3DMark is designed to produce accurate benchmark results from any system that is configured as the user wants (flagging results that are not comparable because the configuration settings affect things - like LucidMVP and ATI/AMD Tessellation slider). 3DMark also includes enough encryption and server side validation to prevent casual score file tampering. However, it isn't hardened against deliberate attempts to game the score by modifying the OS or doing custom driver level (or hardware level) hacks. So if your goal is to produce a meaningless score on purpose and you are willing to go far enough to do so, we can't prevent it. However, you shouldn't be able to get such scores by accident through normal use.

And just FYI: Futuremark is willing to investigate any cases where you think a score uploaded to 3dmark.com is tampered/invalid but 3dmark.com sees the file as valid. We can't automatically catch every single one but we are happy to investigate potential cases and improve our validation routines. Send any requests/information on such cases to info [at] futuremark.com

[Luconosa]

Originally Posted by FM_Jarnis

We've investigated this "cheat" and duplicated it in-house. Effectively if you delete 3DMark 11 from the Lucid MVP application list, that causes the Lucid MVP API to just tell SystemInfo that "no, HyperFormance is not defined to be on for this executable" (which is technically true - it isn't listed) but at that point the global switch for HyperFormance takes priority. This is different from "it is on the list and set as HyperFormance off" in which case it overrides the global switch for that executable.

We'll fix this later this week on 3dmark.com. Practical effect will be that no result that has HyperFormance global switch "on" will be accepted - even if App-specific switch for 3DMark 11 would be effectively disabling it. We'll have to discuss with Lucidlogix if we can further refine the detection later (would require update to SI and probably update to Lucidlogix software as well)

Phishing ?

Nice to see you are working on it, that is one way I guess (did not try it), but that is not my way.

We will see will that fix block my way too.
I'll tell you how I've done it of course, no matter if your fix work for me or not. But latter, now I am curious to see what will happened.
Btw. I am still just asuming that you are 3DM employee, you did not confirm or negate.

Originally Posted by FM_Jarnis

As for the "3DMark 100% accurate" thing, we strive for a simple goal on this; 3DMark is designed to produce accurate benchmark results from any system that is configured as the user wants (flagging results that are not comparable because the configuration settings affect things - like LucidMVP and ATI/AMD Tessellation slider). 3DMark also includes enough encryption and server side validation to prevent casual score file tampering. However, it isn't hardened against deliberate attempts to game the score by modifying the OS or doing custom driver level (or hardware level) hacks. So if your goal is to produce a meaningless score on purpose and you are willing to go far enough to do so, we can't prevent it. However, you shouldn't be able to get such scores by accident through normal use.

And just FYI: Futuremark is willing to investigate any cases where you think a score uploaded to 3dmark.com is tampered/invalid but 3dmark.com sees the file as valid. We can't automatically catch every single one but we are happy to investigate potential cases and improve our validation routines. Send any requests/information on such cases to info [at] futuremark.com

Relax my friend. I know you are doing your best, to improve your service and to prevent cheating. I was just pointing in flaw in your system. No need for so many excuses. I do not have any "goal" or "mission" with your service.

Info e-mails, sorry not in my adress book. Those emails are 95% neglected for various reasons, and I am too old to write unavailling emails.
Company so big should care for their product and reputation, if they are doing it through you as their official here I'll tell you directly. But info emails, no ty.

If you are really their official here they are doing good job, if you are here on your own, then they should pay you, and if they are already paying you but you are not 3DM official here, they should give you a raise.

Best regards,

Luconosa