PCMark05 HB Rules,FM Rules and correct submision for ranking.

[r1ch]

SF3D, you should listen to what your fellow hwbot mods and admins are saying instead of repeating your flawed and blinkered view on the situation. Every argument you put forward has been proved groundless. Massman has this absolutely spot on - read his post again please.

You don't seem to understand that the single justification for your argument against PCM05 would have at least 3DMark01 and Aquamark points removed as well. Is this really what you want?

[SF3D]

Originally Posted by r1ch

SF3D, you should listen to what your fellow hwbot mods and admins are saying instead of repeating your flawed and blinkered view on the situation. Every argument you put forward has been proved groundless. Massman has this absolutely spot on - read his post again please.

You don't seem to understand that the single justification for your argument against PCM05 would have at least 3DMark01 and Aquamark points removed as well. Is this really what you want?

We have a nice discussion going on in crew section about these issues. I know what these two guys are saying and it doesn't change anything for me. I still haven't got any valid argument towards my own.

My argument about PCMark05 being weighted wrongly in the first place haven't been proved groundless. Or if you can do it, please do so. I have to say it again, that it was a design error in first place. They developed this benchmark 2004-2005 and I don't know why they didn't thought about future. If there would have been different weighting method, none of these problems would be here.

And please, explain how 3DMark01 and Aquamark is connected to this issue, what PCMark05 have? I can't see any similarities.

And in general:
It is my job here to evaluate the benchmarks we have in our database. If there is something wrong or if there is even some questions, we have to check them all. It is so funny, when people always think, that I have some personal issues with this. If other moderators who have posted here don't see the problem, it doesn't mean there isn't one. If some benchmark loose points, it is the same situation for everyone. There is no winners or losers in these situations. Aquamark have been under discussion many times, but still there isn't any similar problems present, so there is no need to start taking it down.

[scooter.jay]

I was not going to say anything but pcmark05 is my favorite bench. The problem is that a few think the benchmark gives unfair high scores when a good ssd raid setup or ram drive is used but in think there is nothing wrong with it. Each bench has its own need imo i.e 3d03 does not need cpu speed its all about gpu. So why cant pcmark05 not be about hard drive/drive speed. As long as a piece of hardware is used acard or i-ram or drive setup then whats the problem thats all i think.

[TheKarmakazi]

Originally Posted by SF3D

It is all about weighting a benchmark. PCMark05 was designed to run with normal HDD's. The HDD tests are giving too much gain to the final score and that is making final scores meaningless. Simple as that.

It is same with 3Dmark Vantage --> If you use PhysX, final score doesn't mean anything, cause "new technology" was used and score got a lot better.

It is NOT the same...

PhysX (and MFT or RAMdisk) is a SOFTWARE based bump in points that skews the results of hardware into unrealistic scores. SSD's Irams and Acards are all HARDWARE which gives better performance because it is simply BETTER and more powerful! Your argument is like saying we shouldnt allow dual socket systems to play wprime because it so heavily skews results? Or that dual gpu single card setup is illegal to play 2003...

These are hardware based setups that make the scores so high, not software based like physx/ramdisk. And pcmark rightfully weighted hdd speed heavily because it is the slowest subsystem of 90% of computers! So it makes sense for it to increase scores so much!

Originally Posted by SF3D

If some benchmark loose points, it is the same situation for everyone. There is no winners or losers in these situations.

That is not entirely true! I dont have mega $$ for vantage setups with tri sli and binning 100 cpus. So I bought a few irams to have some fun with pcmark05. I have many results and points invested in pcmark05, since its one of my favorite benches along with 2001. Since you dont seem to even run pcm05 of course you dont care if points are dropped... In fact I would say all of i4memory team is invested in pcmark05 and would be a substantial "loser" if this happens... I know lots of OCF guys and Benchtek UK and Dimastech are hardcore pcmark05 benchers too! So you cant really say no one will be a loser....

[Buckeye]

WoW what another interesting read. It seems the PCMark05 problem just keeps going and going.

I have to add in here also how I feel about this bench. I was an early adopter of SSD's and ended up with my 9x 32gb MTRON PRO SSD - ARC-1231ML some time ago. I love this setup even tho newer SSD's are killing me, haha tech changes.

I played around with using 2x SSD's with onboard controllers with older systems and ran the benches for many, many setups. It's a blast seeing what a simple change in hardware can do for just about any older setup can do for performance and bring those old rigs back to life.

As far as PCMark05 goes I say no limit on XP startup or anything, use what ever you can bring to the table for this bench. This bench is about what the whole computer can do, every piece, no matter if its software or hardware. Be as creative as you can be to make the fastest running system you can.

SSD's, GPU's, PhysicX or MFT or what ever, it make no differance to me.

I get into this same discusion about using 3DMark01, Aquamark etc. with current gen hardware. Or for that matter mixing OS's up in Forum Warz.

Because any bench that is currently used right now can be torn apart and say that this was never designed for this or that so you cannot use this. Yet everyday people bench with hardware that ws never desgned for a given bench.

Those of us that like PCMark05 need a place to race our hardware and compare what each other can do. So what if PCMark05 was never designed to use SSD's, its what we have to use. Leaving a cap on HDD startup or what ever is like going to a F1 race where they put restrictor plates on the carbs to slow them down, I think you might get what I mean there.

If you can say that PCMark Vantage was designed to use SSD's then great I will use that. I am unclear if there is any bandwidth caps on that bench.

I say remove all caps and restrictions and let the races begin.

[knopflerbruce]

It's possible to have limits for the subtests that "kill" the benchmark. It's better than no benchmark, at least.

[jabski]

but thats why we have the limit of 220 on xp start up. Doesnt matter if your using SSD's or what ever. If its over 220 xp start up then the score doesnt count on HWBOT.
I think its best to leave the limit in place and just carry on as we are

[Bustah]

Originally Posted by jabski

but thats why we have the limit of 220 on xp start up. Doesnt matter if your using SSD's or what ever. If its over 220 xp start up then the score doesnt count on HWBOT.
I think its best to leave the limit in place and just carry on as we are

I agree after all i-ram can be bought for quite cheap now, I found one on ebay for £45. I only have one so my xp start up is only in the region of 113mb and im not complaining. If the limit is set at 220 as it is, then that not out of reach of your average bencher [financially like me]

so if that limits stays, its within reach of most people, if the limit is raised then that will make it more difficult for the general hwbot user.

I say keep the bench and keep the limit as it is otherwise maybe hwbot needs to consider some kind of super league?

[r1ch]

Originally Posted by SF3D

And please, explain how 3DMark01 and Aquamark is connected to this issue, what PCMark05 have? I can't see any similarities.

Lets look at the justification you've used...

Originally Posted by SF3D

Not 4 years old tests suite, which wasn't designed to hold this kind of storage power.

We have better hardware nowadays, but it doesn't change the fact that this benchmark wasn't designed for new hardware.

Aquamark, 3DMark01 and 3DMark03 are all older and none are designed to run on 2/4/8 core CPU and 2/3/4 GPU setups.

Originally Posted by SF3D

Now PCMark05 can't give any reliable results with those monster SSD setups. 9000MB/s virus scan is not the value FM had in mind, when they designed this benchmark. It is their error in design, not ours.

To repeat someone else, 388,000 is not the value Aquamark had in mind.

The results ARE reliable. They're consistent with the hardware they're run on and the tweaks used. The hardware has got better, just like CPU's and GPU's.

If you want to spend thousands on Skulltrail, i7 and 4870X2's and and not concentrate on PCMark05, that's fair enough and it's your choice not to buy an i-RAM or an ACARD. Don't spoil it for those who value filesystem performance enough to spend a few hundred quid on a fast operating system 'drive'.

[SF3D]

Originally Posted by r1ch

Aquamark, 3DMark01 and 3DMark03 are all older and none are designed to run on 2/4/8 core CPU and 2/3/4 GPU setups.

To repeat someone else, 388,000 is not the value Aquamark had in mind.

Aquamark and 01/03 are all using only one core. These are single threaded benchmarks and adding cores have no impact. Maybe there is very little gain, but benchmark results are still valid. 03 scales perfectly with multi GPU setups and it is still best benchmark to show the power of CF or SLI. 01 cant use SLI/CF properly. Only Nature test have some noticeable gains. Aquamark does not get too much gain from multi GPU.

So all those benchmarks are giving results which are valid, no matter how you look them.

The examples I have given earlier might not be the best ones, but again I have to repeat myself. It is all about weighting a different parts of benchmark. I have tested PCMark05 for years and I didn't like the situation when I-ram got in to picture, but now I have one of those too. I-ram was already giving huge gains to final score. This benchmark gains too much from HDD tests. That is the point. I will not repeat this again.