Lucid Virtu MVP: Revolution in benching?

[FM_Jarnis]

On the Lucid MVP front, there will be a press release from Lucid and Futuremark this week. Short version: MVP and non-MVP scores should not be compared to each other.

After testing this in-house I also think that anyone with a pair of working eyes can easily watch two runs of a benchmark - any benchmark - and see that nothing really has changed. The only real difference is that you get the same visuals without tearing (courtesy of virtual vsync) and the fps counter shoots up for no apparent reason.

[K404]

Originally Posted by FM_Jarnis

I also think that anyone with a pair of working eyes can easily watch two runs of a benchmark - any benchmark - and see that nothing really has changed. The only real difference is that you get the same visuals without tearing (courtesy of virtual vsync) and the fps counter shoots up for no apparent reason.

Is that a positive or negative comment? Have you tested in a scenario where the (low-end) GPU would normally give a slideshow instead of smooth FPS?

[Massman]

Some comments:

MVP does not work well with multiple GPUs. 2x7970 scores 15K gpu score in 3D11, with MVP enabled that's 10K (~ 1xGPU with MVP). I assume it's just a matter of expanding the MVP software to support multi-gpu configurations, but of course we don't know how much resources Lucid has to do this.

The performance does not go up when overclocking the IGP of the SB cpu. This means that the "coop" between IGP and discrete GPU is only a half-true story; it's quite likely that you could very will do the same thing with any other GPU. But of course, it's nice to say that the IGP and dGPU are working together. I don't expect any performance gain going from SB to IB either ...

Practically, this software requires nothing more than what you do when enabling D3D overrider. You just need a mainboard that has the MVP licence.

Not all applications scale. Heaven DX11 goes up massively (+40%), 3D11 too (+27%), 3DM03 as well (+50%, need to rerun to verify). Vantage crashed all the time (I suspect issue between Virtu and ATI driver), Aquamark3 actually went down (-24%).

Fyi, we've scheduled adding the functionality to indicate whether or not you've used MVP in your benchmark run. MVP scores will not be ranked and will not be receiving any points for the time being. We might add ranks/points in the future, perhaps in a different category.

[Massman]

Originally Posted by FM_Jarnis

On the Lucid MVP front, there will be a press release from Lucid and Futuremark this week. Short version: MVP and non-MVP scores should not be compared to each other.

After testing this in-house I also think that anyone with a pair of working eyes can easily watch two runs of a benchmark - any benchmark - and see that nothing really has changed. The only real difference is that you get the same visuals without tearing (courtesy of virtual vsync) and the fps counter shoots up for no apparent reason.

Will we be able to see whether or not MVP was used in the GUI, or will we have to check the verification link only (like tesselation)?

[FM_Jarnis]

Originally Posted by Massman

Will we be able to see whether or not MVP was used in the GUI, or will we have to check the verification link only (like tesselation)?

Initially verification link only. Benchmark executable changing patches are far more complex to do and QA.

Even this requires SystemInfo update and current target is "early Q2" which probably translates to "sometime in April". Yes, there will be several weeks when people can submit MVP-enabled results and they can't be separated from legit ones. We are considering what to do about that (one option; flag *all* results with compatible chipsets as suspect until the update is ready).

Benchmark GUI update is definitely being talked about. The main problem with it is this; Anything the benchmark does in isolation can be exploited so the only sure way to do various validation bits is to do online connection. Yet it would be somewhat bad if 3DMark advanced/pro editions wouldn't give a result without network connection.

One potential alternate we've been discussing about would be current GUI that is extended with a box that, without network connection, states that the result was made in offline mode and should be considered unverified and preliminary. If network connection exists, parts of the result file are sent to 3dmark.com in the background and validations are received (nothing is saved anywhere at this point) and the box would then display the result status.

[FM_Jarnis]

Originally Posted by K404

Is that a positive or negative comment? Have you tested in a scenario where the (low-end) GPU would normally give a slideshow instead of smooth FPS?

Please try to understand; the number of frames being actually sent to be displayed does not change (as far as I can see) - granted, my testing has so far been fairly limited - but this is my understanding of the underlying tech.

FPS counter number increases because it is counting even those (partial) frames that were skipped/not rendered.

While I have not personally tested the tech in gaming, as I understand it the main benefit is that the time (in milliseconds) between your control input (mouse/key input) and when game can react to that on screen gets shorter - a benefit for gaming - and you get no tearing without the drawbacks of classic vsync. That's it.

In benchmarking there is no control input, so the only practical effect seems to be that anything based on counting the number of frames drawn gets thrown off by the tech (causing it to count frames that in reality were not rendered)

[der8auer]

Originally Posted by FM_Jarnis

FPS counter number increases because it is counting even those (partial) frames that were skipped/not rendered.

If that's true there is no point using MVP for HWBot. Then we can also start just benching 50% of the benchmark and still get a result which is then 200% higher. Doesn't make sense.

[Massman]

Originally Posted by der8auer

If that's true there is no point using MVP for HWBot. Then we can also start just benching 50% of the benchmark and still get a result which is then 200% higher. Doesn't make sense.

It does make sense, that's the problem.

There's no visual difference even though 20-50% less frames are actually rendered. That mean better efficiency, which is a good thing.

The logic that you would then be allowed to disable ALL frames is not really valid, because the idea of this software is that there is NO loss in image quality even if not all frames are rendered.

[der8auer]

I agree on you but who runes Benchmarks for image quality?

We run benchmarks to have comparable results here on HWBot. It's the same like F1 races. Everybody has to do the same course to get a valid time. It's not allowed to leave out obstacles even tho it would be more logic.

MVP might have advantages for gaming but not for us here on HWBot.

[Hondacity]

I like image quality. Otherwise 3dmark06 scores would drop lol