Go Back   HWBOT forum > >

Benchmark software Discuss benchmarking software.


User Tag List

Reply
 
Thread Tools Display Modes

Y-Cruncher Multi-Threaded Pi-Benchmark Updated with GUI and HWBOT Compatibility

y-cruncher is a benchmark program that computes Pi and other mathematical constants to the region of billions or even trillions of digits. The program was created to calculate the vastest Pi numbers possible and in fact holds the world record for the most ever recorded; 13.3 trillion to be exact. Previously only available sans user interface, our man Mystical has taken the time to create a more complete app that integrates a graphical user interface and the ability to submit on HWBOT.

Originally developed in 2009 as a high school project by Alexander J. Yee, y-cruncher is actually quite unique in a number of different ways. It?s the first Pi computing program that can use multiple threads for a near linear boost in calculation speed, it can also use and stress unlimited amounts of system memory and it also uses ISA extensions such as SSE and AVX available on most modern processors.

The new app created by Mystical is called the y-cruncher HWBOT Submitter. It's written in Java and requires Java 8 runtime to run. Note: y-cruncher itself has no requirements other than Windows Vista or later or above. The app includes three presets; Pi to 25 million digits, 1 billion digits and 10 billion digits. The 25 million digit run will complete in just under two seconds, whereas the 10 billion run will require 48GB of memory.

You can learn more about the y-cruncher HWBOT Submitter on the forum thread here.


Mar 28, 2016 - article - forum.hwbot.org
  #101  
Old 08-08-2016
havli havli is offline
robo cop
 
Join Date: Nov 2010
Location: CZECH REPUBLIC
Posts: 278
Thanks: 42
Thanked 74 Times in 29 Posts
havli
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Quote:
Originally Posted by Mysticial View Post
I didn't actually see this until Massman sent me an email. I thought I get email updates to my own threads...

Anyways. What happened was that the benchmark got renamed when it was promoted out of beta. The when program tries to submit using the old name, the server rejects it.

I just pushed out a quick update to fix this issue. But it looks like there's more work to be done on my side of things.
Thank you, it is working now.
__________________
Reply With Quote
  #102  
Old 09-18-2016
Mysticial Mysticial is offline
grunt bot
 
Join Date: Mar 2013
Location: United States
Posts: 114
Thanks: 4
Thanked 45 Times in 26 Posts
Mysticial
Default Re: Math turns benchmark: y-cruncher meets HWBOT

I'd like to bring back the topic of HPET platform timers and open up a discussion about virtual machines (VMs).

y-cruncher submissions to HWBOT currently require either HPET or ACPI for Windows 8 and 10. But I've come to realize that there are other platform clocks out there. When you use bcdedit to force on the platform clock, you're not guaranteed to get either HPET or ACPI.

My very own laptop seems to use an unknown platform clock which I've been unable to identify. Since neither y-cruncher nor the HWBOT submitter is aware of this clock, it blocks me from submitting benchmarks from my laptop.

The broad problem is that only the TSC clock is affected by the clock skew on Windows 8+10. Does anyone mind if I change clock enforcement to blacklist TSC rather than whitelist HPET and ACPI? This should allow all other platform clocks.

The downside of this is that it also allows VM clocks. Some of the Linux benchmarks that I've gotten from people use the "xen" VM clock. But that seems to be mostly a Linux thing. VMs in general are difficult to tackle since they (by design) attempt to trick the software into thinking it's running natively. So everything from clocks to instructions can (theoretically) be faked. y-cruncher v0.7.1 will attempt to detect if it's in a VM. But it currently doesn't block VM benchmarks because:
  1. They aren't a problem right now. I'm not aware of anyone using customized VMs to cheat benchmarks.
  2. The VM detection is not 100% reliable since VMs intentionally make themselves difficult to detect.

Speaking of Linux, the submitter currently does not block any Linux submissions regardless of the reference clock. So you are (technically) allowed to run benchmarks in Linux, take a screenshot there, then transfer the files to Windows and submit to HWBOT. But given that there's no CPUz for Linux, this probably won't work for the majority of the competitions.
__________________

Last edited by Mysticial; 09-18-2016 at 06:35.
Reply With Quote
  #103  
Old 09-18-2016
_mat_ _mat_ is offline
robo cop
 
Join Date: Nov 2008
Location: AUSTRIA
Posts: 330
Thanks: 7
Thanked 48 Times in 35 Posts
_mat_ is on a distinguished road
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Quote:
Originally Posted by Mysticial View Post
The broad problem is that only the TSC clock is affected by the clock skew on Windows 8+10. Does anyone mind if I change clock enforcement to blacklist TSC rather than whitelist HPET and ACPI? This should allow all other platform clocks.
I've researched this topic for some days back when I was developing GPUPI and in my opinion the only manageable option for me was to ban TSC from 8 and 10 and only allow HPET there. Using ACPI as a timer if available is possible but depends on how it's done. I would not advise to use Windows' QPC functions for example.
__________________

Last edited by _mat_; 09-18-2016 at 08:41.
Reply With Quote
  #104  
Old 09-18-2016
Mysticial Mysticial is offline
grunt bot
 
Join Date: Mar 2013
Location: United States
Posts: 114
Thanks: 4
Thanked 45 Times in 26 Posts
Mysticial
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Quote:
Originally Posted by _mat_ View Post
I've researched this topic for some days back when I was developing GPUPI and in my opinion the only available option is to ban TSC from 8 and 10 and only allow HPET there. Using ACPI as a timer if available is possible but depends on how it's done. I would not advise to use Windows' QPC functions for example.
ACPI + QPC is vulnerable to clock skew? I admit that I have no way to test this since the only box I have that lets me disable HPET (to force ACPI) is my AMD box - which isn't affected IIRC.

y-cruncher uses more than just QPC. So even if QPC gets skewed, it theoretically should still be okay. I can PM you the details if you're interested. Either way, I have no way to test this.
__________________
Reply With Quote
  #105  
Old 09-18-2016
_mat_ _mat_ is offline
robo cop
 
Join Date: Nov 2008
Location: AUSTRIA
Posts: 330
Thanks: 7
Thanked 48 Times in 35 Posts
_mat_ is on a distinguished road
Default Re: Math turns benchmark: y-cruncher meets HWBOT

QPC falls back in various ways depending on the hardware and OS version, it can be HPET, ACPI or even RTC. The timer resolution of QPC differs greatly as well so it's difficult to find out which timing method is currently in use.
__________________

Last edited by _mat_; 09-19-2016 at 11:03.
Reply With Quote
  #106  
Old 09-18-2016
Mysticial Mysticial is offline
grunt bot
 
Join Date: Mar 2013
Location: United States
Posts: 114
Thanks: 4
Thanked 45 Times in 26 Posts
Mysticial
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Quote:
Originally Posted by _mat_ View Post
QPC falls back in various ways depending on the hardware and OS version, it can be HPET, ACPI or even RTC. The timer resolution of QPC differs greatly as well so it's difficult to find out which timing method is currently in use. That's why I decided to avoid QPC, it's too unreliable for benchmark measurements. Read around the web, software like VirtualDub banned it as well.
I think we might be talking past each other. I'm fully aware that QPC varies depending on the clock. That's sorta the whole point of this discussion. It's easy (for me at least) to determine whether QPC is backed by HPET, ACPI, TSC, or none of those. But you implied in your previous post that QPC is vulnerable to skew when backed by ACPI.
__________________
Reply With Quote
  #107  
Old 09-19-2016
_mat_ _mat_ is offline
robo cop
 
Join Date: Nov 2008
Location: AUSTRIA
Posts: 330
Thanks: 7
Thanked 48 Times in 35 Posts
_mat_ is on a distinguished road
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Oops, I meant that I am avoiding QPC when HPET is not enabled. Sorry, I have currently a lot on my plate.

I can't remember if it's precisely ACPI that's vulnerable, but on Windows 7 - which is not affected by the RTC bug - QPC gets skewed if HPET is disabled. My best guess is, that it falls back to ACPI, otherwise the fallback to RTC would not produce skewed results. See my results here: https://www.overclockers.at/articles/gpupi-2-1 ... I should have displayed the timer frequencies as well, hrmpf.
__________________
Reply With Quote
  #108  
Old 10-12-2016
Mysticial Mysticial is offline
grunt bot
 
Join Date: Mar 2013
Location: United States
Posts: 114
Thanks: 4
Thanked 45 Times in 26 Posts
Mysticial
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Btw, I've recently received some benchmarks on a 68-core (272 thread) Knights Landing Xeon Phi:
  • 1 billion digits: 41.844 seconds
  • 10 billion digits: 504.873 seconds
As far as I can tell, these are world records for single-socket processors. And despite running at a stock 1.4 GHz, it edges out all the LN2 systems.
It still falls short of the really big Haswell-E and Broadwell-E dualies. But the program is completely untuned for the Xeon Phi. So there's probably a lot more performance that can be squeezed out from just software alone.

I won't submit these to HWBOT since they were done in Linux using the unreleased AVX512-CD binary. The rules currently require Windows with a screenshot. And I prefer to keep benchmarks of unreleased versions separate from the released stuff.

The validation files for these with all the details are available from my website if you want to take a closer look. Just don't try to steal them and submit them to HWBOT. The submitter app can't distinguish a legit benchmark from one stolen from someone else. So it will actually allow such a submission.
__________________
Reply With Quote
  #109  
Old 10-23-2016
IvanCupa IvanCupa is offline
kitchen robot
 
Join Date: Oct 2012
Location: Indonesia
Posts: 4
Thanks: 2
Thanked 1 Time in 1 Post
IvanCupa
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Hello, I've playing little bit for the benchmark. And i have some question.

1. For HWBot submission, It's allowed to run the benchmark via standalone binaries instead of HWBot submitter launcher?
And It's allowed to use Screenshot created by another apps for making the datafile?
Since both of them is possible for now, right?
In GPUPI / HWBot x265 saving the datafile will capture the screen & only the screenshot from datafile is valid.

2. The datafile is created by an TXT validation file + screenshot (or any images file ).
And created when clicked 'Submit to HWBot', deleted by the apps after uploading.

It's possible to directly make / save (multiple) datafile (With Screenshot included) after benchmarking?
I mean, after the benchmark finished, there's no TXT validation file appear first. And the datafile created when I click 'submit to HWBot'. And screenshot is captured & included in the datafile directly when I click 'submit to HWBot'.
I'm aware, if you implement this idea the benchmark must run from the HWBot Submitter. And my first question above is no longer valid.

or

I'ts possible to make multiple validation file (TXT) created by the binaries, to make datafile, after the benchmark running?
Which is easier?
My concern it's, we are talking on OCed system, that anything can happen suddenly. Corrupted TXT Validation file, problems when saving, crash, etc.

3. Why you make the datafile deleted after submitting to HWBot?
I think it's better to keep those file. Make good datafile naming mechanism / scheme. Such as (preset - time.HWBOT). It's related to my idea in 2nd question.
I still can keep it by copying the datafile when it was made though. But it's useless for now since manual submitting doesn't possible.

4. Make manual submit possible.
Many integrated HWBot benchmark could do this, HWBot Prime, GPUPI, HWBot x265 or even XTU, etc. With no issue i think.
It's funny (at least for me) if you willing to submit from another computer you must copy the entire folder which contains the application, and run the apps on another system just for submit. In fact in the end we had to open the browser also to fill in complete submission data form on the HWBot web to finish the submission processes

Many thanks, regards.
__________________
Reply With Quote
  #110  
Old 10-24-2016
Mysticial Mysticial is offline
grunt bot
 
Join Date: Mar 2013
Location: United States
Posts: 114
Thanks: 4
Thanked 45 Times in 26 Posts
Mysticial
Default Re: Math turns benchmark: y-cruncher meets HWBOT

Quote:
Originally Posted by IvanCupa View Post
Hello, I've playing little bit for the benchmark. And i have some question.

1. For HWBot submission, It's allowed to run the benchmark via standalone binaries instead of HWBot submitter launcher?
Yes. The submitter already lets you do this, but you're also free to manually run them. You are also free to fully customize the settings from within the y-cruncher application itself. You aren't required to run the benchmark from the submitter.

And for that matter, you may need to do this anyway since the default settings (as chosen by the submitter) aren't always the best.

(I have a feeling that some people won't like this ability to over-customize the settings. Since it means it's not 100% about hardware anymore. Someone with a sufficient understanding of the program itself and how it works will be able to pick better settings for their hardware setup. But when the goal is to be as fast as possible, I consider that as part of the game.)

Quote:
And It's allowed to use Screenshot created by another apps for making the datafile?
Since both of them is possible for now, right?
In GPUPI / HWBot x265 saving the datafile will capture the screen & only the screenshot from datafile is valid.
The official rules say you need to use the built-in screenshot. But that's impossible to enforce since the built-in screenshot is no different from any other screenshot. (The submitter doesn't watermark the screenshot.)


Quote:
2. The datafile is created by an TXT validation file + screenshot (or any images file ).
And created when clicked 'Submit to HWBot', deleted by the apps after uploading.

It's possible to directly make / save (multiple) datafile (With Screenshot included) after benchmarking?
I mean, after the benchmark finished, there's no TXT validation file appear first. And the datafile created when I click 'submit to HWBot'. And screenshot is captured & included in the datafile directly when I click 'submit to HWBot'.
I'm aware, if you implement this idea the benchmark must run from the HWBot Submitter. And my first question above is no longer valid.
No it's not possible. The submitter is completely separate from the main y-cruncher application. This is by design and I have no intention to merge them.

The y-cruncher application is primarily used outside of HWBOT and it has its own validation system. The submitter is only responsible for integrating with HWBOT. I admit that the integration is less than ideal, but it's the best that can be done without being too intrusive to the main y-cruncher project and with the limited man-power that I have.


Quote:
or

I'ts possible to make multiple validation file (TXT) created by the binaries, to make datafile, after the benchmark running?
Which is easier?
Can you rephrase that? I don't think I understand. The benchmark already makes a new validation file for every computation. And the datafile is made by the submitter - which is after the computation is done running.

Quote:
My concern it's, we are talking on OCed system, that anything can happen suddenly. Corrupted TXT Validation file, problems when saving, crash, etc.
How is that different from any other benchmark? And isn't that just part of the game? Realistically speaking, if the overclock can pass a y-cruncher benchmark, it'll have a pretty decent chance of handling a screenshot as well.

Quote:
3. Why you make the datafile deleted after submitting to HWBot?
I think it's better to keep those file. Make good datafile naming mechanism / scheme. Such as (preset - time.HWBOT). It's related to my idea in 2nd question.
I still can keep it by copying the datafile when it was made though. But it's useless for now since manual submitting doesn't possible.

4. Make manual submit possible.
Many integrated HWBot benchmark could do this, HWBot Prime, GPUPI, HWBot x265 or even XTU, etc. With no issue i think.
It's funny (at least for me) if you willing to submit from another computer you must copy the entire folder which contains the application, and run the apps on another system just for submit. In fact in the end we had to open the browser also to fill in complete submission data form on the HWBot web to finish the submission processes

Many thanks, regards.
Forget about the HWBOT datafile. You aren't supposed to know or care about it. If you're seeing it at all, that probably means that a submission failed. And when that happens, it won't delete it so that you can send it to me as a bug report.

Instead of the HWBOT datafile, you should care about the validation .txt file. With that (and a screenshot), you can submit from anywhere.

Right now, manual submission is disabled because:
  1. You're not supposed to see the HWBOT datafile anyway.
  2. The server doesn't check if the score you submitted is to the right benchmark. In other words, you can submit a 25m score into the 1b bracket.
__________________

Last edited by Mysticial; 10-24-2016 at 01:29.
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Windows 8 Benchmark Records No Longer Accepted At HWBOT - Benchmark Result Veracity Compromised Massman HWBOT Frontpage News 127 07-28-2017 07:48
HWBOT Exclusive: GIGABYTE H77N-WiFi spotted - mini-ITX meets Intel Wi-Di and Dual LAN richba5tard HWBOT Frontpage News 4 09-11-2012 00:33
HWBOT Releases First In-house Benchmark: HWBOT Unigine Heaven Benchmark jabski HWBOT Frontpage News 31 02-06-2011 10:28


All times are GMT +1. The time now is 23:32.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Search Engine Optimisation provided by DragonByte SEO (Lite) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging (Lite) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Copyright HWBOT 2004 - 2015