Hello,

In attemtping to find a proper way to benchmark actual performance of computers (since computers built specifically to work with benchmarking software aren0t usually any better for real), I have decided to bechmark new computers I make using Rosetta@Home processing, and at the same time do something useful for the world.

The value I consider indicative of computer performance, is the time it takes to process a task, when the computer is otherwise idle.

I have a number for my system... 1.7 hours per task on average. But now I need to compare it to something.

If anybody else has a computer sitting idle most of the time (e.g.: your home computer when you're at work), could you take a quick look at your BOINC messages backlog and tell me the time your computer needed to crunch one task. Your help would be much appreciated. ;)

---

Welcome to Rosetta DustWolf. I commend your enthusiasm but I don't think your scheme is going to work for you very well.

There are several reasons:

1) If you are comparing to other peole's machines, then you'll need to make sure their settings are all the same in their Rosetta and general BOINC preferences. This you could do.

2) The more difficult problem is that not all the Work Units are the same. Some require a lot of processing. Some require very little.

3) Even within a single Work Unit, not all of the model predictions will last the same amount of time because their search patterns will direct some into more intensive areas of computation than others.

You might be able to develop a better measurement on one of the other BOINC projects, but Rosetta is probably not the place for such an attempt. (Of course, I think you should stay on Rosetta anyway ;)

[edit] You might also be able to get better numbers on a very long term scale, but I'm not convinced that that would work either. Can someone fill in/correct this?

I think the only "valuable" measurement of Rosetta calculation capacity is to use the "Recent Average Credit" or some other average over time. However, bear in mind that even this is not "static" - it varies depending on the type of work-units that the system has processed recently - some workunits give "better" results than others.

I agree with SOAN's three statements to make the comparison valid.

The time it takes to calculate one workunit (task) or for that matter one decoy (protein model) can only be compared with another workunit of the same type (and further, you can set the amount of time your computer should work on one particular workunit - I keep mine set for 24 hours to reduce the load on the Rosetta network - so one of my units usually takes 85000 or so sectonds to do one "task", but it produces, obviously, quite a few more decoys than someones machine another machine of same specs running for 4 hours (14000 seocnds)...)

In my opinion, the BEST benchmark is the application(s) you actually use the computer for - Rosetta is a good one if you want to figure out how good your computer is at calculating protein folding (in the Rosetta modeling way, at least). It won't tell you a thing about whether the machine is good at servin web-pages or play Quake [12345]. It may not even tell you if it's any good at running SETI or Einstein at home... ;-)

--
Mats

---

It won't tell you a thing about whether the machine is good at servin web-pages or play Quake [12345]. It may not even tell you if it's any good at running SETI or Einstein at home... ;-)

--
Mats

No but Seti@home is easy to benchmark with.

---

Team mauisun.org

No but Seti@home is easy to benchmark with.

Could be but in my book, "looking for aliens" is not in the "doing something useful" category.

Got any better alternatives? Or should I ask that question in another forum? ;)

---

I think the only "valuable" measurement of Rosetta calculation capacity is to use the "Recent Average Credit" or some other average over time.

Credits are based on the initial benchmarks done by BOINC. As I mentioned, I don't like benchmarks due to their general tendency to throw unrealistic results.

Thanks for the answer anyhow, it was most informative. :)

---

I think the only "valuable" measurement of Rosetta calculation capacity is to use the "Recent Average Credit" or some other average over time.

Credits are based on the initial benchmarks done by BOINC. As I mentioned, I don't like benchmarks due to their general tendency to throw unrealistic results.

Thanks for the answer anyhow, it was most informative. :)

Not exactly true at rosetta@home. Credits are based on a rolling average pool of results for each differenct type of target.

So the first person to claim get what they claim, the next gets the average of the two, the 3rd person gets the average of the last average on its own...
Or somthing similar to that i maybe out by a little bit, it is described somewhere.


The RAC will give you an average credit claim per day it will be ~comparative to other computers. Think of it as your throughput.
So the RAC (due to the way Rosetta grants credit) for a PenthlonX CPU at 4GHz should eb around twice as much as for a PenthlonX CPU at 2GHz. BOINC's benchmark should not affect that result*.




*It does a bit.

---

Team mauisun.org

try einstein@home and compare the seconds used for a WU, that should be a good benchmark, because all WUs of a batch are very similar.

---

try einstein@home and compare the seconds used for a WU, that should be a good benchmark, because all WUs of a batch are very similar.

But that would run along the same principle as Seti@home ;-) hence could not be used as a benchmark.

---

Team mauisun.org

The only right way to test your speed is the RAC. To get a good RAC, you need to run few weeks, this mean, you need a fast stable processor and platform ...

my two babies are still number one and 2 on the top list, and the 4 socks, 8 core Opteron 875 can not even get close to it :) hehehehe!

sorry, too fun to do not mention it :)

who?

The only right way to test your speed is the RAC. To get a good RAC, you need to run few weeks, this mean, you need a fast stable processor and platform ...

my two babies are still number one and 2 on the top list, and the 4 socks, 8 core Opteron 875 can not even get close to it :) hehehehe!

sorry, too fun to do not mention it :)

who?

but again it has to be the same project, otherwise there will be a difference between the rac due to the different credit distribution of the projects.

---