rosetta@home in local PC farm

Message boards : Number crunching : rosetta@home in local PC farm

To post messages, you must log in.

AuthorMessage
Jarod

Send message
Joined: 27 May 07
Posts: 5
Credit: 105,258
RAC: 0
Message 65993 - Posted: 6 May 2010, 8:56:14 UTC

hi, Is there any way to set up BOINC distributed computing environment like rosetta@home in local network? I have hundreds PCs in my local network, but they are not Linux nodes (e.g. lab PC connecting to equipments), so it is not possible to set up a grid of Linux farm with SGE or Condor like. I think the best way to make full use of the idle CPU is BOINC framework to mimic the fantastic rosetta@home.

Is there protocol for setting up rosetta@home server in a local network?

I can compile the rosetta++ source codes with boinc extensions, but I don't find any manual for how to run it. When I directly run in in command line, it echoes nothing, very different with the standard rosetta binaries that can dump much messages.

ID: 65993 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65994 - Posted: 6 May 2010, 11:09:22 UTC

What environment are you using? Without having looked at it, I would guess you need a rather old copy of MS Visual C++ (2000 or 2005).

Jochen
ID: 65994 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,178,626
RAC: 3,201
Message 65995 - Posted: 6 May 2010, 11:58:31 UTC - in response to Message 65993.  

hi, Is there any way to set up BOINC distributed computing environment like rosetta@home in local network? I have hundreds PCs in my local network, but they are not Linux nodes (e.g. lab PC connecting to equipments), so it is not possible to set up a grid of Linux farm with SGE or Condor like. I think the best way to make full use of the idle CPU is BOINC framework to mimic the fantastic rosetta@home.

Is there protocol for setting up rosetta@home server in a local network?

I can compile the rosetta++ source codes with boinc extensions, but I don't find any manual for how to run it. When I directly run in in command line, it echoes nothing, very different with the standard rosetta binaries that can dump much messages.


If I am reading you right you want to download a ton of units to a server and then farm them out to other pc's and then have the server send them all back to Rosetta, is that correct? If so this is exactly what Boinc is now working against! The problems, in the past, is that this is exactly how waaaaay too many cheaters mvoed up in the stats. NO I am NOT saying you want to cheat, I am just saying that by doing this, in the past, you could crunch and recrunch the same unit a million times, each time getting credit for it. Yes they have stopped the exact procedure I just laid out but there are still problems. What Boinc does now is send a particular unit to a particular machine, if that particular machine does not upload that specific unit, the unit will not get any credits.

What some people do is called 'sneaker netting' and that is exactly what you are talking about but is also also not 'officially' supported. What you do is set your machine to a very large cache, download a ton of units, farm them out to your non internet connected machines, send the completed units back to the original machine that downloaded them all and then upload them from it. There are many problems involved and it's very work intensive. What you may want to do instead is consider the Dotsch way of computing, it lets lots of machines connect to one server that has all the data on it. Here is the link to where he posted his program https://boinc.bakerlab.org/rosetta/forum_thread.php?id=5283 He has both an individual machine version and a Server version, they are both based on the idea that the individual machine doesn't have any hard drive storage space. I think the Server one is your best bet. I have used his individual machine version and it works just fine, I have talked to him, in the forum, and he is very helpful!
ID: 65995 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jarod

Send message
Joined: 27 May 07
Posts: 5
Credit: 105,258
RAC: 0
Message 65998 - Posted: 6 May 2010, 12:58:45 UTC - in response to Message 65995.  

I don't care about credit.

I just want to localize such computing environment for running my own rosetta simulation jobs. Generally, the simplest way is to buy tens of blade servers and build up linux farm while I already have one such cluster managed by SGE, but since I already have almost one hundred PCs with Windows, I don't want to waste their idle cpu time, esp. for those PCs connected to some equipments.

What I want is setting up my own boinc server, and then distribute my own simulation jobs to the PCs in my internal network, and finally fetch all the results. Every step should be under my control.

At present, I don't care about the system performance, e.g. I/O cache. Because I will control the work unit and jobs into a limited size.


hi, Is there any way to set up BOINC distributed computing environment like rosetta@home in local network? I have hundreds PCs in my local network, but they are not Linux nodes (e.g. lab PC connecting to equipments), so it is not possible to set up a grid of Linux farm with SGE or Condor like. I think the best way to make full use of the idle CPU is BOINC framework to mimic the fantastic rosetta@home.

Is there protocol for setting up rosetta@home server in a local network?

I can compile the rosetta++ source codes with boinc extensions, but I don't find any manual for how to run it. When I directly run in in command line, it echoes nothing, very different with the standard rosetta binaries that can dump much messages.


If I am reading you right you want to download a ton of units to a server and then farm them out to other pc's and then have the server send them all back to Rosetta, is that correct? If so this is exactly what Boinc is now working against! The problems, in the past, is that this is exactly how waaaaay too many cheaters mvoed up in the stats. NO I am NOT saying you want to cheat, I am just saying that by doing this, in the past, you could crunch and recrunch the same unit a million times, each time getting credit for it. Yes they have stopped the exact procedure I just laid out but there are still problems. What Boinc does now is send a particular unit to a particular machine, if that particular machine does not upload that specific unit, the unit will not get any credits.

What some people do is called 'sneaker netting' and that is exactly what you are talking about but is also also not 'officially' supported. What you do is set your machine to a very large cache, download a ton of units, farm them out to your non internet connected machines, send the completed units back to the original machine that downloaded them all and then upload them from it. There are many problems involved and it's very work intensive. What you may want to do instead is consider the Dotsch way of computing, it lets lots of machines connect to one server that has all the data on it. Here is the link to where he posted his program https://boinc.bakerlab.org/rosetta/forum_thread.php?id=5283 He has both an individual machine version and a Server version, they are both based on the idea that the individual machine doesn't have any hard drive storage space. I think the Server one is your best bet. I have used his individual machine version and it works just fine, I have talked to him, in the forum, and he is very helpful!

ID: 65998 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 66003 - Posted: 6 May 2010, 16:19:34 UTC

Jarod sounds like a researcher. You are basically asking how to use the BOINC server code. Those are questions for the BOINC projects EMail list and follow the documentation for researchers, as opposed to the @home crunchers. The BOINC server code runs on Linux, but the crunchers can be other platforms.
Rosetta Moderator: Mod.Sense
ID: 66003 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jarod

Send message
Joined: 27 May 07
Posts: 5
Credit: 105,258
RAC: 0
Message 66013 - Posted: 7 May 2010, 0:51:40 UTC - in response to Message 66003.  

Thanks.

I have already set up the boinc server with the test app "uppercase" according to the guide in Boinc tiki, and it works fine, and the communication between server and crunchers is also ok.

Next, I will try to figure out how to set up the rosetta simulation projects in boinc. I thought there may be some people have such experience or similar interests in this.

Should I forward this thread to BOINC forum?


Jarod sounds like a researcher. You are basically asking how to use the BOINC server code. Those are questions for the BOINC projects EMail list and follow the documentation for researchers, as opposed to the @home crunchers. The BOINC server code runs on Linux, but the crunchers can be other platforms.

ID: 66013 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 66027 - Posted: 8 May 2010, 4:18:40 UTC

There isn't really anything uniquely different to how Rosetta uses BOINC server then an application such as uppercase. You generate work units and assimilate results.
Rosetta Moderator: Mod.Sense
ID: 66027 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jarod

Send message
Joined: 27 May 07
Posts: 5
Credit: 105,258
RAC: 0
Message 66034 - Posted: 8 May 2010, 15:04:44 UTC - in response to Message 66027.  

I think the most tricky thing is how to split a rosetta simulation into workunits.

There isn't really anything uniquely different to how Rosetta uses BOINC server then an application such as uppercase. You generate work units and assimilate results.

ID: 66034 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : rosetta@home in local PC farm



©2024 University of Washington
https://www.bakerlab.org