Message boards : Number crunching : Help Rosetta with FREE 60 days on Google Cloud!
Author | Message |
---|---|
FoldingSolutions Send message Joined: 2 Apr 06 Posts: 129 Credit: 3,506,690 RAC: 0 |
I found this interesting thread on the Einstein@home forum, but I don’t really dig that project as much as Rosetta, so I persisted on finding a way to get Rosetta to run on it instead. In a nutshell, Google Cloud compute offers 60days of free compute time (or $300 of usage, whichever comes sooner). I will copy and paste some of the Einstein thread to avoid repeating unnecessarily. Credit to the user Michgelsen: Here you can create one or more virtual machines, up to a combined total of eight (logical, with hyper threading? don't know.) Xeon cores, with a selection of operating systems to choose from, such as Debian, Ubuntu or Windows Server. I chose to start an eight core machine with 8GB memory with Ubuntu 15.10 (be sure to select the boxes which allow http and https connection to the internet).Once it is created, which doesn't take long, you'll see it under 'Compute Engine' > 'VM instances', and you can see 'SSH' and a few dots to the right, under 'Connect'. Click the dots, then choose 'open in browser window', and now you can simply interact with the machine via a CLI through your browser. Very neat. A few remarks/limitations: - No GPU processing. - No GUI for your VM, as far as I know. - I tried, but failed to run a project (ATLAS or vLHC) needing VirtualBox in the virtual machine. Maybe nesting VMs is not possible, or I did something wrong. Either way, I stuck to Einstein, as a 'normal' CPU project. - Mining cryptocurrency via this method is forbidden by Google. - You will not be billed once the trial period is over, i.e. the trial will not automatically change into a paid subscription. - Whether you can use the full 60 days depends on how powerful the machine is you create. You get $300 to 'spend' on machines in 60 days, and more powerful machines cost more. - I must say, hats of to Google for creating this very easy-to-use browser-based interface and for letting people test it for free! Now, installing BOINC is as simple as copying and pasting the following command into the UI that pops up once you start your instance: sudo aptitude install boinc-client boinc-manager Once it has installed, copy and paste the following into the client: Boinccmd --lookup_account https://boinc.bakerlab.org/rosetta/ email passwd With the email and password being your normal login details (maintain the spacing). It will now spit out an account key for you, so now copy this into notepad or something and copy and paste the following command: Boinccmd --project_attach https://boinc.bakerlab.org/rosetta/ accountkey A successful attachment will be signified by a lack of error messages after this. Now, whenever you type in the following command into the interface: Boinccmd You will get a list of possible commands, such as viewing tasks etc. I found that the 8 core, 8GB machine located in the US using Ubuntu 15.10 was the best option (value and ease etc). If you get issues with the BOINC client in the interface, just uninstall using the below command then start again, being sure to look up the commands as per above: sudo aptitude remove boinc-manager boinc-client Let me know what you guys think and if you have any issues following the instructions below, I will edit this post to amend any inaccuracies, cos this is awesome! You can also see the machine I created in my computers in my account (instance-1). Links: http://boinc.berkeley.edu/wiki/Installing_BOINC_on_Ubuntu https://cloud.google.com/compute/ https://einstein.phys.uwm.edu/forum_thread.php?id=11737#150772 |
Timo Send message Joined: 9 Jan 12 Posts: 185 Credit: 45,649,459 RAC: 0 |
I did this previously. I may actually pony up a little money once CASP starts to see if I can't give the team here a 'push' of an extra 40-50 cores for a couple weeks. Google compute is awesome, and with their Preemptible instances (similar to Spot instances on AWS) it's even more competitively priced. **38 cores crunching for R@H on behalf of cancercomputer.org - a non-profit supporting High Performance Computing in Cancer Research |
Chilean Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
That's amazing. 150 per month is pretty expensive vs electricity using your own hardware... I wonder if the prices will drop to the point it'd be easier to just "rent" some hardware to run rosetta instead of using your own. Unless it is winter. In that case I'd rather use my PCs to heat up my house. |
FoldingSolutions Send message Joined: 2 Apr 06 Posts: 129 Credit: 3,506,690 RAC: 0 |
I agree, I also use my machine to heat my front room! It is pretty pricey and probably doesn't make much sense from a cost perspective. Unless you're in a situation like time and want a short term boost :) |
Eric_Kaiser Send message Joined: 22 Sep 13 Posts: 3 Credit: 341,141 RAC: 0 |
Indeed this is very pricey. I've rent some i7 980x (6-core + HT) 24 GB RAM and at least 2 1,5 TB HDD for 35€/month with no setup fee. |
Chilean Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
|
FoldingSolutions Send message Joined: 2 Apr 06 Posts: 129 Credit: 3,506,690 RAC: 0 |
Microsoft Azure also gives out 200 free credits. Which amounts to this host for 1 month. Nice link :) thanks will check it out |
FoldingSolutions Send message Joined: 2 Apr 06 Posts: 129 Credit: 3,506,690 RAC: 0 |
I tried the Azure console, but found it very confusing to use. Looks like you have to use some sort of third party SSH protocol to access the server. Any tips @Chilean? |
Timo Send message Joined: 9 Jan 12 Posts: 185 Credit: 45,649,459 RAC: 0 |
Since Chilean posted a reference to what Azur trial gives you, I thought I'd share my own experience about the Google cloud compute trial. It gives you 60 days or $300, whichever you use first, and will get you 6 hosts like this one for almost 2 months. I preferred Google's setup as no software was needed (they have an SSH console that runs right from your browser, and you just follow the steps). **38 cores crunching for R@H on behalf of cancercomputer.org - a non-profit supporting High Performance Computing in Cancer Research |
Chilean Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
I tried the Azure console, but found it very confusing to use. Get putty and puttygen (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html). When creating a new VM in Azure, look for the SSH protocol /public key "box" and paste the public key generated by putty gen (putty gen will output something like "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAgv5Oi3+EHqqevSAROKKzmpqmcjf7x7l2FTyvWmWQK51Ri3aEVCJCL9DTw8GRR2hRNmGvgbPHgLKY55Hye75YkcmxPfghz02chPbgdbW3gUBSlseIut02R6GD1D0Xd9lJ0t0x9KtFA6zT8zEjga7ryUAUOX3QM5EjXt8oiL5rSXKz17mIFCzKnRn23OHCmN+eWaFY76Jkekcw555vjyKJuZwpCIeKCSPay8X9TgSgG/bUzHvrA194E+KJw9pXuxrOex3ijAFObX6XrGH2Dxl5T+MbUrzeAqRY+BsHmm4A2Mh8hvoUsCBc9aL6vyg7pzyfEi1pR5eVp106xiu5ZmET4Q== rsa-key-20160214" copy this whole thing into Windows Azure). Putty Gen will also generate a private key. Save it as a .ppk file somewhere on your PC. So: Create your VM using the public key created by puttygen. Look for the VM's IP. Run Putty. In Putty, paste your machine's IP, then go to Connection/SSH/Auth and browse for the private key you saved in Putty Gen. Click "Open". If everthing was done correctly then you should get the command window from your VM logged in as admin/root under the username than you used during the creation of the VM. Google automtically does this for you. You have to do it manually under Windows Azure. There are plenty of guides on Google. Just search for: "Putty azure windows ssh" if the above didn't work for you. |
Chilean Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
Since Chilean posted a reference to what Azur trial gives you, I thought I'd share my own experience about the Google cloud compute trial. 6 hosts? |
Timo Send message Joined: 9 Jan 12 Posts: 185 Credit: 45,649,459 RAC: 0 |
Since Chilean posted a reference to what Azur trial gives you, I thought I'd share my own experience about the Google cloud compute trial. (Note: Chiliean - posting the SSH key in the thread made it super super wide, now I have to scroll across horizontally to read anything else in this thread.. booo XD, lol) Yeah, I set up 6 instances - I started with 1, but soon noticed that the remaining dollar amount was not ticking down fast enough to be used up in the 2 month period, so created more hosts until it started using the trial credit at a pace that would take it to around 2 months.. ended up being able to run 6 hosts for maybe 7 weeks or so if memory serves... They have 'preemptive instances' now that may be even cheaper for the same CPU power (they can be 'randomly paused' when demand on Google's cloud is high though, the whole idea of these pre-emptive instances are to use up any 'excess' compute power Google has and sell it off for super cheap since the physical hardware is powered on anyways.. anyways, when I did it they didn't have those yet, so it was 6 regular full time instances, with pre-emptive you may be able to squeeze out more though.....) |
Chilean Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
Since Chilean posted a reference to what Azur trial gives you, I thought I'd share my own experience about the Google cloud compute trial. Yeah, I saw. I can't edit it though because it was posted last night :( I'll check my credit and see how fast it's going down. I remember it was about 150 bucks per month (estimated) upon the creation of the instance. Could it be that since running nothing but rosetta uses nothing but CPU power (no networking, HDD IO, blah blah) it's cheap? EDIT: My single 8-core instance is eating about 5 bucks per day. I can't create new instances wihtout trading off the number of days of the free trial. |
FoldingSolutions Send message Joined: 2 Apr 06 Posts: 129 Credit: 3,506,690 RAC: 0 |
Great instructions Chilean, managed to connect no problem! I went for the "basic" 4 core option as it was the cheapest for 4 cores, however I have ended up with this sketchy Opteron machine xD Which one did you go for as you seem to have a Xeon with 16GB memory? WRT the Google one - I also seem to be chewing through the credit at about $5 a day, so will just about last out the 2 month period. |
rjs5 Send message Joined: 22 Nov 10 Posts: 273 Credit: 23,054,272 RAC: 5,361 |
Great instructions Chilean, managed to connect no problem! Looks like the performance of your 4-core machine is VERY similar to his 2-core machine. Much more important for Rosetta is the 512kb cache size vs 20mb and ... Measured floating point speed 1353.73 million ops/sec vs 3043.46 Measured integer speed 5963.12 million ops/sec vs 11811.97 |
Chilean Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
Great instructions Chilean, managed to connect no problem! I think I chose the D3 v3 machine (https://azure.microsoft.com/en-us/pricing/details/virtual-machines/), which is "Compute Optimized". I'm pretty sure creating new accounts just for this isn't morally acceptable... or within the rules. If you have any close relatives that use Android, you could ask them if you can use their free trial (most won't even know what IaaS is, unless they are nerds like us lol). I skimmed thru Google's rules and it doesn't mention anything about "borrowing" someone else's free trial. So I immediately asked my gf for hers. Looks like the performance of your 4-core machine is VERY similar to his 2-core machine. "My" Xeon from Azure is using 4 physical cores btw. Not 2 with HT (Unlike the 4 core Xeons from Google with HT). "Mine" is the one with huge cache (Level 3 cache that is, not 2... I suppose Rosetta could still benefit from the Level 3 cache as it does from the level 2). What's very insteresting, is seeing how this 2.6 GHz Xeon with 20MB cache competes with this slightly slower 2.3 GHz Xeon but with 45MB cache. Both have the same type of VM instance running (with slight linux kernel difference), 4 cores w/ HT and 8GB of RAM. I think the 2.6 GHz one is a Sandy bridge with 8 cores (16 threads), while the 2.3 is a Haswell with 18 cores (36 threads). |
FoldingSolutions Send message Joined: 2 Apr 06 Posts: 129 Credit: 3,506,690 RAC: 0 |
Ok after much messing about, I now have a Xeon (I went for the D3 option) but not the same Xeon as Chilean, Oh well, it has to be better than the Opteron before! |
Timo Send message Joined: 9 Jan 12 Posts: 185 Credit: 45,649,459 RAC: 0 |
Ok after much messing about, I now have a Xeon (I went for the D3 option) but not the same Xeon as Chilean, Oh well, it has to be better than the Opteron before! I hope you didn't orphan the tasks that the Opteron had picked up. You can run the nomorework parameter against boinccmd as described in this thread to do the equivalent of hitting the 'No new tasks' button in the BOINC manager UI. :) |
River~~ Send message Joined: 15 Dec 05 Posts: 761 Credit: 285,578 RAC: 0 |
Two comments on the original post. First about nesting VMs. You can't if you are using the special CPU flags to facilitate running in a virtual machine - these are either "on" or "off". There is no way the processer can understand that a level could be the virtual guest of one machine and still be the host of another VM. If you don't use hardware acceleration you can do it (but not on any of the commercially hosted VM servers. Qemu without the KQemu extensions will quite happily run itself running itdelf running itself, just getting slower at every layer of emulation. SEcond, if you are not using a GUI I don't think you need boinc-manager. As far as I know boinccmd comes with the client. So I am suggesting you adjust those apt-get's accordingly. I will try it myself in a month or so, when I have time. As you seem to be aallowed only one trial, there is no rush for me to get my free trial in. But I will do, if only for the stats... Thanks for this thread R~~ |
Message boards :
Number crunching :
Help Rosetta with FREE 60 days on Google Cloud!
©2024 University of Washington
https://www.bakerlab.org