Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 327 · 328 · 329 · 330

AuthorMessage
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5746
Credit: 5,978,093
RAC: 2,150
Message 112473 - Posted: 4 Apr 2025, 21:54:07 UTC - in response to Message 112469.  

How do you come up with these numbers? If your looking at Robetta page, did you exclude Domain Prediction jobs? As far as I know we only do Structure Prediction.
This is Rosetta, and i use the numbers that are displayed on the main page under the Server Status section there for everyone to see.
Presently
Total queued jobs:  9,316,772


Oh! I was looking at server page and couldn't find it and Robetta I could not see anything.
Robetta shows you what is being sent out to which project.
Structure prediction is us and you will see RoseTTA fold on some of them and ordinary user names on the others.
I think fold comes from the computer or AI that is looking at stuff in Structure prediction.
Ordinary user stuff is further usage of a existing studied protein in there specific line of study.
I've seen this over at Folding.
A lab at a university someplace around the world uses a protein from over here to develop something new on the next molecular level up. A gene or something.

No idea what the Robetta page is.

https://robetta.bakerlab.org/queue.php
Where the proteins for us and for I guess the AI to study.

I haven't taken the time to study it.

Robetta is a protein structure prediction service that is continually evaluated through CAMEO

Features include relatively fast and accurate deep learning based method, RoseTTAFold, and an interactive submission interface that allows custom sequence alignments for homology modeling, constraints, local fragments, and more. It can model multi-chain complexes using RoseTTAFold (user must provide paired MSA) or comparative modeling (CM) and provides the option for large scale sampling. The CM method uses the PDB100 template database, a co-evolution based model database (MDB), and also provides the option for custom templates. Computing resources are provided by the Baker lab and by volunteers from the distributed computing project Rosetta@home. You can help this service by joining Rosetta@home.

For more information please visit our Frequently Asked Questions.
PDB templates last updated June 14 2024
ID: 112473 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1814
Credit: 18,534,891
RAC: 2
Message 112475 - Posted: 5 Apr 2025, 0:15:08 UTC

While the Assimilator backlog has pretty much cleared, the Ready to Send buffer has fallen to 0 again.
At least things worked for a few hours.
Grant
Darwin NT
ID: 112475 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2303
Credit: 43,287,441
RAC: 27,466
Message 112478 - Posted: 5 Apr 2025, 1:07:20 UTC - in response to Message 112457.  

I've only just added bwsrv1 to that personalDNSfilter app I was told about wrt the alternate boinc-files server

That, and Rosetta Beta doesn't have Android applications to begin with. Rosetta traditional (Rosetta 4.20) just started to have some tasks today.

Oh! I didn't realise that.
That explains a few more things - thanks for pointing it out
ID: 112478 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2303
Credit: 43,287,441
RAC: 27,466
Message 112479 - Posted: 5 Apr 2025, 1:14:58 UTC - in response to Message 112475.  

While the Assimilator backlog has pretty much cleared, the Ready to Send buffer has fallen to 0 again.
At least things worked for a few hours.

I was about to write the same - that In Progress is in the mid 190ks and Unsent was zero - just double-checked and Unsent is reported at 2200, so up from zero again.

With the current active userbase, when everyone's filled up, the In Progress was around 225k so we're doing pretty well with a little way still to go.
Not quite back to 'normal' but still slowly trying to get there after a good few days.
Let's hope it keeps going for a little while longer
ID: 112479 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2303
Credit: 43,287,441
RAC: 27,466
Message 112480 - Posted: 5 Apr 2025, 1:26:02 UTC - in response to Message 112456.  

So from my view they got the bug fixed whatever it was.
Or something else has happened, and it's helped whatever was going on.
There are still issues with the Assimilators, but they're not as bad as they were. And the Ready to send has been over 2,000 for 8 hours now (it's actually well over 17,000. Normal value is 5000).

Total Queued jobs is now below 9.5 million. Whether it was a case of the system choking on the huge number of Queued jobs, we'll probably never know.
If things continue to improve and stay good as the Queued jobs number reduces, and if they turn to crap again if another big dump of work comes along, then that's very probably what's been going on.
Now if they could just fix the DNS & download server issues (and then fix the servers so that a large Queued jobs number won't cause everything else to grind to a halt in the future) things would be so much better all around...

I'm inclined toward 'something else' too. I'm not relying on it.
I haven't seen the assimilator issue for a while tbh.
On Ready to Send, the target for Betas has always been 5k. I haven't seen too many Rosetta 4.20 tasks for a long while but for some reason I seem to think the figure they target is 25k of that type.
No idea why they're different numbers, but that's why the overall quantity can get quite high iirc.
I don't think the size of the queued tasks is the issue. When we had all those VirtualBox tasks the queue was over 20million for months without seeing the problems we have recently.
ID: 112480 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2303
Credit: 43,287,441
RAC: 27,466
Message 112481 - Posted: 5 Apr 2025, 4:51:07 UTC - in response to Message 112480.  

I'm inclined toward 'something else' too. I'm not relying on it.
I haven't seen the assimilator issue for a while tbh.

I'm now going to correct (disagree with) myself

The hardest question is usually "tell me what you can't see"

What I can't/didn't see is the validators going down at all this week - so no-one had to wait for them to come back either

So I'm now wondering if something actually <was> fixed, which explains why we're getting a regular supply of tasks from that large queue

Does anyone want to double-check on all the other issues we've been struggling with recently?
Might they be fixed too?
That would explain one other thing I noticed this week
ID: 112481 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1814
Credit: 18,534,891
RAC: 2
Message 112482 - Posted: 5 Apr 2025, 6:09:57 UTC - in response to Message 112481.  

Does anyone want to double-check on all the other issues we've been struggling with recently?
No need, it's still broken.
Ready to send has been around zero for a few hours now, and the Tasks in Progress is taking a dive again.
Grant
Darwin NT
ID: 112482 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5746
Credit: 5,978,093
RAC: 2,150
Message 112483 - Posted: 5 Apr 2025, 10:18:34 UTC - in response to Message 112482.  

Does anyone want to double-check on all the other issues we've been struggling with recently?
No need, it's still broken.
Ready to send has been around zero for a few hours now, and the Tasks in Progress is taking a dive again.



Because there are so many systems here we burn through the tasks faster than they can transfer them or however they do it.
With 27,400+ "computers" online with chips like mine 16 "core" we can chew up the tasks faster than they can process them.

Since your last post with the total queued tasks and as of this post we have processed 81,809 tasks in 12 hours or so. That's about 6800 tasks an hour
Thats huge.

I don't know where they get their queued number from or how they move tasks from the queue to the server, but I bet its human.
And now its the weekend and if its not automatic, then you have to wait until monday morning or afternoon (US time) or for us here in Europe, afternoon to evening.
And the way things are not refilled automatically, it looks like human.

If you want this project to keep your system busy all the time then you have to hog a bunch of work.
ID: 112483 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1814
Credit: 18,534,891
RAC: 2
Message 112484 - Posted: 5 Apr 2025, 10:32:39 UTC - in response to Message 112483.  

Because there are so many systems here we burn through the tasks faster than they can transfer them or however they do it.
With 27,400+ "computers" online with chips like mine 16 "core" we can chew up the tasks faster than they can process them.

Since your last post with the total queued tasks and as of this post we have processed 81,809 tasks in 12 hours or so. That's about 6800 tasks an hour
Thats huge.
Actually it's bugger all.
Seti with much older and slower server hardware was handling 200k+ per hour.
Here it's less than 195,000 per day. At it's peak it was around 300,00 per day.



I don't know where they get their queued number from or how they move tasks from the queue to the server, but I bet its human.
You just lost your bet.
The Tasks supplied to the Queued Tasks are automated, however the part that supplies them is human- the Researchers.
But moving them from the Queued Tasks to the Ready to send is fully automated.
So the lack of work at present is purely due to project server issues. The work is there, the system just isn't moving it along to where it can be allocated & downloaded.
Grant
Darwin NT
ID: 112484 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1814
Credit: 18,534,891
RAC: 2
Message 112488 - Posted: 5 Apr 2025, 21:09:04 UTC

Assimilator backlog is back.
Ready to send is back to being empty, with the occasional burst of work for an hour or so every now & then.
Grant
Darwin NT
ID: 112488 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 327 · 328 · 329 · 330

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2025 University of Washington
https://www.bakerlab.org