Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 172 · 173 · 174 · 175 · 176 · 177 · 178 . . . 300 · Next

AuthorMessage
computezrmle

Send message
Joined: 9 Dec 11
Posts: 63
Credit: 9,680,103
RAC: 0
Message 104701 - Posted: 7 Feb 2022, 18:48:43 UTC - in response to Message 104700.  

This may help - if it gets accepted someday:
https://github.com/BOINC/boinc/issues/4602
https://github.com/BOINC/boinc/pull/4603

Over the weekend I was running a permanent load of 29 VMs concurrently on a 32 core box with 128 GB RAM.
Disk I/O load - close to zero since /slots/ was mounted as tmpfs (similar to a ramdisk).
But the project admins would have to create a new app version with reduced limits and the modified vboxwrapper.
ID: 104701 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
kotenok2000
Avatar

Send message
Joined: 22 Feb 11
Posts: 258
Credit: 483,503
RAC: 133
Message 104702 - Posted: 7 Feb 2022, 19:10:03 UTC - in response to Message 104701.  

I hope differencing hard drive images support is coming soon.
ID: 104702 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104703 - Posted: 7 Feb 2022, 19:39:47 UTC - in response to Message 104700.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

[snip]

Good grief..do I need 4 massive sticks of memory now? I just dropped a 150 on two big sticks. Plus 65 on a new 550GB drive (78% free) I think I might put a project limit of 12 tasks into play unless someone has a way of isolating Python with a code? I've put to much money into this computer now.
Lightweight, I have 64GB in my main machine. 3 of my machines will take 128GB if necessary.
ID: 104703 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104704 - Posted: 7 Feb 2022, 19:41:37 UTC - in response to Message 104698.  

Oops , I forgot about that , I was thinking in Linux // Vs
And so it duz I put two backslashes after the Vs to try it
I suppose it is seen as a part of the BB forum tags and so is hidden
Arrr :-) , but if you put fore in you get one \
so lets try ten \\ Hmm got two
Ok twenty backslashes \\\\\ I know I am being silly . . . .
Ok so I got five
Game over :-}
Weird, tags have square brackets, why is it trying to interpret backslashes? Boinc forums are the most screwed up I've ever seen.
1
2
3
4 \
5 \
6 \
7 \
8 \\
9 \\
10 \\
Looks like every 4th one is shown. WTF?!
ID: 104704 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
kotenok2000
Avatar

Send message
Joined: 22 Feb 11
Posts: 258
Credit: 483,503
RAC: 133
Message 104705 - Posted: 7 Feb 2022, 19:43:52 UTC - in response to Message 104704.  

Maybe you can post an issue on boinc github?
https://github.com/BOINC/boinc/issues
ID: 104705 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104706 - Posted: 7 Feb 2022, 19:44:59 UTC - in response to Message 104705.  

Maybe you can post an issue on boinc github?
https://github.com/BOINC/boinc/issues
I'll let them work on the important stuff.
ID: 104706 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104707 - Posted: 7 Feb 2022, 19:47:20 UTC - in response to Message 104701.  

This may help - if it gets accepted someday:
https://github.com/BOINC/boinc/issues/4602
https://github.com/BOINC/boinc/pull/4603

Over the weekend I was running a permanent load of 29 VMs concurrently on a 32 core box with 128 GB RAM.
Disk I/O load - close to zero since /slots/ was mounted as tmpfs (similar to a ramdisk).
But the project admins would have to create a new app version with reduced limits and the modified vboxwrapper.
Sounds like a brilliant suggestion. I hope they make it work.
ID: 104707 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 104708 - Posted: 7 Feb 2022, 19:59:31 UTC - in response to Message 104703.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

[snip]

Good grief..do I need 4 massive sticks of memory now? I just dropped a 150 on two big sticks. Plus 65 on a new 550GB drive (78% free) I think I might put a project limit of 12 tasks into play unless someone has a way of isolating Python with a code? I've put to much money into this computer now.
Lightweight, I have 64GB in my main machine. 3 of my machines will take 128GB if necessary.



LOL you got money to burn or a good connection on parts.
ID: 104708 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104709 - Posted: 7 Feb 2022, 20:10:05 UTC - in response to Message 104708.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

[snip]

Good grief..do I need 4 massive sticks of memory now? I just dropped a 150 on two big sticks. Plus 65 on a new 550GB drive (78% free) I think I might put a project limit of 12 tasks into play unless someone has a way of isolating Python with a code? I've put to much money into this computer now.
Lightweight, I have 64GB in my main machine. 3 of my machines will take 128GB if necessary.



LOL you got money to burn or a good connection on parts.
RAM costs less than a tank of petrol.
ID: 104709 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104711 - Posted: 7 Feb 2022, 20:38:47 UTC - in response to Message 104700.  
Last modified: 7 Feb 2022, 21:28:52 UTC

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

BOINC complained with this:

LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space. You currently have 0.00 MB available and it needs 7629.39 MB.
2/7/2022 7:55:21 AM DESKTOP-LFM92VN
--------------------------------------------------------------------------------
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space. You currently have 0.00 MB available and it needs 19073.49 MB.
2/7/2022 7:55:34 AM DESKTOP-LFM92VN

snip ,snip..
does that meen you don't have "Leave non gpu tasks in memory while suspended" set , you will loose work past the last checkpoint but it uses more ram.
see my thread https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14903
All about the dreded "messing you about from server"
The trick iz Give it loadza disk space , everything the disk has got.
and my memory settings are "when in use 99%" when not in use 100%"
The story continues . . . .
ID: 104711 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1673
Credit: 17,615,377
RAC: 22,356
Message 104712 - Posted: 7 Feb 2022, 21:17:10 UTC - in response to Message 104700.  

Good grief..do I need 4 massive sticks of memory now?
No, you just need to read the messages that you are getting.
Every single one of the messages you posted refers to disk space, not RAM. And it's not just Pythion- it is all of your projects complaining about a lack of disk space
That is disk space, not RAM.

All of your projects need more disk space, but there is 0 MB available.


LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space.
LHC@home: Notice from server
ATLAS Simulation needs 9536.74MB more disk space.
QuChemPedIA@home: Notice from server
NWChem needs 4096.03MB more disk space.
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space.
SiDock@home: Notice from server
CurieMarieDock on BOINC + zipped input, checkpoints and progress bar needs 128.00MB more disk space.

And the common denominator in all of those messages.
You currently have 0.00 MB available
All of your projects need more disk space.
Grant
Darwin NT
ID: 104712 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104713 - Posted: 7 Feb 2022, 21:25:17 UTC - in response to Message 104712.  
Last modified: 7 Feb 2022, 21:25:53 UTC

Good grief..do I need 4 massive sticks of memory now?
No, you just need to read the messages that you are getting.
Every single one of the messages you posted refers to disk space, not RAM. And it's not just Pythion- it is all of your projects complaining about a lack of disk space
That is disk space, not RAM.

All of your projects need more disk space, but there is 0 MB available.


LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space.
LHC@home: Notice from server
ATLAS Simulation needs 9536.74MB more disk space.
QuChemPedIA@home: Notice from server
NWChem needs 4096.03MB more disk space.
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space.
SiDock@home: Notice from server
CurieMarieDock on BOINC + zipped input, checkpoints and progress bar needs 128.00MB more disk space.

And the common denominator in all of those messages.
You currently have 0.00 MB available
All of your projects need more disk space.
No, Boinc has a severe bug. I very much doubt he has precisely 0.00MB available. I think it's certain sizes of disk, it wraps around (as though they've used a small integer instead of a large one). I get around it by adjusting the amount of disk Boinc is allowed to use by a small amount. Last time I changed it from 95% to 100%. The disk was only half full, but crappy Boinc can't add up.
ID: 104713 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104714 - Posted: 7 Feb 2022, 21:27:19 UTC - in response to Message 104711.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

BOINC complained with this:

LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space. You currently have 0.00 MB available and it needs 7629.39 MB.
2/7/2022 7:55:21 AM DESKTOP-LFM92VN
--------------------------------------------------------------------------------
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space. You currently have 0.00 MB available and it needs 19073.49 MB.
2/7/2022 7:55:34 AM DESKTOP-LFM92VN

snip ,snip..
does that meen you don't have "Leave non gpu tasks in memory while suspended" set , you will loose work past the last checkpoint but it uses more ram.
see my thread https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14903
All about the dreded "messing you about from server"
The trick iz Give it loadza disk space , and my memory settings are "when in use 99%" when not in use 100%"
The story continues . . . .
I put 100% everything, since it never uses what it says it will.
ID: 104714 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104715 - Posted: 7 Feb 2022, 21:36:38 UTC - in response to Message 104714.  

I put 100% everything, since it never uses what it says it will.

Put a silly big number in the "use no more than" box , like 1000GB , untick the other two , boinc works it out and can use all possible.
at least that's the theory I am working with at the moment.
ID: 104715 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104716 - Posted: 7 Feb 2022, 21:43:58 UTC - in response to Message 104715.  
Last modified: 7 Feb 2022, 21:46:04 UTC

I put 100% everything, since it never uses what it says it will.
Put a silly big number in the "use no more than" box , like 1000GB , untick the other two , boinc works it out and can use all possible.
at least that's the theory I am working with at the moment.
I think you can put 0 in a lot of boxes to mean infinity. But I seem to be using:

Use at most 1000GB of disk space
Leave at least 0.1GB of disk space
Use at most 99% of total disk space
Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)
Use at most 100% of page file (I really don't understand this option. The page file is managed by windows, Boinc doesn't know about it. Boinc sees it all as RAM.)
Use at most 100% memory while computer is in use
Use at most 100% memory while computer is idle
tick Leave application in memory while suspended.

These options may differ slightly from what you see as I don't use the standard Boinc Manager. It's a horrid outdated simple black and white interface which doesn't even group a queue of 300 tasks into one line, it's totally unusable. I use Boinctasks.
ID: 104716 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 104717 - Posted: 7 Feb 2022, 23:01:48 UTC - in response to Message 104709.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

[snip]

Good grief..do I need 4 massive sticks of memory now? I just dropped a 150 on two big sticks. Plus 65 on a new 550GB drive (78% free) I think I might put a project limit of 12 tasks into play unless someone has a way of isolating Python with a code? I've put to much money into this computer now.
Lightweight, I have 64GB in my main machine. 3 of my machines will take 128GB if necessary.



LOL you got money to burn or a good connection on parts.
RAM costs less than a tank of petrol.



Not for me...32 gigs at my usual store from Corsair is 148 euro.
Tank of fuel 98 unleaded 104 for at 1.71/L
ID: 104717 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104718 - Posted: 8 Feb 2022, 0:04:24 UTC - in response to Message 104717.  
Last modified: 8 Feb 2022, 0:05:35 UTC

RAM costs less than a tank of petrol.
Not for me...32 gigs at my usual store from Corsair is 148 euro.
Tank of fuel 98 unleaded 104 for at 1.71/L
113 Euros here for 1 x 32GB DDR4 3200MHz. Tank of 50 litres standard unleaded is 83 Euros. Similar. Ok, it's less than a seatbelt or speeding fine.
ID: 104718 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104719 - Posted: 8 Feb 2022, 1:21:10 UTC - in response to Message 104717.  

Greg, technical questions are being asked, you can respond here, or tell me what to say if you don't want to create an account.
https://github.com/BOINC/boinc/issues/4629#issuecomment-1032102228
ID: 104719 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104720 - Posted: 8 Feb 2022, 1:27:33 UTC
Last modified: 8 Feb 2022, 1:51:49 UTC

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)

If those kind of settings don't work , nothing will
Boinc does do many small writes to the disk all the time , I am sad enough to have sat and watched `Resource Monitor` trying work what it was doing.
The wording in the boinc manager settings is for a full check point
"Request tasks to checkpoint at most every 60 seconds"
Having had a read this thread Rosetta Python floods disks with snapshots
I changed my checkpoint interval to 3600 seconds to reduce the number of checkpoints it does to save on disk wrights , a few seconds of time
and for the sake of generaly messing about with the system.
and reading the output files of valid work units it now only does one full checkpoint per hour instead of six.
The default for python in the output file is 600 the BM setting does change it
I only restart them when I have to , not a setting recommended for regular shutdown and restarted Pc`s
ID: 104720 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 11,717,270
RAC: 11,974
Message 104721 - Posted: 8 Feb 2022, 2:08:25 UTC - in response to Message 104720.  

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)

If those kind of settings don't work , nothing will
Boinc does do many small writes to the disk all the time , I am sad enough to have sat and watched `Resource Monitor` trying work what it was doing.
The wording in the boinc manager settings is for a full check point
"Request tasks to checkpoint at most every 60 seconds"
Having had a read this thread Rosetta Python floods disks with snapshots
I changed my checkpoint interval to 3600 seconds to reduce the number of checkpoints it does to save on disk wrights , a few seconds of time
and for the sake of generaly messing about with the system.
and reading the output files of valid work units it now only does one full checkpoint per hour instead of six.
The default for python in the output file is 600 the BM setting does change it
I only restart them when I have to , not a setting recommended for regular shutdown and restarted Pc`s
Well mine says 10 minutes, but the writing is continuous. I just ran some rare covid GPU units from WCG. The disk was doing 20MB/sec write non-stop. Another Boinc screwup.
ID: 104721 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 172 · 173 · 174 · 175 · 176 · 177 · 178 . . . 300 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org