Message boards : Number crunching : Shorter WU deadlines
Author | Message |
---|---|
Team TMR Send message Joined: 2 Nov 05 Posts: 21 Credit: 1,583,679 RAC: 0 |
I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines. Is this a permanent change? It's not a big deal for us in the most part, except on a couple of PCs which are only switched on a few hours a day. |
Hoelder1in Send message Joined: 30 Sep 05 Posts: 169 Credit: 3,915,947 RAC: 0 |
I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines. Great ! I have always wondered why such a fast-paced project permits work to potentially sit in user cashs for a week or more to perhaps be returned after the project has performed the final analysis on a particular numerical experiment and moved on to other things (I keep my cash size at 0.1 days to give them quick feedback for that reason). I suspect that some of these new WUs will now sit in large user cashs, happily awaiting their expiration. ;-) I guess it would have been a good idea to warn those users who keep large cash sizes, ahead of sending out the new WUs (like in the news section on the home page ?) to give them a chance to reduce their cash sizes... ;-)) |
STE\/E Send message Joined: 17 Sep 05 Posts: 125 Credit: 4,100,301 RAC: 206 |
All the WU's I have Downloaded in the last 24 hours still have the 1 Month Deadline. |
River~~ Send message Joined: 15 Dec 05 Posts: 761 Credit: 285,578 RAC: 0 |
I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines. er, not great. The big advantage of a long deadline with a short wu is that it makes an ideal backup project, either to tide over periods with no work on another project (notice how my Rosetta stats slump when LHC has work) or when a machine's availablility is unpredictable for other reasons. For example, there is a machine that is dual boot, it usually runs windows but is occasionally booted into Linux for maybe a half day or full day once a week. Ive been running Rosetta on it when it is in Linux because the long deadlines mean that the work does not time out while the box is running windows. I will now have to use Einstein for that role. Also, if a project deadline is less than double the cache size, the box will only run in EDF mode whenenver that project is present. This makes it impossible to run a single CPDN WU on the same 2cpu box as Rosetta - EDF makes ROsetta take over the cpu permanently. With 7 day deadlines this happens at a cache size of 3.5 days - needed sometime locally due to intemittent power issues that affect the network between me and the Internet. My preference would be for the project to set varying dealines according to how urgently they need the work back. LHC does this and it works very well for crunchers, and I believe for the project. When the scientists want the turnround fast they set a short deadline -- those WU then jump the queue in long caches. If the EXAMPLE_WU series are needed back by the end of FEb, they'd be sent with a 28 day deadline now, but shortened by 7 days eech week till they were on down to a 7 day deadline, then perhaps shortened to a 2 day deadline to pick up the stragglers really fast. The LHC scheduler seems to have no problem keeping a mix of long and short deadline work in my longer caches - I think they use the standard scheduler code for this. River~~ |
Hoelder1in Send message Joined: 30 Sep 05 Posts: 169 Credit: 3,915,947 RAC: 0 |
I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines. ... anyway, thanks for the clarifications. ;-) I had naively (and it seems erroneously) assumed that the cache works on a first-in-first-out basis. As I said, personally short expiration dates are fine with me. So lets see what we hear out of Seattle on this - oh and sorry for misspelling 'cache' in my last post. ;-) |
River~~ Send message Joined: 15 Dec 05 Posts: 761 Credit: 285,578 RAC: 0 |
I had naively (and it seems erroneously) assumed that the cache works on a first-in-first-out basis. ... In normal running (when projects are run in Round Robin mode) the cache for each project does indeed run first-in-first-out. However, as soon as that is predicted to make any WU late for a deadline, the machine switches to EDF - earliest deadline first - mode, and then the WU from all projects are put into deadline order. In this mode the short deadline WU from LHC get crunched before long deadline WU that came in earlier. In particular, if you have a cache of (say) 3 days work and you get a WU with a 2 day deadline, it instantly puts the box into EDF precisely because the new work would otherwise go to the back of the queue. On Pirates@home they have sent out WU with 6-hour deadlines - these usually run instantly on download as with almost any mix of work they would otherwise be 'late'. In addition, the server will not send out more short deadline work that your box can crunch in 90% of the alloted time - so a batch of short deadline work gets spread around many boxes even when each box is asking for a lot of work. This combination of behaviour built into the client and server means that the scientists can in fact combine 'urgent' work (short deadline) with 'background' work (long deadline) and BOINC gets it right most of the time. As you say, it is up to UW how to use this, but my hope would be that they mix the deadlines and look into varying them even within the same series of WU (if this is possible) R~~ |
David Baker Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 17 Sep 05 Posts: 705 Credit: 559,847 RAC: 0 |
I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines. Sorry for the lack of warning! we switched to one week for just the reason you point out--we want to analyze the results and be able to draw conclusions as soon as possible! another change is that the maximum work unit length has been increased to eliminate (hopefully!) the time out problem. |
Moderator9 Volunteer moderator Send message Joined: 22 Jan 06 Posts: 1014 Credit: 0 RAC: 0 |
The following two posts have been moved to Forum discussion thread Reason - Off Topic rbpeake - Posted in reply to David Baker This is a great example for having an Announcements section ... Angus - Replied to rbpeake I respectfully disagree. ... Both posts moved to this thread. {MODERATOR NOTE: This move announcement will be deleted} Moderator9 ROSETTA@home FAQ Moderator Contact |
mikus Send message Joined: 7 Nov 05 Posts: 58 Credit: 700,115 RAC: 0 |
I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines. I guess change is to be expected, even though it is not always welcome. I run rosetta as the __only__ BOINC project on a machine that is offline (except when I temporarily change it, I've specified in BOINC Manager: "Network activity suspended"); I have to both dial-in (from another machine) and set up a proxy before my rosetta machine can talk to the server. So communication takes place when I set it up, NOT when the machine feels like it. I was mystified to see (when I connected today) that the current WU the machine was working on got 'preempted'. I did not for a while realize that a just-downloaded "short deadline" WU (several pages further down the work list) had been started in its stead. And I was further confused by seeing more and more "short deadline" WUs being downloaded, even though the client kept telling me that the machine was OVERLOADED !! For me, the four-week-deadline WUs were ideal - they could chug along without me paying much attention. I could connect at my convenience (based on when the queue of WUs for 'uploading' got long). With the change to one-week-deadline WUs, I feel I am being FORCED to pay more attention and to connect to rosetta at shorter intervals. In fact, had I anticipated that I would come to feel rosetta deadlines "breathing down my neck", I might well have chosen a different project. |
Angus Send message Joined: 17 Sep 05 Posts: 412 Credit: 321,053 RAC: 0 |
The following two posts have been moved to Forum discussion thread I'm sorry, but my post is not in the thread you referred to: I can't remember the exact words (since I can't find it), but if I think it would be meaningless taken out of context in which it was posted. Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :) "You can't fix stupid" (Ron White) |
Angus Send message Joined: 17 Sep 05 Posts: 412 Credit: 321,053 RAC: 0 |
I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines. Dr. Baker - Could you PLEASE post information about substantial changes in WU size or duration in the NEWS or Technical News section? Many of us who have been around BOINC a long time look first to the News section when something unusual appears. Something as radical as changing the deadline from a month to a week can play havoc with work queues, and require some adjustment by us crunchers. Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :) "You can't fix stupid" (Ron White) |
STE\/E Send message Joined: 17 Sep 05 Posts: 125 Credit: 4,100,301 RAC: 206 |
Something as radical as changing the deadline from a month to a week can play havoc with work queues, and require some adjustment by us crunchers. ========== I have to agree on that Angus, but then thats what the Abort Tab is for I figure, which I have no regrets on using when a Project pulls something like that ... I don't like my other Projects getting put on the back shelf because 1 Project decides to Override the other Projects by cutting their Deadline down so dramatically without warning ... |
Team TMR Send message Joined: 2 Nov 05 Posts: 21 Credit: 1,583,679 RAC: 0 |
For us, we've had to rejig the projects that some PCs work on, taking some PCs off Rosetta completely. We had Rosetta running on a few slow PCs, which take 1-2 days to complete a WU. When you factor in that those PCs are only on for less than 8 hours a day, it becomes 4-6 days to complete - and then a weekend arives (PCs are off) and it takes over a week to complete. If the short deadline WUs were smaller so they completed quicker it wouldn't be such a problem. |
Jeff Gilchrist Send message Joined: 7 Oct 05 Posts: 33 Credit: 2,398,990 RAC: 0 |
Sorry for the lack of warning! we switched to one week for just the reason you point out--we want to analyze the results and be able to draw conclusions as soon as possible! This is causing me some problems. I have some machines that are not frequently connected to the Internet so I have my cache set to queue 10 days of work and I sometimes can only connect once a week to upload. I am now not able to fill up my cache because I cannot complete the work in time for the deadline. Could you reconsider this and possibly select 2 weeks instead of 1 so that you still get your work back reasonably fast but you dont penalize some people? |
[B@H] Ray Send message Joined: 20 Sep 05 Posts: 118 Credit: 100,251 RAC: 0 |
Dave Any chance of making the deadlinc 2 weeks or even 10 days rather than 7 days? That short deadline sent one of my systems into EDF and it only has 3 Rosetta units. Summery of other units on that system: SETI: 6 units with deadlines of Feb. 9, 2006 SETI BATA: 1 unit with a deadline of Mar. 7, 2007 (2007 is correct) CPDN: 1 Sulpher unit with a deadline of Dec. 18, 2006 Just the 3 Rosetta units with a deadline of Feb. 2 and Feb. 3 sent it into EDF, but of course that is good for Rosetta getting the time when in EDF until someone suspends it to get some other work done. What surprised me was when the Rosetta units I had were almost finished it called for more and got two more while in EDF. Currently the cache is set for 3 days. Ray Brown Pizza@Home Rays Place Rays place Forums |
[B@H] Ray Send message Joined: 20 Sep 05 Posts: 118 Credit: 100,251 RAC: 0 |
I faially had to suspend Rosetta to get the system back to doing CPDN till I get up in the morning. Ray |
Keck_Komputers Send message Joined: 17 Sep 05 Posts: 211 Credit: 4,246,150 RAC: 0 |
What surprised me was when the Rosetta units I had were almost finished it called for more and got two more while in EDF. Currently the cache is set for 3 days. NWF and EDF are not mutually exclusive. Although you rarely see EDF without NWF it is possible. BOINC WIKI BOINCing since 2002/12/8 |
Stanley A Bourdon Send message Joined: 17 Sep 05 Posts: 3 Credit: 112,907 RAC: 4 |
I faially had to suspend Rosetta to get the system back to doing CPDN till I get up in the morning. If you Waite a little while negative debt will build high enough that no more Rosetta WU will be downloaded until the negative debt is payed off by other projects running. For this to happen the negative debt for a project must be grater than the connect to time. for a 60 minute switch time debt must be more negative than -3600. May take a day or two to work out. I use 1500 minutes so I need to be more negative than 90,000. This can take over a week to re-balance. this assumes that you are using a client after 4.71 ? or for sure a 5.x.x client. Stanley Boinc Wikipedia - the FAQ in active change |
Joe Send message Joined: 26 Sep 05 Posts: 3 Credit: 588,731 RAC: 1,006 |
I loved the one month deadlines. I understand why shorter may be better but unfortunately they dont work for me. I suppose the percentage of folks who stop crunching the project due to the change is negligable. |
Johnathon Send message Joined: 5 Nov 05 Posts: 120 Credit: 138,226 RAC: 0 |
|
Message boards :
Number crunching :
Shorter WU deadlines
©2024 University of Washington
https://www.bakerlab.org