I've started this thread since the other one was getting too big to wade through and I couldn't easily see the answer to my question(s). I know 4.35 is development.
------------------------
A question on the debt, long_term_debt and resource share.

On my laptop I have 37 LHC wu's, three SAH wu's and no predictor wu's with the following set up.

Project Resource Share
LHC 500
Predictor 250
SETI 250

Client_state.xml show the following for debt and long_term_debt.

Project Debt Long_term_debt
LHC 42440 28736
Predictor 0 -15053
SETI 0 -13683

Questions.

1. Are these debt numbers in seconds?

2. BOINC has not downloaded Predictor in 24hrs and I have none in the cache (which defeats the purpose of a cache). Will BOINC download work when the long_term_debt gets to be above 0?

3. Why does BOINC allow a cache to run down to zero work - doesn't this defeat the purpose of BOINC?

4. How is the long_term_debt calculated?

5. How often are the debt numbers calculated? Is it every time a project update happens?

6. Do the long_term_debt numbers mean that BOINC will continue crunching LHC for another 28736 seconds?

7. How is the resource share used with the debt calculations?

8. When BOINC eventually downloads predictor work, it's deadline period is 7 days whereas both LHC and SETI are 14 days, since the predictor deadline will be sooner than both LHC and SETI, will BOINC only crunch predictor until the predictor wu's run out and then only crunch LHC and SETI to reduce their long_term_debt and not download any more predictor since it will have a negative long_term_debt?

9. Will BOINC debt scheduling only really work properly once the LHC and SETI deadlines are within the same time frame as a predictor wu and then project resource sharing and wu caching will work effectively ? So in effect BOINC will be bouncing between having pred work to no pred work until the LHC and SETI wu's are 7 days old?

Live long and crunch!

---

Paul
(S@H1 8888)
And proud of it!

I'll take a stab at some of this:

1. Are these debt numbers in seconds?

Everything else is in seconds so I think it is safe to assume this is as well.

2. BOINC has not downloaded Predictor in 24hrs and I have none in the cache (which defeats the purpose of a cache). Will BOINC download work when the long_term_debt gets to be above 0?

According to an answer I got here, yes. This is how BOINC is intended to work.

4. How is the long_term_debt calculated?

No idea here. Probably something like cpu time (in seconds) spent on one project divided by total cpu time since last change of resource share compared to resource share of project divided by total of all resource share values.

5. How often are the debt numbers calculated? Is it every time a project update happens?

My guess is that this is calculated when switching between projects. Modified in preferences under: "Switch between applications every..."

6. Do the long_term_debt numbers mean that BOINC will continue crunching LHC for another 28736 seconds?

Not sure. My guess is yes, but with the stipulation that it will switch to another project when the WU deadlines for that project get close. (So you don't run the risk of not reporting on units you have downloaded.) After it completes the ones that are close to deadline, it probably won't download more until the LHC debt is "paid off."

7. How is the resource share used with the debt calculations?

See question 4.

8. When BOINC eventually downloads predictor work, it's deadline period is 7 days whereas both LHC and SETI are 14 days, since the predictor deadline will be sooner than both LHC and SETI, will BOINC only crunch predictor until the predictor wu's run out and then only crunch LHC and SETI to reduce their long_term_debt and not download any more predictor since it will have a negative long_term_debt?

See questions 2 and 6.

9. Will BOINC debt scheduling only really work properly once the LHC and SETI deadlines are within the same time frame as a predictor wu and then project resource sharing and wu caching will work effectively ? So in effect BOINC will be bouncing between having pred work to no pred work until the LHC and SETI wu's are 7 days old?

By switching between applications, BOINC will even out all the debt to managable amounts. This may take some time (especially after adding a new project. And because different projects require different times to cruch respective units, you may find that projects with long times (eg CPDN) will operate with zero cache.

Hope this helps, I'm not totally sure about any of this being correct but it makes sense to me. Hopefully someone with the project or with greater knowledge at least can confirm or deny my answers.

---

7,049 S@H Classic Credits

<blockquote>I've started this thread since the other one was getting too big to wade through and I couldn't easily see the answer to my question(s). I know 4.35 is development.
------------------------
A question on the debt, long_term_debt and resource share.

On my laptop I have 37 LHC wu's, three SAH wu's and no predictor wu's with the following set up.

Project Resource Share
LHC 500
Predictor 250
SETI 250

Client_state.xml show the following for debt and long_term_debt.

Project Debt Long_term_debt
LHC 42440 28736
Predictor 0 -15053
SETI 0 -13683

Questions.

1. Are these debt numbers in seconds?
</blockquote>
Yes.
<blockquote>

2. BOINC has not downloaded Predictor in 24hrs and I have none in the cache (which defeats the purpose of a cache). Will BOINC download work when the long_term_debt gets to be above 0?
</blockquote>
Yes.
<blockquote>

3. Why does BOINC allow a cache to run down to zero work - doesn't this defeat the purpose of BOINC?
</blockquote>
It doesn't really defeat the purpose of BOINC. This is done to ensure that deadlines are all met. BOINC is still doing multiple WUs during the same day. BOINC has never guaranteed that all projects would have work at the same time. One side effect of this is that if the debt is small and negative when the WU is completing, the report will be made fairly quickly.
<blockquote>

4. How is the long_term_debt calculated?
</blockquote>
The same way that debt is with the exception that LT debt is shifted so the average is always 0.
<blockquote>

5. How often are the debt numbers calculated? Is it every time a project update happens?
</blockquote>
Once a second. Along with debt and a query whether anything has to be done like download work, upload files, report work... There is a polling loop that does all of this once per second. It was already there, I just made a couple of modifications.
<blockquote>

6. Do the long_term_debt numbers mean that BOINC will continue crunching LHC for another 28736 seconds?
</blockquote>
No. The short term debt <debt> in the state file determines which runs next (highest wins) unless the CPU needs to be in crunch earliest mode in which case the earliest deadline is used instead.
<blockquote>

7. How is the resource share used with the debt calculations?
</blockquote>
The resource fraction and the CPU time used for each project determine the offset in the debt. Resource fraction for a project = resource share for the project / total resource share. debt += wall time * resource fraction for project - CPU time for project. All of the debts are shifted after they are all recalculated. ST debts are shifted so the smallest is 0, LT debts are shifted so the mean is 0.
<blockquote>

8. When BOINC eventually downloads predictor work, it's deadline period is 7 days whereas both LHC and SETI are 14 days, since the predictor deadline will be sooner than both LHC and SETI, will BOINC only crunch predictor until the predictor wu's run out and then only crunch LHC and SETI to reduce their long_term_debt and not download any more predictor since it will have a negative long_term_debt?
</blockquote>
Not necessarily. It depends on whether the CPU scheduler determines that a deadline is in danger of being missed if it does not use Earliest Deadline First mode. Normal mode (highest ST debt next) is preferred.
<blockquote>

9. Will BOINC debt scheduling only really work properly once the LHC and SETI deadlines are within the same time frame as a predictor wu and then project resource sharing and wu caching will work effectively ? So in effect BOINC will be bouncing between having pred work to no pred work until the LHC and SETI wu's are 7 days old?
</blockquote>
It should work just fine in just about all cases. There are a couple of pathological cases that are not protected, but I am hoping that these are rare. If a project is allowed to download work, and there is already some work that has a deadline of say 7 days, and the current work will take 2 days, and the new work that is downloaded will take 7 days and has a 7 day deadline, there will be trouble.

In general normal mode where the highest ST debt gets the next time slot (think about the way that 4.25 works, and this is what normal mode is). Only if there is a danger of a WU being late will Earliest Deadline First be used.

The criteria for Earliest Deadline First:
1) A deadline is earlier than 24 hours from now.
2) A deadline is earlier than 2 * the queue size. This allows modem users to report work on time more often.
3) If you order the WUs by deadline and start adding the remaining processing times, is there any place in the chain where the sum is greater than 0.8 * (the deadline - now).

The criteria for no more work from anywhere.
1) See #3 above.
2) More than a maximum number of projects on the host (default is 5, and it can by changed by editing the global_prefs.xml file).
3) If the sum of required time fractions is greater than 0.8. The time fraction for a WU is the processing time remaining / the wall time remaining.
<blockquote>

Live long and crunch!

</blockquote>

---

BOINC WIKI

Thanks John,

On another pc (3.2GHz HT) I currently have my resource share set up like this;
LHC 1500
Einstein 250
Predictor 250

For the last 10hrs it has only crunched Einstein on both cpu's (deadlines not a problem yet). Not something I'm impressed with really.

Can we have the old way back please, but with BOINC then concentrating on wu's singularly like this only when they run into deadline problems. I think that is what we were expecting.

Live long and crunch!

Paul.

Paul, I'm starting to agree. It can be assumed that the actions we are seeing are "Self" inflicted. In that We have "Long" (by some reference) "Connect To" settings. I'm using 3 days and most the complaints are coming from people with "Long" connect to times. With my that "High" the day after I download some Einsteins the Ccheduler jumps into Panic mode. I'm using Dial Up, but also connect every day, so I really don't need 3 days worth of work (really 2 days due to benchmarking) per project. I like the longer cache in case of outage.

I also think if I let this run long enough, I'll see a disparity of higher credit towards the shorter deadlined projects (Einstein and PPAH) even though resource share is set to 100 each (seti, Einstein, PPAH, LHC).

what do you think?

My name may not be Paul, but I notice I am constantly downloading LHC units after they have run their full cache. Seti and Einstein wait as always until the last unit is about 85-90% done, before downloading 2 new units per project. Still much on a 0.5 connection setting.

---

I have found that the new scheduler works much better with a connect time of less than 1. I had mine at 1.2 and it would not keep WUs for all projects on my system but after lowering it to 0.9 it started working, like it did before installing new scheduler, and now keeping work from all projects :)

---

98SE XP2500+ @ 2.1 GHz Boinc v5.8.8

And God said"Let there be light."But then the program crashed because he was trying to access the 'light' property of a NULL universe pointer.

The idea of the BOINC cache is to outlast any outages and to keep your computer crunching while your away for a few days. If I have a cache of 3 days I don't want BOINC to only try and connect every 3 days, I want BOINC to connect when I want it to and to continually hold 3 days of work.

If I'm on dial-up then when ever I dial in I want it to upload and download work (probably once per day and in reality whenever I'm connected), if I have an always on connection then I want BOINC to hold a cache of 3 days work and update whenever a wu is completed or at most every 12 hrs automatically. Currently a 3 day cache in reality is much less since Predictor and SAH severely overestimate the crunch times, Einstein under estimates it a little while LHC is pretty close but still a little over estimation.

I don't think it is unreasonable to ask that BOINC caches 3 to 5 days of work where deadlines are at 7 days, anything else is trying to work within some arbitrary limitations set by the BOINC devs.

Live long and crunch!

Paul.

<blockquote>The idea of the BOINC cache is to outlast any outages and to keep your computer crunching while your away for a few days. If I have a cache of 3 days I don't want BOINC to only try and connect every 3 days, I want BOINC to connect when I want it to and to continually hold 3 days of work.

If I'm on dial-up then when ever I dial in I want it to upload and download work (probably once per day and in reality whenever I'm connected), if I have an always on connection then I want BOINC to hold a cache of 3 days work and update whenever a wu is completed or at most every 12 hrs automatically. Currently a 3 day cache in reality is much less since Predictor and SAH severely overestimate the crunch times, Einstein under estimates it a little while LHC is pretty close but still a little over estimation.

I don't think it is unreasonable to ask that BOINC caches 3 to 5 days of work where deadlines are at 7 days, anything else is trying to work within some arbitrary limitations set by the BOINC devs.</blockquote>
While BOINC may not do exactly the way you describe, the new scheduler in 4.35 does a really good job of keeping relevant work in the cache and not caching work that it can't complete without going into a panic.

If you let go of the idea that "my cache must be full" and replace it with "I want to do all available work in the proportions I've asked" then you've got it. If BOINC is starved by a SETI outage, SETI builds debt, and when SETI is back, BOINC will crunch only SETI until it gets caught up.

In other words, it does do what you want, just not the way you expect.

---

<blockquote>
If you let go of the idea that "my cache must be full" and replace it with "I want to do all available work in the proportions I've asked" then you've got it. If BOINC is starved by a SETI outage, SETI builds debt, and when SETI is back, BOINC will crunch only SETI until it gets caught up.</blockquote>

Ok, but why should I give that up? I mean, I do have high speed "always-on" connection, but why must I surrender my desire to keep the tank topped off?

I am playing devil's Advocate here ...

<blockquote>
In other words, it does do what you want, just not the way you expect.</blockquote>

But, I would argue that I do want it to maintain the proportions, but why can it not extrapolate into the future better?

This goes back to my argument about the client "knows" about its performance and could/should track this as part of its normal processing. Tracking the average processing time per work unit is possible (though LHC@Home does have that wrinkle of the same application with the spread of times - though that information is stored somewhere and could be tracked by the application).

Any way ... my brain stopped so I lost track of my argument

---

<blockquote>Ok, but why should I give that up? I mean, I do have high speed "always-on" connection, but why must I surrender my desire to keep the tank topped off?

I am playing devil's Advocate here ...
</blockquote>

I agree with your points Paul. With the new scheduler activity, BOINC is less dial up friendly than ever.

I don't run a huge cache, (0.5 days) and while attached to more projects than probably the average user, I have never returned a WU past deadline. So I'm absolutely mystified as to why now the CC has decided I simply can not have any more WUs from almost every project. (no, is not due to the max_projects setting) I have had no trouble for well over a year, so why give me grief now?

I've mentioned before in another thread, surely there can be some sort of a dumb*** filter implemented in the CC. The CC is obviously aware if you've missed deadlines, so why not institute the 'harsher' scheduling methods for those that are having 'issues' meeting deadlines, and otherwise leave normal operation alone?

Hindering user options because some people can't get their settings right is never a good idea IMO.

<blockquote>
I agree with your points Paul. With the new scheduler activity, BOINC is less dial up friendly than ever.
</blockquote>

I am having troubles today so I am not able to "speak" clearly, but the intent, I think, of what John was trying to do is needed. I am just not sure we have truly covered the ground correctly.

Like you I do the short buffer but I do have the advantage of 5 projects where there are only 4 fully open for participants right now and that hepls a lot ...

sigh, spelling bad quit

---

<blockquote><blockquote>Ok, but why should I give that up? I mean, I do have high speed "always-on" connection, but why must I surrender my desire to keep the tank topped off?

I am playing devil's Advocate here ...
</blockquote>

I agree with your points Paul. With the new scheduler activity, BOINC is less dial up friendly than ever.</blockquote>

I think dialup is a seperate issue.

... and we aren't talking about limiting options.

We're talking about not trying to stay __too far__ ahead. Let me suggest an example, and hopefully John will help me out if I screw up too badly.

Let's say that you crunch Einstein and SETI, with a share of 10 for Einstein and 230 for SETI. You crunch 24 hours/day, so 1 hour of Einstein per day, and 23 hours of SETI. You have "connect every 1 days" set.

SETI goes down on Friday at midnight, and stays down all weekend.

By midnight Saturday you are out of SETI. You are now crunching Einstein exclusively, and on Sunday at midnight SETI has 23 hours "owed."

When SETI comes back up, BOINC downloads SETI exclusively and does not crunch Einstein until the debt is "paid back."

It doesn't download another Einstein unit until it has time to crunch.

So, over the long term (3 or 4 weeks?) everything gets crunched in the proportion you specify -- it's just that you have a whole weekend of Einstein followed by weeks of 100% SETI.

... and you always have a day's worth of something in your cache.

---

I tried to make it a bit more dialup friendly, however, it works best if the cache and the time between connections is about the same (so the cache can be mostly emptied before requesting more work). The CPU scheduler tries to make certain that there is enough work from somewhere to fill the cache, even if it means that I am ignoring trouble.

In order to get work returned in time, the CPU scheduler enters crunch earliest first if a deadline is less than 2* the period between connections. This is supposed to ensure that the work that will otherwise be late for the next connection will get done this connection.

---

BOINC WIKI

Ned, John,

Ok, I think I got it. For whatever reason I was seeing this as a someting it wasn't.

I told youo my brain is not working well.

---

<blockquote>Ned, John,

Ok, I think I got it. For whatever reason I was seeing this as a someting it wasn't.

I told youo my brain is not working well.</blockquote>
I don't think it's exactly intuitively obvious -- or we wouldn't have so many folks saying "it can't possibly be right!"

... but it looks great once you see what it does.

---

I don't know if I got it right...

my settings are as follow:

host1 (voyager) 4.37
BURP
(debt)0.000000(/debt)
(long_term_debt)646665.260049(/long_term_debt)
(resource_share)200.000000(/resource_share)
(dont_request_more_work/)

CPDN
(debt)0.000000(/debt)
(long_term_debt)-546572.438743(/long_term_debt)
(resource_share)20.000000(/resource_share)

SETI
(debt)0.000000(/debt)
(long_term_debt)-100092.821306(/long_term_debt)
(resource_share)80.000000(/resource_share)

my host voyager has completed all SETI workunits and doesn't even think of getting new ones.
BURP is down for developing and bug-fixing so I stopped the scheduler requests.
I have one WU for CDPN which deadline is 25th march 2006 and it's going to be done at about 16th may 2005.

Why the hell boinc doesn't load workunits for SETI?
I want to run SETI at 80% the time when BURP isn't up, but it doesn't.
It crunches CDPN all the time...

Andy

---

Want to know your pending credit?


The biggest bug is sitting 10 inch in front of the screen.

<blockquote>I don't know if I got it right...

my settings are as follow:

host1 (voyager) 4.37
BURP
(debt)0.000000(/debt)
(long_term_debt)646665.260049(/long_term_debt)
(resource_share)200.000000(/resource_share)
(dont_request_more_work/)

CPDN
(debt)0.000000(/debt)
(long_term_debt)-546572.438743(/long_term_debt)
(resource_share)20.000000(/resource_share)

SETI
(debt)0.000000(/debt)
(long_term_debt)-100092.821306(/long_term_debt)
(resource_share)80.000000(/resource_share)

my host voyager has completed all SETI workunits and doesn't even think of getting new ones.
BURP is down for developing and bug-fixing so I stopped the scheduler requests.
I have one WU for CDPN which deadline is 25th march 2006 and it's going to be done at about 16th may 2005.

Why the hell boinc doesn't load workunits for SETI?
I want to run SETI at 80% the time when BURP isn't up, but it doesn't.
It crunches CDPN all the time...

Andy</blockquote>
Because CPDN has the least debt of all active projects (by quite a bit).

---

ok, it has the least debt, but...
a) the deadline is far away
b) it should work only a fifth of the time SETI should

After reading the posts in here I thought the higher the debt the more work it gets.
So wrong here?

Andy

---

Want to know your pending credit?


The biggest bug is sitting 10 inch in front of the screen.

The project that is most positive works first... since BURP is down, the "most positive" is CPDN. Over the LONG RUN it will work ~20% of the time but in any specific time frame it may run more or less. If you have been crunching significantly more SETI in the past while, then the system will switch over to CP to "make up" for the time it was "supposed" to get

---