Granted Credits are not correct calculated!

Message boards : Number crunching : Granted Credits are not correct calculated!
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Honie

Send message
Joined: 22 Jan 04
Posts: 141
Credit: 29,681,066
RAC: 0
Germany
Message 62579 - Posted: 9 Jan 2005, 13:34:39 UTC

With the new 3 of 4 sended WU calculation, the
granted credit is not working correct.

look at this example:

http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7159111

the four claimed are: 38.15;18.77;27.11 and 43.93

the granted is 27.11;

Because of the formular the correct granted should be: (27.11 + 38.15) / 2 = 32.63

more examples:

http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7159111

http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7159102

and so on..

any ideas about this behaviour??
ID: 62579 · Report as offensive
Profile kinnison
Avatar

Send message
Joined: 23 Oct 02
Posts: 107
Credit: 7,406,815
RAC: 7
United Kingdom
Message 62582 - Posted: 9 Jan 2005, 13:39:17 UTC

The way I believe it works, Honie, is that it is only on the first three usually that the credit is worked out. So in this case, the first three were 38.15, 18.77 and 27.11. The middle of those is 27.11 and that is what is granted. The fourth result just gets whatever was granted for the first 3.
Haven't looked at the other two examples of yours, but I would presume they are the same.
<img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=268&amp;prj=1&amp;trans=off" /><img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=268&amp;prj=4&amp;trans=off" />
ID: 62582 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 62583 - Posted: 9 Jan 2005, 13:41:28 UTC - in response to Message 62582.  

> The way I believe it works, Honie, is that it is only on the first three
> usually that the credit is worked out. So in this case, the first three were
> 38.15, 18.77 and 27.11. The middle of those is 27.11 and that is what is
> granted. The fourth result just gets whatever was granted for the first 3.
> Haven't looked at the other two examples of yours, but I would presume they
> are the same.
>

That's correct - credit is calculated on the first 3 returned results only. The 4th result to be returned, if validated, get's whatever the other 3 got. No recalculation is performed.

Ned


*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 62583 · Report as offensive
Profile Thierry Van Driessche
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3083
Credit: 150,096
RAC: 0
Belgium
Message 62585 - Posted: 9 Jan 2005, 13:44:07 UTC
Last modified: 9 Jan 2005, 13:46:25 UTC

I believe it it still OK.

For the WU with id 71591113 fhe first result that came in had a claimed credit of 38.15, the second a claimed credit of 18.77 and for the third it was 27.11. The granted credit if I am correct is the mid value of these 3, so it is 27.11. The fourth returned one receives this value as well.

For the WU with id 7159102, the numbers are 33.88 then 22.58 and finally 35.15, so the mid value is 22.58, corresponding to the granted credit.
ID: 62585 · Report as offensive
Honie

Send message
Joined: 22 Jan 04
Posts: 141
Credit: 29,681,066
RAC: 0
Germany
Message 62586 - Posted: 9 Jan 2005, 13:48:00 UTC - in response to Message 62582.  

> The way I believe it works, Honie, is that it is only on the first three
> usually that the credit is worked out. So in this case, the first three were
> 38.15, 18.77 and 27.11. The middle of those is 27.11 and that is what is
> granted. The fourth result just gets whatever was granted for the first 3.
> Haven't looked at the other two examples of yours, but I would presume they
> are the same.
>

Ok, would make sense, but in the past, if a fourth or fifth result is returned in time, the granted credit was calculated backwards by eliminating the highest and lowest and building the average of the rest, even when the granted credit was calculated before.

Example:

3 WU were send and 2 results were returned with 20.5 and 24.5 claimed and the 3rd ran out of time. The 4th WU was sended and BEFORE the 4th was sending a result the 3rd returned his result with 22.5 claimed. Now the validator calculats the granted with 22.5 which is correct.
Now the 4th returns its result with claimed 21.5 the granted is corected with
(21.5+22.5)/2 = 22 by the validator. And now the granted is decreased by 0.5 to 22 granted.

That's the way it worked in the past. And I don't know any reason why it should changed.
ID: 62586 · Report as offensive
Honie

Send message
Joined: 22 Jan 04
Posts: 141
Credit: 29,681,066
RAC: 0
Germany
Message 62588 - Posted: 9 Jan 2005, 13:55:28 UTC

Please look at this thread: (3 Months ago)

http://setiweb.ssl.berkeley.edu/forum_thread.php?id=5525
ID: 62588 · Report as offensive
Profile NickBrownsFan

Send message
Joined: 28 Sep 01
Posts: 24
Credit: 1,705,461
RAC: 0
United States
Message 62609 - Posted: 9 Jan 2005, 15:41:04 UTC

Also dont forget that with the backlog of work some unit atm are getting all 4 results in before the credit is granted. When this happens then it does use an avg of the middle 2.
Once things are caught up I doubt you'll see to much more of these types of conflicting granted credits.
<a href="http://www.teampicard.net"><img src="http://boinc.mundayweb.com/seti2/stats.php?userID=2205&amp;trans=off"></a>
ID: 62609 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 62648 - Posted: 9 Jan 2005, 18:09:16 UTC - in response to Message 62579.  

> With the new 3 of 4 sended WU calculation, the
> granted credit is not working correct.
>
> look at this example:
>
> http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7159111
>
> the four claimed are: 38.15;18.77;27.11 and 43.93
>
> the granted is 27.11;
>
> Because of the formular the correct granted should be: (27.11 + 38.15) / 2 =
> 32.63
>
NO formula, NO division is done with the credits! Straight numbers here, 3 results, all within statistical limits, the middle number is what everyone gets!
Computer A requests 34.75, Computer B 35.98, Computer C 43.22, ALL users are granted 35.98 credits!


>
> any ideas about this behaviour??
>
ID: 62648 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 62651 - Posted: 9 Jan 2005, 18:12:15 UTC - in response to Message 62586.  

> > The way I believe it works, Honie, is that it is only on the first three
> > usually that the credit is worked out. So in this case, the first three
> were
> > 38.15, 18.77 and 27.11. The middle of those is 27.11 and that is what is
> > granted. The fourth result just gets whatever was granted for the first
> 3.
> > Haven't looked at the other two examples of yours, but I would presume
> they
> > are the same.
> >
>
> Ok, would make sense, but in the past, if a fourth or fifth result is returned
> in time, the granted credit was calculated backwards by eliminating the
> highest and lowest and building the average of the rest, even when the granted
> credit was calculated before.
>
> Example:
>
> 3 WU were send and 2 results were returned with 20.5 and 24.5 claimed and the
> 3rd ran out of time. The 4th WU was sended and BEFORE the 4th was sending a
> result the 3rd returned his result with 22.5 claimed. Now the validator
> calculats the granted with 22.5 which is correct.
> Now the 4th returns its result with claimed 21.5 the granted is corected with
> (21.5+22.5)/2 = 22 by the validator. And now the granted is decreased by 0.5
> to 22 granted.
>
You are still trying to divide! NO FORMULAS are used in the granting of credits!

> That's the way it worked in the past. And I don't know any reason why it
> should changed.
>
No it didn't EVER work that way! I have been here since the Beta was still going on and we have NEVER done it the way you are thinking.

ID: 62651 · Report as offensive
Profile KWSN - MajorKong
Volunteer tester
Avatar

Send message
Joined: 5 Jan 00
Posts: 2892
Credit: 1,499,890
RAC: 0
United States
Message 62666 - Posted: 9 Jan 2005, 18:52:29 UTC

The four results come back in order:

38.15
18.77
27.11
43.93

Normally, the unit is validated after the 3rd one comes back, so discarding the high and the low values, this sets the 'granted credit' for this work unit to be the middle value, 27.11

If the 4th result comes back while the work unit is still active, it too gets the granted credit of 27.11 (if it matches). Granted credit is not recalculated... Ever.

If the validator is backlogged (as has been the case over the last week or so) and the 4th one comes back BEFORE the validator 'gets to' this work unit, then I believe the 'middle two' of the four claimed credits are averaged to get the granted credit. We haven't seen this behavior before because up until recently, there has only been three possible candidates for validation (the initial 3, plus any single issues to replace 'errors' -- past deadline, or bad results).

Rather recently, they set the initial issue from 3 to 4, but kept the validator quorum at 3. Only in the case of the 4th result being returned before a backlogged validator gets to the work unit should this averaging occur. Hopefully, it will only happen rarely (if the validator can 'keep up' after S@H-Classic goes offline).
https://youtu.be/iY57ErBkFFE

#Texit

Don't blame me, I voted for Johnson(L) in 2016.

Truth is dangerous... especially when it challenges those in power.
ID: 62666 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 62669 - Posted: 9 Jan 2005, 19:11:47 UTC - in response to Message 62666.  

Also if a wu is re-issued due to "no consensus yet", you can have 4 or more results being validated at the same time wu is validated, and therefore be part of the credit-decision-process.
ID: 62669 · Report as offensive
Profile kinnison
Avatar

Send message
Joined: 23 Oct 02
Posts: 107
Credit: 7,406,815
RAC: 7
United Kingdom
Message 62670 - Posted: 9 Jan 2005, 19:12:51 UTC

Has anyone actually seen this "averaging for 4 units" happen?
I've had a few results in now that were waiting validation, with 4 units there, an d they've all gone by the middle of the first 3.

I've not yet seen an average of the middle 2 of 4 taken.. i think this is a myth!
<img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=268&amp;prj=1&amp;trans=off" /><img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=268&amp;prj=4&amp;trans=off" />
ID: 62670 · Report as offensive
JAF
Avatar

Send message
Joined: 9 Aug 00
Posts: 289
Credit: 168,721
RAC: 0
United States
Message 62671 - Posted: 9 Jan 2005, 19:15:55 UTC - in response to Message 62648.  

> NO formula, NO division is done with the credits! Straight numbers here, 3
> results, all within statistical limits, the middle number is what everyone
> gets!
> Computer A requests 34.75, Computer B 35.98, Computer C 43.22, ALL users are
> granted 35.98 credits!
>
But then how is this one figured if no divisions are done?
http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7401955

24.17
28.51
26.49
28.77

granted = 27.50

That looks a lot like (28.51 + 26.49)/2
<img src='http://www.boincsynergy.com/images/stats/comb-912.jpg'>
ID: 62671 · Report as offensive
Profile kinnison
Avatar

Send message
Joined: 23 Oct 02
Posts: 107
Credit: 7,406,815
RAC: 7
United Kingdom
Message 62679 - Posted: 9 Jan 2005, 19:36:33 UTC - in response to Message 62671.  

> > NO formula, NO division is done with the credits! Straight numbers here,
> 3
> > results, all within statistical limits, the middle number is what
> everyone
> > gets!
> > Computer A requests 34.75, Computer B 35.98, Computer C 43.22, ALL users
> are
> > granted 35.98 credits!
> >
> But then how is this one figured if no divisions are done?
> http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7401955
>
> 24.17
> 28.51
> 26.49
> 28.77
>
> granted = 27.50
>
> That looks a lot like (28.51 + 26.49)/2
>


I agree, it does look like the average of the top/bottom of the first 3 results
It's the first result I've seen like this, I'd love an explanation from one of the SETI crew


<img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=268&amp;prj=1&amp;trans=off" /><img border="0" src="http://boinc.mundayweb.com/one/stats.php?userID=268&amp;prj=4&amp;trans=off" />
ID: 62679 · Report as offensive
Profile MattDavis
Volunteer tester
Avatar

Send message
Joined: 11 Nov 99
Posts: 919
Credit: 934,161
RAC: 0
United States
Message 62680 - Posted: 9 Jan 2005, 19:37:20 UTC - in response to Message 62670.  

> Has anyone actually seen this "averaging for 4 units" happen?
> I've had a few results in now that were waiting validation, with 4 units
> there, an d they've all gone by the middle of the first 3.
>
> I've not yet seen an average of the middle 2 of 4 taken.. i think this is a
> myth!
>

I've seen one. Here it is:

http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7355335
-----
ID: 62680 · Report as offensive
Honie

Send message
Joined: 22 Jan 04
Posts: 141
Credit: 29,681,066
RAC: 0
Germany
Message 62689 - Posted: 9 Jan 2005, 20:03:00 UTC

Thanks Matt, that's exactly what I mean.

Maybe it has to do something with the backlog of the validator,
but I think it is not a very good idea, if the granted credits
depends on if the validator has past this wu or not.

So this should be correct:

if the validator didn't pass the WU and 4 or more results are present,
the granted credtis are the average of the claimed after the min and max
are eliminated.

if the validator passed the WU the 4th or higer result will get the once
calculated credit of the middle result from the first three results.

So the granted credit depends on the time, the validator is passing the
results.



ID: 62689 · Report as offensive
Charles Doubrava / cld66
Volunteer tester

Send message
Joined: 16 Jul 99
Posts: 7
Credit: 451,914
RAC: 0
United States
Message 62690 - Posted: 9 Jan 2005, 20:04:09 UTC

Yes it did happen here is my recent WU.

http://setiweb.ssl.berkeley.edu/workunit.php?wuid=7324144


Computer A - 6.79
Computer B - 31.57
Computer C - 40.07
Computer D - 34.27

Remove Computer A & B
Add Computer C & D for a total of 74.34
Then divide 74.34/2 = 32.92

otherwise it should have been 34.27.
ID: 62690 · Report as offensive
Honie

Send message
Joined: 22 Jan 04
Posts: 141
Credit: 29,681,066
RAC: 0
Germany
Message 62694 - Posted: 9 Jan 2005, 20:09:35 UTC

Sorry?

Remove Comp A and Comp C

You get:

(31.57+34.27)/2 = 32.92

(Try calc.exe)

74.34/2 = 37.17 !!!!!!!!
ID: 62694 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 62721 - Posted: 9 Jan 2005, 20:54:06 UTC
Last modified: 9 Jan 2005, 20:54:24 UTC

The BOINC-crediting haven't changed, the credit is decided at the same time a wu is validated, based on how many of the results passed the validation:

If only 2 passed validation, grant the lowest claimed to everyone.
If 3 or more, remove highest and lowest claimed and average the rest.

Any results returned later and passes validation get the same credit as the others, no re-calculation of credit is done.

The system will wait for late arrivals of results after wu validated, but only till all results is either reported and tried validated, or reached their deadline.


Seti is now issuing wu to 4 users, but tries validation then got 3 "success"-results, so normally the 3 results returned 1st will decide the credit. But, since the validator currently is backlogged, many wu have now 4 results before validator keeps up, for these 4 is used for deciding the crediting.
There's also a small group of wu being re-issued due to no consensus, and these can have upto 6 results passing validation then wu at last is validated.

BTW, in SETI@home it's also possible only 2 of the results passes the validation. ;)
ID: 62721 · Report as offensive
Profile Benher
Volunteer developer
Volunteer tester

Send message
Joined: 25 Jul 99
Posts: 517
Credit: 465,152
RAC: 0
United States
Message 62751 - Posted: 9 Jan 2005, 22:03:33 UTC
Last modified: 9 Jan 2005, 22:04:29 UTC

Ingleside is correct,

Code for granting hasn't changed recently.

Has allways been:
A. remove highest claimed
B. remove lowest
C. add up remainder of claims (however many that may be).
D. Divide by count of claims in C (average).

Seti had allways been 3 results per WU, so 3-high-low = 1, and average of middle score = middle/1 .

Backlog caused a few of the 4 result WUs to all be returned and be in database before validation...and for those, it was average of middle two.

This is from looking at the server source code.
ID: 62751 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Granted Credits are not correct calculated!


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.