Panic Mode On (102) Server Problems?

Message boards : Number crunching : Panic Mode On (102) Server Problems?
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 25 · Next

AuthorMessage
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1758842 - Posted: 23 Jan 2016, 23:11:46 UTC

V8 has been successfully rolled out and Lunatics has released their new version...

ID: 1758842 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1758844 - Posted: 23 Jan 2016, 23:14:30 UTC

... so there's nothing to panic about at all!

(ducks)
ID: 1758844 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1758851 - Posted: 23 Jan 2016, 23:36:08 UTC

Dang, locked out the old thread while I was composing a reply! ;^)

Okay, reply to Message 1758837:

Both the GTX cards are running on Haswell rigs with the Intel GPU also running 3 jobs and the CPU running 7 jobs. The Xeon machine is running 3 CPU jobs. The Intel Haswell GPU rigs are averageing 2.5 to 2.79 turnaround days.

Turn around days are like Credits, they are a lagging indicator & it will take several days for the numbers to reflect reality.
That's why I prefer to use actual run times.

It also seems to me that "Average turnaround time" can be affected by numerous factors unrelated to what you're trying to observe. Such as the size of your work buffer (especially if you happen to change it), whether you run less than 24/7, other BOINC projects that may use GPU time, or the normally changing mix of angle ranges for the tasks you receive (with "normal" ARs taking longer to turn around than the "shortie" higher ARs that you often see mentioned here).

I agree with Grant that manually recording actual run times (spreadsheet, database, scratch pad, napkin, or whatever) gives you the best picture of performance. Just pick two or three common Angle ranges (I like tasks around 0.41nnnn for a normal AR and perhaps 2.72nnnn for higher ones), check your recent tasks for Stderrs with those ARs which don't show an overflow condition, and record them. You should find that the run times and CPU times for any given AR will stay pretty consistent, with only an occasional one that's abnormally high.
ID: 1758851 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1758854 - Posted: 23 Jan 2016, 23:46:12 UTC - in response to Message 1758851.  

The underlying problem with 'averages', is not a new or mysterious thing. They are susceptible to various kinds of noise, and don't respond to change well (promptly without overshoot). Distilling things down as much as I could on a Sunday morning.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1758854 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1758872 - Posted: 24 Jan 2016, 0:31:06 UTC

One way to get a good look at your runtimes is to simply disable you network activity for awhile, then you can get a good look at what is in your upload queue.
ID: 1758872 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1758875 - Posted: 24 Jan 2016, 0:38:55 UTC - in response to Message 1758872.  
Last modified: 24 Jan 2016, 0:40:26 UTC

One way to get a good look at your runtimes is to simply disable you network activity for awhile, then you can get a good look at what is in your upload queue.

But that won't tell you what the AR is for each task, which is the only way you can be sure you're comparing apples to apples. You still have to dig for that info.

EDIT: You also have to know whether a task was a -9 overflow or not.
ID: 1758875 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1758919 - Posted: 24 Jan 2016, 2:46:44 UTC - in response to Message 1758875.  

Inconclusives are certainly a lot less than before.

So far most of the ones I've got are against x86_64-apple-darwin systems, and one Windows system.

2038109353
2038051505
2038004355
2037962132
2037732068
2037292502
Grant
Darwin NT
ID: 1758919 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1758937 - Posted: 24 Jan 2016, 3:33:49 UTC - in response to Message 1758919.  

By design, Eric amped up the precision, and we follow suit as the manpower and technology allows,
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1758937 · Report as offensive
Kevin Olley

Send message
Joined: 3 Aug 99
Posts: 906
Credit: 261,085,289
RAC: 572
United Kingdom
Message 1758969 - Posted: 24 Jan 2016, 8:46:16 UTC - in response to Message 1758919.  

Inconclusives are certainly a lot less than before.

So far most of the ones I've got are against x86_64-apple-darwin systems, and one Windows system.



+1
Kevin


ID: 1758969 · Report as offensive
Filipe

Send message
Joined: 12 Aug 00
Posts: 218
Credit: 21,281,677
RAC: 20
Portugal
Message 1759057 - Posted: 24 Jan 2016, 17:27:16 UTC

Matt talked about MB tasks getting 4 time bigger than now.

Does the Tasks recorded at Green Bank with the new recorder, will be bigger?(And minimize the overlap?)
ID: 1759057 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22190
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1759061 - Posted: 24 Jan 2016, 17:37:59 UTC

As far as I'm aware the GBT data sets will be bigger, the band width of the receivers is bigger, overlap will be a similar proportion of the sample.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1759061 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34257
Credit: 79,922,639
RAC: 80
Germany
Message 1759073 - Posted: 24 Jan 2016, 18:17:03 UTC
Last modified: 24 Jan 2016, 18:18:52 UTC

The tasks i have processed so far at beta were only a few KB bigger but are merely VLAR`s.

I downloaded a few for testing and file size is 365xxx bytes.


With each crime and every kindness we birth our future.
ID: 1759073 · Report as offensive
Terror Australis
Volunteer tester

Send message
Joined: 14 Feb 04
Posts: 1817
Credit: 262,693,308
RAC: 44
Australia
Message 1759102 - Posted: 24 Jan 2016, 20:56:41 UTC
Last modified: 24 Jan 2016, 21:03:34 UTC

Just looked at my active rig and it had 104 download errors, all on V8 CUDA tasks.

While I watched it successfully reported and downloaded new CPU tasks.

Is anyone else getting these download errors ?

EDIT: Running stock apps with no app_info file. It appears the V8 CUDA 4.2 app downloaded and installed ok. It's only the WU's that failed.

T.A.
ID: 1759102 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1759103 - Posted: 24 Jan 2016, 20:59:41 UTC - in response to Message 1759102.  

This corner of the upper left coast is download error free.
ID: 1759103 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1759105 - Posted: 24 Jan 2016, 21:02:10 UTC

None to report here....
All quiet on the kitty front.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1759105 · Report as offensive
Profile Louis Loria II
Volunteer tester
Avatar

Send message
Joined: 20 Oct 03
Posts: 259
Credit: 9,208,040
RAC: 24
United States
Message 1759108 - Posted: 25 Jan 2016, 0:17:22 UTC

WOW! What was that all about?
ID: 1759108 · Report as offensive
Admiral Gloval
Avatar

Send message
Joined: 31 Mar 13
Posts: 20243
Credit: 5,308,449
RAC: 0
United States
Message 1759113 - Posted: 25 Jan 2016, 0:25:14 UTC

Don't know. Everything went wonky.

ID: 1759113 · Report as offensive
Profile Cactus Bob
Avatar

Send message
Joined: 19 May 99
Posts: 209
Credit: 10,924,287
RAC: 29
Canada
Message 1759115 - Posted: 25 Jan 2016, 0:33:49 UTC

Possibly a small glitch ... glitch .... glitch ... in the matrix. Probably nothing to worry about.

Possibly a small glitch ... glitch .... glitch ... in the matrix. Probably nothing to worry about.

Possibly a small glitch ... glitch .... glitch ... in the matrix. Probably nothing to worry about.

Bob
Sometimes I wonder, what happened to all the people I gave directions to?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SETI@home classic workunits 4,321
SETI@home classic CPU time 22,169 hours
ID: 1759115 · Report as offensive
OTS
Volunteer tester

Send message
Joined: 6 Jan 08
Posts: 369
Credit: 20,533,537
RAC: 0
United States
Message 1759118 - Posted: 25 Jan 2016, 0:59:04 UTC
Last modified: 25 Jan 2016, 1:03:36 UTC

Hmmm, what ever happened is still affecting the AP results out in the field reporting. Looks like it is not updating.


AP Tasks were available on 24 Jan 2016, at 20:50:04 UTC [Channels Total=42, Prog=5, Done=24, To Do=13] ROITF=101,300, RRLH=1,193
AP Tasks were available on 24 Jan 2016, at 21:00:04 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=101,619, RRLH=988
The  SSP Time Did Not Change as of 24 Jan 2016, 21:26
AP Tasks were available on  [Channels Total=, Prog=, Done=, To Do=] ROITF=, RRLH=
The  SSP Time Did Not Change as of 24 Jan 2016, 21:36
The  SSP Time Did Not Change as of 24 Jan 2016, 21:46
The  SSP Time Did Not Change as of 24 Jan 2016, 21:56
The  SSP Time Did Not Change as of 24 Jan 2016, 22:06
The  SSP Time Did Not Change as of 24 Jan 2016, 22:16
AP Tasks were available on 24 Jan 2016, at 22:10:04 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=102,035, RRLH=949
AP Tasks were available on 24 Jan 2016, at 22:30:03 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=102,035, RRLH=949
AP Tasks were available on 24 Jan 2016, at 22:40:03 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=102,035, RRLH=949
The 24 Jan 2016, 22:40:03 UTC SSP Time Did Not Change as of 24 Jan 2016, 22:52
AP Tasks were available on 24 Jan 2016, at 23:00:03 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=102,035, RRLH=949
AP Tasks were available on 24 Jan 2016, at 23:10:03 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=102,035, RRLH=949
AP Tasks were available on 24 Jan 2016, at 23:20:03 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=102,035, RRLH=949
The 24 Jan 2016, 23:20:03 UTC SSP Time Did Not Change as of 24 Jan 2016, 23:32
AP Tasks were available on 24 Jan 2016, at 23:40:03 UTC [Channels Total=42, Prog=5, Done=26, To Do=11] ROITF=102,035, RRLH=0
AP Tasks were available on 24 Jan 2016, at 23:50:06 UTC [Channels Total=42, Prog=4, Done=26, To Do=12] ROITF=102,035, RRLH=0
AP Tasks were available on 25 Jan 2016, at 0:00:05 UTC [Channels Total=42, Prog=4, Done=26, To Do=12] ROITF=102,035, RRLH=287
AP Tasks were available on 25 Jan 2016, at 0:10:05 UTC [Channels Total=42, Prog=4, Done=27, To Do=11] ROITF=102,035, RRLH=287
AP Tasks were available on 25 Jan 2016, at 0:20:06 UTC [Channels Total=28, Prog=4, Done=17, To Do=7] ROITF=102,035, RRLH=1,226
The 25 Jan 2016, 0:20:06 UTC SSP Time Did Not Change as of 25 Jan 2016,  0:32
AP Tasks were available on 25 Jan 2016, at 0:40:04 UTC [Channels Total=28, Prog=4, Done=17, To Do=7] ROITF=102,035, RRLH=2,149
AP Tasks were available on 25 Jan 2016, at 0:50:04 UTC [Channels Total=28, Prog=4, Done=17, To Do=7] ROITF=102,035, RRLH=2,289
ID: 1759118 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1759120 - Posted: 25 Jan 2016, 1:15:22 UTC

Work requests are getting "no tasks available." SSP ready to send hasn't updated in 4 hours.
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1759120 · Report as offensive
1 · 2 · 3 · 4 . . . 25 · Next

Message boards : Number crunching : Panic Mode On (102) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.