Message boards :
Technical News :
Sleepless in Oakland (Aug 27 2007)
Message board moderation
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
Minor issues over the weekend. One night penguin (the download server) got in a snit with the network and needed to be rebooted. No big deal there, except that traffic was vastly reduced for several hours there. Of greater concern was the swelling ready-to-assimilate queue. Normally this wouldn't be that big a deal and could wait until Monday to diagnose, but this backlog left extra workunits on disk (since they have to be assimilated before they can be deleted). Add this to our lower quorums and rising results-to-send queue, and the workunit file system almost filled up! I had to halt splitting for a while to keep this from happening. I also tried adding extra assimilator processes but this didn't help. Jeff found the problem this morning: some new assimilator code to update the "hot pix" table in the science database was doing sequential scans for row updates. A simple "update stats" on the informix table cleaned that right up quick. The "hot pix" table will be used for the near time persistency checker (yep - we're actually working on that stuff slowly but surely). The queue, and therefore the workunit storage usage, should be draining now. Today I've been working on getting new disk volumes on line (a continuation from my last post). Not sure why I didn't know this already, but it turns out the ext3 filesystem has an 8 Terabyte limit. So we had to adjust certain plans for volume configuration until they come out with ext4. I have no time or interest in trying any other filesystems at this point. Last night woken up around 3:00am by a nearby 2.3 earthquake and again at 3:10am by a 2.4 at the same exact location. Actually this has been an active hot spot for the past year - right at the base of the Claremont Hotel (about a mile or two away from campus). Tonight I'll be up again around the same time to catch the full lunar eclipse, or at least I'll try to be. I'm kinda wrecked. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
speedimic Send message Joined: 28 Sep 02 Posts: 362 Credit: 16,590,653 RAC: 0 |
There are 48bit patches on sourceforge to make 1024 PByte possible - but I think still marked experimental... However, from what I read - and experienced! - the e2fsck time will be incredibly large - about 2 hours for 3TB in my case. mic. |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
Thanks Immensely for the Update Matt - It is Appreciated Sir! > reminders of 25 Years Experience w/ California Quakes - oh Lord - i won't go there . . . < nice work by Each of You @ Berkeley - Keep it up . . . |
Lois Petrolito Send message Joined: 9 Nov 03 Posts: 10 Credit: 68,170 RAC: 0 |
Matt The clogged servers could probably explain a problem I've been having lately....v-e-e-r-r-y slow uploads and downloads. I have a cable modem connection and have been noticing download and upload speeds of less than 5000 kbps. I thought maybe my connection was having the issue, but all my other downloads(software, my other BOINC project, etc) have been up to speed. Any feedback would be helpful. Thanks. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
Matt 5000 kbps or 5mbps or 5,000,000 bps? This does not seem to bad to me at all. If you meant 5000 bps or 5kbps, then there is a problem someplace. BOINC WIKI |
Ghery S. Pettit Send message Joined: 7 Nov 99 Posts: 325 Credit: 28,109,066 RAC: 82 |
Thanks for the update. You must be a light sleeper. 2.3 wakes you up? Loma Prieta in 1989 allowed me to calibrate what it takes to wake me (we lived in south San Jose at the time, 6 miles from the epicenter of the main quake) and anything less than 5.0 and I don't even stir. :-) Keep up the good work. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
Matt My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet. I think the difference is that there currently seems to be a long period of 'waiting time' (of the order of a minute) after BOINC has made the connection, but before the actual data starts flowing. 65 seconds of waiting, followed by 10 seconds transfer at 36.7 KBps, makes the figures average to the 5 KBps shown in the logs. Edit: it's a bit like air travel - the planes themselves are fast, but if you add in all the time spent hanging around in the terminal, and collecting your bags at the other end, then the average doesn't look so hot anymore. There's some more discussion of this and related questions in Number Crunching at the moment, in the 21 second timeout question.... thread, if anyone wants to continue the conversation. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet. Yes, 50Kbps is slow. 5000Kbps isn't. BOINC WIKI |
Don Farry Send message Joined: 12 Jul 05 Posts: 5 Credit: 674,437 RAC: 0 |
I've been getting several days of the following: 8/28/2007 5:33:13 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40 8/28/2007 5:33:35 PM||Project communication failed: attempting access to reference site 8/28/2007 5:33:35 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect 8/28/2007 5:33:35 PM|SETI@home|Backing off 36 min 24 sec on download of file 11ja07aa.11719.7843.8.5.40 8/28/2007 5:33:41 PM||Access to reference site succeeded - project servers may be temporarily down. 8/28/2007 5:44:24 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209 8/28/2007 5:44:46 PM||Project communication failed: attempting access to reference site 8/28/2007 5:44:46 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect 8/28/2007 5:44:46 PM|SETI@home|Backing off 57 min 57 sec on download of file 11ja07aa.12318.4980.9.5.209 8/28/2007 5:44:47 PM||Access to reference site succeeded - project servers may be temporarily down. 8/28/2007 6:10:00 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40 8/28/2007 6:10:22 PM||Project communication failed: attempting access to reference site 8/28/2007 6:10:22 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect 8/28/2007 6:10:22 PM|SETI@home|Backing off 1 hr 2 min 16 sec on download of file 11ja07aa.11719.7843.8.5.40 8/28/2007 6:10:26 PM||Access to reference site succeeded - project servers may be temporarily down. 8/28/2007 6:42:45 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209 8/28/2007 6:43:08 PM||Project communication failed: attempting access to reference site 8/28/2007 6:43:08 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect 8/28/2007 6:43:08 PM|SETI@home|Backing off 3 hr 21 min 17 sec on download of file 11ja07aa.12318.4980.9.5.209 8/28/2007 6:43:10 PM||Access to reference site succeeded - project servers may be temporarily down. 8/28/2007 6:51:38 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209 8/28/2007 6:51:41 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40 8/28/2007 6:51:59 PM||Project communication failed: attempting access to reference site 8/28/2007 6:51:59 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect 8/28/2007 6:51:59 PM|SETI@home|Backing off 3 hr 24 min 32 sec on download of file 11ja07aa.12318.4980.9.5.209 8/28/2007 6:52:00 PM||Access to reference site succeeded - project servers may be temporarily down. 8/28/2007 6:52:02 PM||Project communication failed: attempting access to reference site 8/28/2007 6:52:02 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect 8/28/2007 6:52:02 PM|SETI@home|Backing off 3 hr 46 min 18 sec on download of file 11ja07aa.11719.7843.8.5.40 8/28/2007 6:52:04 PM||Access to reference site succeeded - project servers may be temporarily down. I have a DSL connection and 2 GB of RAM. Any thoughts? Thanks. |
Lois Petrolito Send message Joined: 9 Nov 03 Posts: 10 Credit: 68,170 RAC: 0 |
My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet. You're right-I just checked again, and it should have been 5000 bytes, not kbytes. |
Lois Petrolito Send message Joined: 9 Nov 03 Posts: 10 Credit: 68,170 RAC: 0 |
I've been getting several days of the following: No answers, but the same messages....I don't have as much RAM, but am using a cable modem connection. |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
have a try @ this: With Windows: Shut down BOINC-Manager. On the Commandline (run) type: ipconfig /flushdns <enter> then On the Commandline (run) type: ipconfig /registerdns <enter> then . . . Restart BOINC-Manager This solves HTTP-Problems . . . |
Keith T. Send message Joined: 23 Aug 99 Posts: 962 Credit: 537,293 RAC: 9 |
have a try @ this: The ipconfig /flushdns is definitely worth doing on Windows XP. The registerdns bit is not necessary as discussed in a previous thread some months ago. This will not do anything on Windows 98, I tried it yesterday when my Win 98 box had a stuck download. Not sure for other flavors of Windows. I also noticed that a lot of Results/WUs are being sent out within 2 - 5 seconds of creation. I think this indicates that one of the processes (the splitters ? ) is or was under pressure to keep up. Sir Arthur C Clarke 1917-2008 |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
Matt I think this supposition is confirmed. I've just watched this download happening: 29/08/2007 11:39:53|SETI@home|[file_xfer] Started download of file 11ja07ab.927.4980.11.5.122 As you can see, the download apparently lasted for 1 minute 37 seconds, but by my reckoning nothing happened in the first 1 minute 25 seconds: all the data arrived in the final 12 seconds, at a perfectly respectable speed. Unfortunately, the discussion thread I linked to yesterday has been hidden, for an unfathomable and unexplained "administrative purpose". I suggest future discussion should take place in Number Crunching, not here, but someone will need to start a new thread if they want to carry on. |
Lois Petrolito Send message Joined: 9 Nov 03 Posts: 10 Credit: 68,170 RAC: 0 |
Matt I had a recent download come down at 3200 bps(+-) However, the uploads are going at 50000+ bps. The way my provider works is usually the downloads are faster (?) |
Marko Send message Joined: 2 Jun 99 Posts: 10 Credit: 659,205 RAC: 0 |
have a try @ this: This should work on Win98: Shutdown BOINC WIN-R ---> cmd ipconfig /renew_all |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
This would renew all the DHCP leases, but I don't think it flushes any local DNS cache. Windows 98 used WINS to resolve NETBIOS names, 2K and later use a Microsoft-kluged DNS, which is why the later versions of IPCONFIG can flush DNS (and have the registerdns command to update the DNS server). |
Cosmic_Ocean Send message Joined: 23 Dec 00 Posts: 3027 Credit: 13,516,867 RAC: 13 |
As for the ext3 8TiB limit, ext3 will only go to 8TiB limit if you set it up for 2KiB block size. 4KiB block size will take you to 16TiB, and only some systems support 8KiB (32TiB) blocks. Personally I prefer ReiserFS. Dynamic i-node allocation, on-the-fly defragging, tail-packing (gets rid of the wasted space at the end of blocks when files aren't in even block-sized increments), and online expansion of volumes through the use of LVM. Info: http://en.wikipedia.org/wiki/ReiserFS Specs: http://namesys.com/faq.html#reiserfsspecs |
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
Nope - 8TB is all the ext3 version we have (and all others I know about) can handle at maximum block size. Not a big deal - 8TB is fine. And I'm not sure I can bring myself to use ReiserFS given the author's current criminal status (guilty or otherwise). - Matt As for the ext3 8TiB limit, ext3 will only go to 8TiB limit if you set it up for 2KiB block size. 4KiB block size will take you to 16TiB, and only some systems support 8KiB (32TiB) blocks. -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
speedimic Send message Joined: 28 Sep 02 Posts: 362 Credit: 16,590,653 RAC: 0 |
Nope - 8TB is all the ext3 version we have (and all others I know about) can handle at maximum block size. Not a big deal - 8TB is fine. And I'm not sure I can bring myself to use ReiserFS given the author's current criminal status (guilty or otherwise). I don't know if it's good to judge the software by it's author... However, what is more important is that ReiserFS - with the given 8TB - will finish a fsck in 10 min and ext3 will take 4-5 hrs. Long outage if it goes down hard... mic. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.