Message boards :
Technical News :
upload issues and other news (Oct 5, 2007)
Message board moderation
Author | Message |
---|---|
Jeff Cobb Send message Joined: 1 Mar 99 Posts: 122 Credit: 40,367 RAC: 0 |
Matt is still away on his well deserved vacation so I will summarize the week. Last weekend we had 3 servers go down, as Eric described in the previous tech note. Two of these were attached to a UPS that malfunctioned. Not good, but at least we understand what happened. The third machine, bruno, crashes every week or two and hangs on reboot for reasons we have yet to understand. Our best guess at this time is that the fiber connection to the disk array that holds the upload directory is sometimes throwing garbage onto the bus that the machine cannot gracefully handle. This is an old fiber array that we would like to phase out anyway, so we thought about different storage devices that we currently have that could hold the uploads. We came up with the underutilized disk space on the master science database machine, thumper. This could have the added benefit of hosting the assimilators on the same machine that hosts the back end science database. Eric ran a script that gradually migrated the uploads over to thumper. This worked fine until the migration reach a critical point, at which time the loads on the two download machines shot up to the 80-100 range (they are usually at 5 or less). The high loads were because each instance of the file_upload_handler was taking a long time to write the uploaded results over to thumper. To make a long story short, it turns out that the volume on thumper that held the new upload directory was getting slammed by the uploads. It was running at nearly 100% utilization (local disk, not network, utilization). This was, and still is, a bit surprising. The volume on bruno is software RAID50 and on thumper the volume is software RAID5, the latter having 2 more spindles than each of the RAID50 mirrors on bruno. At any rate, we are migrating back to the fiber array on bruno and have already seen download performance normalize. We'll have to figure this one out... The other systems news of the week involves database replication on both of our production databases. The seti_boinc database (users, hosts, teams, recent results) replica was lost to a machine crash. We restored from the master and the replica is once again running normally. We are getting very close to having a replica of the back end science database. The initial data load is nearly complete. We will turn on replication either over the weekend or early next week. Over in science development we are getting the splitter ready to handle the radar blanking signal that will be embedded in all new data once Arecibo comes back on line later this month. -- Jeff |
Blurf Send message Joined: 2 Sep 06 Posts: 8962 Credit: 12,678,685 RAC: 0 |
Thank you for the update Jeff. |
petrusbroder Send message Joined: 2 Dec 01 Posts: 9 Credit: 55,701,905 RAC: 1 |
Thanks for the info, I'll post a link @ TeAm AnandTech - there have been a few ;) questions ... |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
Thanks Immensely for the Update Jeff . . . Nice Work Berkeley . . . |
DJStarfox Send message Joined: 23 May 01 Posts: 1066 Credit: 1,226,053 RAC: 2 |
I think that damn military radar has been the source of the -9 errors and noise in a lot of the WU in the last 6 months or so. |
Jim-R. Send message Joined: 7 Feb 06 Posts: 1494 Credit: 194,148 RAC: 0 |
I think that damn military radar has been the source of the -9 errors and noise in a lot of the WU in the last 6 months or so. It is in *all* of the multibeam data. As I recall, the linefeed array was lower down on the structure so the radar signal was blocked by a nearby hill. The multibeam array is positioned higher so it receives the signal. The splitters have been configured to remove most of it, but some still gets through. Right now it's been a tradeoff between the amount of radar signal in the work units and the amount of good data that is not split due to the possibility of radar contamination. When they get the new setup working they can remove the radar signal and *only* the radar signal. Jim Some people plan their life out and look back at the wealth they've had. Others live life day by day and look back at the wealth of experiences and enjoyment they've had. |
JLDun Send message Joined: 21 Apr 06 Posts: 573 Credit: 196,101 RAC: 0 |
I think that damn military radar has been the source of the -9 errors and noise in a lot of the WU in the last 6 months or so. Thanks for this explanation.
I know I've seen an explanation before, but how can SETI 'be sure' they've removed the "d*** military radar" from the WU's? Without going through every second of WU data before it's sent? |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
I know I've seen an explanation before, but how can SETI 'be sure' they've removed the "d*** military radar" from the WU's? Without going through every second of WU data before it's sent? There's a Scientific Newsletter article about it. |
Scrat Send message Joined: 12 Jul 00 Posts: 2 Credit: 254,582 RAC: 1 |
If you have pending problems with garbage on the fibre cables you could check the mass points on the system. One problem could be that the shielding of the fibre cables is directly connected on both ends of the cable. Its strongly recommended to connect ONLY one point directly with the mass point and the other over a spezial surge protector. Another problem could be that the mass points of the server array is to long. If you connect too many servers over a larger distance with the same mass point the equalizing current of the mass wire could also cause serious problems on the sending and receiving modules. (You know these problems if you ever connected your hifi with the television....) These two problems can occour and cause such problems. Mostly this problems are not taken care of at the construction of the servers, cause of the increased costs of building. I already had such problems..... maybe i could help..... Excuse me for the english.... its a long time since last using it ;) |
Clyde C. Phillips, III Send message Joined: 2 Aug 00 Posts: 1851 Credit: 5,955,047 RAC: 0 |
I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective. |
Odan Send message Joined: 8 May 03 Posts: 91 Credit: 15,331,177 RAC: 0 |
If you have pending problems with garbage on the fibre cables you could check the mass points on the system ..... Scrat, the fibres are _optical_ fibres so electrical problems such as grounding (mass?) do not have any effect. Excuse me for the english.... its a long time since last using it ;) Your English is fine :~} |
Jim-R. Send message Joined: 7 Feb 06 Posts: 1494 Credit: 194,148 RAC: 0 |
I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective. Hmm, good suggestion. It wouldn't have to be very big, just big enough to block the radar signal from the receiving antennas. Unless there's reflections from some other structure. It would have to be made of something opaque to the freq of the radar. Maybe the sort of radar absorbing material they use on the stealth and other planes. Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them! Jim Some people plan their life out and look back at the wealth they've had. Others live life day by day and look back at the wealth of experiences and enjoyment they've had. |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective. The First Invisibility Shield . . . Technology mighten be 'incorporated into' said 'meshin' . . . hmmm? |
kittyman Send message Joined: 9 Jul 00 Posts: 51469 Credit: 1,018,363,574 RAC: 1,004 |
If you have pending problems with garbage on the fibre cables you could check the mass points on the system. Sorry, but you are confusing electrical signal cables with fiber optic cables. The grounding advice you gave is well applied to electrical cables, but fiber optics are immune to RFI and other electical interference, and do not require shielding which is earthed (grounded). "Freedom is just Chaos, with better lighting." Alan Dean Foster |
Andy Lee Robinson Send message Joined: 8 Dec 05 Posts: 630 Credit: 59,973,836 RAC: 0 |
Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them! Well I'd send it back to them amplified a few billion times... but that's me! Problem with any amount of terrestrial rfi, it massively outweighs those we are looking for. Blocking rfi is difficult because of diffraction and reradiation, and the pylons will also resonate and reradiate all kinds of frequencies and pick up much more than just the radar. A huge multiwalled Faraday cage might help. Damned inconvenient to have one within line-of-sight-and-a-bit-more near Arecibo. The power of these things can be 10s of kW so even when not facing the antenna, leakage could still be considerable. Those responsible for the radar should also be responsible for it not interfering with their neighbours, and a radar absorber/attenuator/blocker or switcher offer would be more effective closer to the source. Saving grace is that the signal can be neutralized somewhat, or skipped over but there can still be annoying traces. Ideally would need to build a really big mountain around the dish and turn it into a large tube, or move to the far side of the Moon and recruit a few convenient craters. |
kittyman Send message Joined: 9 Jul 00 Posts: 51469 Credit: 1,018,363,574 RAC: 1,004 |
Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them! If it were a civilian installation, they might be held responsible for the problem, but being a military installation, no such restrictions apply. "Freedom is just Chaos, with better lighting." Alan Dean Foster |
Scrat Send message Joined: 12 Jul 00 Posts: 2 Credit: 254,582 RAC: 1 |
:) I know that fibre cables normally have no shielding, but sometimes the rodent protection of the cables is connected to the sending and the receiving modul, or to the server rack. This can cause such problems, not because of electrical interference in the cable itself, but in the sending or receiving modul. I just wanted to mention it, cause i had such problems twice (with fibre optic) and it took me 2 months to locate the problem. |
Gary Charpentier Send message Joined: 25 Dec 00 Posts: 30687 Credit: 53,134,872 RAC: 32 |
Don't worry about sending it back to them. Their software will just remove it as ground clutter. Gary I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective. |
kittyman Send message Joined: 9 Jul 00 Posts: 51469 Credit: 1,018,363,574 RAC: 1,004 |
I suppose if the 2 interconnected pieces of equipment were not on the same ground bus there could be some differential between them thus inducing some current flow from one to the other. Fair enough. When you don't have an obvious solution to a problem, you must look for other obscure causes. "Freedom is just Chaos, with better lighting." Alan Dean Foster |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13750 Credit: 208,696,464 RAC: 304 |
I suppose if the 2 interconnected pieces of equipment were not on the same ground bus there could be some differential between them thus inducing some current flow from one to the other. All too common. Earth loops can be an nightmare to track down & resolve. Grant Darwin NT |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.