Message boards :
News :
Unexplained database slowness
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · Next
Author | Message |
---|---|
![]() Send message Joined: 21 Jul 15 Posts: 1 Credit: 12,746,155 RAC: 0 ![]() |
You'd think giving us more than 8 hours worth of work would be a no brainer.... |
Sirius B ![]() ![]() Send message Joined: 26 Dec 00 Posts: 24922 Credit: 3,081,182 RAC: 7 ![]() |
Dangerous road to travel. Been debated many times in the past. Green stars, donations, "super" crunchers, cpu crunchers, gpu crunchers. Introduce limits & users & just watch all those donations disappear. |
mrchips ![]() Send message Joined: 12 Dec 04 Posts: 17 Credit: 26,590,842 RAC: 8 ![]() |
Good point, I've only donated once or twice, now I am retired, maybe I should donate again. Slippery Slope........... |
rob smith ![]() ![]() ![]() Send message Joined: 7 Mar 03 Posts: 22682 Credit: 416,307,556 RAC: 380 ![]() ![]() |
The current limits of 100 for the cpu and 100 per gpu were introduced to protect the working database (BB2 in Richard's description) from overloading, which it is now suffering (or at least appears to be suffering from). Increasing these limits at this time would not help the situation, indeed might have undesirable effect of making things even worse. Thus there may be an argument for reducing these limits, and strictly enforcing them. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
Sirius B ![]() ![]() Send message Joined: 26 Dec 00 Posts: 24922 Credit: 3,081,182 RAC: 7 ![]() |
Sorry Rob, I wasn't referring to WU lmiits. |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 ![]() |
Has anyone considered making the WU's longer in order to reduce the average number of transactions on the database? Figuratively, for example, making new WU's that contain 3x the data to reduce on the client than is normal today. A significant increase would be needed, I suspect. And, is it possible to send such large WU's only to clients running GPU's? It seems absurd that GPU's should be treated the same as CPU's when sizing WU's or specific client compute tasks. Other projects do this, I think, e.g. PrimeGrid. |
mmonnin Send message Joined: 8 Jun 17 Posts: 58 Credit: 10,176,849 RAC: 0 ![]() |
Has anyone considered making the WU's longer in order to reduce the average number of transactions on the database? Figuratively, for example, making new WU's that contain 3x the data to reduce on the client than is normal today. A significant increase would be needed, I suspect. This fixed MW in the past. 5 tasks were bundled together since they only took a few seconds on a 79xx AMD card. Now it has other DB issues. |
![]() ![]() Send message Joined: 16 Jun 02 Posts: 6895 Credit: 6,588,977 RAC: 0 ![]() |
MultiMILLIONAIRE and BILLIONaire LEFTIEs are SPENDING Their DOLLARs fO ManyManyMany 'Causes'. Where Are dA LEFTY BIG BUCKs fO SETIatHOME? Certaninly dA 'SCIENCE' is MEGA COMPELLING to ANY RichieRICH LEFTY. There 'Is' 'SCIENCE' Here WORTH 'Promoting'; 'PUBLISHING'; and Donating HARD CASH to? Correct? What Do dA SUPER RichieRICH Know dAt DEDICATED CRUNCHERs Do Not? Only People Believing 'IT" CAN Happen Any Second or In 100s of Years Have dA COMMITMENT and FORTITUDE to 'KNOW" dA EFFORT is WORTH dA Coming REWARD? While KNOWING Hardware Is NOT Up to Date fO Task, and 'RESULTs' NOT EVER Going to be 'ANALYZED' Further. Where is dA Scientific Method to this MADNESS? Is Da Moon Hollow? Yap@Home ![]() May we All have a METAMORPHOSIS. REASON. GOoD JUDGEMENT and LOVE and ORDER!!!!! ![]() |
![]() ![]() ![]() Send message Joined: 29 Jun 99 Posts: 11447 Credit: 29,581,041 RAC: 66 ![]() ![]() |
Oh, I see not your green star. |
![]() ![]() Send message Joined: 3 Jun 99 Posts: 2422 Credit: 12,323,733 RAC: 1 ![]() |
Oh, I see not your green star. Book'um Dano! har... ;-) ... ![]() |
![]() ![]() Send message Joined: 20 Dec 05 Posts: 3187 Credit: 57,163,290 RAC: 0 ![]() |
...The tasks which make the database 'messy' will be the ones which hang around, and their wingmates - the errors, the failed validations, the ghosts, the users who walk away without clearing their caches... Speaking of ghosts: Is anyone working on removing the 71 ghost MB 7.0 WU's? That might clear up some space, as not only the space for those WU's could be re-allocated, but once they are gone, the entire set of MB 7.0 executables could be deleted... These ghosts have been bothering me ever since the number of "in the field" 7.0 WU's went to zero - and I'm on one of the ghosts! (as in: I processed the WU, and a wingman also came in, but the WU never got deleted!...) And as far as "Walking away without clearing the cache" goes: sometimes you have no choice about it - sometimes a computer will fail on you without warning, and you have to either buy or put together a new one, and let those WU's that were on the failed computer get picked up when they exceed deadline... . ![]() Hello, from Albany, CA!... |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13889 Credit: 208,696,464 RAC: 304 ![]() ![]() |
But on server-grade kit like this, they probably have access to multi-channel RAID controllers, and lots of drive bays, so my 'two database containers' suggestion doesn't mean two physical servers or even disk access contention. Yep. But the more storage pools there are, the greater the work of the server to manage them, and the more tweaking required to balance available RAM (for caching) between the pools (I've no idea what automatic resource allocation is available for such systems, nor what it's capabilities are if it exists). The big advantage of multiple storage pools on a given server is that existing HDDs can be replaced with Flash storage so that at the very least indexes & logs can be placed on them, or if there is enough space in the server for the capacity necessary, one or more of the tables with the greatest I/O load could be moved to the Flash storage pool and save the HDDs for the not so busy tables. Has Seti gone back to Hardware RAID controllers? I recall they did have some, but when one died they didn't have any spares of the same model & version & so were unable to rebuild that particular array, hence Matt went for Software RAID. Cheaper and easier to deal with hardware failures, but certainly not the performance of dedicated hardware controllers. Grant Darwin NT |
![]() Send message Joined: 7 Apr 01 Posts: 172 Credit: 23,823,824 RAC: 0 ![]() |
Hello all of you. I think everyone see the problems y differents ways. All of uou, experimented tester knwns much ore than me. My only idea is only to stop producing work. Arecibo is out. So no new tasks at this time. Stopping producing work will for sure be not apreiated to all of us. But. A least, it will give time to server to absorbe all WU in queue. Servers will be more able to give more power to all other application. So I really thik, after a few days, all very late works will be resorbed, the bottlneck reduced. Then staff can work more easily on DB. DB try to work, but receive more and more WU ! It is never ending. Why not to FULLY stop the project for a few days and so give fresh air to server to breaths (also to staff) I think everyone become tired about all the problems. We, normal cruncher as me, probably, we will lost CR and AVG due to longer outage. And then ???? We will not die ! It is my suggestion : stopping producing work. And let the time , the time to solve by self, with help from staff. Sorry, I stay with my idea : stopping producing work, Disable up/down loas server. I know, some WU will be out of time. Or perhaps, no any work sending to us, upload server active to let finish WU out in field, And then closing up/down after one week. Then let the staff working. It need one week ? two weeks ? 3 weeks ? no problem. As long staff give information. Best regards from Belgium ![]() |
![]() Send message Joined: 30 Mar 00 Posts: 13 Credit: 60,615,800 RAC: 163 ![]() ![]() |
Greetings to all and greetings to Belgium. I cannot understand that with so many millions of donations in the last few years technically hardly anything has happened. In any case, not really positively noticeable. It was purchased and that (what you can find and read on the internet about the purchases at SETI) and sometimes the software improves from 7.8.x to 7.9.x but with SETI only a “handful of people work†... - where is all the money there? Already consumed everything? And if there are so many problems with hardware and software - what do they do all day? Look at the monitors and wait for the big WOW-two signal to come? The database problem and hardware difficulties have existed for months. It looks like Eric has to do it all by himself. Maybe it would make sense to make a comprehensive improvement plan and then shut down SETI for 2-3 weeks. Clearing old stuff, clearing up statistics and "corpses", processing all outstanding data (WUs), then reworking the entire hardware and software and then going back online clean. ET will certainly not send a signal in the 3 weeks. But when it comes - and it will come - and all the hardware and software (as it is now) does not work properly, the whole years were really free for each and every one of us. Much more likely it is for me that the signal is already there and is not recognized ... But I'm not giving up yet and continue to search with you - if I get WUs. So many good people write here who are familiar with databases and hardware but are not from SETI. Why are not these experiences, ideas and helps flowing into the project? Do the ideas evaporate here in the chat? Science means to create knowledge and that is what many do here! It is clear to me that in scientific projects, which are also subject to a certain level of security, not everyone can and should be allowed to work and have insight. But there is always talk of so many donations (money). Why are the experiences, ideas and knowledge donations not accepted? Certainly, very creative and intelligent people work at SETI. But even these are limited with their knowledge and their performance. Maybe SETI will start a donation call - Mining of knowledge - for thoughts and ideas of professionals? I am sure that we can find a solution so quickly. Well, for the most successful and implemented ideas there are no credits but instead of a donation star, graphics card symbol or disk icon, a bronze, silver or golden, pulsating brain as an achievement 😉 As reminder: - (Info: 2011) Every year SETI needs about 2.5 million US dollars to operate. - (Info: 2015) Milner and Hawking: "A Russian billionaire donates $ 100 million for the search for aliens." The lack of money cannot really be. English: https://motherboard.vice.com/en_us/article/pga5zv/who-is-the-russian-billionaire-who-just-pledged-100-million-to-find-alien-life German: https://motherboard.vice.com/de/article/8q8ye4/russischer-milliardaer-geht-mit-stephen-hawking-auf-aliensuche-387 and http://breakthroughinitiatives.org/news/1 So, there should be enough money for hardware, software and specialists - at least in the last 3 years. Or did I misunderstand something? But maybe all the faults are there too? March 31, 2018: “Signal Received From Deep Space is Strongest Ever Recorded†http://exonews.org/signal-received-from-deep-space-is-strongest-ever-recorded/ . . . 04.04.2018: Meanwhile, after a long period of maintenance, something has changed - it is running again. 😊😊😊 |
![]() Send message Joined: 26 Mar 00 Posts: 47 Credit: 5,127,861 RAC: 2 ![]() |
I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me. |
![]() ![]() Send message Joined: 20 Dec 05 Posts: 3187 Credit: 57,163,290 RAC: 0 ![]() |
I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me. They recently (about 6 months ago, IIRC...) doubled the size of the WU, first on Beta, then on production. . ![]() Hello, from Albany, CA!... |
![]() ![]() ![]() ![]() ![]() Send message Joined: 25 Dec 00 Posts: 31138 Credit: 53,134,872 RAC: 32 ![]() ![]() |
I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me. I remember that too. The other issue we have is this project wants to support crunching on smart phones. They are snails compared to high end GPU's. W/U size has to stay reasonable for them as do deadlines considering their flop counts. Now if some of you programmer types want to change BOINC so it can issue dynamic deadlines based on reported flop count, the code is on github ... Making W/U sizes dynamic is much harder as that is the science code. |
![]() Send message Joined: 26 Mar 00 Posts: 47 Credit: 5,127,861 RAC: 2 ![]() |
I appreciate the info guys. Odd though, I haven't noticed my geriatric crunchers taking twice as long to get the work done. Must be the organic, non-GMO, 100% natural electricity I'm feeding them. |
rob smith ![]() ![]() ![]() Send message Joined: 7 Mar 03 Posts: 22682 Credit: 416,307,556 RAC: 380 ![]() ![]() |
The "doubling" of the task size was triggered not by increasing the number of data points but by increasing the resolution. This change has had very little impact on the execution time of a task. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
![]() ![]() ![]() Send message Joined: 6 Nov 99 Posts: 716 Credit: 8,032,827 RAC: 62 ![]() ![]() |
The Endurance Expériment ;) https://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead |
©2025 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.