Message boards :
Number crunching :
Version 3 of faster SETI cruncher for Linux
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 8 · Next
Author | Message |
---|---|
Ned Slider Send message Joined: 12 Oct 01 Posts: 668 Credit: 4,375,315 RAC: 0 |
Noted. But don't belittle your contribution - you've helped bring these amazing performance increases to the masses. I couldn't have built fftw3 versions without your significant contribution integrating these routines into the source code! Ned *** My Guide to Compiling Optimised BOINC and SETI Clients *** *** Download Optimised BOINC and SETI Clients for Linux Here *** |
S.L.Chia Send message Joined: 3 Apr 99 Posts: 98 Credit: 2,667,122 RAC: 1 |
this new version is really fast... in my p4 3.0ghz only take 50m-1h15m to cruch 2 wu.. :) great job!! |
trux Send message Joined: 6 Feb 01 Posts: 344 Credit: 1,127,051 RAC: 0 |
Harold modestly wrote: More discomfort... Actually, I saw the source code in both versions - before and after, and although I see that the principal changes or ideas came from Tetsuji (FTT@IPP) and Hans (caching), I see very well that you modified many places in the code and cleaned it up, where many changes contributed to the speed up too. And even if it did not, it was tremendous work and a lot of time putting it all together. As you wrote, this is a collective work, and credits belong to all - from the original scientists and programmers writing the software, through the official team and their countless volunteer (and often anonymous) helpers, 3rd party developers and compilers like Tetsuji, Hans, Harold, Alex, Crunch3r, Ned,... to many testers and critics here on the forum. Btw, I purchased both MSVS 2003 and Intel ICC+IPP, and have now a Windows version working, using the code from Harold's server with some additional modifications. The performance gain is amazing - for example at my 1.8GHz P4M (without SSE3/HT), it dropped from 1:35 with TMR's r8 to 51 minutes. At AMD Opteron 248 it dropped from 1:07 to 0:48, and at Opteron 140 even more - from 2:20 to 1:24 (probably thanks to fast RAM that has now bigger influence with the caching). I am still working on it; there may be still minor improvements. As soon as I build and test static applications, and builds for multiple platforms, I release them for public (probably at the beginning of the week). Since I had to do many modification in the source code to make it compilable in the Windows development environment, I think the changes should be incorporated into the source code on Harold's server. I do not think it makes sense publishing them on my server, multiplicating so the number of places, each with a different version of the source code. I tried to make the changes so, that they do not influence the original Unix version, so aside from some testing, it should not cause any problems to Unix developers. trux BOINC software Freediving Team Czech Republic |
Harold Naparst Send message Joined: 11 May 05 Posts: 236 Credit: 91,803 RAC: 0 |
I actually had done this work for MSVS, and it's on a branch in the subversion repository, svn://hnaparst.homelinux.com/seti_boinc/branches/windows The branch isn't up-to-date, but it shows what I think one needs to do to get it to compile under MSVS. It definitely compiles and runs, but MSVS is slow as death. Anyway, send me the patches and I'll add them to the source tree. I don't have the Windows compiler, so I won't be able to verify that it works, but I can certainly integrate them. Should be a very popular addition. Harold Naparst |
trux Send message Joined: 6 Feb 01 Posts: 344 Credit: 1,127,051 RAC: 0 |
Actually, these changes were done for MSVS 2003 + Intel ICC/IPP (yes, I bought legal copies of all of them). And I do not think it makes sense keeping two different branches of the source code. I unified some of my earlier attempts with your build #126, so when I spoke about testing the source code, I meant compiling it for Unix (not Windows - I did that already) to see if I did not messed up something on the other part (but I hope not). I am going to send you the current version to the email address listed on your website. PS: I saw you dropped Tetsuji from the list of credited people in the version message (resp. the error text). I suppose it was accidental. However, it looks like the list of contributors may grow, so putting them all there, and flooding so the field and messages on the S@H server may become uncomfortable for some. Maybe it would be better, simply putting there an URL to the page with credits on your website? trux BOINC software Freediving Team Czech Republic |
Harold Naparst Send message Joined: 11 May 05 Posts: 236 Credit: 91,803 RAC: 0 |
Actually, these changes were done for MSVS 2003 + Intel ICC/IPP (yes, I bought legal copies of all of them). And I do not think it makes sense keeping two different branches of the source code. I unified some of my earlier attempts with your build #126, so when I spoke about testing the source code, I meant compiling it for Unix (not Windows - I did that already) to see if I did not messed up something on the other part (but I hope not). Right. Of course the idea of a branch is that it eventually gets merged back into the trunk. Branches are just used for extensive changes that demand testing and shouldn't affect ongoing development. It's a standard software development technique. The right way to do it is to take a branch, do the testing for windows, and then "svn merge" them into the trunk. I'll do that, of course, and I can verify that your changes work on MSVS and Unix, too. By the way, it appears to me that MSVS 2005 Express is basically free, so you didn't need to buy it. Am I wrong? And at a price of zero, it is still overpriced... Harold Naparst |
trux Send message Joined: 6 Feb 01 Posts: 344 Credit: 1,127,051 RAC: 0 |
OK, I've sent the source code. I forgot to wrote - the modifications I did are easy to spot, because I marked them all with a comment beginning with "// TX" trux BOINC software Freediving Team Czech Republic |
Harold Naparst Send message Joined: 11 May 05 Posts: 236 Credit: 91,803 RAC: 0 |
OK, I've sent the source code. I forgot to wrote - the modifications I did are easy to spot, because I marked them all with a comment beginning with "// TX" Thanks, that will help. However, I use and recommend meld which is an absolute joy to use when merging versions. Also, regarding my contributions: Yes, there is a lot of my code in the source, however.... all of it does not add to the speed. It is like poetry. It looks nice, and is aesthetically pleasing, but it doesn't solve any problems. Premature optimization is the root of all evil. That's the motto of callgrind. Harold Naparst |
trux Send message Joined: 6 Feb 01 Posts: 344 Credit: 1,127,051 RAC: 0 |
By the way, it appears to me that MSVS 2005 Express is basically free, Afaik, MSVS 2005 is still beta and has many bugs. Besides it the Express version has many limitations and restricted libraries, and afaik also the licensing has some clues that may make the redistribution problematic. I am using MSVS 2003 Enterprise. And I do not agree about the quality - both the programming workbench and the compiler are excellent products. They produce much smaller binaries than ICC, often they are even faster (especially for non-Intel CPU's), and especialy it compiles about 10-20 times faster than ICC, what is also rather important in the main development stage. The programming workbench (Visual Studio) is really an amazing piece of software, and allows you to work with impressive efficiency, once you learn using it. Some 12 years ago I used to use its precedestor - the text based MS PWB and was rather disapponted when they dumped it with the first GUI based Visual Studio, because it had none of the advanced options of PWB. Fortunately, the current VS is as good as PWB was, resp. it is even much better. trux BOINC software Freediving Team Czech Republic |
trux Send message Joined: 6 Feb 01 Posts: 344 Credit: 1,127,051 RAC: 0 |
Thanks, that will help. However, I use and recommend meld Yes, that's possible, but still I think it is better keeping the comments of idividual developers in the source anyway - it makes it easier to see who changed what at which point and for what reason. trux BOINC software Freediving Team Czech Republic |
Martin A. Boegelund Send message Joined: 4 Jul 00 Posts: 292 Credit: 387,485 RAC: 1 |
Harold must be an artist :-D I think I've heard a story about an artist, who carved an amazing peace of art out a stone. The artist, however, didn't understand what all the fuss about his artistic skills were about - "I really didn't do anything special" he said, "The sculpture already was inside the stone, all I did was to take it out from there!" FWIW, I think you've contributed something of value, Harold. By the way, the 3.0 client did bring a performance improvement over 2.8 in my case (se my previous post). I'm now moving towards 3.1. "Are you suggesting coconuts migrate?" |
michael37 Send message Joined: 23 Jul 99 Posts: 311 Credit: 6,955,447 RAC: 0 |
Here is an SSE Benchmark. The workunits were executed at the same time. CPU type GenuineIntel Intel(R) Pentium(R) III CPU family 1400MHz Number of CPUs 2 Operating System Linux 2.4.21-20.ELsmp Memory 1001.31 MB time ./setiathome_4.07.3-P3-fftw3-static-cache-linux-gnu Edit: version 2.06S real 227m35.061s user 223m49.650s sys 0m7.710s ../rescmp result.sah ../reference_result_unit.sah Result: these are weakly similar. time ./setiathome_SSE-naparst-r3.1 real 223m21.474s user 219m4.010s sys 0m7.170s ../rescmp result.sah ../reference_result_unit.sah Result: these are strongly similar. CONCLUSION: The preferred client for Pentium III is Harold's SSE with IPP version 3.1. FFTW produces weakly similar results in the same time. |
JohnB175 Send message Joined: 15 Oct 03 Posts: 124 Credit: 321,769 RAC: 0 |
Well here are my updated SSE numbers... I must say great work... Little RAM usage and still the great performance. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx harold 3.1 setiathome_SSE-naparst-r3.1 real 241m10.409s user 238m29.500s sys 1m0.850s wu_cpu_time=15300.200000 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx harold 3.0 setiathome_SSE-naparst-r3.0 real 240m50.496s user 238m38.940s sys 0m50.760s wu_cpu_time=15319.340000 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx harold 2.8 setiathome_SSE-naparst-r2.8 real 272m52.589s user 262m51.860s sys 2m46.550s wu_cpu_time=16773.180000 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx crunch3r setiathome-4.7.pentium3_sse_fftw3_V2.02s_cache-pc-linux-gnu real 285m27.216s user 277m20.010s sys 3m11.880s wu_cpu_time=17678.820000 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx tmr setiathome-4.07-ipp.i686-p3-linux-gnu real 355m50.345s user 350m57.890s sys 1m12.430s wu_cpu_time=22130.310000 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |
JohnB175 Send message Joined: 15 Oct 03 Posts: 124 Credit: 321,769 RAC: 0 |
Btw, I purchased both MSVS 2003 and Intel ICC+IPP, and have now a Windows version working, using the code from Harold's server with some additional modifications. The performance gain is amazing - for example at my 1.8GHz P4M (without SSE3/HT), it dropped from 1:35 with TMR's r8 to 51 minutes. At AMD Opteron 248 it dropped from 1:07 to 0:48, and at Opteron 140 even more - from 2:20 to 1:24 (probably thanks to fast RAM that has now bigger influence with the caching). Awesome... I can't wait :) |
Crunch3r Send message Joined: 15 Apr 99 Posts: 1546 Credit: 3,438,823 RAC: 0 |
Hello, once again I've updated all setiathome Linux clients for i686,P3,Athlon(K7) and Athlon XP. And all setiathome clients for EV5, EV6 and EV67 running Linux. All are based on Harolds 3.1 source. URL: I WANT VERY FAST LINUX CLIENTS :-) Have fun, Happy crunching P.S. And hopefully Harold will not come up with a new source right after i'm posting this :-) Join BOINC United now! |
Harold Naparst Send message Joined: 11 May 05 Posts: 236 Credit: 91,803 RAC: 0 |
Well, this isn't an accident. You might remember how harshly I was criticized for my first client, which produced weakly similar results. I'll never forget being told that "weakly similar isn't science." So, since then, I only release strongly similar clients. If it is weak, I figure out why and fix it. I'll still release memory hogs, though. That's still possible. Harold Naparst |
Harold Naparst Send message Joined: 11 May 05 Posts: 236 Credit: 91,803 RAC: 0 |
I'm sure you're correct. However, this was my experience. I purchased MSVS 2003 and ported seti_boinc to Windows about a month ago. The sources are all in svn://hnaparst.homelinux.com/seti_boinc/windows It compiled very quickly (as you noted), and the IDE was very slick. I did all this on an AMD64 x2 running 32-bit Windows XP. When I ran the client on the reference work unit (under the IDE debugger), the client crashed every time, but in a different place on each run. The error was really not helpful. I went out an got an evaluation license for Purify to try to track this down, but Purify said there were no errors. This was exactly the same code that ran flawlessly under valgrind in Linux. What should I conclude from this experience? I obviously didn't release the client. I concluded that I should wait and buy ICC. Harold Naparst |
Ned Slider Send message Joined: 12 Oct 01 Posts: 668 Credit: 4,375,315 RAC: 0 |
My GCC-4/FFTW3 builds are still giving weak similarity. Not sure why and I haven't had time to play with it yet to try and figure it out. Obviously they're built from the same source (Harold's) that gives strong similarity for him, so it's definately not Harold's source :) Nice work Trux on the Windows clients - I bet that will make a lot of people very happy as us *nix users are definately still in the minority. But at least we're able to do some decent testing and development and see the improvements filter through :) Regards, Ned PS - Harold - please, no more new sources until next weekend. I need some sleep this week!! *** My Guide to Compiling Optimised BOINC and SETI Clients *** *** Download Optimised BOINC and SETI Clients for Linux Here *** |
ML1 Send message Joined: 25 Nov 01 Posts: 20872 Credit: 7,508,002 RAC: 20 |
...PS - Harold - please, no more new sources until next weekend. I need some sleep this week!! Well, I just get Ned's compiled latest super-client running, to see my meagre 1.8GHz AthlonXP scream at x2 the speed of a shiny new 3GHz P4 behemoth and... (See this WU.) Hark the YAOC new Herald Community Super Clients! I'll just have to get smug for a short while over the results for the latest again ;-) Great work! And what's the total speedup now over the original standard client? Regards, Martin (Sorry for the bad puns on Harold, herald, and "Hark The Herald Angel Sings" ;-P ) See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Hans Dorn Send message Joined: 3 Apr 99 Posts: 2262 Credit: 26,448,570 RAC: 0 |
On a 3.2G P4 it's about 3.3 times faster than the standard Windows client. (The standard linux client is even slower) Regards Hans |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.