Philosophy: To DeviceQueueOptimize or NOT (with a focus on: is it "micro managing"?)

Message boards : Number crunching : Philosophy: To DeviceQueueOptimize or NOT (with a focus on: is it "micro managing"?)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Stubbles
Volunteer tester
Avatar

Send message
Joined: 29 Nov 99
Posts: 358
Credit: 5,909,255
RAC: 0
Canada
Message 1811849 - Posted: 23 Aug 2016, 13:31:09 UTC

In order to remove an unresolved tangents from the thread: Panic Mode On (103) Server Problems?
and in the spirit of Stephen's new thread "Philosophy: To CPU or NOT to CPU",
this thread is to focus on the benifits of DeviceQueueOptimisation (such as Mr Kevvy's guppiRescheduler) and if it is to be considered as a form of "micro managing".

Please see the following posts (in another thread) for the debate's background:

Grant's post mentioning "micro managing":
https://setiathome.berkeley.edu/forum_thread.php?id=79575&postid=1811758

My 1st reply: https://setiathome.berkeley.edu/forum_thread.php?id=79575&postid=1811819

Grant's reply: https://setiathome.berkeley.edu/forum_thread.php?id=79575&postid=1811825

I'll take the time to write a concise reply (as I have a tendency to write way too much at times).
Until then, I invite anyone to comment.

Cheers,
RobG :-)
ID: 1811849 · Report as offensive
Profile Stubbles
Volunteer tester
Avatar

Send message
Joined: 29 Nov 99
Posts: 358
Credit: 5,909,255
RAC: 0
Canada
Message 1811859 - Posted: 23 Aug 2016, 14:06:21 UTC

Here is Grant's post in the last link provided in this thread's OP, with my reply below it.
I welcome your reply and hope this time we can come to a conclusion on "micro managing", especially if I'm still not understanding your perspective well.

There won't be any conclusion because you don't consider what you are doing to be micro management, when it would qualify as a text book example.
Just because you automate something doesn't disqualify it from being micro management.

From the Wikipedia,
Micromanagement
In business management, micromanagement is a management style whereby a manager closely observes or controls the work of subordinates or employees.

BOINC allows you to set your desired levels of CPU usage, network activity, resource share etc & then it just gets on with the job.
You find it necessary to shuffle work around, on top of what you have already set the manager to do.
ie a manager (yourself) closely observes or controls the work of subordinates or employees (BOINC).
As I said, a text book example.

I think I'm starting to see why we have a different perspective.
I'll try to use similar Business Management terminology.
Keep in mind that mine is mostly from experience in Canada's Federal Govt hierarchy with the following generic titles from bottom going up:

1. non-supervisor (the worker-bee)
2. supervisor
3. manager
4. director
5. DG (Director General)
6. ADM (Assistant Deputy Minister)
7+ ...irrelevant for the moment

For me:
6. the ADM is the BOINC commitee;
5. the DG is the Boinc architecture;
4. the Director is the S@h project;
3. the Managers are the SETIzens;
2. the Supervisors are the Boinc Managers (or BoincTasks or SETIspirit); and
1. the worker-bee is the Boinc host (PC) and the Boinc Client

As for an automated DeviceQueueOptimizer (such as MrKevvy's guppiRescheduler),
I place that at the #2 level alongside the Boinc Manager (or BoincTasks).
If it wasn't fully automated or prone to bugs, I would include that with #3 and I could consider that as a form of "micro managing".
But if it doesn't require any intervention during a whole week/month, I don't see how it is a form of "micro managing".

Does that help or change anything on your end?

With all due and intended respect,
RobG
ID: 1811859 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1849
Credit: 268,616,081
RAC: 1,349
United States
Message 1811905 - Posted: 23 Aug 2016, 21:33:48 UTC

Guess I better subscribe to this one, eh?
:)
ID: 1811905 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1811926 - Posted: 23 Aug 2016, 23:08:54 UTC - in response to Message 1811905.  

@jimbocous

. . :)
ID: 1811926 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1849
Credit: 268,616,081
RAC: 1,349
United States
Message 1812005 - Posted: 24 Aug 2016, 3:14:50 UTC

So as I mentioned to Kevvy, been running this on 5 crunchers since the release of 0.51, and RAC was 84k or so when I started. Over 90 now, I think. For just a few days, that's substantial.
As I may have mentioned, I'm working with Stubbles to further develop his front end script, which I guess we're going to call DQO (Device Queue Oprimizer). I dived in because it's been a while since I had a fun project to dive into.
Is it micromanaging? Not to my definition, because to me micromanaging implies an excessive amount of effort relative to the results obtained. Stepping up to each of 5 machines 2-3 times a day, at most, and clicking a shortcut to watch a script run to completion doesn't seem that excessive:)
As long as we're not in any way impacting the Project or the Science, it's hard to see this as a bad thing, imho.
Later, ...
ID: 1812005 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1812012 - Posted: 24 Aug 2016, 3:34:58 UTC - in response to Message 1812005.  

Is it micromanaging? Not to my definition, because to me micromanaging implies an excessive amount of effort relative to the results obtained.

Which differs from the actual definition of micro management.
Grant
Darwin NT
ID: 1812012 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1812015 - Posted: 24 Aug 2016, 3:43:51 UTC - in response to Message 1812012.  
Last modified: 24 Aug 2016, 3:46:38 UTC

Is it micromanaging? Not to my definition, because to me micromanaging implies an excessive amount of effort relative to the results obtained.

Which differs from the actual definition of micro management.


So does this example of automated hardware and efficiency based dispatch meet your definition of micromanagement, despite no user intervention ?

Optimal function choices:
--------------------------------------------------------
name timing error
--------------------------------------------------------
v_BaseLineSmooth (no other)

v_GetPowerSpectrum 0.000202 0.00000 test
v_vGetPowerSpectrum 0.000073 0.00000 test
v_vGetPowerSpectrum2 0.000081 0.00000 test
v_vGetPowerSpectrumUnrolled 0.000064 0.00000 test
v_vGetPowerSpectrumUnrolled2 0.000079 0.00000 test
v_avxGetPowerSpectrum not supported on CPU
v_vGetPowerSpectrumUnrolled 0.000064 0.00000 choice

v_ChirpData 0.016097 0.00000 test
fpu_ChirpData 0.018683 0.00000 test
fpu_opt_ChirpData 0.016175 0.00000 test
v_vChirpData_x86_64 0.053370 0.00000 test
sse1_ChirpData_ak 0.009806 0.00000 test
sse1_ChirpData_ak8e 0.007956 0.00000 test
sse1_ChirpData_ak8h 0.008675 0.00000 test
sse2_ChirpData_ak 0.007877 0.00000 test
sse2_ChirpData_ak8 0.005676 0.00000 test
sse3_ChirpData_ak 0.008085 0.00000 test
sse3_ChirpData_ak8 0.005683 0.00000 test
avx_ChirpData_a not supported on CPU
avx_ChirpData_b not supported on CPU
avx_ChirpData_c not supported on CPU
avx_ChirpData_d not supported on CPU
sse2_ChirpData_ak8 0.005676 0.00000 choice

v_Transpose 0.003496 0.00000 test
v_Transpose2 0.002163 0.00000 test
v_Transpose4 0.001617 0.00000 test
v_Transpose8 0.004306 0.00000 test
fftwf_transpose 0.002496 0.00000 test
v_pfTranspose2 0.001863 0.00000 test
v_pfTranspose4 0.001719 0.00000 test
v_pfTranspose8 0.003851 0.00000 test
v_vTranspose4 0.002151 0.00000 test
v_vTranspose4np 0.001786 0.00000 test
v_vTranspose4ntw 0.006173 0.00000 test
v_vTranspose4x8ntw 0.006612 0.00000 test
v_vTranspose4x16ntw 0.003466 0.00000 test
v_vpfTranspose8x4ntw 0.006093 0.00000 test
v_avxTranspose4x8ntw not supported on CPU
v_avxTranspose4x16ntw not supported on CPU
v_avxTranspose8x4ntw not supported on CPU
v_avxTranspose8x8ntw_a not supported on CPU
v_avxTranspose8x8ntw_b not supported on CPU
v_Transpose4 0.001617 0.00000 choice

FPU opt folding 0.003601 0.00000 test
ben SSE folding 0.002227 0.00000 test
AK SSE folding 0.001666 0.00000 test
BH SSE folding 0.001821 0.00000 test
JS AVX_a folding not supported on CPU
JS AVX_c folding not supported on CPU
AK SSE folding 0.001666 0.00000 choice

Test duration 6.33 seconds


Either way, looks to me like heuristic strategies are encouraged by the stock CPU implementation.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1812015 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1849
Credit: 268,616,081
RAC: 1,349
United States
Message 1812031 - Posted: 24 Aug 2016, 5:05:16 UTC - in response to Message 1812012.  

Is it micromanaging? Not to my definition, because to me micromanaging implies an excessive amount of effort relative to the results obtained.

Which differs from the actual definition of micro management.

Result, not definition :) :)
ID: 1812031 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1849
Credit: 268,616,081
RAC: 1,349
United States
Message 1812034 - Posted: 24 Aug 2016, 5:22:19 UTC - in response to Message 1812015.  
Last modified: 24 Aug 2016, 5:24:00 UTC

Is it micromanaging? Not to my definition, because to me micromanaging implies an excessive amount of effort relative to the results obtained.

Which differs from the actual definition of micro management.


So does this example of automated hardware and efficiency based dispatch meet your definition of micromanagement, despite no user intervention ?

Optimal function choices:
--------------------------------------------------------
name timing error
--------------------------------------------------------
v_BaseLineSmooth (no other)
.
.
.

AK SSE folding 0.001666 0.00000 choice

Test duration 6.33 seconds


Either way, looks to me like heuristic strategies are encouraged by the stock CPU implementation.


Wouldn't have a clue, dude. I am not, and have not held myself out to be, a programmer on that level. Which I think you were well aware of. I have a tremendous respect for those like you and Raistmer who are. I think that probably doesn't get said enough.

Not sure what point you were trying to make, other than perhaps to say "go away and let the experts deal with it"? Don't think that's where you were coming from ... hopefully.
But folks wanting to do what we're doing shouldn't be misconstrued as an insult, I would hope ... Lotsa prickly folks here ...
Later ...
ID: 1812034 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1812036 - Posted: 24 Aug 2016, 5:44:20 UTC - in response to Message 1812031.  

Is it micromanaging? Not to my definition, because to me micromanaging implies an excessive amount of effort relative to the results obtained.

Which differs from the actual definition of micro management.

Result, not definition :) :)

Micro management isn't about the result obtained.
Grant
Darwin NT
ID: 1812036 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1812037 - Posted: 24 Aug 2016, 5:48:37 UTC - in response to Message 1812034.  

Is it micromanaging? Not to my definition, because to me micromanaging implies an excessive amount of effort relative to the results obtained.

Which differs from the actual definition of micro management.


So does this example of automated hardware and efficiency based dispatch meet your definition of micromanagement, despite no user intervention ?

Optimal function choices:
--------------------------------------------------------
name timing error
--------------------------------------------------------
v_BaseLineSmooth (no other)
.
.
.

AK SSE folding 0.001666 0.00000 choice

Test duration 6.33 seconds


Either way, looks to me like heuristic strategies are encouraged by the stock CPU implementation.


Wouldn't have a clue, dude. I am not, and have not held myself out to be, a programmer on that level. Which I think you were well aware of. I have a tremendous respect for those like you and Raistmer who are. I think that probably doesn't get said enough.

Not sure what point you were trying to make, other than perhaps to say "go away and let the experts deal with it"? Don't think that's where you were coming from ... hopefully.
But folks wanting to do what we're doing shouldn't be misconstrued as an insult, I would hope ... Lotsa prickly folks here ...
Later ...


Nah Jim, that question was directed at Grant, trying to establish where the line between user micromanagement and automation lies.

We've got a situation where the technology and users 'want' more sophisticated capabilities to make life simpler and more efficient, while the status quo resists change.

It's not something I'd pretend to know the absolute best solutions for, though at the same time I can point to an example of how it happens already, and say 'What's wrong with that?'.

Nothing wrong with it then theres the answer --> Something wrong with it then Boinc and the project need to look at it too.

If it were all black and white, like 'rescheduling bad, accepting poor scheduling good', then I think the discussion would be moot. In this case I think the discussion leads to questions about suppressing of technology, and assumptions made under the current mechanisms, that probably should be discussed.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1812037 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1812039 - Posted: 24 Aug 2016, 5:58:44 UTC - in response to Message 1812015.  

So does this example of automated hardware and efficiency based dispatch meet your definition of micromanagement, despite no user intervention ?

Nope.
And it's not my definition, it's the definition.

For those that missed it the first time around-

From the Wikipedia,
Micromanagement
In business management, micromanagement is a management style whereby a manager closely observes or controls the work of subordinates or employees.

Grant
Darwin NT
ID: 1812039 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1812040 - Posted: 24 Aug 2016, 6:03:58 UTC - in response to Message 1812039.  
Last modified: 24 Aug 2016, 6:04:31 UTC

So does this example of automated hardware and efficiency based dispatch meet your definition of micromanagement, despite no user intervention ?

Nope.
And it's not my definition, it's the definition.

For those that missed it the first time around-

From the Wikipedia,
Micromanagement
In business management, micromanagement is a management style whereby a manager closely observes or controls the work of subordinates or employees.


Then I would argue the provided (stock CPU 'dispatcher') example precisely matches that definition.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1812040 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1812041 - Posted: 24 Aug 2016, 6:06:03 UTC - in response to Message 1812037.  

Nah Jim, that question was directed at Grant, trying to establish where the line between user micromanagement and automation lies.


BOINC is the worker- we are the Managers. We tell it what we want it to do & how to do it, then let it get on with the job.
Us changing things above & beyond that, are perfect examples of micro management, whether it's automated & runs once weekly, or it's someone endlessly changing the web and/or local host settings to get it to do things in a particular way (why FIFO bothers some people I have no idea).

BOINC having the ability to do rescheduling would just be BOINC doing it's job, the way we ask it to (through the web/local host settings). Of course when people then decide they don't like the way BOINC is handling that, and do their own {whatever}, then they are micro managing.
Grant
Darwin NT
ID: 1812041 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1812042 - Posted: 24 Aug 2016, 6:10:09 UTC - in response to Message 1812040.  
Last modified: 24 Aug 2016, 6:10:48 UTC

From the Wikipedia,
Micromanagement
In business management, micromanagement is a management style whereby a manager closely observes or controls the work of subordinates or employees.


Then I would argue the provided (stock CPU 'dispatcher') example precisely matches that definition.

Not even close.
BOINC is the worker, we are the manager.
We tell it what to do (through web & local host settings) & then let it get on with it's job. When we then go & change the way it does things, that is micro management.

In the definition replace manager with user, subordinates & employees with BOINC.
micromanagement is a management style whereby a user closely observes or controls the BOINC manager.
Grant
Darwin NT
ID: 1812042 · Report as offensive
Kiska
Volunteer tester

Send message
Joined: 31 Mar 12
Posts: 302
Credit: 3,067,762
RAC: 0
Australia
Message 1812043 - Posted: 24 Aug 2016, 6:12:13 UTC - in response to Message 1812042.  
Last modified: 24 Aug 2016, 6:13:13 UTC

In the definition relace manager with user, subordinates & employees with BOINC.
micromanagement is a management style whereby a user closely observes or controls the BOINC manager.


replace manager with client, the manager just a visual interface in which we control BOINC, we could do it with boinccmd :)

EDIT: I see Stubbles got banished :(
ID: 1812043 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1812044 - Posted: 24 Aug 2016, 6:14:30 UTC - in response to Message 1812041.  
Last modified: 24 Aug 2016, 6:19:31 UTC

Nah Jim, that question was directed at Grant, trying to establish where the line between user micromanagement and automation lies.


BOINC is the worker- we are the Managers. We tell it what we want it to do & how to do it, then let it get on with the job.
Us changing things above & beyond that, are perfect examples of micro management, whether it's automated & runs once weekly, or it's someone endlessly changing the web and/or local host settings to get it to do things in a particular way (why FIFO bothers some people I have no idea).

BOINC having the ability to do rescheduling would just be BOINC doing it's job, the way we ask it to (through the web/local host settings). Of course when people then decide they don't like the way BOINC is handling that, and do their own {whatever}, then they are micro managing.


OK, so let's go with that. So Stock CPU app automatically changing from FPU code to SSE, SSE2, SSE3, AVX etc (which are different devices) is micromanagement->rescheduling, which you'd be in good company with Boinc developers in thinking, since Boinc assumes all x86 CPUs run at single threaded FPU Whetsone (which happens to be provably false, especially stock CPU AVX)

So we're saying micromanagement/rescheduling breaks boinc, since it doesn't support the needs of even stock CPU applications.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1812044 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1812047 - Posted: 24 Aug 2016, 6:33:24 UTC - in response to Message 1812044.  
Last modified: 24 Aug 2016, 6:34:27 UTC

OK, so let's go with that. So Stock CPU app automatically changing from FPU code to SSE, SSE2, SSE3, AVX etc (which are different devices) is micromanagement->

Incorrect.
It is not micro management. How can it be? It is doing, what it has been programmed to, according to the settings it has been given through the Web/locally.
Someone changing those settings back and forth in order to change how BOINC does it's job, that would be micro management- whether they do it manually or automate it.
refer back to the definition-
micromanagement is a management style whereby a user closely observes or controls the BOINC manager.
The user changing the Manager's decision about which code to use would be micro management- which is what I did on Beta when the manager picked the slowest of 4 possible applications to use as a result of the work crunched when it was figuring out which was the best application to use. BOINC determining which code is best to use isn't micro management.



So we're saying micromanagement/rescheduling breaks boinc,

Who is, where? If so, is it true or not?
And that in itself is a completely different discussion.
Grant
Darwin NT
ID: 1812047 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1812049 - Posted: 24 Aug 2016, 6:42:17 UTC - in response to Message 1812047.  
Last modified: 24 Aug 2016, 6:42:48 UTC

OK, so let's go with that. So Stock CPU app automatically changing from FPU code to SSE, SSE2, SSE3, AVX etc (which are different devices) is micromanagement->

Incorrect.
It is not micro management. How can it be? It is doing, what it has been programmed to, according to the settings it has been given through the Web/locally.
Someone changing those settings back and forth in order to change how BOINC does it's job, that would be micro management- whether they do it manually or automate it.
refer back to the definition-
micromanagement is a management style whereby a user closely observes or controls the BOINC manager.
The user changing the Manager's decision about which code to use would be micro management- which is what I did on Beta when the manager picked the slowest of 4 possible applications to use as a result of the work crunched when it was figuring out which was the best application to use. BOINC determining which code is best to use isn't micro management.



So we're saying micromanagement/rescheduling breaks boinc,

Who is, where? If so, is it true or not?
And that in itself is a completely different discussion.


Well let me try to understand your rationale here. You're saying if a user does it it's micromanagement, or a tool they employ does it, still micromanagement, but if they make an application that does it internally, then it's not ?
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1812049 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1812055 - Posted: 24 Aug 2016, 7:04:18 UTC - in response to Message 1812049.  

Well let me try to understand your rationale here. You're saying if a user does it it's micromanagement, or a tool they employ does it, still micromanagement, but if they make an application that does it internally, then it's not ?

Correct.

BTW- it's not my rationale, it's the actual definition of micro management.
micromanagement is a management style whereby a manager closely observes or controls the work of subordinates or employees.

Of course, as I mentioned in another post about myself & Beta; if someone were to change the web/local settings to counteract a decision made by the manager, that in itself would be micro management.



You're a programmer.
If you were to work for another company, they would have their own procedures, libraries, tools, formats & definitions for documentation & the like.

For a particular project, the goals & results expected would be stated. The tools & procedures & formats for documentation etc would be decided, then you (and any one else on the team) would be left to do the job. It is entirely up to you & the team how to accomplish the outcome, within the guidelines originally outlined.
The manager overseeing the job sees something you're doing and says to do it differently.
You point out that what you're doing is according to what was decided at the start of the project.
They don't care, it has to be done their way.

That is micro management.
If they were to change the agreed procedures tools etc while in the middle of the project, to get it done the way they want- that would be micro management.

Micro management doesn't mean lots of interventions. Even one is a case of micro management.
Grant
Darwin NT
ID: 1812055 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Philosophy: To DeviceQueueOptimize or NOT (with a focus on: is it "micro managing"?)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.