survey cybernetics

Organizations regularly use surveys to steer themselves. Here I examine the cybernetics of doing so:

Open Loop

In control theory, a branch of cybernetics, we model the inputs to and outputs from systems using block diagrams, e.g.,

Inside the block resides system F which operates on inputs x to produce output y. For example, F might be an automobile and x the position of its accelerator. y might then be the automobile’s speed, a function of how the accelerator position x influences movement of the system F (the automobile). Similarly, F might be a medical patient that experiences outcome y of therapeutic intervention x. Finally, F might be a corporation and x HR initiatives that result in y amount of employee satisfaction.

The arrows in the block diagram above indicate the direction which information flows. With this in mind we immediately see that the diagram shows no feedback, i.e., no information about y is used to adjust x. For reasons that will be apparent shortly, we call such systems open loop.

Closed Loop

In the automobile example above, to achieve a desired speed a driver must incorporate information about the vehicle’s current speed into her adjustments of the accelerator position x. We describe this by adding a feedback loop to the block diagram:

The addition of the feedback loop creates a closed loop system, meaning that information about y can now be used to adjust x, with the goal of controlling y. We then say that the system is under feedback control; information flow through this closed circuit enables control over y.

The feedback can be as simple as observation:  For example, a medical doctor observing effect y on patient F can adjust treatment x to pursue the desired therapeutic outcome. Or the feedback can be very complex, e.g., using multiple inertial measurements to adjust thrusters x of a spacecraft F to track desired orbit y.

Finally, the feedback can be an employee survey used by an HR department of corporation F to adjust initiatives x in pursuit of desired employee satisfaction level y.

Measurement

This all relies on correct and timely measurement of y. In the case of the automobile example above, measurement is no problem. Measuring y requires only a speedometer, and any lag between a change in vehicle speed and a change in the speedometer reading is imperceptible to the driver. Furthermore, today’s speedometers measure speed accurately enough for the job. However, below I briefly identify three measurement shortcomings that impact real control problems:

First, consider what happens when there is a significant lag between input x and outcome y. For example, a depressed patient F must wait about a month for many antidepressants to show therapeutic outcome. A medical doctor monitoring the patient’s condition y and tweaking the medicine dose x is at the mercy of this lag in information. The result of such a lag typically causes changes to x that cause y to overshoot the desired value.

Second, consider how inaccuracies in the measurement of system state y impact the ability to control y. If a velocity sensor reads an aircraft’s speed as 5% slower than the true speed, the pilot (or autopilot) will overshoot the desired velocity and probably burn extra jet fuel.

Finally, consider that the system F itself may change form over the time period during which control is attempted. This does not happen with automobiles and aircraft—we are ignoring tilt-rotor aircraft here—but can happen with corporations which must constantly restructure to face changes in their business environment. Unless the feedback measurement changes congruently with the corporation’s change, information retrieved will not reflect the true condition of the organization.

Feedback Control Using Surveys Risks all Three Shortcomings

Feedback control of business organizations using surveys risks all three of the above stated shortcomings.

First (and third), a lot can happen in a business between writing a survey questionnaire and retrieving results from its delivery. Therefore, the corrective action prompted by the survey results may lag behind the true system state so much that the survey loses relevance. This is especially true for corporations changing form in response to the unsteady business climate. The system F itself may change between the time the survey questionnaire is distributed and the time corrective measures are prescribed based on the survey results.

Second, since survey questions may not be interpreted the same way by every respondent, surveys offer an imperfect description of the state y that they attempt to measure. Therefore corrective action is prescribed in the presence of “sensor error”, some of which may be consequential.

Recommendations

Despite these shortcomings, surveys remain one of the best available tools for accessing the state of an organization. I offer four recommendations to help mitigate the risks described above.

First, deploy the survey through an online form if possible. This reduces the opportunity to make errors induced by lag by accelerating the analysis of results.

Second, have a professional survey company design the questionnaire and interpret the results. This reduces the risk of asking questions which are vulnerable to misinterpretation, and reduces the risk of improper statistical analysis of the results.

Third, be sure to ask the same question in a few different ways, so that Cronbach’s consistency measurement may be computed to access the reliability of the answers.

Finally, survey results should be combined with other ways of assessing an organization. In other words, make sure the results satisfy intuition before relying on them.

This entry was posted in marketing, survey analysis and tagged , . Bookmark the permalink.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>