By Megan Hustad
FORTUNE -- People in the business of reviewing business advice get it by the truckload. After a while, every book delivered to your door looks just like the last one. All strategy prescriptions are backed by comprehensive research, and every author is impressively credentialed. It is hard to determine who is adding value to the conversation for two reasons: One, no one has time to read all these books; two, there's tremendous incentive for an author to spin hard conclusions out of mucky data.
"It's very tempting, consciously or subconsciously, to impose a pattern on data that isn't really there in order to support a hypothesis," says Michael Raynor, co-author with Mumtaz Ahmed of The Three Rules: How Exceptional Companies Think. "After all, if you stare at the poundcake long enough, Elvis's profile will surely appear."
The Three Rules conforms to type by citing impressive study numbers -- 25,000 companies over 45 years -- then allocates several pages to unpacking their study methodology. So I asked Raynor how he reads business books. Is there a way to assess research claims quickly, respectfully, but skeptically?
Raynor's first prescription is to remember that persuasive storytelling requires that the storyteller leave out the weeds. This is especially relevant to corporate biographies, since the form requires the narrator to omit people and events that turn out to be irrelevant only in hindsight.
His second note of precaution is about what to do when presented with causal claims. Most smart people know not to mistake correlation for causality, but we do it all the time. Or we dismiss someone else's claims by saying that they haven't proved causality (just because one event happened after another doesn't mean the first happening caused the second). True enough, says Raynor, but "nobody has evidence of causality." Causation exists -- there would be less incentive to leave the house in the morning if it didn't -- but it's difficult to prove in complex systems (and any system that includes humans is a complex system).
Raynor also advises watching out for what Phil Rosenzweig dubbed "the halo effect." In other words, make sure you aren't letting the reflected glory of a company's signature achievement in one arena color your view of their performance in other areas.
Next, be aware of the data's limitations and your own. Why dwell on your own limitations? Our intuition as to what's statistically significant can be terrible. When we pick up a book that profiles certain companies, we tend to assume that the companies being profiled have, in fact, delivered noteworthy performance.
But "that's an assumption that's worth questioning," says Raynor. "If two companies differ in profitability by 0.1% in return on assets over a five-year period, would you study those two companies to understand behavioral differences that drive performance differences? Of course not. Because it's too small a difference over too short a period of time."
So watch for sample selection and time frame. "In a short season, luck can overcome skill."
Raynor's last note concerns an all too common criticism of business success studies. Say a company praised in a popular business book -- for example, Circuit City in Jim Collins's 2001 Good to Great -- ultimately disappoints. Critics then pile on to say that the author botched the analysis. ("Hey wait a minute, you said that company was great and then three years later they're in bankruptcy. You don't know what you're talking about.") That's unfair -- and shortsighted. "This whole notion that you have to study a company that is perpetually excellent before you can learn something [from it] is nonsense," Raynor says.
The best rebuttal, he says, is to point out that Usain Bolt will probably not be an Olympic gold medal winner at age 60, but that doesn't mean the techniques he uses now will not be worthy of study in years to come.
Our best defense against seeing Elvis in the poundcake, however, is one both authors and readers can use daily: Realize that the smartest people in any room appreciate it when you acknowledge data that doesn't support your conclusions. So, in cases where the rules you've devised don't appear to hold up, say so. Mention how you might be wrong, and then present a case for why you believe what you believe anyway, says Raynor. That kind of candor is flattering to your audience's intelligence and -- most importantly -- memorable.
Forget the falling-backwards "trust" exercises and other hackneyed stuff, says one CEO. Give people time to enjoy the locale, and keep the agenda down-to-earth.
By Anne Fisher, contributor
FORTUNE -- Dear Annie: I guess it is true that no good deed goes unpunished, because our office holiday party -- which took me six weeks to organize -- was such a hit that now my boss has put me in charge of a three-day MOREDec 21, 2011 12:39 PM ET
Most companies say they have trouble finding the right people these days. A new book argues that identifying brilliant hires depends on a willingness to abandon the usual criteria.
By Anne Fisher, contributor
FORTUNE -- In retrospect, the biggest blunders often seem inexplicable. Four different book publishers, for instance, passed on J.K. Rowling's first Harry Potter novel. A weird story about the adventures of a juvenile wizard and his friends just didn't MORENov 1, 2011 11:26 AM ET
Most of the conventional wisdom about how to generate world-beating business ideas is wrong, says a new book. Which practices belong on the scrap heap? By Anne FisherOct 6, 2011 2:28 PM ET
Three new books explain how to swipe market share from rivals, why some brands stand out, and the sneaky tricks great marketers use. By Anne FisherSep 19, 2011 10:07 AM ET
Scared about giving a speech? Forget trying to relax. Instead, a new book says, put your stage fright to work -- or camouflage it. By Anne FisherJul 26, 2011 1:13 PM ET
|Michaels hack hit 3 million|
|Wealthy investors flock to fine art funds|
|Detroit pension cuts hit civilian workers hardest|
|Obama would cut deficits by another $1 trillion|
|GM's recalled Cobalt was a failure from the start|