Beware junk research. I was just asked to advise a client as to whether they should buy a recent law firm compensation survey.

The Methodology description says the questionnaire “was sent to 8327 US and Canadian … members.” Further inside, it reports that about 950 responded; a participation rate of 14%, resulting in a potential for non-response bias that is past the point of being incalculable. And even if it were measurable, only one of the eight regions studied has more than 100 participants, resulting in unacceptably wide margins of error. Worst of all, they report some results at the state level without reporting the number of participants per state. Many states for which stats are reported must have fewer than 10 participants, and that’s “junk research.”

Always study the Methodology section of any syndicated research. In many cases, the results are more likely to be misleading than to be helpful.

Caveat emptor.