Here’s CLOC’s latest research.

I have a few thoughts. It is an interesting report, but be careful. With 200+ participants, we can can have SOME faith in the generalizability of the findings, but with caveats.

Note that half of the participating companies not classified as “other” are either tech or financial services. If those aren’t

Beware junk research. I was just asked to advise a client as to whether they should buy a recent law firm compensation survey.

The Methodology description says the questionnaire “was sent to 8327 US and Canadian … members.” Further inside, it reports that about 950 responded; a participation rate of 14%, resulting in a potential

Have you ever noticed client service ratings that change substantially from year to year, or market research findings that just don’t seem to make sense? There’s a reason.

Historically, few buyers of market research for professional service firms have had formal training in research methodology or statistics. This has improved somewhat inSnake Oil recent years, but serious expertise remains unusual. As a result, law firms and other professional service organizations often fall prey to vendors selling research services (and research-based products such as attorney ratings) with little or no methodological rigor. This causes firms to make important strategic decisions believing they are acting from sound information when in fact they probably would be better off making “judgement calls;” at least then they would recognize the fragile underpinnings of their decisions and not be misled into unjustified certainty by shaky research.

LMA GuidelinesWhen the purveyors of these dubious research services started to gain serious traction more than a decade ago, the Legal Marketing Association wisely directed its Market Research Committee to draft a set of guidelines to use when commissioning such services. The principles of statistics and sound research methodology have not changed so these recommendations are still valid today.

If you are considering commissioning any sort of market research services, I urge you to read the complete document (LMA Research Guidelines). Everyone in marketing should be conversant with these Top 10 Recommendations:
Continue Reading Sound Market Research or Snake Oil?