When you need your taxes prepared, you hire a CPA. When you’ve got a bridge to build, you hire a PE (professional engineer). Even your hairdresser has a license. But anyone can hang up a shingle and call herself an analyst. When you need an analyst, how do you choose?
A business manager came to me in desperation looking for advice. He was responsible for developing a thorough risk analysis for a massive engineering project. Finding that some parts of the analysis were beyond his own skills, he had engaged the help of a costly consultant.
The consultant had given the manager worst news possible – the analysis could not be done.
Had the consultant determined that the risk was unacceptably high, plans could have been modified to reduce risks to tolerable levels. But if the risk could not be estimated at all, there would be no way to save the project.
The manager carefully explained the knowns and unknowns of the analysis. He wondered if there might be an even more sophisticated consultant, one whose special expertise could help to complete the picture. As we went talked over the problem, I realized I knew exactly where he could obtain the help he needed – in an introductory statistics textbook. The “unsolvable” problem was actually a common issue in statistics, and not at all a difficult analysis to perform.
The good news – the analytic problem was solved.
The bad news – the manager had paid handsomely for consulting services that were worthless and potentially damaging to his business.
You can prevent this kind of bad news.
Before you act on analytics advice, evaluate the advisor. Whether you are hiring an outside consultant, working with an advisor within your company or seeking help from a personal contact, you will be responsible for the results of your decisions. If you were seeking business advice from a lawyer or accountant, you’d ask plenty of questions. Unlike lawyers and accountants, most data analysts have no professional certification, making it even more important to ask questions.
But… what are the right questions to ask? And what are the right answers?
When evaluating a data analyst, you are looking for three things:
1) Technical competence
2) Business understanding
3) Communication skills that meet your needs
Here’s how to go about it.
1) Technical competence.
Here, “technical” refers to knowledge of mathematics, analytic methods, analysis software and other skills that are useful for data analysis. This is the toughest area for most people to investigate, because most people don’t know much about it. It’s also challenging because the advisor may be a friend or respected colleague, and it takes some thought to probe without offending.
Consider the model of professions that require professional licensing or certification. These professions have formal and clearly-defined minimum requirements for proof of competence. You can follow that lead by looking for evidence of competence such as relevant education and experience. You probably won’t be giving any tests, but you can seek input from others who have worked with the analyst you are evaluating.
Does the analyst have adequate training? If you’re considering a paid consultant, ask directly about formal education. A degree in a relevant field such as statistics, mathematics or economics is a meaningful qualification. Many capable analysts have degrees in other fields, but have completed significant coursework in statistics. (A Ph.D. is meaningless if the studies aren’t appropriate to the analytics you need. The consultant in the story above was a professor at a respected university, but his field was not statistics and he did not have sufficient knowledge of statistics to do the job.)
If you’re seeking advice from a friend or coworker, probing will require a more tactful, indirect approach. You may be able to get basic information about your friend’s education through online sources, or by reminiscing about college days over lunch. If your friend’s education isn’t an obvious match, probe in a friendly way: “Say, Brenda, how does an art history major end up in this line of work, anyway?”
Ask about the analyst’s relevant work experience. Again, some situations require more tact than others, but there’s always a nice way to ask. You can get a surprising amount of information with a simple, open-ended question like “Does this issue come up often?”
Got doubts? Feel good about what you’ve heard, but wish you had more to support your decision? It’s time to seek out others who know the analyst’s work. When it comes to getting worthwhile assessments of an analyst’s work, life is easiest when the analyst is a coworker. Chances are you’ll have a few trusted colleagues in the company who are familiar with the analyst’s work and who will, in a private setting, give you an honest appraisal. Outside consultants will only refer you to people who love their work. Tap into your own network or search online for leads to others who may be able to provide additional information.
2) Business understanding
A brilliant theoretician is no help to your business.
You need someone who is realistic and practical, someone who “gets it.” Don’t hold out searching for the analyst who has years of experience in your tiny niche. But do sit down, talk over your issues, and keep doing this until you are satisfied that you have found an analyst with a good grasp of business in general (bonus points for businesses similar to yours), one who listens, makes the effort to challenge you for clear and thorough explanations of your concerns, and demonstrates the motivation and ability to understand your specific business issues.
3) Communication skills that meet your needs
What will you need this analyst to do?
Make a presentation to your boss?
Explain and document the analysis clearly so that you can make the presentation yourself?
Meet with your clients?
Testify in court?
You don’t need anybody to tell you how to do this part. Just take the time to anticipate what you need. Then sit down, reflect on your impressions of conversations and written materials shared by the analyst, and ask yourself if you’ve found a match for your needs.
At the end of the day, it’s your profit and your reputation that is on the line. If you need analytics help, make the effort to assure yourself that the analyst is technically competent, understands your business and has the communications skills that you need.
© 2010 Meta S. Brown
