Does anyone have any experience in the analysis or testing of the information being produced by a CRM system? Or, is there a suggested procedure or process that an Audit Committee should pursue with such none financial reporting systems that are used to inform and advise the Board?
CRM system data and information review
Answers
I think the simplest answer is an audit. Yes, an audit of the CRM data, no different than you would audit physical vs. "book" or system-level information. There are any number of audit companies who could do that for you, probably including your
You could likewise have an internal audit run on the system, but the key, as you note above, is the procedure. In theory, the audit of an actual vs. system comparison you would need to test 1-n samples of data. Thus you would need a procedure to capture the actual info being entered by your control group into the CRM (which means double-entry for them - once into the CRM and then a second time into whatever you use for validation, like
As for SugarCRM, I have used it at a past company and did some research on it at the time. It's no Salesforce.com, but it is very widely used and by some very large companies. I would be surprised if there is an issue with it, although there could always be a configuration problem. Which brings me to another alternative for you which would be to bring in a SugarCRM SI (System Ingegrator) to check your configuration. They could also help troubleshoot in general as they would be highly conversant with the platform.
1
Can I ask, what have you communicated about your discomfort about the information, and were the responses sufficient? It sounds like you have not received sufficient responses. Very appropriate to inquire about the value of the information that you are being provided. How can you make a decision if it is not sufficiently understood that you have been provided with the information that you believe is necessary. Have other deciding members also expressed uncertainty? Or, perhaps the information being provided is sufficient, but you might need additional background explanation--nothing wrong with that. Do you have an internal audit function, or has any somewhat independent third party looked into the information to determine validity?
David Tate
I have communicated my discomfort and the Audit Committee has reviewed the system with the internal staff's assistance. The difficulty with CRM data is that it is recording of activities and estimates made by sales professionals that know the system is being monitored and used for evaluation. While the Audit function has reviewed the data in the system it is difficult to "test" opinion other than by subjective challenge or gut feel review. I have asked for some specific analytics including the identification of old data that has not been updated, old data that has recently (say end of quarter) been updated, data that appears wrong (a specific "last contact" date that was out three months), opportunities that have not changed status, etc. I am still looking for general advice. The large providers of systems do not have great advice to offer on the subject.
It may be that CRM is useful for tracking activity of sales but is of modest use for projecting revenues as it is far to subjective. How do you adjust business budgets for revenue opportunities that have 80, 50 or 20 percent chances of occuring?
1
Age old question, Sanford. Those pipeline stages and deal completion percentages are highly subjective and are not terribly useful unless you are in a very steady state business with a lot of deal flow. The percentages tend to reflect growing accurracy as deals move through the pipeline, so you could use weighted averages and weigh the higher % (or higher stage) deals more than early stage deals to reflect that more is known about them since they are further down the pipe.
You could also try to do a cohort analysis based on aging of deals that have already closed, and tracking of how long they were in each stage. SF.com makes note of every stage change so you can track the dates. Then you could apply the historic win percentages and the "time in stage" to the current pipeline and test it for a couple of months to see if the results fit the model (I sound like a stock trader here).
You could also institute an up or out model whereby all opportunities must move up in stage every X days or they will be closed out, in order to get rid of the noise. In my few decades I have rarely seen a stalled opportunity regain life and close. Typically when that happens there is some exogenous force which would qualify it as a new opportunity. This won't work 100% of the time, but will draw attention to data
At the end of the day I have found the only way to drive accuracy in CRM data is to do rigorous reviews. Noone likes them b/c they lay your performance bare and they take time (weekly), but they keep your sales leaders informed and the salespeople honest. And if people aren't willing to maintain their data, then it's sayonara.
I was just looking at the Proformative "Ask the Experts" page. It looks like there might be an expert category that would fit your question, or perhaps another group like budgeting would be a good direction.
Filed Under:
Technology