"Which measures do you recommend tracking for measuring forecast accuracy? Revenue, Profit or others?"
This question was asked at a recent webinar, now available on-demand: "Calculating Financial Forecast Accuracy"
Please add your thoughts about it below. Thanks!
What I always recommend is to forecast the entire chart of accounts. If you have the right software, your revenue and expense forecasting will also populate a set forecasted balance sheet accounts and you will be able to track your entire company's performance against the budget, in each budget period.
This becomes very useful when you have your actual
When you have sufficient historical data and have developed specific KPIs, you can use them in the budget model by implementing drivers to arrive at actual budgeted amounts in many of your key accounts.
The key thing is to forecast with enough granularity so that you can go back afterwards to compare results to what was forecasted.
Forecast sales by person responsible for those sales.
Don't overcomplicate it.
Forecasting post mortems are extremely important and you want everyone to see where you hit and where you missed. The post mortems are one of the most important things you can do.
1
Lyle you hit on something, but didn't expand on the thought...
(Over) Complication.
We tend to over complicate everything. We want to be ready for every possible scenario. Not only is this impossible but drags out the process and distorts the picture.
I'm a firm believer in the KISS principle. Once you get that right, then you add layers.
When measuring forecasting accuracy, it is useful to compare each line of the forecast that is material (significant), because if you look only at the key KPIs (e.g. ebit), you may miss several large variances above which offset each other - even though the KPI variance is small. Next month you may not be that lucky.
With regards to the accuracy of the forecasting process, you may consider measuring separately the accuracy of the initial forecast (for example the first draft prepared based on statistical analysis) and then the accuracy of the final approved plan. This is because
Last, it is useful to distinguish between ordinary and systematic variances. Systematic variances occur for example due to bias - when all your variances have the same sign (e.g. all are adverse or all are favourable). In my experience this often occured with forecasts coming from sales department or when forecasts were tied to compensation or appraisal (which they should not - if you want a honest and realistic forecast, it must be cleared from personal politics). Ordinary variances will always happen and its the CFOs job to make them as small as possible.
It is helpful to remember that the future is uncertain.
As a result your forecast will always be wrong. The real trick is to determine how wrong it can be and still be useful in obtaining your objectives (e.g. gaining market share, meeting investor expectations, avoiding economic potholes in a so-so economy, etc)
What to forecast - certainly profits but also cash flow and selected balance sheet items (e.g. bank debt). As has been said a forecast should be in enough detail to drill down several layers, but not too much detail you loose site of where you wish to go in relation to the market or others expectations.
When to forecast - there is an old economics joke that goes if yuo cannot forecast accurately, forecast often. How often can depend on your industry and yuor own internal capabilities.
Accuracy - selecting a $ or % variance target is the tricky part; there is no right answer, but looking toward your peers' results could give you a clue. Also, consistency is a hallmark of a good forecasting process. In other words if you are always 10% off then you will know what to adjust the next time.
As important as what to forecast is the relationship between one set of foretasted figures (e.g. sales) with another, say ARs. Getting sales right may help with income forecasts, but getting ARs right will help with cash flow forecasts. I will leave the importance of any variance of these items to yourself.
Bottom line - a forecast with a variance of X% from actual results maybe acceptable if it is better than a previous period or a competitor's results and doesn't adversely impact some other performance element, say EBITDA or operating cash flow?
2
Bruce - I'm with you on your 1st and 3rd paragraphs: the future is uncertain, forecasts (just like budgets) will always be wrong, and updating forecasts as often as is reasonably economical to do (allowing improvement of forecast assumptions to reflect changes in business conditions) is better than long gaps between forecasts.
The problem I see developing is that there are companies in which forecast values accuracy becomes more important than correctness of management's analysis of business conditions &
Lyle Newkirk's comment that forecasting post mortems are extremely important is correct. However, the focus should not be on where (the values) that were hit and missed. Instead it should be on why the values were hits and misses. A forecast that accurately predicted a specific value could still be the result of a flawed forecast process if the business conditions and actual events where different than those used in creating the forecast. In that situation, the forecast accuracy is the result of luck, not of ability and knowledge.
If Boards and senior management remember that the future is uncertain and that forecasts will always be wrong and, as a result, focus on evaluating forecast accuracy based is on whether or not the underlying assessment of business conditions and resulting forecast assumptions are accurate, then the forecasting process can add value and support development of management skills & effectiveness. If instead, they focus on value accuracy, then they may as well stop wasting time on forecasting, because all they will create is a double set of budgeting processes.
Isn't the point of forecasting to set measurable targets which are in line with the organization's strategic plan? And therefore anything that is deemed worthy of measuring (i.e., forecasted) should be assessed against the organization's plan goals.
So the question, "Which measures do you recommend tracking for measuring forecast accuracy?" actually answers its self as it mentions "measures". If you have elected to measure "It" then assessing accuracy of results against organizational goals is what is next.
Once results are measured and compared against the forecast/budget determining accuracy of forecast prediction, by analyzing the underlying drivers to explain causes of variance, is the next step. It is the explanation of the underlying drivers and their impact on performance (a.k.a., results) that explains forecast accuracy, so having a handle on what drives each measurement is essential to understand accuracy.
Any spread sheet can spit out percent variance up or down. They are useful to show the delta +/- from forecast targets, but not vary useful by themselves in explaining the 'how' and the 'why'. The 'how' and the 'why' is found in the underlying drivers, and understanding them will make the forecast model that more robust and improve accuracy of prediction.
I disagree almost completely with the first answer above from Mr Hart: I loath the idea that we are trying to forecast our accounting. Accountants (and I am one) have an unfortunate habit of getting caught-up in our own importance and in the importance of book-keeping. Unless your business is book-keeping, your customers aren't buying your products because you have an efficient month-end close process, or swift re-forecasting capability, or an integrated reporting system: those are hygiene factors not value creators.
If you told your customers that you were increasing your prices above those of your competitors because you want a better forecasting system, will they pay the premium or switch to your competitor?
Find out from your colleagues what are your customers thinking when they buy your product? How do you ensure they are happy? What are the internal and external indicators that would signal your successes and failures in delivering that happiness? Not all of these items (eg market share) will be captured by your accounting system so leave your chart of accounts behind and understand the business. What are the KPIs for your other stakeholders?
There will almost certainly be KPIs which come from the accounts such as debt covenants, working capital management, EPS. But some of these cannot wait till the month end and the accurate close of your books. I've worked in organisations with severe cashflow restrictions: forecasting was done on an almost hourly basis, not releasing payments till receipts have been banked. Waiting till accounts have been compared to forecast was not an option. So, think also about your timescales and whether you are forecasting the right period.
I think it depends on the intended usage of the measurement. Is it a KPI for performance evaluation? Are you using it to determine what needs to change about your assumptions on future performance? Do you have a culture of under commit and over deliver that biases the process and is that the behavior you want to reward?
Thank you for the many good points above. In the spirit of the question above, I thought I would throw another out there that might lead to further discussion.
Granularity has it's pros & cons as pointed out above. It allows for a much better visibility of the variances, yet becomes increasingly difficult to attain the more data you have. Yet, on the flip side, the more data you have the greater accuracy your forecast.
I am living this dilemma currently with my working capital and cash forecasting as my company's smaller size is subject to greater fluctuations due to my A/R aging. It is nearly impossible to get a bead on cash flow until the money actually hits the account. So my question to the group is, how do you deal with that which is so significant and so far out of your immediate control (ie-client payments)?
2
If you can't develop a collection days model that approximates based on poor performance future performance, you might look at different scenarios that may entice your customers to pay you in terms.
Depending on industry, Factoring (whether you take advances or not) may entice payment.
Who is responsible for collecting A/R in your company?
