Predictive analytics is a rapidly growing field. Around the time of its inception, data scientists made bold predictions about its capabilities. Predictive analytics has proven to be invaluable for organizations of all sizes across all industries. Yet, over the past few years, we have discovered a number of limitations that must be taken into consideration. This is particularly relevant for small businesses that use predictive analytics for risk management.
Before you begin over relying on predictive analytics to forecast risks for your small business, you must educate yourself about the limitations. Here are some factors to assess.
Many macro level predictive analytics solutions poorly suited for real world applications
The predictive analytics models that attempted to forecast the results of the 2016 election proved to be incredibly wrong. This was one of the biggest lessons in humility the global data science community ever had to face. It also has some important lessons for small businesses that depend on predictive analytics.
One of the biggest issues with the electrical data models was that they place too much weight on polls conducted at the national level. The United States has an electoral college system that determines the victor by state results, rather than the popular vote. Therefore, the models that predicted candidate Donald Trump only had a 20% chance of winning were significantly underestimating his potential. They failed to account for the margin of error in key swing states, which gave him much better odds than these models suggested.
More Read
Similar challenges could affect small business owners that depend on predictive analytics. When they try to assess the risk of a given outcome, they may use predictive analytics models that make system-wide risk assessments. Instead, it would make more sense to have a model that accounts for all events that could trigger a given outcome and the weighted probability of each of those events occurring. This can help small businesses get a better understanding of the actual measures that they need to take to prevent undesirable outcomes from arising. This is important for actuaries underwriting business insurance for contractors, as well as businesses to mitigate the risks.
Unfortunately, these more nuanced, weighted probability models are much more difficult to develop. Here are some of the problems:
- It takes time to pinpoint all of the factors that could lead to particular event. You may believe that you have identified all of the relevant variables, but only 20% of them have actually been discovered.
- While you may have a lot of data on the actual real risks at a macro scale, data on specific events may be a lot more limited. This can create a much larger margin of error when it comes to forecasting the rest of the system as a whole.
Consider a couple of scenarios where you would need to build a risk profile with predictive analytics:
- You are trying to predict the risks of a cybersecurity breach. There are number of ways that malicious third parties could coordinate a cyberattack. In order to protect your system, you must identify the weak parts of your cybersecurity infrastructure and gauge the risk that a breach will occur at any of them. You may have conducted penetration testing to determine the risk that a “typical” cyberattack will be successful. However, you may not have adequate data on the likelihood that a cybercriminal will target your system in the first place. You also may have sufficient data on the security of your firewall, but lack data on other elements of your cybersecurity system.
- You are expanding four of your most successful products in five new regions. After testing the markets in United States, you discovered that 20% of your products were profitable. Based on this data, you assume that four of the 20 campaigns that you are lunching will also be profitable. Your existing sales reports suggest that you need to achieve profitability with at least three of them to be profitable. However, the problem with your predictive analytics model is that you are assuming that the probability of success in the new regions you’re operating is the same as in the United States. The adoption rate could actually be much lower. You also making assumptions about the ROI and sales volume based on your US data, which may prove to be unreliable overseas.
Getting the data to fine-tune your predictive analytics is going to take time and resources. Unfortunately, even after collecting those, our predictive analytics capabilities may not be sufficient to draw accurate forecasts, which can be a risk for many small businesses.
