Why A Big Data Strategy is Critical for AI
Big data describes the volumes of data that your company generates, every single day—both structured and unstructured. Analysts at Gartner estimate that more than 80 percent of enterprise data is unstructured. Meaning, they can be text files from IT logs, emails from customer support, direct Twitter messages from customers, and employee complaints to your HR department. This type of diverse and scattered data sources is true of almost every enterprise.
A big data strategy, on the other hand, is a glorified term for how you’ll collect, store, document, manage, and make the data accessible to the rest of the company. When companies don’t have a good data strategy, they spend enormous amounts of time just getting their data into a usable form when needed.
But, you may be wondering, what’s this “Big Data” got to do with AI?
Modern AI applications thrive on data. Depending on the problem, it can be your very own structured or unstructured data.
In fact, according to IBM’s CEO, Arvind Krishna, data-related challenges are the top reason IBM clients have halted or canceled AI projects. Forrester Research also reports that data quality is among the biggest AI project challenges. This goes to show how critical data or rather big data is for AI.
A Support Ticket Routing Example
Let’s take a machine learning model that automatically routes support tickets to the appropriate support agents. In order to build this model, you’d need a large volume of historical support tickets and the corresponding routing. Historical here means all the old, resolved tickets.
This historical routing data is then used to automatically learn patterns so that the machine learning model can make predictions on new incoming tickets.