Automation is the key to unlocking a big, sustainable advantage in companies in different sectors.
Big data can be a big nothing without a strategic automation approach.
On the one hand, we are in a turbulent time of information wealth, with unprecedented amounts of data on everything from equipment performance to consumer behavior on social media (more than half of all citizens in the world are on social media). But without careful automation – the use of machines and algorithms to process, process and analyze the available data – your business will lose great potential.
Well done, automation turns “dead” big data into a living, breathable resource that you can use to drive value. So it’s no surprise that many companies are striving to automate anything that can be automated, as a top Google CEO said recently.
To help you think about automation in your business context, I present the three main ways in which this technology-driven activity helps you create value.
The first thing that automation helps you do is function extraction, or download critical information pins from massive haystack data. Imagine that your organization needs to review patent applications for information about a specific technology and related. You can browse thousands or tens of thousands of applications, each running 30 or more pages, for millions and millions of words. But only a small fraction of these words and the interrelationships between patents are relevant, such as what the patented technology depends on or the qualifications of the inventors and past patents.
So this task, like many in the business world, involves a very small signal-to-noise ratio and would require thousands of people hours to complete them manually – something too expensive and unbelievable. But a machine-based algorithm can be trained to retrieve the necessary key information relatively quickly, saving considerable time and effort. Also, say that in the future you want to search for the same set of patents or related ones, but for different information, such as the size of the patent applicant’s team. You can easily reprogram or retrain the algorithm to take on this task, gaining economies of scale and a higher return on your initial investment.
Second, automation helps data inspection and cleaning. Datasets often need work. There are errors and missing values, anomalies and sometimes evidence of bias. For example, if the algorithm is trained to notice the characteristics of offenders but uses data only on captured offenders, the algorithm will be addictive because there is no data on offenders who have not been caught – a particular problem for white-collar crimes. tend to be under accounted for. Again, checking and dealing with this huge amount of potential problems is too much to take on manually. But automation allows for the rapid deployment of testing and cleaning tools, again saving time while creating value.
Third, and this is great, automation is the most important driving engine of the analyzes. Yesterday’s simple regression analysis turned into today’s machine-driven clusters and random forests, whether to understand product consumers, forecast sales next month to optimize inventory, or forecast the impact of a new advertising campaign. Machine-based automation not only allows you to regularly replicate standardized analytical processes at a low cost, but it can also spot nonlinear patterns that we humans cannot.
For example, my lab has studied more than 5 million patents, using algorithm-driven analyzes to see if we can predict the debut of revolutionary future technologies based on information about their patent application. We assumed that the machine would identify future hit patents from the application data if the invention had stand-alone, “miracle-like” possibilities or ideas. In the end, the algorithm did find the hit patents of the future with high accuracy, but not in the way we humans imagined it. This means that the algorithm has not identified a future hit patent based on its stand-alone capabilities; rather, it identifies successful patents based on whether they are part of a cluster of related patents, which together could solve specific problems in combination, which no single patent could solve alone.
For example, ultrasound technology has had a major impact on healthcare several years after it was first discovered, allowing non-invasive imaging and treatment of physical conditions such as kidney stones and even some cancers. But this progress would not be possible without minor inventions outside the core technology – applicators, static reduction processes, specialized medical pads and staples, which have been developed independently of ultrasound technology but are crucial for its successful application in medicine. Our automated analysis reliably recognizes the existence of these groups of related patents in over 5 million health product patents to the latest golf ball technology and that these clusters are related to the likelihood that patents in them will become future dominant technologies in the future – conclusions that have not been evaluated before.
My northwestern colleague, Andrew Papachristos, uses similar analyzes to show that police corruption in Chicago does not stem from a few “bad apple” employees, but from a network of connected police officers acting in bad faith; its operation allows earlier detection of such problems.
I hope I have clarified the mutually reinforcing benefits of automation and how it can help you transform data into great, sustainable value. Indeed, the more data you have, the more you need automation; once you have strong automation capabilities, you can collect and harness even more data and the cycle continues.
The bottom line: automation is an increasingly critical capability and can be key to the short-term and long-term efficiency of your business. But it is important to understand how it leads to value and to take steps to mitigate its very real shortcomings for the good of your company and the wider community in which it operates.
In the second part of this article, I will discuss the three main disadvantages of automation – clarity, transparency and cost – and how to deal with them.