The volume of data in the world is growing massively – 2.5 exabytes (2.5 billion gigabytes) a day, by some calculations. It equates to 530 trillion songs, or the contents of 5 million laptops, every day. Huge volumes of data now end up on the drives of a range of organisations which previously collected far less, and are now looking for ways to make sense of, or still better, use of the growing data lakes in their possession.
At the same time, techniques are emerging, fuelled partly by the rapid increase in availability of cheap computing power, to find patterns in this data that are not immediately apparent from normal analysis. Inferences and surprisingly accurate predictions can be made from
- consumer behaviour
- financial markets
- medical records
- Performance monitoring data in engineering
Some of the techniques, e.g “deep learning” or neural networks, are modelled on the functions of the biological brain, rather than relying on conventional computing algorithms which explicitly prescribe what a program should do in every eventuality. The emerging machine learning techniques, whether conventional or not, are not only well adapted to process large volumes of data, they require large volumes of data to work, and with these large datasets, they make possible recent major advances in
- natural language interpretation
- computer vision.
They are the building blocks of artificial intelligence (AI).
True, complete AI, as depicted in science fiction, is still some way in the future, although marketing departments, eager to capitalise on the commercial potential, have not been slow to use the term, and much hype surrounds it. What is certain is that, with today’s techniques, computers are capable of doing tasks which a few years ago could only be done by humans. Computers are becoming ever better at transcribing and interpreting speech, recognising faces, reading printed and handwritten characters, and making inferences from data which are not obvious and whose possibility was not necessarily foreseen by the programmers who wrote the software.
The applications of these new capabilities are far reaching. They include
- Self-driving cars: although completely autonomous cars are not yet road-legal, the technology is complete, or close to it. Autonomous cars are as safe or safer than human-driven ones, and the greater problem now is in defining a legal framework for them to operate alongside human-driven cars.
- Advanced warehouse and factory robots: robots have existed for decades, but performed actions pre-defined to the millimetre, and were not capable of handling unexpected eventualities. Newer robots are better able to do this, adapting their paths around obstacles, and compensating for components which are not in exactly in a predefined place.
- Chatbots: chatbots can increasingly automate customer service. Since they do not have to imitate the look and sound of a human, they can be hard to distinguish from one. They are increasingly capable of dealing with routine enquiries, and, once the customer’s question becomes too complex for the machine to understand, it can be handed over almost seamlessly to a human operative.
- Medical diagnosis: machine learning can help with the analysis of medical scans, a form of image recognition. Computers’ capabilities in this area are growing, and are already valuable in supplementing human expertise. Computers can classify the easier cases and draw attention to anomalies in large volumes of data, allowing highly trained medical practitioners to concentrate on the more difficult, value-added work. Another form of medical diagnosis consists in computer analysis of a patient’s symptoms, and comparison with the large body of medical knowledge and precedents. This technique can help analyse all the possible causes from a set of complex symptoms, and help reduce the chance of missing important diagnoses, which human doctors might occasionally miss.
- Predictive maintenance: complex machinery such as aircraft and their engines, other vehicles and forms of transport and factory equipment, require predictive maintenance. Predictive maintenance an be expensive because it takes valuable machines out of use before they are broken in order to repair them pre-emptively. Machine learning can help optimise the timing of predictive maintenance by analysing the great volumes of performance data now produced by advance machinery. In some circumstances, machine learning can detect early signs of the failure of a machine, and conversely provide reassurance that the machine is not about to fail, even though its scheduled maintenance is due, allowing the maintenance to be deferred safely, with resultant cost savings.
- Financial market prediction: financial institutions are among the leaders in the adoption of machine learning. Being able to make even very short-term predictions about market movements brings obvious financial benefits. Machine learning techniques for that purpose are typically kept secret, but machine learning has other financial applications, such as fraud prevention by detection of anomalous spending patterns; credit analysis of borrowers; and matching financial products to consumers based on their spending and saving behaviour, and their income patterns and lifestyle.
Automation Consultants has knowledge of many of the data science and machine learning techniques currently transforming the world. If your organisation has large datasets and is looking to use them better, you can find out more about our capabilities here.
