The data that will change the world is scattered all around us

This article was contributed by Dr. Roman Sandler, CTO and cofounder at Ravin AI.

It’s no secret that AI is changing industries and businesses of all types. Medicine, education, retail, manufacturing, automotive, and many others are being impacted by advances in the area of machine intelligence, also known as machine learning, neural network technology, natural language processing, or simply AI.

AI-powered technologies have already been responsible for significant efficiencies and improvements in a wide variety of areas — but this is just the beginning; the AI-wrought changes we’ve seen so far utilize, by many estimates, only a small amount of all data available. It’s safe to say that when we use more data — much of it unstructured — things will really get interesting.

Most AI data-analysis efforts center around text, audio, and videos collected via the web, mostly to provide insights for business, marketing, and customer service, with only a growing minority of organizations now using tools to understand and organize unstructured data from the physical world. But there’s a whole world of unstructured data that could be a boon to many other industries — medicine, agriculture, transportation, construction, to name just a few.

Sensors that are currently in use — and the expected explosive growth of IoT devices — will collect huge amounts of data, much of it unstructured, and much of it in non-text forms. Such data, by definition, is “computer friendly,” but not AI-analysis friendly. While data collected by sensors and machines are easily readable by systems, it cannot provide insights in this “raw” state. In order for AI systems to be able to analyze data and provide those insights, it needs to be deployed in a structure that will enable scientists to mine it for information that will provide the answers they seek. The first step is to apply an initial layer of AI to transform this unstructured data into structured data that can then be exploited by additional types of AI for insights into solutions in a wide variety of areas. 

For example, unstructured data will be essential to further the development and use of autonomous vehicles. Utilizing data from cameras and sensors, autonomous vehicles currently do quite well on well-maintained roads with clear markings and signage, where driving is done in a “predictable” manner. A bigger challenge to expanded usage of autonomous vehicles is their performance in non-standard driving situations — where the roads aren’t smooth, neat, straight, or properly signed and marked. 

And it’s here that unstructured data could make a difference. By utilizing the data pulled into the system and applying it to the structures that autonomous vehicles can understand, AI systems can enable vehicles to navigate driving those challenging roads just as they would the easy, standardized highways. Given that a mass reconstruction of country roads, urban streets, and long-distance expressways to accommodate autonomous vehicles is unlikely, utilizing unstructured data in this way — converting it to structured data — will be an important component in the growth of autonomous vehicle usage.

With the power of AI unleashed on this newly-structured data, not just self-driving cars, but many types of businesses and organizations will have many more resources to work with — no longer leaving what may be their most important and valuable insights on the table.

Here are some other ways unstructured data can be used to improve insights: 

Agriculture: Sensors and IoT devices on equipment and in the field could yield data that AI systems can structure for advanced analysis, yielding insights that could help farmers grow more crops, harvest them at exactly the right time and maximize resources and profits. For example, sensors mounted on farm equipment could collect data on sound waves, and analyze them for malfunctions; temperature and soil readings, when combined with images of crops, could yield insights on ideal growing environments; analysis of social media posts could provide clues on which crops are likely to have the highest market demand and fetch the highest prices. While some of this data (like temperature and weather information) are likely already in structured databases, much of it likely isn’t — and by applying AI structuring and analysis to this large mass of data, farmers — and consumers — will benefit.

Healthcare: If there’s one area that should be a model for the potential power of unstructured data, it’s healthcare. While much data collected by doctors and hospitals is properly coded and labeled for use in structured databases, much more data remains unstructured — and in many cases even currently unrecorded.

Among the sources for unstructured data that healthcare could be taking advantage of are emails, text files, meeting transcripts, videos, photos, videos, data from chat apps — even handwritten notes. Each of these could be sources for insights into a large number of areas — from quality of care to efficiency, to whether a physician is at risk of making a mistake. Analyzing this data on a patient level could provide insights for healthcare workers on an individual’s true situation, emotional or economic issues that could be affecting their wellbeing, or a full picture of their health and lifestyle.

In addition to traditional sources of data outlined above, other sources of data, including the temperature of rooms, could be analyzed and correlated with data on patient recovery, length of stay in the hospital, diet, and other factors, to optimize the environment and ensure the speediest recovery and most effective treatment for patients. Here, too, AI systems can be used to structure the huge resources out there that could yield life-saving insights for millions.

Road and Vehicle Safety: Vehicles today have dozens of sensors, gathering data on everything from speed to atmospheric conditions to traffic. Data is uploaded to computers (onboard or cloud-based) for analysis and quick turnaround, cautioning drivers when they get too close to the vehicle ahead of them or of dangerous road conditions. But again, the data that goes unused could be used to make driving safer and more efficient. For example, AI systems can correlate data on merging traffic with vehicle collision prevention settings. Using machine learning, the systems could provide ever-greater degrees of safety, ensuring that vehicles merge into oncoming road traffic in the safest manner possible. AI systems that structure and analyze this data could help save lives.

Fleet managers could benefit from unstructured data as well. Currently, fleet management systems analyze structured data on speed, driver safety behavior, and routes. But unstructured data — properly “treated” with AI structuring and analysis systems — too, could be used to understand the influence of atmospheric conditions on driver behavior, the connection between physical road conditions and vehicle depreciation, and provide a full picture of how preventive maintenance can ensure safe operation of vehicles, by correlating dozens of pieces of data with vehicle and driver performance.

AI, even in its currently limited analytic capacity, has improved life in numerous ways — but it will have an even bigger impact in the coming years when it can include this currently unstructured data coming from millions of sensors. By AI structuring the data from the sensors reading and recording the real world, the hard problems of the real world will benefit from the existing AI solutions that are already working really well in the structured world. This will immediately boost the number of problems that can be solved by the AI solutions that have already proven themselves in traditional AI domains.

Dr. Roman Sandler is the CTO and cofounder at Ravin AI.