Can AI diagnose problems like a human?
by Erik Pfannmöller and Max Breitsprecher
Artificial Intelligence (AI) can be great at answering questions. Ask Google who invented 3D printing and boom! You get your answer: “Chuck Hull”. Not many people who could answer this question – especially not in a fraction of a second.
But what if you were to ask Google a different question? Let’s try it with, “why is my 3D printer not working?” It turns out that all you’ll get will be the standard list of Google search results.
Why the difference? One way of looking at it is that they are different types of question. While question one can be answered with a single fact, it requires a diagnosis process to come up with a good answer for question two.
Your first thought might be, “Okay, but diagnosing 3D printer issues is a very uncommon case. But is this really true? Many interesting tasks are in fact problems that require diagnosis. Here are a few examples:
- A doctor diagnoses your illness.
- The mechanic diagnoses a fault with your car.
- A technician diagnoses why your Internet router is not working.
- Customer service agents “diagnose” why your package hasn’t been delivered on time.
How humans diagnose problems
All four examples share some common characteristics. Firstly, the user starts out with only a vague idea of the problem. Perhaps he or she might be able to observe some the symptoms. Often, the user struggles to even formulate the problem, let alone understand the underlying causes. Finding an appropriate solution would cost him or her a great deal of time and effort.
Secondly, the user seeks the advice of a subject specialist. Training and experience provide the expert with an understanding of the relationship between problems and symptoms. The expert knows which questions to ask, and when to ask them. Furthermore, he or she knows when sufficient information has been gathered to make an informed decision and provide a solution.
Thirdly, all four of the example situations are highly repetitive. Every day, millions of illnesses are diagnosed, cars are fixed, connection failures are remedied and customer problems solved.
How AI systems diagnose problems
Just a minute! If a task is highly repetitive then surely it would be correct to assume that there’s a good chance we could automate it using AI? Exactly. This is precisely what numerous companies are trying to achieve. Actually, Google gave it a shot in 2015. They attempted to automate IT Helpdesk troubleshooting using multi-layer neural networks (Deep Neural Net). To achieve this, they trained a long-term/short-term memory network with 1024 memory cells, using a stochastic gradient process on a dataset of 30 million tokens. The outcome was as follows:
While the achievement by the Google team was impressive and some of the automatically generated replies made sense, AI was clearly not ready for use in a real-world Helpdesk situation. Why was that? What makes building a high-quality diagnosis system so difficult? In this respect, two main challenges need to be overcome:
Two significant challenges
Since the user is not an expert on the subject, the AI system needs to initiate the dialogue and move it forward. It does this by asking questions to gather additional information about the problem. However, herein lies the challenge: Which questions should the AI ask? Moreover, at which point in the conversation should it ask which question?
Let’s look once more at the dialogue example from earlier. As a rule, a good question to ask might be, “is it a windows machine?” However, this makes no sense if the user has already stated that the operating system was Linux.
Therefore, to generate a good question, the machine needs to properly consider multiple factors:
- What information do I already have about the situation?
- Which answer would help me in the elimination of potential solutions?
- Which answer would direct me to common solutions faster?
These are only a few of the important factors to be considered. It turns out that asking the right question is actually the hardest part of the diagnosis process.
On top of this, AI systems face a second challenge. They need to decide quickly when they should stop asking questions and concentrate on the solution. Even if threshold values are trivial, some interesting questions arise under real world conditions:
- Should the AI system always endeavour to find the correct solution, even if this would entail a long and protracted questioning process?
- Or should it take a risk and come to solution faster? Could the required behaviour perhaps be dependent on the circumstances (medical diagnosis vs. 3D printer troubleshooting)?
- Should all solutions be treated equally? Or should the AI system discriminate against unusual solutions?
- Should the AI system trust all user answers (equally)? What happens if the user makes a mistake?
Artificial intelligence experts from fredknows.IT
Here at www.fredknows.IT, we are of the opinion that both of these challenges might be difficult for the current technological capabilities of Deep Neural Nets – especially if only a few thousand training samples are at hand. However, we’d like to make one thing clear: we are convinced that Deep Learning will get there. It might be in 5 years’ time; perhaps it will be 10. But until this happens, we need another approach to building Artificial Intelligence systems that have the capability to diagnose almost anything.
This is why we have launched a platform upon which anyone can develop an AI expert. Our typical customers are businesses that receive hundreds of troubleshooting requests every week. The platform allows them to build smart AI experts that help their users to help themselves. What are the benefits?
- Users receive instant 24/7 and cost-free troubleshooting support.
- Companies can keep support costs in check.
- Furthermore, service quality is enhanced because the AI system learns and improves with every user interaction.
To give you an impression of the look and feel of our diagnosis AI systems, we have created two demo projects for you (see below). The first one lets you challenge “Zookinator”. Think of an animal and see if Zooki can guess correctly what it is. It’s a lot of fun, we promise!
The second AI system has been trained in 25 common 3D-printer problems. So perhaps AI can find an answer to the question, “why is my 3D printer not working?” after all.