Knowledge is Power - Business Analytics using Text Analysis

Knowledge is Power – Business Analytics using Text Analysis

In the article Artificial Intelligence Powers Quality Journalism we already discussed the opportunities Artificial Intelligence (AI) holds for journalism. Thanks to robot journalism, for example, hyperlocal weather reports, real-time sports news and stock exchange listings are feasible.

But not only consumers, also journalists benefit from AI; machine learning and other technologies are capable of sorting and classifying huge amounts of data for their self-written texts. That way, entirely new analyses are feasible and it suddenly opens up a new view of the world and its interrelationships. The ability to process gigantic amounts of data is also used in other areas.

Rethinking communication

Media consumption has changed as a result of digitalisation. The general public has divided itself into many small groups. What used to be a reader’s letter at most, i.e. a consumer's reaction to the media offer, has now multiplied. The former one-way street has become a busy motorway; the crowd is answering. Highly accurate researched content is placed in "aggregators” like Facebook next to or above any personal status reports. Mass media becomes "media of the masses", as Nathalie Wappler-Hagen, programme director of MDR once said. [1]

That matter of fact makes it necessary to rethink and reorganize the communication with the reader. At the Munich Media Days 2016, Jochen Wegner, chief editor of ZEIT ONLINE, said that 100,000 readers comment on articles every week. Each of these comments needs to be read to sift out racist, sexist or other criminal opinions. "It would be nice if something could help us; where exactly should we look?"

A mathematician then designed a digital helper for the editorial staff. "Several neural networks are now talking to each other analyzing comments." The result: what the machine marks as dubious is actually questionable with 75 percent certainty and is then reviewed by a colleague. Does this make man superfluous? Or does it create conditions for redundancies? "The opposite is the case," says Wegner, "the moderators can finally concentrate on what is really important and look at specific contributions calmly, instead of reading thousands a day. [2]

However, it is not enough to react to certain words. What is important is the context, the intention. If someone seems to be too enthusiastic about something, maybe the comment was meant to be ironical and for that reason a satirical undertone was used by the author. Intentions that humans understand almost immediately and intuitively, must be learned by machines at great expense if you want to deploy them in customer service.

A firework of different technologies

The Berlin-based start-up parlamind GmbH has developed a technology that is able to conduct human-like dialogues. It enables machines to enter into a dialogue with customers and interested parties in a human-oriented manner. Parlamind can be booked as a service and then runs as an additional team member in the customer service team. It can receive requests and understand them at different levels: Why did the customer write to the customer service? What was the request about? What's the user’s mood? Are they upset? Are they expecting an apology? Which factual information do they need?

Dr. Tina Klüwer, co-founder of the start-up, says: "We can discern precisely the reason for contacting. We can also recognize if there are several reasons. The machine then generates different answers, as the case may be."

Parlamind works with three main categories. Klüwer explains: "One of them is sentiment, another analyses the subject. Afterwards we perform a basic contact analysis. It recognizes the users’ intentions in fine granular form. It is about how customers complain. Or if they would like to change their address for the next delivery and the like. At this point it becomes very specific and complex. In short, we're flaring off a huge firework of different technologies."

The company works with neural networks, but also uses classical machine learning such as Maximum Entropy and Bayesian Networks, all combined into one machine. "The challenge is to make a maximum number of manifold conversations machine-readable. The machine has already read three million conversations, most of them originating from e-commerce customer service. It reads along another hundred every day. The more data it reads, i.e. received messages and their replies, the better the models will become," says Klüwer.

Facebook and its text analysis tool DeepText

Facebook also uses a similar technology to interpret and understand status updates and context. "According to the manufacturer, DeepText recognizes context and the user’s intention almost as well as man. But unlike humans, the AI is able to process several thousand posts per second in more than 20 languages. The programmers are using Deep Learning and in addition, drawing on Recurrent and Convolutional Neural Networks.

A Recurrent Neural Network (RNN) is a type of Artificial Neural Network in which units are connected and thus form a cycle. This enables a dynamic temporal behaviour. Unlike Feedforward Neural Networks, RNNs can use their internal memory to process any sequence of input. They can be used for segmented, connected handwriting recognition or speech recognition. As a result machine learning takes place at both levels; the word and drawing level.

This approach has the advantage of the AI not needing to learn from humans which data means what by annotating the data, which is extremely time consuming. Facebook can roll out the trained models scaled to its infrastructure at the flick of a switch. [3]

TUI gets to know its customers

Whilst ZEIT Online fulfills its journalistic mission and involves the reader - which is basically also a marketing measure - Facebook wants to get to know its users better in order to be able to sell more and better. TUI, the giant in tourism, also uses intelligent text analysis for marketing in order to examine large amounts of text according to different questions quickly and automatically. The company had conducted an extensive survey of its nationwide sales network, resulting in a high number of comments and feedback in open text form; the evaluation would have been extremely time-consuming for humans.

The industry leader therefore commissioned Neofonie GmbH, a specialist for semantic text analysis, with automated evaluation. Peter Adolphs, Head of Research at the agency in Berlin, reports: "Using sentiment analysis, all comments could be examined for a positive or negative statement about various aspects of travel booking and visualized for all scenarios. Methods such as text mining, data analytics and product review mining were deployed. We gained a profound overall picture of the travel agents’ feedback."

Business analytics

The expert abstracts fundamental options for text analysis procedures from specific assignment: "We analyse unstructured customer and user data thus generating data for corporate management to help understand and improve one's own performance. It's about business analytics by using text analysis," says Adolphs. This can be useful on different levels. A supplier of consumer goods for example commissioned Neofonie to help them evaluate their product reviews. "We were asked to find out what product aspects users are talking about: Is it handling, design or sound level? From this user-generated content, which basically consists of countless strings, we have to obtain relevant information."

Companies need to understand what their users really ask for and require. Only then will they be able to permanently hold their ground in the ever faster changing markets. Of course you can rely on your luck, but it’s more sustainable to work with data. "Look at a company's exchange rates, for example" says Adolphs, "what percentage of goods are returned? This data is structured and legible, but does not necessarily provide deep insight into reasons. You can only discern the whys and wherefores from all the information around it: in social media channels, in forums, in e-mails that customers send to the company. This huge flood of content can only be handled by text analysis."

It’s an old problem; knowledge is power, but how do you make all this information accessible and thus use your power? Companies are sitting on data silos; unstructured databases, CRM data, delivery addresses, instructions, white papers. All this information is already available to them in their own databases, but it isn’t linked. Text analysis provides uniform access to this information. Adolphs says with a smile: "Basically, every company only needs to create an Artificial Intelligence to google its own data. Because Google search also uses text analysis, as every good search does. You just don't notice."



[1] YouTube: Medien und künstliche Intelligenz – Wie sieht die Zukunft aus?
2016-10-29, (accessed 2018-02-25)

[2] YouTube: Medien und künstliche Intelligenz – Wie sieht die Zukunft aus?
2016-10-29, (accessed 2018-02-25)

[3] Facebooks KI liest Nachrichten der Nutzer
2016-06-02, (accessed 2018-02-25)

Add a comment

your browser is not up to date
to enjoy this website you will need to install a modern browser.
we recommend to update your browser and to install the latest version.

iOS users, please male sure you're running at least iOS 9.

Mozilla Firefox Google Chrome Microsoft Edge Internet Explorer