Technology advancements in Natural Language Processing have enabled NLP models to perform much better than humans in understanding, interpreting, and extracting information from unstructured communication channels.
Organizations use many communication channels every day to perform various activities. Emails and call center conversations are among the most common channels in any organization where people spend hours processing information.
Yet, there are inefficiencies around these conversational channels. However, it is also a source for discovering hidden opportunities for enhancements and automation initiatives.
Topic Covered in the Article
1. Introduction
Data is one of the integral parts of any organization. All the software applications, Robotic Process Automation (RPA) tools, and Artificial Intelligence (AI) models run on data. Many organizations have invested in automation to efficiently and accurately run their repetitive and high-volume processes, freeing users to focus on more value-added activities. These automation solutions and RPA platforms require structured data to process. However, large volumes of data processed in these organizations are unstructured. In addition, the most critical information is hidden between different communication channels, such as emails, chat messages, call transcripts, etc. These communication methods are identified as one of the most inefficient in the organization. People usually spend hours daily processing emails alone, causing inefficiencies.
Further, these communications are the triggers for other downstream processes. Inefficient communication leads to inefficient execution of downstream processes. Conversational Intelligence is a solution to eliminate these inefficiencies by understanding unstructured data and making them meaningful using state-of-the-art technologies.
2. What is Conversational Intelligence
Advanced technology is used in almost all complex areas of automation. Some of those include processing documents that require Optical Character Recognition (OCR) and many machine learning models to process documents, text classification, language analysis, object detection, and many more. The technology has given so much benefit in improving the efficiency and accuracy of these processes. Further, Natural Language Processing (NLP) has enabled users to implement RPA on processes requiring human language understanding without needing someone to interpret the communications to the automated processes. NLP is one crucial technology that supports understanding conversations carried out by business users across all industries. However, extracting vital information from communications such as emails, chat messages, and call logs and analyzing them comes with challenges.
The use of technologies such as Artificial Intelligence (AI), Machine Learning (ML), Deep Learning, and Natural Language Processing (NLP) to understand unstructured conversational data and draw meaningful insights in a structured form is known as Conversational Intelligence.
Many conversational intelligence platforms were introduced in the last few years to support and extract meaningful information from communication data. These platforms are designed to integrate easily with other applications and perform mining on communication data to uncover process improvements and automation opportunities.
Common Difficulties in Analyzing Conversations and How NLP Can Overcomes These Challenges Better than Humans
Language analysis on communications is not as straightforward as language analysis performed on other mediums like books, official documents, websites, and blogs. These mediums follow standards in representing information clearly for all users. However, communication between people is casual and may contain small talk, emotions, idioms, and figures of speech. These could be a challenge in understanding and extracting accurate information.
Keywords are essential in conversational analysis to understand and accurately classify the information extracted. However, identifying such keywords can be tricky due to the nature of the conversations described above.
In general, the NLP models are tested against some of the following:
Named Entity Recognition (NER): Which word or group of words in a sentence are proper names of people, organizations, amounts, dates, locations, etc.
Semantic Similarity: Understand whether a given sentence has the same meaning as another or if it contradicts one another.
Conference Resolution: Determining linguistic expressions that refer to the same real-world entity in natural language. For example, identifying the phrase "it" in a sentence with multiple real-world objects and determining which object it refers to.
Understand Sentiments: Determine whether the data represents positive, negative, or neutral feedback.
The interesting fact is that NLP has reached a point today where it surpasses human capabilities in understanding information and overcomes most challenges.
The General Language Understanding Evaluation benchmark (GLUE) is a collection of datasets used for training, evaluating, and analyzing NLP models relative to one another. The models available today are also compared against human performance. The GLUE benchmark clearly shows how the models outperform human understanding of natural language.
These advanced capabilities of NLP have enabled conversational intelligence platforms to understand human conversations and extract key information effectively.
Benefits of Conversational Intelligence
Monitor the performance of support agents in organizations
Get more insights for product development, process improvement, and automation
Reduce customer churn
Increase productivity of customer-facing departments like call centers
Ability to introduce continuous improvements in processes and ways of working
3. Conversational Intelligence with Re:infer
Re:infer is a conversational intelligence platform powered by state-of-the-art NLP technologies that convert unstructured electronic textual communication data into structured data. The structured information extracted through Re:infer generates insights and identifies improvement opportunities.
It is a no-code platform that supports creating fully customizable machine-learning models within a shorter period without ML expertise. The inbuilt analytics allows users to create customizable dashboards to visualize business conversations to generate in-depth insights. Further, pre-built connections of Re:infer enable easy connection with leading enterprise communication platforms and RPA solutions.
Re:infer is GDPR compliant, ISO 27001 certified and allows encrypted data storage on standard GCP practices
What Data Re:infer Works WIth
Re:infer works with the following data types:
Emails
Tickets
Chat messages
Customer feedback
Call transcripts
However, it does not support input data such as images, email attachments, Excel files, and information on websites on other media. These are not supported because they require extra technologies such as OCR and other functionalities to extract information.
Re:infer only focus on electronic textual communication data for analysis
Stages of Re:infer
Every communication data passed into Re:infer goes through a series of stages to generate structured, meaningful information as the output.
Ingest & Store: Connect with various unstructured communicated data sources and pull the data for processing. Some of the input sources are Microsoft Exchange, Outlook, Salesforce Data, and any structured data via Re:infer API integration
Clean & Parse: Parse and understand communication using NLP technology.
Unsupervised Learning: Using pre-trained models with unsupervised learning, semi-supervised learning, and active learning are used to extract entities and intents from the input data
Discover & Train: Define and train the model with customized taxonomy of entities and labels to capture important information using semi-supervised and supervised learning
Report & Analyze: Use customizable dashboards available in Re:infer to visualize, monitor and generate insights and automation opportunities.
Validate & Deploy: Use advanced yet easy-to-use features of Re:infer to monitor the performance of the models and continuously train them for better predictions. In addition, use the Re:infer API to integrate with downstream applications like RPA or data pipeline applications like Kafka to perform analysis.
Re:Infer as a conversational intelligence platform uses a few concepts to understand and process the data. The terms mentioned in these concepts are common across many NLP machine-learning models.
Core Concepts of Re:infer
Similar to UiPath Automation Cloud or Automation Suite, Re:infer also follows a similar Organization and Tenant structure. It is important to understand the architecture and the concept behind it to plan and prepare for the conversational intelligence initiatives using Re:infer.
The following diagram illustrates the architecture at a high level.
An Organization can consist of multiple Tenants. Each tenant refers to a different environment, like Production, UAT, and Development.
Users and projects are defined inside specific tenants. Users who have access to a specified tenant may not have access to other tenants.
Source:
The Projects may contain more than one Source for data. These sources could be direct integrations with emails, call center logs, or data gathered through Re:infer API integration.
Comments:
A Souce may contain thousands of data elements. A Comment refers to each piece of text communication within a Source. A Comment may also contain additional information describing what type of data it represents.
DataSet:
A Dataset may connect with multiple data sources to label the data. Each dataset holds labeled data.
Entities:
Entities are elements of structured data which can be extracted from the dataset. A Comment may have zero or more Entities defined. Entities include data points such as organizations, people, email addresses, URLs, amounts, dates, and many more. You may find pre-trained and custom entities within Re:infer.
Labels:
Labels are a way of classifying information into different types. For example, communication could be based on insurance claims, financial reporting, etc. Further, if we consider one of those, there can be more varieties of information that you can extract. For example, an insurance claim-related conversation may focus on claim confirmations, payments, accidents, etc. Re:infer uses Labels in a hierarchical structure to capture all of this information accurately.
Model:
Model refers to the actual machine learning models deployed after training. These models are continuously updated as the users label more data.
Concluding
Conversational Intelligence uses multiple state-of-the-art technologies to understand, interpret, and extract valuable information from communication data within organizations. The article explains the importance of conversational intelligence and how NLP is powering unstructured data mining, followed by a brief explanation of how Re:infer platform supports conversational intelligence and its features.
We are not yet sure how Re:infer will work with UiPath as that information is not yet publicly available. However, this is Re:infer for now :)