This white paper describes a Dell Technologies DellNLP, an in-house Natural Language Processing (NLP) product. It discusses how DellNLP leverages machine learning and artificial intelligence to derive insights from text data within Support Services and helps improve the customer experience.

• A reduction of about 17% for call handle times
• A centralized, scalable solution for processing text data in various applications, resulting in reduction of about 50% in development time
• Improved customer experience by modernizing multiple touchpoints in customer journey

Due to massive amounts of data and the technical challenges associated with it, analyzing text data is an increasingly difficult, tedious, and time-consuming task. Accurate analysis frees up time for business analysts and helps improve the customer experience.

DellNLP helps solve these text data problems by adding a layer of structured data on top of text data, allowing business analysts to parse more than 1,000 documents in minutes rather than 500 documents a week.

The purpose of this document is to introduce Dell Technologies DellNLP and provide an outline of its purpose and benefits, along with user reaction information.

This document is intended for data scientists and business analysts working with textual data.

Dell Technologies and the authors of this document welcome your feedback on the solution and the solution documentation. Contact the Dell Technologies Solutions team by email. 

Authors: Asanka Wasala, Mridul Garg, Prashant Shaw, Anish Arora, Shivani Singh, and Rohitt Punjj

Contributors: Kshitiz Khatri, Evangelos Giakoumakis, Amanda Heirderman, Kate Brierley

Note: For links to additional documentation for this solution, see Dell Technologies Solutions Info Hub for Artificial Intelligence.

Figure 1.            Real-world examples of spelling and grammar issues

Spelling and grammar mistakes are problematic for NLP tasks. Some mistakes also have a negative impact beyond the world of computing, and spelling issues may even end up costing business millions of dollars in lost revenue. The following figure shows real-world examples of spelling and grammar issues.

An analysis of a random sample of 1,000 textual symptom entries by the customer support agents revealed that a staggering 15% of all entries have one or more spelling mistakes in them, which means about two out of every 10 entries have one or more spelling errors.

Noisy and erroneous text make the automated processing of textual data difficult. Noisy text is text with any difference in the surface form from the intended text, including abbreviations, acronyms, and more. It has been proven to reduce the performance of various automated text processing applications and algorithms including the following:

• Semantic and syntactic parsing
• Information retrieval
• Machine translation
• Text classification

Research indicates that addressing these issues by pre-processing text help improve the accuracy of many NLP applications.

Text pre-processing is usually a prerequisite for NLP tasks such as speech processing, information retrieval, and text classification, as shown in the following figure.

Figure 2.            Text pre-processing: A common step in NLP applications

Automated processing of textual data is increasingly important within Dell Technologies. Tools employ various ML and Deep Learning (DL) techniques to extract useful information that inform and streamline business operations. NLP applications implement their own bespoke text cleaning and pre-processing techniques in various levels of maturity.

Still, most of these applications employ primitive key-value based mapping techniques for spelling corrections, which can only deal with a limited number of vocabulary items contained in a database. Moreover, different teams working on similar applications can duplicate the same text-preprocessing functionality, as shown in left-hand side of the Figure 3 in the Methodology section. 

This issue of different teams having to develop same text-processing solutions over and over again highlights the need for a central, robust, high-performance automated query correction and text pre-processing API for processing textual data logged within Dell Technologies.

DellNLP solution facilitates the extraction of insights from text by proposing a central scalable API, including a novel automated query correction system that addresses various text pre-processing challenges and requirements. The main benefit DellNLP provides is incorporating Dell context which is missing from industry solutions and usually trained on open text corpora such as Wikipedia.

A novel automated query correction algorithm created for the project primarily performs detection and automated correction of spelling errors while minimizing false positives.

The solution was benchmarked with other widely available solutions including NLTK and spaCy.

In comparison, DellNLP provides:

• Accuracy gains of more than 10% over NLTK and spaCy, reducing spelling errors from 15% to 2%
• Major performance gains with a processing time of 1-2 minutes compared to 3-4 hours 

DellNLP can be immensely useful for current text processing applications as well as future applications, such as chatbots. Further, the algorithm and resource derivation process can be generalized. The same algorithm has the potential to process similar text found in other industry or academia contexts outside of Dell Technologies. The following figure compares the old and new processes.

Figure 3.            DellNLP as a central text pre-processing engine

Key tasks associated with building NLP solutions include:

1. Query and spell correction

2. Text normalization including query and spelling correction

3. Extracting keywords and named entities from text

4. Data labeling for building ML models

5. Building highly accurate DL models for tasks such as classification, text summarization, natural language understanding, and natural language generation

6. Predictive text for features like auto-complete and Smart Compose

DellNLP implements several text pre-processing functions and incorporates NLP resources such as n-gram dictionaries and word embeddings trained on Dell data.

The primary task of DellNLP is to automatically correct spelling mistakes. The other tasks implemented include text sanitization, predictive text capability, and retrieval of semantically similar terms.

The subsequent sections offer details on design decisions and algorithm components, some of which are included in the following figure:

Figure 4.            DellNLP consists of seven key modules to help extract insights from text data

Automated Query Correction component - DellSpell, is a key DellNLP algorithm component that involves the detection and auto-correction of spelling errors. Several exploratory analyses helped identify key issues found in textual data within Dell Technologies.

The analysis revealed that the following types of issues are prominent in Dell Technologies text:

Addressing these issues meant investigating readily available libraries that perform automated spelling correction. The SymSpell library could handle issues such as spelling mistakes, compounds, and decompounds, but was not expected to work on regular Dell Technologies text because of proprietary and internal terminology that does not exist in SymSpell’s language resources. This led to the creation of custom language resources from Dell Technologies data that are compatible with SymSpell. The key language resources developed include a unigram database and a bigram database. 

Still, even after constructing SymSpell-compatible dictionaries, empirical analyses showed a substantial number of false positives, where false positive means the algorithm unnecessarily correcting a correct term, leading to an incorrect term. The SymSpell word segmentation feature also has low accuracy when operating on Dell Technologies data. 

In response to these issues, the language resources were leverages to derive and devise custom algorithms to perform compounding and decompounding tasks.

Minimizing false positives required a mechanism to semantically validate syntactic corrections proposed by the SymSpell module. For this purpose, a word embedding model was trained and devised as an algorithm to predict spelling corrections based on word embeddings trained on Dell Technologies data. This completely novel algorithm acts as a key component that reduces false positives.

The development of these functionalities led to several useful Python functions that diverse teams can leverage in various NLP applications, leading to the decision to expose these functionalities through a package and an API, later named DellNLP.

The following figure shows the main components of our algorithm, some of which are discussed in detail within this document:

Figure 5.            DellNLP auto-correction algorithm steps

Step 1. Expansion and correction of contractions

The concordance analysis of a large amount of Dell Technologies text data helped identify mappings for contraction expansions. This analysis also used trained word embeddings to identify terms like “doesn” instead of “doesn’t,” for example.

The first operation that DellNLP performs on text is expansion of contractions. This is based on the simple but efficient use of compiled regular expressions. Any contractions found in the input text are corrected or expanded with the aid of derived mappings, as shown in the following figure.

The current mapping list consists of 270 contractions similar to this sample list.

Figure 6.            Contraction mappings

Step 2. Dell Technology terminology expansion and correction

To address Dell-specific terms within text, a mapping was leveraged and constructed with the aid of concordance analysis to expand frequently used abbreviations by support agents.

The default mapping list includes more than 900 Dell Technology terms with an option to augment these mappings depending on project requirements. DellNLP also supports custom language packs that can be installed separately.

A sample of Dell Technology terminology expansion and correction mapping is shown in the following figure:

Figure 7.            Dell Technologies abbreviations and acronyms mapping

The following table illustrates the application of these mappings:

Steps 3 and 4. Compounding and decompounding

Exploratory data analysis revealed that some support agents pay less attention to the proper use of spaces between words or proper word segmentation due to time pressure. In these steps, the algorithm determines the appropriate placement of spaces between words in a sentence. 

By leveraging the collated bigram and unigram frequency information, the probability of segmenting or amalgamating given terms is computed. For example, given the term “lap top,” the algorithm determines whether the typical usage is “laptop” or “lap top.” Next, depending on the probability of segmenting the word, these modules either break the term into several terms (as in the bigram dictionary) or amalgamate it into a single term (as in the unigram dictionary). 

The following illustrates the application of these corrections:

Step 5. Individual term correction

This algorithm is the most complex component of the project and includes a novel, newly created word-embedding-based spelling correction algorithm. Here given an incorrect term, we use spelling corrections by four different algorithms independently and select the term suggested by the majority of the algorithms as the final correction. The details of the algorithm are given below:

1. After carrying out Steps 1-4, the algorithm consumes each term as a separate token and analyzes each token for a possible non-word.
2. Next, three separate functions available through SymSpell are employed for obtaining possible syntactic corrections for the non-word.
3. In parallel, the custom word-embedding-based spelling correction algorithm is also executed. The latter suggestion is primarily used to semantically validate the SymSpell syntactic suggestions.
4. Finally, a simple decision tree determines the final term correction.  This decision tree mainly implements a voting mechanism where, if suggestions by multiple independent methods agree on the same correction, it will be used as the final term correction.

The algorithm also segments terms if previous bigram or unigram analysis failed to do the correction. Empirical results show that this novel algorithm minimized false positives but increases recall, including the detection of true misspellings, in Dell Technologies text. The following figure illustrates the novel term correction process:

Figure 8.            Individual term correction by different modules

The invention of the word-embedding-based spelling correction method, which determines semantically accurate spelling correction for a given term, uses fuzzy matching techniques and meta-phone features of the terms under analysis. 

The following table illustrates the outcome of this step:

Step 6. Dell Terminology expansion and correction

As a result of individual term correction, some terms may need to be segmented that did not get resolved during the compounding and decompounding stages. An example of this is “optiaio” in the figure below. Terms like this undergo the same terminology mapping as described in Step 2.

The following table illustrates the outcome of this step:

Step 7. Repetitive phrase deletion

Exploratory data analysis showed that repetitive phrases are a common issue in Dell Technologies text. These repetitions may occur due to database augmentations, by appending or inserting data into same table, for example.

Since repetitive phrases usually add little value in semantic parsing, the repeated phrases are removed in text using simple but effective regular expression technique.

The following table illustrates the outcome of this step:

The following figure shows how DellNLP handles individual cases as implemented in a notebook.

Figure 9.            DellNLP auto-correct function

Optimizations

As highlighted in the preceding figure, the auto-correct algorithm performs several complex operations to detect and correct various issues found in Dell Technologies text. The initial implementation of DellNLP primarily focused on accuracy and less on performance, yet due to the increasing demand for use in real-time applications, there was a focus on identifying algorithmic optimizations.

The optimizations included comprehensive time complexity analysis of individual functions, including profiling of each function. Appropriate steps were then taken to optimize code performance, including optimizing O(n2) operations into O(n), the use of appropriate data structures, and more. 

The above steps failed to reach desired performance targets (for example, 200-400ms for processing an average sentence, including API call time, which was longer than desired), the algorithm was redesigned as a Cython implementation. This redesign resulted in a more than 50 percent faster performance, outperforming existing Python pre-processing algorithms in terms of speed within some projects.

The current implementation in production is the Cython-based DellNLP implementation. Subsequent optimizations focused on code maintainability, such as identifying and implementing key unit test cases, linting (with 10/10 rating), and PEP8 formatting of code.

Systematic derivation of language resources

Several language resources were collated and built for this project, such as unigram and bigram dictionaries. As a first step to the systematic derivation, textual data was collected and a text corpus was compiled, primarily containing text logged by Dell Technologies support agents and service providers. The collected data was later used for training FastText-based word embeddings due to its ability to provide embeddings for even out-of-vocabulary data.

Next steps involved identification of Dell Technologies terminology using the trained word-embeddings to retrieve semantically similar terms. For example, if the given term was “cx,” similar ones such as “cust”, “customer”, “cst” are retrieved.

A list of unique words found in the textual corpus was also derived. This list originally contained more than 198,000 terms, but any terms found within an English dictionary were filtered out, at about 80,000 words. Only 28,248 words were common between the Dell Technologies unique word list and English dictionary. The rest of the entries contained of Dell Technologies-specific terms, multilingual entries, and erroneous or misspelt terms.

These 170,000 entries could be further segregated into Es, Pt, It, De, Fr language terms and short terms with just two to three letters. The remaining entries (~132,000) consisted mainly of terms with genuine spelling mistakes, specific to Dell Technologies, person names, place names, erroneous compounds, and some transliterated terms. The following figure outlines term analysis.

Figure 10.        Dictionary construction - term analysis

Identifying Dell Technologies-specific terms meant reviewing ~132,000 terms to construct a custom dictionary. After filtering out less frequently occurring terms, the rest were manually reviewed to identify legitimate terms including acronyms, abbreviations, and products to be included. The following figure shows a sample of the word review process.

Figure 11.        Manual analysis of unique words

Concordance analysis and word-embeddings were used to examine the usage of words in context, as shown in the figure below:

Figure 12.        Concordance analysis of the term "weve"

These steps resulted in the following:

• A Dell Technologies dictionary containing more than 83,000 Dell Technologies-specific terms
• A word-embeddings model
• A bigrams dictionary with more than 303,000 words

These systematically derived resources are leveraged in algorithms and models and are consistently being updating, with new Language Packs released as necessary. This ensures that model predictions are updated to reflect any changes or new patterns in the data.

As the requirements of DellNLP evolved, a text sanitization functionality was incorporated into the library and API. This functionality can be used for the following: 

• Anonymization of personally identifiable information
• To reduce cardinality of text by replacing non-relevant terms or textual content with pre-defined tokens (for example, replacing dates such as 01/10/2020 by <DATE> token)

The text sanitization functionality is primarily implemented using regular expressions, however, the capability has been empirically evaluated on random samples of Dell Technologies text data to ensure these patterns cover more than 70 percent of data.

Currently, the sanitizer supports following content:

• URLS
• Emails
• Currencies
• Phone numbers
• Dates
• Service request numbers

DellNLP provides a highly configurable sanitization class for implementing similar cleaning steps in NLP pipelines.

DellNER is a custom Named Entity Recognition (NER) model that helps identify pre-defined entities in text data, as shown in the following figure. Currently, DellNER helps identify the following entities:

• Hardware entities such as motherboards, hard disk drives, memory
• Software entities such as operating systems, BIOS, Microsoft Office
• Diagnostic tests carried out by technical support agents such as ePSA, BIST
• Diagnostic codes generated during troubleshooting such as:

2000-0141, amber white-off-of

• Technical issues encountered during troubleshooting such as no power, slow performance, no video

Figure 13.        DellNER helps identify pre-defined entities in text data

DellNER was trained on a corpus of more than 1 million case logs generated by technical support agents during troubleshooting for the client business. The training data was bootstrapped using a combination of static term lists and regular expressions. For evaluating model performance, a sample of 1,000 case logs was manually labelled. The model performance is shown in the following table:

Note: DIAGCODE was not included in test results above since there was a very small sample of it in the test set (<10 occurrences).

Dell Technologies Support Services must analyze various text data sources to provide a better customer experience. Analyzing text data is challenging due to scale as well as the technical challenges associated with its inherently noisy nature. Analyzing this data is also a tedious, time-consuming process for business analysts.

DellNLP helps solve these text data problems by adding a layer of structured data on top of text data, allowing business analysts to go from parsing approximately 500 documents a week to more than 1,000 documents in minutes. This helps analysts focus on solving customer problems rather than spending time mining data.

DellNLP has led to the following outcomes for our business partners as well as Dell Technologies customers:

Business Impact

• 22% fewer No Fault Found (NFF)
• 6% lower Repeat Dispatch Rates (RDR)
• 46,000 repeat call reductions
• 500-person hours saved every month to label text data using DellNLP Elixir
• 12x increase in cases analyzed for quality assurance improvements
• 200-person hours per application saved using real-time text processing service, eliminating redundant solutions developed for each AI/ML application

Customer Impact

• 17% reduction in resolution time
• 7% improvement in issue identification
• Improving customer experience by driving innovation of new applications, including online chatbots for troubleshooting, automated case prioritization, and early intervention to avoid case escalations

“I was impressed by DellNLP because it gets so many things right: solves a key opportunity, utilizes advanced data science techniques, is re-usable, running on a cloud-native stack and production deployed. It sets the path for us to build a blueprint to help more data science teams build, scale, and deploy additional solutions like this. DellNLP is contributing to advance Dell data science to the next level.” -Dell Technologies Director, Enterprise Data Science & AI Capabilities

DellNLP is a building block in the development of innovative applications across Dell Support Services. Multiple touchpoints across the customer journey have been modernized, such as automated issue identification, next best action recommendations to resolve customer issues, and identifying failure themes for improving quality assurance.

In continuing to modernize and digitalize the customer experience, the next priority is to enhance coverage to support both multimedia and multilingual data. Work is underway to expand the solution to process the top five non-English languages and voice data by 2022, and offer DellNLP as a cloud native service to Dell Technologies customers.

The following external documentation provides additional information:

• Coughlan, S. 2011. Spelling mistakes 'cost millions' in lost online sales.
• Bernoff, J. 2016. Bad Writing Costs Businesses Billions.
• Subramaniam, L. Venkata, et al. 2009. A Survey of Types of Text Noise and Techniques to Handle Noisy Text - Proceedings of The Third Workshop on Analytics for Noisy Unstructured Text Data.
• Apostolova, Emilia, and R. Andrew Kreek. 2018. Training and Prediction Data Discrepancies: Challenges of Text Classification with Noisy, Historical Data.
• Jianqiang, Zhao and Gui Xiaolin. 2017. Comparison Research on Text Pre-processing Methods on Twitter Sentiment Analysis.
• Camacho-Collados, Jose, and Mohammad Taher Plehvar. 2017. On the Role of Text Preprocessing in Neural Network Architectures: An Evaluation Study on Text Categorization and Sentiment Analysis. 
• Eler, Danilo Medeiros, et al. 2018. Analysis of Document Preprocessing Effects in Text and Opinion Mining.
• Banushka, Aliaksandr, and Petr, Hajek. 2019. The Effect of Text Preprocessing Strategies on Detecting Fake Consumer Reviews.
• Haddi, Emma, Xiaohui Liu, and Yong Shi. 2013. The Role of Text Preprocessing in Sentiment Analysis. 
•  Etoori Pravallika, Manoj Chinnakotla, and Radhika Mamidi. 2018. Automatic Spelling Correction for Resource-scarce Languages Using Deep Learning.
• Huang, Yinghao, Yi Lu Murphey and Yao Ge. 2013. Automotive Diagnosis Typo Correction Using Domain Knowledge and Machine Learning.
• Freitas, Diamantino, et al. 2002. A Project of Speech Input and Output in an E-commerce Application.” International Conference for Natural Language Processing in Portugal.
• Xie, Xia, et al. 2020. A Novel Text Mining Approach for Scholar Information Extraction from Web Content in Chinese.
• Zaki, Taher, Mohamed Salin EL Bazzi, and Driss Mammass. 2018. An Evolutionary Model for Selecting Relevant Textual Features.
• Galitsky, Boris. 2019. Chatbot Components and Architectures.