In the domain of tech-support and services, Knowledge Base (KB) articles are indispensable resources for both customers and support agents. These articles typically encompass step-by-step troubleshooting guides, supplemented with multimedia aids such as video guides and images, aimed at providing self-support instructions to common technical issues. For example, Dell Services is a repository of KB articles that address a broad spectrum of "How to" questions, providing general troubleshooting steps for common technical issues. There are also highly specialized articles like Product Support Quick Notes (PSQN), which tackle specific issues unique to hardware configurations or product lines. Specialized articles, such as the "Yellow Paint Defect," known only to Inspiron 7348 models, provide targeted diagnostic and troubleshooting steps for prominent issues associated with specific brands or models.

The diverse nature of these articles, coupled with the varying levels of complexity and specificity, poses significant challenges in Information Retrieval (IR). Some KB articles are designed exclusively for support agents and contain advanced diagnostic procedures that are not publicly accessible. The sheer volume of over 100,000 available articles, translated into over 20 languages, compounds this issue. Therefore, efficiently identifying the most relevant articles based on symptom descriptions or specific keywords becomes a critical task.

To address this challenge, Dell Technologies Services (DTS) introduce an improved PSQN search facility, which leverages contemporary embedding-based IR techniques. This solution is designed to pinpoint and surface the ten most pertinent PSQN articles that align with a given symptom description and brand or model specification. DTS's approach enhances the efficiency of issue resolution by enabling support agents to quickly access the most relevant information, thus reducing the time spent sifting through an extensive array of articles. Additionally, DTS has automated much of the data management process, minimizing human intervention while maintaining, and even surpassing, the accuracy of previous models. This document presents the detailed implementation of the solution, highlighting its architecture, technical details, and the substantial business value it delivers in streamlining support operations.

Dell Technologies and the authors of this document welcome your feedback on the solution and the solution documentation. Contact the Dell Technologies Solutions team by email.

Authors: Nick Caldwell, Renold Devaraj, Ahmed Hani, Kinjal Maru, Abderahmane Naidjate, Akshay Raina, Bipana Thapaliya, Ruwan Asanka Wasala

To best serve customers, Dell Services leverages in-house built, AI and Generative AI tools such as Next Best Action (NBA), which is part of a larger ecosystem of support optimization AI tools known as Contact Connect AI. The customer support journey typically begins when a customer encounters a problem with their purchased hardware, leading them to seek self-support solutions. Once these self-support options are exhausted, they contact official customer support through various communication channels. Tools like NBA have been trained with historical data to predict the best resolution based on symptom and telemetry data. The PSQN Retrieval solution supplements NBA capabilities and aids agents in troubleshooting specific hardware issues faster and more efficiently by surfacing and ranking the most relevant KB articles according to their relevance, while filtering out irrelevant ones.

Historically, this solution employed a primarily lexical-based, direct database implementation for IR, which proved to be suboptimal. It retrieved the top ten PSQN articles based solely on keyword similarity matches against a manually curated keyword set for each specific article. These articles were then shown to the support agent through the NBA user-interface. However, since many of these articles were not always relevant to the given symptom, support agents had to sift through each of the 10 articles to find the most applicable article for resolving customer issues. This suboptimal retrieval process, coupled with the need to manually search through multiple articles, introduced significant inefficiencies. DTS’s enhanced article retrieval strategy, based on semantic embedding, addresses these inefficiencies by ensuring that only the most relevant articles are surfaced.

In addition to improving the retrieval strategy, substantial optimizations were made to the PSQN article data preparation and organization workflow, eliminating error-prone and tedious human interventions in manual keyword generation and article tagging for applicable hardware configurations by leveraging Generative AI technologies. Before DTS’s new solution, experts analyzed specific articles, manually identified keywords, and tagged articles with specific hardware configuration identifiers using a manually curated product taxonomy. This solution fully automates both the article tagging process and the article content preparation for database indexing, yielding substantial accuracy gains and process improvements. Now, only the most relevant articles—up to a maximum of 10 (or none if no articles are relevant), are shown reducing the time agents need to sift through the filtered top known articles.

DTS’s improved implementation enables faster diagnosis and troubleshooting of customer issues, leading to higher customer satisfaction.

From a business standpoint, the most critical aspects of support as a service are Customer Satisfaction (CSAT) and company cost. Historically, the previous model caused many issues in DTS's support space due to the constant need to validate and ensure that the correct articles were shown to support agents. Multiple escalations and churns occur due to critical PSQN articles not showing, causing avoidable hardware or part dispatches and CSAT dropping due to high case turnaround times.

With over 4,000 PSQN articles and anywhere between 50 to 100 new articles or revisions each week, automation is essential. Leveraging machine-learning and algorithmic decision-making can expedite issue resolution and enhance confidence in accurate results. The business process to determining and publishing an article under this category is complex and has created downstream impacts, with multiple organizations in silos around the revision and selection of these documents. This leads to data spread out across sources, resulting in outdated content and misalignment.

The process that exists today is ever-changing, and this new model update is at the forefront of collaborating with various groups, slowly fixing the previously complex process. With creating a cross-functional model that affects various groups and introduces a new process, it is critical to have a shared vision and outcome. The implemented solution addresses company costs while transforming previously complex and inefficient processes into multiple streamlined workstreams across various organizations.

The process of data understanding involves analyzing and understanding the data and their respective sources that are being used. This includes identifying the strengths and weaknesses of each source and exploring ways to combine them for a more comprehensive and accurate system. An initial analysis of the existing full-text search-based PSQN article retrieval system, followed by engagement with the data teams involved, revealed that the data management and curation process was predominantly manual and multi-stage. It also had unstructured points of entry, leading to some data quality issues.

Building on this information, an Exploratory Data Analysis (EDA) was conducted, uncovering a taxonomy misalignment related to the applicable product/band hierarchy between the article content management system and the target application (where UI/system articles are displayed for support agents). This misalignment impacted the design and effectiveness of the existing search system, prompting consideration of alternative approaches. The analysis also revealed substantial data gaps, highlighting the lack of complete data understanding.

The initial EDA was conducted on the article dataset from the existing system. The EDA involved examining the distribution of values, identifying missing data, and visualizing key aspects of the dataset. The primary goal was to ensure data quality and understand the underlying patterns and distributions before proceeding with further analysis and implementing DTS’s enhanced article retrieval strategy. Specific attention was paid to columns that provide valuable information about product categories, issue descriptions, symptoms, article title text, and associated URLs. DTS's analysis revealed substantial missing data in the higher-level product category (Prod Cat1) and article content fields, each with approximately 20 percent missing values. In contrast, the more specific product category (Prod Cat2) column had only 0.9 percent missing values, while the description, technical symptoms, and article link columns, were complete without any missing values. These findings, along with less than 50 percent accuracy in the higher-level product category and more specific product category data, highlighted significant quality concerns.

Figure 1.   Initial EDA results

Subsequent analysis following engagement with data owners, it was understood that partial data is expected as not all articles are considered relevant at these product taxonomy levels. Further, analysis that incorporated the taxonomy misalignment understanding showed that where values did exist, they had a high (but not complete) 94 percent level of correctness. These insights lead to further analysis into upstream data which bridged the data gaps and were all brought into the data processing pipeline design.

DTS's solution can be divided into several key components: 1) article retrieval, 2) article thresholding, 3) article re-ranking, and 4) evaluation of the solution with the help of subject matter experts. The Implementation details section covers the infrastructure and data pipeline that prepares the system for these components. This includes the article “Tagging” and “Search data generation” elements of the article preparation for search.

Article search and retrieval at solution runtime primarily involves receiving customer issues or symptoms as a query using the NBA tool (the UI for support agents), converting these queries to semantic embeddings, and searching in DTS’s indexed database. The results are then refined by applying similarity, score-based thresholding and re-ranked by historical user feedback. Details of these processes are discussed in the sub-sections Article retrieval, and Thresholding and Re-ranking articles.

The Validation sub-section elaborates on the process DTS adopted to evaluate the performance of the new solution, comparing it to the previous implementation with the assistance of subject matter experts.

The major components outlined above are depicted in the solution overview shown in Figure 8.

Figure 2.   Solution overview

The NBA tool which forwards user queries to this solution for article retrieval, also provides product information relevant to the queried asset (such as laptop or hardware commodity). This product information includes the specific asset service tag, and the product hierarchy-related tags such as the “Precision Notebook.” The article search process begins by filtering for articles associated with specified tags. The details of this tagging process are further elaborated in the Data pipeline section.

Once articles are filtered for relevance to the product/asset, they are ranked based on semantic similarity. The embedding model plays a key role, in understanding the query once provided, for example, “Keyboard key stuck.” The model’s function is to transform (or embed) the given text, which may be in one of several supported languages, into numerical vectors which capture the semantic meaning of the text, supporting measurement of semantic similarity between the model and article texts. An Embedding Store (an efficient database for managing embeddings) performs the similarity calculation, leveraging its vector indexing functionality, calculating the cosine distance from the query embedded at runtime and the article summaries each embedded and indexed before runtime.

With articles filtered and ranked based on query relevance, they are shortlisted for inclusion in the final search results.

Thresholding filters the most relevant PSQN articles for a given query based on a cut-off value applied to the semantic similarity scores. A common baseline approach is Global Thresholding, which determines a constant threshold applied to all queries.

By setting this threshold, only PSQNs with similarity scores above a certain value are considered relevant. This means that streamlining the article filtering process and providing tech support agents with targeted, useful information pertinent to the customer’s query, enhances efficiency and effectiveness of troubleshooting.

In cases where there are many PSQNs available, thresholding helps manage the volume of information by excluding less relevant notes. This makes it easier for users to focus on the most significant articles without being overwhelmed. This improves the quality of support and overall user experience.

Applying a threshold specific to each query results at run time is an effective approach to improve the relevance and quality of the PSQN articles retrieved. While it may introduce some complexity in terms of additional overhead or latency, the benefits in terms of improved precision, user satisfaction, and adaptability to diverse troubleshooting scenarios can surpass these constraints.

As DTS supports multi-lingual input and semantic scoring, relevance scoring can vary significantly between language pairs. In addition, query specificity can vary by tool. Examples of possible queries include:

• “No boot”
• “due to battery swelling the touchpad part is floating”

Dynamic thresholds allow the system to adapt to these differences, providing more tailored and accurate results. The complexity and nature of queries can vary greatly, and dynamic thresholding is a diverse approach. It matches the complexity and commonality of the issues and the retrieved relevant PSQN articles for each query and product combination.

Statistical techniques such as Kernel Density Estimation (KDE) are employed to perform dynamic thresholding for each query-retrieved article. KDE is fitted to estimate the probability distribution of the semantic similarity scores for each retrieved PSQN article. The peaks and valleys are identified that represent the high-density and low-density regions in the curve, respectively. Change points between these peaks and valleys indicate the natural distinctions among the scores. These change points can be interpreted as cluster boundaries in the data indicating change in relevance of the retrieved articles.

The best threshold based on the KDE curve determines significant change points, percentile of scores, and more. An experimental retrieved PSQN result set and a validation set determine the best kernel parameters. These include the best kernel function fitting the data, and the best bandwidth capturing the internal structure and nuances of data.

The plot below shows a gaussian kernel density curve of semantic similarity scores for a ‘no boot’ query. The probable thresholds based on the least distance between the density curves of different bandwidths and the respective symptoms can be found are tabulated in the image below. The dotted line shown in the plot corresponds to the best threshold which is chosen based on the least distance between the density curves across a bandwidth range.

Figure 3.   KDE curves and associated candidate thresholds on example query results

For each query, DTS’s system returns up to 10 articles which are first ranked and thresholded. This ensures that the articles are relevant to the user's query in terms of content and context.

Next, DTS performed re-ranking of the articles towards of likelihood of success, leveraging historical statistics related to articles and the individual user query.

The article ranking algorithm is a calculation that assigns a score to each article based on the number of thumbs up it has received relative to the number of times it was presented to customers, as well as its view count. The algorithm is designed to give new articles a chance to rank high by considering the view count, while also rewarding popular and effective articles with higher ranking scores.

The algorithm can be represented mathematically as follows:

Ranking Score = Thumbs Up Rate * (1 + View Count Adjustment)

Where:

* ` Thumbs Up Rate = Thumbs Up Count / Presentations `

+ `Thumbs Up Count` is the total number of thumbs up received by the article

+ `Presentations` is the total number of times the article was presented to customers

* ` View Count Adjustment = 1 - (View Count / Max View Count)

+ `View Count` is the total number of views for the article

 + `Max View Count` is the maximum view count for all articles

The algorithm provides a simple yet effective way to rank articles based on their popularity and effectiveness relative to the number of times they were presented to customers. By considering the view count, new articles have a chance to rank high and gain visibility, while popular and effective articles are rewarded with higher ranking scores.

The solution was validated from both functional and non-functional perspectives. The functional aspects were tested throughout the development stages, in addition to acceptance testing by independent stakeholders. This is in accordance with DTS's test plan which captures testing activities, data sources, and roles and responsibilities.

DTS's development test data consists of a diverse sample of 25 user queries, spanning across 20+ products and the product families representing historical user queries related to the most common products encountered in technical troubleshooting. Subject Matter Experts (SME) determine the ground truth for each query, focusing on the relevance and effectiveness of the retrieved articles in resolving the user queries given the products. This fully annotated data is used to evaluate the effectiveness of dynamic thresholding, and the selection of the best embedding model for semantic similarity tasks, including article retrieval. It also supports metric evaluation, such as Precision, Recall, and F-Score.

For acceptance testing, SMEs conducted comprehensive tests across 178 unique products and 393 unique symptoms. These tests were carried out on both the legacy system and the new system. The results were annotated, highlighting the instances where the new system either returned irrelevant results or failed to return relevant results successfully retrieved by the legacy system.

A metrics-based approach was used in both development and acceptance phases to drive decision making. Precision, Recall, and F1-Score were leveraged to quantify the effectiveness of the system. While recall is typically a costly metric to calculate for IR systems, due to the requirement to annotate all system documents on a per query basis, DTS mitigated cost. For the development set, DTS reduced the document set and annotated only the filtered article set for each query. In acceptance testing, only the results from the legacy system and the new system were annotated as relevant or not. With this, DTS can measure precision metrics over both the dataset’s “absolute” recall measurements for the development test set, and recall “improvement” over the legacy system for the acceptance test set.

The new system improves retrieval precision from less than 40 percent up to 75 percent.

Data pipeline

PSQN articles undergo regular updates by the business support teams in collaboration with content teams, providing targeted solutions for specific model issues. To incorporate these updates into a PSQN search solution, DTS requires an efficient pipeline that ensures frequent synchronization. Once prepared for search, the pipeline generates indexed data within the Embedding Store, facilitating real-time retrieval.

Figure 4.   The stages of the data pipeline

Tagging

Articles undergo tagging to associate them with specific products and corresponding assets. DTS extracts these tags/associations at various levels (product family and asset). While the NBA tool provides the product information with the search query, it originates from a distinct taxonomy compared to the one used during article creation. To address this, DTS has implemented a product taxonomy mapping that bridges the two taxonomies, facilitating the translation to the NBA tool’s taxonomy within a data pipeline.

Before implementing a comprehensive solution for taxonomy mapping, a short-term static mapping was developed, which leveraged a Large Language Model (LLM) and fuzzy search tools. This approach facilitated a semi-automated process to align the two taxonomies. The need for automation arose due to the substantial volume of article data and nuanced labels, which necessitated human review.

Search data generation

To enhance article search, DTS augments human-generated data to represent each article within the search index. This augmentation combines an article summary with additional keywords to emphasize key concepts of the article. An LLM is employed in cases where human-generated summary is not available. Similarly, DTS generates keywords for all articles and appends them to their respective summaries.

The example below compares such generations and observes the LLM's ability to reduce an article of approximately 300 words into concise, accurate summaries as good as a human. Unlike alternative methods such as document chunking, DTS's approach minimizes data processing requirements and results in a smaller search index, enabling faster inference. Future work could explore further improvements if precision or adaptability issues arise, as highlighted in the section, Finding 4: Precision loss and adaptability in summarizations.

Table 1.      Example of human compared to model generated article summary and keywords

DTS deployment consists of two primary components: Software Development Kit (SDK) and Web Service.

SDK

For this solution, DTS adopted an SDK-First development approach that uses Data Science (DS) functions, which are implemented as a library that integrates into a larger system. The larger system as NBA is a real-time support platform, tightly coupled with the business infrastructure to leverage PSQN predictions. The PSQN SDK is designed to align with how the NBA Application Programming Interface (API) web service would consume it. DTS’s primary goal for the SDK is to expose functionalities related to article retrieval from the indexed data within the PGVector database as an offline operation. It facilitates query inference, which the PSQN web service uses to generate article predictions. Creating an SDK involves close collaboration with the DS team, understanding the source code, and converting it into an efficient and optimized ‘consumable package’ for installation within the web service dependencies container image.

PSQN SDK releases are decoupled from API releases. This approach provides flexibility for the PSQN core DS development team to keep refining the model without frequent web service updates, if the API contract remains unchanged. The SDK undergoes rigorous unit-tests, integration-tests, performance testing, security, and quality checks.

Web service

The web service provides an API-endpoint for PSQN model inference (such as POST /api/v0/psqn-retrieval). The NBA application side invokes the endpoint to retrieve the PSQN predictions as a list of article data items in a JSON serializable format.

The following figure illustrates the major solution-deployment components of the PSQN web service:

Figure 5.   Web service components

DTS's deployment process for the PSQN web service relies on a set of essential DevOps tools. These tools and infrastructure are used across other tools to check the code quality and security checks. The key components include:

• GitLab: GitLab and its CI/CD pipeline are leveraged to host source code and manage version releases for both the PSQN SDK package and web service.
• Docker/Containerization: The web service, model package, and associated dependencies are containerized using Docker. The resulting container image is stored in the on-premises Harbor registry.
• JFrog Packages Registry: A JFrog repository is maintained for model source code, dependencies, and metadata. This repository allows the model service to retrieve and install components, making them available for integration into the NBA web service.
• Pivotal Kubernetes Service (PKS): The on-premises PKS cluster serves as the deployment target for DEV, PERF, and PRD environments.
• PostgreSQL PGVector: PSQN is stored in the multi-node PostgreSQL database with PGVector extension enabled. This setup facilitates efficient semantic searches by storing text vectors and building IVFFLAT indexes. For PSQN releases, an IVFFLAT index with a searching probe of 10 is employed.
• PostgreSQL: PostgreSQL is used to store the requests, responses, and logs, supporting monitoring and model refinement.
• Locust Load Testing: For load and stress testing of the PSQN endpoint, an on-premises Locust load testing tool is deployed within the PKS clusters.
• Dynatrace: Dynatrace is relied upon to monitor deployment logs and capture the logs from all PSQN model service pods.

From a business standpoint, DTS's improvements in semantic, search-based PSQN article retrieval have resulted in significant cost savings across various domains. These savings include reduced manual work hours, fewer dispatches, and faster handle times. In the support space, several factors contribute to efficient and satisfactory service delivery.

SMEs support tech-support agents through the NBA tool, ensuring the accessibility and validation of essential documents. These tasks involve manual symptoms/keywords generation, downstream validation of affected products, and validation of online articles. Collectively, manual validation processes previously consumed over 500 hours annually. However, DTS's model has effectively eliminated the need for these activities, resulting in substantial time savings.

As a large tech corporation selling products and services globally, DTS encounters significant operational costs and a high volume of users seeking technical support, with issues ranging from malfunctioning peripherals to faulty motherboards. As a first line of support, DTS’s primary aim is to resolve issues through “soft troubleshooting,” which addresses problems without hardware replacement, favoring software-based solutions wherever applicable. While hardware dispatches can resolve certain specific issues, low confidence levels may lead to unnecessary costs such as shipping costs or write-offs.

In the consumer space, where over 1,900 different product models exist, the need for focused and efficient troubleshooting guides is important for support agents. By reducing “troubleshooting noise” (irrelevant documents or articles) by approximately 20 percent, users can quickly access relevant answers without sifting through extensive documentation.

Given the number of support agents, average support cases, and overall support costs, along with an annual dispatch cost exceeding approximately $60 million, this solution addresses approximately 12 percent of these cases, resulting in estimated annual savings of approximately 7 million.

Finding 1: Semantic search relevancy

Semantic search, which enables advanced natural language understanding, has transformed how DTS interprets user queries. Unlike lexical search, which strictly depends on keyword matching, semantic search considers the context and intent behind a query. This nuanced approach significantly enhances the accuracy of search results, ensuring that the articles retrieved are directly relevant to the specific issues described by customers.

Consider a customer query like "no video," which in a lexical search might return less relevant articles such as "4K video playback stuttering on XPS 15 9520 and Precision 5570 Computers." It could also return "Low Audio Volume from Internal Speakers on Precision 5570," where an article about audio issues might erroneously rank in the top three due to keyword overlap. Semantic search, however, discerns that the user is likely experiencing issues with video output rather than video file playback or audio problems. It prioritizes articles like "Troubleshooting Video Output Issues on Dell Laptops," directly addressing the customer's actual concern.

As a result, support agents equipped with semantic search capabilities can resolve customer issues more swiftly and accurately, markedly enhancing the user experience and operational efficiency.

Finding 2: Large Language Models for rapid prototyping

Technological advancements in Machine Learning and Natural Language Processing, specifically LLMs, allow for efficient content generation, keyword enhancement, and comprehensive understanding of textual data. By employing LLMs, organizations can significantly expedite the development process by generating high-quality content swiftly. This rapid prototyping capability not only accelerates product innovation, meeting and beating customer demands, but also fosters a competitive edge in the marketplace, driving business growth.

Finding 3: Data alignment across business units

Data quality and availability are central to ML systems. Like many others, DTS’s system leverages data from one business unit to benefit another, and alignment is key to its success. This was demonstrated following DTS's data understanding and article tagging efforts, which required alignment on specific taxonomy data fields. Several stages of development required alignment, including bringing data into accessible data stores for processing, and centralization of the source data and coordination of its ongoing maintenance in line with DTS's system’s expectations.

Finding 4: Precision loss and adaptability in summarizations

The use of LLMs for summarizing KB articles has led to several key improvements in precision, adaptability, and searchability. LLMs accurately distill essential information from the articles, resulting in concise summaries that reduce redundancy and focus on the most critical aspects of the content. Moreover, the creation of concise summaries and well-defined keywords has greatly improved the searchability of the articles. DTS can efficiently find the most relevant articles for a query based on the summaries generated by model. Although automated summarizations have reduced manual labor and enhanced processing speeds, summaries may occasionally experience precision loss, particularly when dealing with complex technical content. In future iterations, DTS plans to mitigate these challenges by employing alternative automated approaches, and a human-in-the-loop strategy that integrates automated outputs with expert reviews. This approach aims to enhance the accuracy and adaptability of generated summaries while preserving the advantages of automation.

Finding 5: Global thresholding limitations

While global thresholding is simple and easy to implement, it has several limitations, such as effectively handling diverse queries, adapting to different relevance requirements, and maintaining consistent performance. These limitations eventually make global thresholding less suitable for retrieving PSQNs in a dynamic and varied tech troubleshooting domain. A single threshold cannot account for the varying levels of relevance and specificity required by different queries. Some queries may need a higher threshold to ensure only highly relevant PSQNs are retrieved, while others may require a lower threshold to include more general information. For instance, the query "Error code 404 on Product X," might need highly relevant PSQNs, suggesting a higher threshold. The more generic query, "Slow performance on Product Y," might require a broader range of PSQNs, suggesting a lower threshold. Lack of customization, inconsistent relevance of the retrieved PSQNs, inflexibility in adapting to data distribution changes over time, and handling diverse data are important limitations of the global thresholding approach. A global threshold can also lead to underfitting and overfitting certain queries. This can eventually lead to reduced agent and customer satisfaction. Conversely, dynamic thresholding adapts to the specificities in the data and provides a tailored and an effective approach for PSQN retrievals with improved relevance.

Finding 6: ML library configuration in deployment

This section highlights deploying the embedding in a multi-CPU environment using PyTorch, a significant optimization that substantially improves inference speed.

Problem: In virtualized environments (such as Kubernetes or Docker), PyTorch's reliance on system tools for CPU cores can be inaccurate. Virtualization layers present a virtualized view, leading to suboptimal thread generation and hindering PyTorch's performance. Accurate CPU core detection is crucial for optimal PyTorch performance in virtualized environments.

Solution: Levering Linux kernel features, specifically Control Groups (cgroups), allows us to manage and allocate resource. The /sys/fs/cgroup/cpu (CPU Cgroup) directory contains control files for CPU management within a cgroup. Notably: cpu.cfs_quota_us defines the allowed CPU time (in microseconds) for a process within a period set by cpu.cfs_period_us. cpu.cfs_period_us which defines the CPU usage accounting period length in microseconds). This ratio of quota/period provides a relative measure of allocated CPU share, enabling optimization of PyTorch thread usage.

Environment Variables for Thread Control

• OMP_NUM_THREADS: Controls OpenMP threads for parallel computing.
• MKL_NUM_THREADS: Controls Math Kernel Library (MKL) threads for optimized math functions.
• NUMEXPR_NUM_THREADS: Controls NumExpr threads for high-performance numerical evaluation.
• OPENBLAS_NUM_THREADS: Controls OpenBLAS threads for linear algebra operations.

Setting these variables prevents oversubscription and efficiently uses available CPU cores for PyTorch computations.

The solution has had a significant impact, as highlighted by key stakeholders. According to the Sr. Director of Support Readiness, “PSQNs are a crucial part of our troubleshooting process, serving direct resolutions to ongoing/trending issues, and the improvements with the model have clearly impacted our business in a positive way.” Additionally, an SME in Quality Engineering noted, “When validating the past and present model, it is clear to see the improvements made regarding noise reduction and accuracy of the articles shown.”

This document provides an in-depth analysis of the challenges encountered by Dell Services in managing knowledge base articles for technical support. It emphasizes the role of efficient information retrieval systems capable of matching customer queries with relevant knowledge base articles.

Embedding-based semantic Information Retrieval techniques were implemented alongside Generative AI to streamline article tagging, indexing, and improve search relevance, resulting in:

• Reduced time spent by support agents on irrelevant articles
• Accelerated diagnosis and resolution of customer issues
• Enhanced customer satisfaction
• Significant operational savings

Future work includes automating the daily indexing process, refining retrieval algorithms, expanding system capabilities for dynamic thresholds, and leveraging telemetry data to offer personalized recommendations through Generative AI technologies.             

In this whitepaper, DTS presented an enhanced approach to retrieving and organizing knowledge base articles for Dell Services, aimed at improving the efficiency and accuracy of support provided to customers. This solution leverages contemporary embedding-based information retrieval techniques to surface the most relevant articles based on symptom descriptions and hardware configurations. By automating the article tagging and indexing process with Generative AI technologies, DTS has eliminated error-prone manual interventions, thus increasing the overall accuracy and relevance of retrieved articles. The improved Top Known Issues implementation not only reduces the time support agents spend sifting through irrelevant articles but also facilitates faster diagnosis and resolution of customer issues. This leads to higher customer satisfaction and significant operational savings.

DTS’s near future work will focus on fully automating the end-to-end indexing of new articles, leveraging Generative AI to optimize and automate key stages such as content generation, product mapping, filtering, ranking, and thresholding. This pipeline will be performed on a daily cadence, ensuring that the most recently introduced articles are promptly accessible to support agents. DTS also plans to further refine the retrieval algorithms, automate the taxonomy alignment process, and expand the system's capabilities to dynamically adjust thresholds based on the search query. This will minimize the display of irrelevant or non-applicable articles.

Future work will also involve leveraging telemetry data and other technical support data, such as online search journey data, to further enhance the relevance of retrieved articles. DTS aims to use customers' previous historical data, profile data, and machine telemetry profile to suggest personalized recommendations leveraging Generative AI technologies. Furthermore, DTS plans to expose applicable top-known issues articles not only using NBA support agents, but also directly to customers. For example, by integrating these articles into Customer Support chatbots, DTS can enable enhanced self-support, providing a more comprehensive and accurate support experience. This approach will ultimately lead to better customer satisfaction and more efficient support operations.

The following documentation provides additional information:

• Empowering Support Agents with Next Best Action Recommendations
• Transforming Customer Support with Contact Connect AI
• Omni-Channel Customer Support Troubleshooting Solutions Using Generative AI