Best practices to get a Now Assist in AI Search Genius Results using the Multi-Content Response (MCR) pipeline with a synthesized answer containing information from the right documents - Support and Troubleshooting

Best practices to get a Now Assist in AI Search Genius Results using the Multi-Content Response (MCR) pipeline with a synthesized answer containing information from the right documents Issue .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Specific search terms entered by users in the Service Portal that triggers AI Search (Now Assist for AI Search) are not returning the expected synthesized answers from the Large Language Model (LLM). The LLM-generated Genius Results on the Service Portal may display incorrect information, reference the wrong knowledge articles, or fail to extract the relevant content from the appropriate source documents.

Symptoms .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Genius Results displays catalog items instead of expected knowledge articles Search queries using acronyms (e.g., "SAP PR1", "AD", "MS Visio") return irrelevant or generic Genius Results The LLM extracts answers from the wrong knowledge article even when the correct article exists Short or vague search queries (e.g. "I need Visio") do not produce step-by-step instructions that exist in the knowledge base whereas the longer search query (i.e. "I need to install Visio on my laptop") does return the step-by-step instructions The expected knowledge article appears in search results, but is not ranked high enough to be selected for answer extraction The same search term may produce slightly different outputs on repeated searches due to non-deterministic behaviour of the LLM Facts .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Environment Now Assist for AI Search with Multi-Content Response (MCR) pipeline Knowledge articles and catalog items indexed in AI Search with semantic indexes for answer extraction How AI Search Works AI Search removes stop words from search queries (e.g. "I", "the", "on", "a") Synonyms are applied for semantic search expansion Results are ranked based on Relevancy Ranking in AI Search , with the title/short description fields carrying the most weight The LLM extracts answers from the top-ranked extracts from documents returned by AI Search MCR (Multi-Content Response) does not use caching, meaning the same query may yield slightly different outputs in the Genius Result Release .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } All

Cause .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Poor search results from AI Search typically stem from one or more of the following causes:

1. Acronyms Not Expanded Users frequently search using acronyms (e.g., "PR1", "AD", "MS Visio") that do not exist in the knowledge articles. Without synonym dictionary entries to expand these acronyms to their full terms, the search cannot match the relevant content.

2. Missing Keywords in Knowledge Articles If the search term used by the user does not appear in the knowledge article (especially in the title or description), the article will not rank highly enough to be selected for answer extraction.

3. Result Improvement Rules Not Configured Without proper Result Improvement Rules to promote specific documents for certain search terms, generic or less relevant content may rank higher than the intended article.

4. Improper Synonym Configuration Overly broad synonyms (e.g., grouping "laptop", "PC", and "Mac" together) can cause unrelated articles to surface, diluting the relevance of search results.

5. LLM Non-Deterministic Behaviour Large Language Models are inherently non-deterministic, meaning identical search queries may produce slightly different synthesized answers. This is expected behaviour and cannot be fully eliminated.

Resolution .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Follow these steps in the recommended order to analyze and improve search results for AI Search Tuning - Making AI Search Analytics Actionable

Step 1: Analyze Search Behavior Using Analytics Tools User Experience Analytics:

Navigate to User Experience Analytics Select the target portal/search application Open AI Search Analytics Dashboard to view: Top searched queries Queries with no results Queries with no clicks Top clicked results and more

Now Assist Admin Console - User Search Analyzer:

Navigate to Now Assist Admin Console > Performance Select User Search Analyzer Review: queries with Genius Results, engagement metrics, feedback (thumbs up/down), response times and more

Clustering Solution (Optional):

Navigate to Clustering > Solution Definitions Create a new clustering solution on the [sys_search_event] and/or [sys_search_signal_event] and/or [sys_search_signal_result_event] tables Set the input field to "search_query" or dot-walk to it in table [sys_search_event] Run the solution to visualize search term patterns and identify common queries Step 2: Create or Update Synonym Dictionaries When to use synonyms:

Expanding acronyms to full terms (e.g., "AD" = "Active Directory")

Best practices for synonyms:

Use similar words, not hierarchical terms Avoid overly broad groupings that may return irrelevant results Consider use-case-specific requirements (e.g., Mac users may not want PC articles) Reference the Community article on AI Search Synonyms

Configuration steps:

Navigate to the AI Search Profile Open the Synonym Dictionary Add synonym entries (e.g., "AD, Active Directory") Publish the Synonym Dictionary and the Search Profile Step 3: Configure Result Improvement Rules When to use Result Improvement Rules:

When synonyms alone do not resolve the issue To promote specific documents for particular search terms To ensure critical knowledge articles appear in Genius Results

Configuration steps:

Navigate to the AI Search Profile > Result Improvement Rules Create a new Result Improvement Rule Define the trigger condition (e.g., "Query contains 'Visio' OR Query contains 'install' OR Query contains 'software'") Select action type: Promote Documents Search and add the target knowledge article(s) to promote You can promote multiple documents in the same rule using the "Promote Documents" button Publish the search profile

Important: Use OR conditions to capture variations of search terms. Add specific keywords (like application names) to ensure the rule triggers for specific user queries.

Step 4: Modify Knowledge Article Content (If Necessary) When to modify the knowledge article:

As a last resort after synonyms and Result Improvement Rules have been tried When the search term needs to be explicitly present in the article for relevancy

Recommended modifications:

Add the specific keyword or acronym to the article title (highest weight) OR include the term in the short description/article body Add the contextual phrase in the article body that is used in the search term (e.g. "If you need software, please check...")

Steps:

Edit the knowledge article Add the relevant keywords to appropriate fields Submit for publishing and approve Wait approximately 30 seconds for incremental indexing to complete Step 5 (Optional): Verify Indexing After making changes, verify the updates have been indexed:

Copy the sys_id of the modified knowledge article Navigate to the sysevent table Filter by: instance = [sys_id of article] Filter by: queue = "ais_index" Verify the most recent events shows the state = "Processed"

Note: The AIS Index Event Processor scheduled job runs every 30 seconds to process incremental indexing events.

Step 6: Test and Iterate Test the search query in the Service Portal environment using Multi-content Response (MCR) In the Service Portal, verify the Genius Result displays the expected content and sources the correct knowledge article If results are still unsatisfactory, iterate through Steps 2-5 Test with multiple variations of the search query