Overview

The Automation Flows Report provides a schematic representation of the bot's steps, accompanied by Key Performance Indicators (KPIs) that track various aspects of customer-bot interactions. This report allows agents and managers to gain deeper insights, which can lead to improvements and optimizations in the bot. This guide outlines its components and capabilities with relevant examples.

The Report Structure

The report visually maps each step of the flow (also known as a node) with arrows indicating the progression to subsequent steps. Hovering over any step reveals a dialog box displaying related data.

Note the step's name (heading) and ID (n_18) are listed.

For the selected time, the data presents:
Total Ins – Shows how many customers came into this step.
Total Outs - This shows how many customers continued from this step within the flow (and did not drop out).
Start Through Entry Point – Counts the number of times this step was used as an entry point to this flow from other flows (bots), agents, or directly from the customers.
Dropouts – Counts how many conversations with the bot were dropped by the customer when reaching this step.
Unknown Transactions – Counts the incorrect or unrelated information customers enter during a conversation with a bot.
Errors – Counts events with an API that result in an error or no returned data.
External script exits – Counts the actual transfers to other flow from this step.
Identified keywords – Counts the identified keywords (if applicable) at this point of the flow.

When clicking on a step, a dialog box lists the steps that can lead to this step and the steps that could follow it.

The heading includes the step's title and number, followed by the step's statistics. The body of the dialog box displays the preceding and following steps, along with the number of customers who followed those paths.

You can click any step before or after the one you selected and jump to that step to investigate the customer’s path and see what was chosen.

Viewing Menu

The top filters limit the schematic display to show specific routes within the bot.
By clearing the unwanted routes, it is easier to follow the different customer journeys and gain insights into them.
Options for filtering the display:

Insights examples:

Based on the bot's goals and expectations, managers can analyze customer paths to identify opportunities for improving both the bot and the overall customer experience.

Most Common Route:

• Optimization Opportunities: Analyze the most common path to ensure it aligns with the bot's design goals. If the path diverges from expectations, consider adjusting the flow to better guide customers through the intended experience.
• Streamlining: Look for opportunities to simplify or shorten this route to enhance efficiency and customer satisfaction.

Route with the Most Errors:

• Error Analysis: Investigate the specific errors occurring on this route. Determine whether they stem from input issues, unclear instructions, or technical issues (bugs) in the system.
• Bot Training: Consider retraining the bot on error-prone scenarios or updating the logic to handle these cases more effectively.

Route with the Most Unknowns:

• Customer Expectation Mismatch: Examine why the bot encounters so many unknowns on this route. This may indicate that customers are not presented with the options they expect or need.
• Language Refinement: Revise the bot's language to ensure clarity and intuition, minimizing the occurrence of unknown responses.

Route with the Most Dropouts:

• Customer Frustration: High dropout rates may signal customer dissatisfaction or confusion. Investigate whether the content or flow needs adjustment to better meet customer needs.
• Technical Evaluation: Assess for potential bugs or technical issues that could be causing customers to abandon the interaction.

Route with the Most Agent Transfers:

• Agent Transfer Points: Identify steps where the bot transferred customers to an agent or recognized a request for agent assistance instead of resolving the issue autonomously.
• Customer Segmentation: Determine if certain customer segments are more likely to request agent assistance, and tailor the bot's approach to address their specific needs.

Route with the Most Bot Resolves:

• Success Replication: Analyze what makes this route successful in resolving customer issues without agent intervention. Apply these best practices to other routes to improve overall efficiency.
• Customer Satisfaction: Evaluate customer feedback for this route to understand the aspects that contribute to a positive experience and replicate these elements across other parts of the bot.

Transitions

Types of transitions

There is a key at the side of the screen for all the transition types between the steps. By selecting a few types, you can limit the investigation scope to specific aspects of the bot.

Intents – The bot identified intents.
Keyword – The bot identified keywords.
Direct – The move to the next step was due to the customer’s selection or input. For example, the customer clicked on the “contact us” button.
Automatic – The move to the next step was not based on the customer’s selection or input. For example, conditions to enter the next step.
End – A default ending action (set in the Flow Settings) is used when the flow does not end with an end step or when jumping to a different step or flow.
Error – An API in the step had an error.
Unknown – The customer entered irrelevant or incorrect input, so the bot is heading to a specific step.
Jump Delay – If the customer does not respond within a set time, the bot will continue to this step.
Jump API – An API that creates a jump to a different step.
Chat – Start Over – The flow returns to the start step (only in Chat channels).
Show deleted nodes – When applicable, selecting this key will show deleted nodes throughout the flow.

Hidden steps

The report sometimes skips steps that work mainly behind the scenes or have no direct interaction with the customer.
When hovering over the step, you will see the magnifying glass icon.

Click on that path between the steps to see the additional steps between them.

Key Performance Indicators (KPIs)

At the top of the screen, different KPIs tally essential information about the bot’s performance.

Total Conversations – Counts the conversations that started in this flow
Reopen Conversations – This KPI counts the conversations that ended and restarted by the customer (within the time limit set in the split conversation settings).
Manual – Manual handover counts the transfer of the conversations from an agent to a bot (usually to complete a process automatically).
Reply After Compose – Count the conversations that started when the customer replied to the agent’s communication with a setting directing them to a bot.
Reply After Campaign – Count the conversations that started when the customer replied to a campaign with a setting directing them to a bot.
Transfer To Agent – Counts the conversations that were transferred to an agent.
Closed – Counts how many conversations were resolved by the bot. That figure includes the number of conversations that ended with an Irrelevant status.
Intents – Count the Intents identified throughout the conversations.
Key Words – Count the Key words identified throughout the conversations.
Direct Clicks – Counts the number of times the customer selected an option (clicking a button).
Unknown – Counts the total misspelled words and fragmented sentences entered by customers.
Errors – Counts the total number of API-related errors in the conversations.
New Items – Count how many new conversations (objects) were created by the bot.
Customer Messages – Counts the total number of messages the customer sent.
Bot Messages – Total number of messages by the bot.

Cross Flows Paths

Sometimes, the customers move between different flows.

When the previous step comes from a different bot, the icon will appear with the bot’s name (as opposed to the step title).

Before leaving that flow, the following warning displays:

When a step jumps to another flow, the heading includes the flow’s name and the title of the step it will enter. The icon and the warning mentioned above are added.

Management of Unknown items

The system notes entries by customers that it does not recognize. Click the Unknown KPI to open a list of those entries to decide if they can be used to teach the system to identify an intent.
For example, a customer entered, “speak to an egent”. You can teach the bot that the misspelled word is “good enough” to activate the “request for an agent” intent or just delete it.

To maintain the accuracy of your data, it's advisable to periodically remove unknown entries from both the Automation Flow report and the Intent module, as the brand has a limited capacity for these entries.