Chatbots

Chatbots

This section gives you detailed insights into the performance of your bot operators. Most metrics recorded for human operators are also recorded for chatbot operators. Only a few metrics tracking the interaction between human and chatbot operators are recorded separately, such as “Number of chatbot conversations reassigned to human operators”. Note that some metrics related to conversations without operators are by default assigned to the human operator category, such as “Missed opportunities”. This may be important to interpret raw data pulled using our Analytics API.

Chatbot conversation overview

What does it measure?	An overview of several conversation-related chatbot metrics: “Conversations”, “Number of chatbot conversations reassigned to human operators”, “Number of conversations ended by a chatbot” and “Number of timed out chatbot conversations”.
Unit	Conversations
When is it counted?	Depending on respective metric.
Why is it useful?	Provides you with a quick overview of the amount and flow of your chatbot conversations.
Example	See descriptions of respective metrics below.

Average conversation duration

What does it measure?	The average time span between the first message sent in a conversation and the moment its status is set to “Ended” for a certain number of live and asynchronous conversations.
Unit	Seconds
When is it counted?	Whenever a conversation with operator messages is set to “Ended.” Resumed conversations are counted anew. If a conversation was handled by human and bot operators, the duration is recorded for each operator type separately.
Why is it useful?	Whether you’re aiming for quick resolutions or ongoing conversations, with this metric you keep track of these goals.
Example	A conversation starts and ends 20 minutes later. A second conversation takes 40 minutes, ends, is resumed later on and ends again after another 40 minutes. Considering those two conversations, your average conversation duration is 50 minutes.

Total conversation duration

What does it measure?	The total amount of time your chatbots handled ongoing conversations. This metric’s visualization compares chatbot data with that of human operators.
Unit	Seconds
When is it counted?	Whenever a conversation with operator messages is set to “Ended.” Resumed conversations are counted anew. If a conversation was handled by human and bot operators, the duration is recorded for each operator type separately.
Why is it useful?	Shows you how much your chatbots relieve the workload of human operators. To allow you to compare the performance of chatbot and human operators, we display a graph for each operator type in this metric’s visualization.
Example	A conversation starts and ends 20 minutes later. A second conversation takes 40 minutes, ends, is resumed later on and ends again after another 40 minutes. Considering those two conversations, your total conversation duration is 100 minutes.

Average conversation session count

What does it measure?	The average number of conversation sessions your chatbots handled. This metric is displayed in relation to the average number of all conversation sessions led by human operators.
Unit	Conversation sessions
When is it counted?	Whenever a new conversation is created
Why is it useful?	Shows you how often your chatbots step in to support your human operators.
Example	Two contacts start a conversation session with a bot operator, another starts one with a human operator. Both values will be displayed in this metric’s chart.

Operator chat slot utilization

What does it measure?	An operator’s average chat slot utilization during a given time in percent.
Unit	Chat slot utilization in percent
When is it counted?	Whenever the number of occupied chat slots changes for an operator.
Why is it useful?	It shows you how busy or idle individual bot operators were during a certain time. A low slot utilization might call for reducing the number of operators while a high slot utilization might suggest raising it.

Conversations

What does it measure?	The number of conversations started in a given time. This metric’s visualization compares chatbot data with that of human operators.
Unit	Conversations
When is it counted?	Whenever the first message is sent in a conversation. This will be roughly equivalent to its creation date in the Message Center. There might be discrepancies if, for example, the contact filled out the pre-conversation survey but then waited some time before sending the first message. The value is counted for both assigned and unassigned conversations.
Why is it useful?	Allows you to measure your overall chat traffic and identify your peak times.
Example	A contact starts a conversation on your website via the Website Messenger by sending a first message. A contact reaches out to you through Facebook Messenger. The conversation will be counted when they send the first message. To allow you to compare the performance of chatbot and human operators, we display a graph for each operator type in this metric’s visualization.

Post-conversation survey results

What does it measure?	The number of times and when contacts selected certain answers in your surveys after conversations with bot operators.
Unit	Survey answer
When is it counted?	When a contact answers a post-conversation survey.
Why is it useful?	Allows you to gather representative results of your surveys.

Feedback count

What does it measure?	The number of times your contacts left feedback during or after conversations with bot operators.
Unit	Feedback
When is it counted?	Whenever a contact leaves feedback. Multiple entries per conversation are possible. Feedback submitted while no operator is assigned will be omitted.
Why is it useful?	Customers tend to give feedback on particularly good or bad service experiences. So, this metric can give you insights on how common they are.

Satisfaction rating

What does it measure?	The average rating your operators receive from contacts.
Unit	Star rating (from 1 to 5)
When is it counted?	Whenever a contact rates a conversation. Multiple entries per conversation are possible. Ratings submitted while no operator is assigned will be omitted.
Why is it useful?	Gives you quantitative insights into your contacts’ satisfaction with your operators and messaging support.

Number of leads

What does it measure?	The number of times contacts submit their contact details for the first time.
Unit	Number of leads
When is it counted?	The first time a contact leaves contact information like name, email or mobile number.
Why is it useful?	Shows you how effective your chatbots are in generating leads.

Number of chatbot conversations reassigned to human operators

What does it measure?	The number of times a conversation is reassigned to a human operator.
Unit	Conversations
When is it counted?	When a chatbot reassigns a conversation to a human operator.
Why is it useful?	It shows you how many contact requests require additional human support and how often contacts are willing to continue when the bot was unable to solve their issue. Depending on your setup and expectations, a higher number can be a positive result (e.g. contacts are forwarded to sales representatives who can close the deal) or a negative one (the chatbot could not handle the request on its own).
Example	A contact navigates through your chatbot questions until the bot has enough information and reassigns the conversation to a designated human operator.

Number of conversations ended by a chatbot

What does it measure?	The number of times a chatbot sets a conversation to “ended”.
Unit	Conversations
When is it counted?	Whenever a conversation with chatbot messages (and none by human operators) is set to “ended” by a chatbot.
Why is it useful?	It shows you how effective your chatbots are in successfully handling contact requests without any help from humans.

Number of timed out chatbot conversations

What does it measure?	The number of times conversations time out while contacts interact with chatbots.
Unit	Conversations
When is it counted?	Whenever conversations with chatbot operators reach the bot conversation timeout.
Why is it useful?	A high value may indicate that many contacts abandoned their conversations because your chatbot was unable to handle their requests. This can point you to necessary tweaks on your chatbot setup.
Example	A contact starts a conversation on your website and your chatbot can neither answer the question nor reassign the conversation. The contact loses interest and leaves the website, the conversation runs into its defined timeout.