Laura Meschi

We were, I think, the first customer in Europe to go live with AgentForce. It's learning by doing. It's a completely revolutionary technology. We're in the very beginning of it. And it's something that you need to take as a long-term investment.

We were, I think, the first customer in Europe to go live with AgentForce. It's learning by doing. It's a completely revolutionary technology. We're in the very beginning of it. And it's something that you need to take as a long-term investment.

It's funny you ask because it's a very much a current discussion that we're having because my role, of course, is I'm mainly responsible for the quality and performance of our external vendor that does the frontline support. And then I got involved into the AI project, but it really takes focus to be able to work on this and improve it in a short amount of time.

It's funny you ask because it's a very much a current discussion that we're having because my role, of course, is I'm mainly responsible for the quality and performance of our external vendor that does the frontline support. And then I got involved into the AI project, but it really takes focus to be able to work on this and improve it in a short amount of time.

So having a pool of experts that is focused on this full time, or at least the majority of their working time, I think it's key to get the best results in the fastest way. way possible.

So having a pool of experts that is focused on this full time, or at least the majority of their working time, I think it's key to get the best results in the fastest way. way possible.

My take on this is rather than seeing the AI as a way to cut jobs, is I think a way to scale up the company without growing the workforce so much, but not cutting what we have. So it's bringing the company to the next level without having to add more resources, but definitely having what we have focusing on the more interesting side of the interaction with customers.

My take on this is rather than seeing the AI as a way to cut jobs, is I think a way to scale up the company without growing the workforce so much, but not cutting what we have. So it's bringing the company to the next level without having to add more resources, but definitely having what we have focusing on the more interesting side of the interaction with customers.

My goal is to take away all the repetitive and admin-y kind of tasks that agents have to perform. It's not just... in the realm of repetitive queries and easy queries, but it's also workload management and what they have to do to flip through a case. I think AI is key, summarizing a case, case logging, all these things that take up two, three minutes per interaction.

My goal is to take away all the repetitive and admin-y kind of tasks that agents have to perform. It's not just... in the realm of repetitive queries and easy queries, but it's also workload management and what they have to do to flip through a case. I think AI is key, summarizing a case, case logging, all these things that take up two, three minutes per interaction.

We could be spending more time on the phone with a customer, helping them, upselling, giving an advice of where to go next for their holiday. So yeah, investing more time in building that long-term relationship is the way I look at this. AI should be taking away the redundant tasks and giving the agents more time to do what humans are good at. So building relationship.

We could be spending more time on the phone with a customer, helping them, upselling, giving an advice of where to go next for their holiday. So yeah, investing more time in building that long-term relationship is the way I look at this. AI should be taking away the redundant tasks and giving the agents more time to do what humans are good at. So building relationship.

Yeah, definitely. And I, I think when it comes to, you know, a, near future and thinking of using AI on voice, for example, the way I would approach this is not saying, oh, now the machines are just going to handle all our conversations and we will deflect everything. It's actually the opposite.

Yeah, definitely. And I, I think when it comes to, you know, a, near future and thinking of using AI on voice, for example, the way I would approach this is not saying, oh, now the machines are just going to handle all our conversations and we will deflect everything. It's actually the opposite.

It's kind of understanding actually what is it that the virtual agent can do and filter that out and make whatever the humans are needed for you know, at closer reach so that members and we call our customers members can, you know, be closer to the agents when an agent is actually needed rather than trying to funnel everything through the AI and probably frustrating customers.

It's kind of understanding actually what is it that the virtual agent can do and filter that out and make whatever the humans are needed for you know, at closer reach so that members and we call our customers members can, you know, be closer to the agents when an agent is actually needed rather than trying to funnel everything through the AI and probably frustrating customers.

But because there is there are certain things that AI will not be able to help our customers with. But the rooting of those questions, picking up the tone

But because there is there are certain things that AI will not be able to help our customers with. But the rooting of those questions, picking up the tone

of someone, whether a customer is escalated or angry on the phone and immediately passing it through the right type of agent with the right set of skills, that's how I think we should be looking at AI helping us making good connections between customers and agents. Mm-hmm.

of someone, whether a customer is escalated or angry on the phone and immediately passing it through the right type of agent with the right set of skills, that's how I think we should be looking at AI helping us making good connections between customers and agents. Mm-hmm.

Yeah, so the way we approached this was very efficiency driven. As I said, we looked at the highest reason for contacts and we came up with a set of use cases. What we tried to do in the instructions and scope and description of the use cases was to give it enough boundaries so that we have enough control on what it could and couldn't do.

Yeah, so the way we approached this was very efficiency driven. As I said, we looked at the highest reason for contacts and we came up with a set of use cases. What we tried to do in the instructions and scope and description of the use cases was to give it enough boundaries so that we have enough control on what it could and couldn't do.

but there is a fine line between putting boundaries and between putting limits to it. So it took a lot of refinement, but it's basically based on those use cases that we picked at the beginning. In hindsight, probably we could have opted for less complicated use cases, as I said, and start from simply FAQs, question and answer kind of use cases.

but there is a fine line between putting boundaries and between putting limits to it. So it took a lot of refinement, but it's basically based on those use cases that we picked at the beginning. In hindsight, probably we could have opted for less complicated use cases, as I said, and start from simply FAQs, question and answer kind of use cases.

However, our industry, unlike, for example, retail, there is much less that you can actually resolve in an FAQ kind of way. So the personalized kind of question about a specific booking, about a specific deal where you actually need to action something, it's much bigger than the simple FAQ environment. So

However, our industry, unlike, for example, retail, there is much less that you can actually resolve in an FAQ kind of way. So the personalized kind of question about a specific booking, about a specific deal where you actually need to action something, it's much bigger than the simple FAQ environment. So

I think we did the best thing we could do, and we were quite brave in exploring also the actions and making it available to the virtual agent to be able to do something and not just answering based on the knowledge base. And that's something we want to leverage on. That's how we picked it.

I think we did the best thing we could do, and we were quite brave in exploring also the actions and making it available to the virtual agent to be able to do something and not just answering based on the knowledge base. And that's something we want to leverage on. That's how we picked it.

But it's a mix of type of queries and type of interaction that you can have with the virtual agent that we got to, but quite unconsciously. Like I wasn't aware of conscious that we were mixing up FAQs and personalized type of questions and access.

But it's a mix of type of queries and type of interaction that you can have with the virtual agent that we got to, but quite unconsciously. Like I wasn't aware of conscious that we were mixing up FAQs and personalized type of questions and access.

Now I realize it in hindsight when I look back and look at the different type of use cases we have and when you go to conferences and you see that we should start from FAQs only and then move forward. I was like, okay, we were quite brave and brave. you know, mixed things up a little bit.

Now I realize it in hindsight when I look back and look at the different type of use cases we have and when you go to conferences and you see that we should start from FAQs only and then move forward. I was like, okay, we were quite brave and brave. you know, mixed things up a little bit.

Because of the use cases that we picked, we had to have a knowledge base that could sustain the FAQs type of questions. We were already using a knowledge base and we had articles that belonged to a menu-based bot that we retired when we implemented the agent force agent. So we used a little bit of that.

Because of the use cases that we picked, we had to have a knowledge base that could sustain the FAQs type of questions. We were already using a knowledge base and we had articles that belonged to a menu-based bot that we retired when we implemented the agent force agent. So we used a little bit of that.

Of course, it might be in need of a cleanup, but we were already at a very advanced stage when it comes to having the knowledge required for this project inside Salesforce. Some of the use cases required more personalized information, as I said, information about your flights or luggage or your cancellation terms for your specific bookings.

Of course, it might be in need of a cleanup, but we were already at a very advanced stage when it comes to having the knowledge required for this project inside Salesforce. Some of the use cases required more personalized information, as I said, information about your flights or luggage or your cancellation terms for your specific bookings.

So we had to push that type of data inside Salesforce for us to be able to then have the virtual agent look at the data depending on the question that he was asked and being able to refer it back to the customer. So it took a little bit of preparation and I would say the more data that is in Salesforce, the better it is for you to start the agent force journey.

So we had to push that type of data inside Salesforce for us to be able to then have the virtual agent look at the data depending on the question that he was asked and being able to refer it back to the customer. So it took a little bit of preparation and I would say the more data that is in Salesforce, the better it is for you to start the agent force journey.

It's all about the quality of the data you have.

It's all about the quality of the data you have.

So we worked with a development company that has helped us in the past with Salesforce development. And I strongly recommend companies to assess that possibility. because it was really helpful for us to be able to count on them when it comes to the prep work.

So we worked with a development company that has helped us in the past with Salesforce development. And I strongly recommend companies to assess that possibility. because it was really helpful for us to be able to count on them when it comes to the prep work.

So it took, as I said, longer than we expected, but we were able to go live with the first complex use case in the space of two months since we started the whole process. operation on this. And then it continued for the next three months.

So it took, as I said, longer than we expected, but we were able to go live with the first complex use case in the space of two months since we started the whole process. operation on this. And then it continued for the next three months.

So it took longer than the six weeks everyone goes on about, but it was because we picked quite complex use cases, as I said, and the preparation in terms of data was quite extensive.

So it took longer than the six weeks everyone goes on about, but it was because we picked quite complex use cases, as I said, and the preparation in terms of data was quite extensive.

Exactly. Especially because within companies, it's kind of a new role. So there isn't anyone that has done this before. Whereas if you start speaking to a development consultancy company, they have worked with our other clients and kind of bring their knowledge and good practice. So they can spare you a lot of time.

Exactly. Especially because within companies, it's kind of a new role. So there isn't anyone that has done this before. Whereas if you start speaking to a development consultancy company, they have worked with our other clients and kind of bring their knowledge and good practice. So they can spare you a lot of time.

Yeah, so the reason why we started the journey with AgentForce was mainly around trying to deflect, or at least the initial thought was around deflecting the contacts for the highest reasons for contact that we were getting through our live chat channel.

Yeah, so the reason why we started the journey with AgentForce was mainly around trying to deflect, or at least the initial thought was around deflecting the contacts for the highest reasons for contact that we were getting through our live chat channel.

And, you know, there are many companies out there where if your values align, then it's an extension of your own team. And it's very easy to work with them because they understand the business and They also understand the technology much better than you and the company.

And, you know, there are many companies out there where if your values align, then it's an extension of your own team. And it's very easy to work with them because they understand the business and They also understand the technology much better than you and the company.

One of the surprises or like welcome surprise, I was hoping this was going to happen, was that the negative CES score on how easy it is to get in touch with us via live chat has increased massively. Before, around 15% of the negative entries when it comes to negative CES were about how clunky it was to get in touch with us via live chat because mainly of the menu-based bot. That was my baby.

One of the surprises or like welcome surprise, I was hoping this was going to happen, was that the negative CES score on how easy it is to get in touch with us via live chat has increased massively. Before, around 15% of the negative entries when it comes to negative CES were about how clunky it was to get in touch with us via live chat because mainly of the menu-based bot. That was my baby.

I liked working on it, but it wasn't the best when it comes to smoothness. Whereas now that type of negative feedback is virtually dissolved.

I liked working on it, but it wasn't the best when it comes to smoothness. Whereas now that type of negative feedback is virtually dissolved.

It was great to see that, especially because we were struggling with it so much in the menu-based bot era. Of course, The virtual agent is a little bit more sensitive to escalating the issue to humans because it can pick up the tone. A menu-based bot where you click through the options, it can't really feel your sentiment and tone of voice. So you have to be careful.

It was great to see that, especially because we were struggling with it so much in the menu-based bot era. Of course, The virtual agent is a little bit more sensitive to escalating the issue to humans because it can pick up the tone. A menu-based bot where you click through the options, it can't really feel your sentiment and tone of voice. So you have to be careful.

careful in tuning the instructions of the agent to avoid escalating as much as it would naturally do or organically do. Our escalation percentage in terms of interactions that are passed to humans has increased. But as I said, overall, I can see a better perception of the interaction with the virtual agent versus the bot.

careful in tuning the instructions of the agent to avoid escalating as much as it would naturally do or organically do. Our escalation percentage in terms of interactions that are passed to humans has increased. But as I said, overall, I can see a better perception of the interaction with the virtual agent versus the bot.

I have to say that partially one of the reasons why we got on this journey is also because we wanted to be at the forefront of the new technology. So it wasn't just about the efficiency side and the big promises around AI, but also we wanted to be on the AI wagon for everyone else. So we started the pilot with Salesforce. We were, I think, the first company

I have to say that partially one of the reasons why we got on this journey is also because we wanted to be at the forefront of the new technology. So it wasn't just about the efficiency side and the big promises around AI, but also we wanted to be on the AI wagon for everyone else. So we started the pilot with Salesforce. We were, I think, the first company

And the number of repeated contacts from the same customer has decreased, meaning that the customer is not giving up on us on chat and reaching out via phone. But this new journey allows them to stay in the channel of choice and hopefully get to a nice resolution.

And the number of repeated contacts from the same customer has decreased, meaning that the customer is not giving up on us on chat and reaching out via phone. But this new journey allows them to stay in the channel of choice and hopefully get to a nice resolution.

Exactly. And also an increase in escalated interactions. And by escalated, I mean passed over to a human, means a decrease in abandoned interactions. So as I said, you don't want your customers to give up on you. You want them to be able to reach a resolution and do it fast. So if the AI can do it, just interacting with the agent, Fine. If not, it's fine to pass it over to a human.

Exactly. And also an increase in escalated interactions. And by escalated, I mean passed over to a human, means a decrease in abandoned interactions. So as I said, you don't want your customers to give up on you. You want them to be able to reach a resolution and do it fast. So if the AI can do it, just interacting with the agent, Fine. If not, it's fine to pass it over to a human.

What you don't want is abandonment. So customers giving up on your agent, virtual agent, and choosing to phone us or email us.

What you don't want is abandonment. So customers giving up on your agent, virtual agent, and choosing to phone us or email us.

I am trying to find different ways to leverage the FAQs side of things, as I said, so to increase and improve our knowledge base for the agent to have additional grounding, but also exploring more of

I am trying to find different ways to leverage the FAQs side of things, as I said, so to increase and improve our knowledge base for the agent to have additional grounding, but also exploring more of

different type of grounding so that we could expand that side of action and support that the agent can provide to our customers when it comes to helping them about their specific booking, whether they're looking at a specific holiday or whether they have issues with their own specific account.

different type of grounding so that we could expand that side of action and support that the agent can provide to our customers when it comes to helping them about their specific booking, whether they're looking at a specific holiday or whether they have issues with their own specific account.

So, you know, I'm looking at different strategies we could take, but these are the main two avenues that I would like to explore.

So, you know, I'm looking at different strategies we could take, but these are the main two avenues that I would like to explore.

And I have to say that I'm also looking at some of the new functionalities that will be coming up in Salesforce when it comes to workload management and co-piloting for agents when it comes to case management and workload management, as I think that would be key for us to free up time from the agents. So case summaries and case logging, analyzing reasons for contacts,

And I have to say that I'm also looking at some of the new functionalities that will be coming up in Salesforce when it comes to workload management and co-piloting for agents when it comes to case management and workload management, as I think that would be key for us to free up time from the agents. So case summaries and case logging, analyzing reasons for contacts,

customer in Europe to go live with AgentForce. So it felt really good to be at the forefront of something completely unknown and exciting like AI, but also it was a learning curve and excitement turned into realization that it wasn't exactly what we expected, but also the excitement is not over and there is a long way It's a long-term investment. Of course.

customer in Europe to go live with AgentForce. So it felt really good to be at the forefront of something completely unknown and exciting like AI, but also it was a learning curve and excitement turned into realization that it wasn't exactly what we expected, but also the excitement is not over and there is a long way It's a long-term investment. Of course.

taking that away from humans.

taking that away from humans.

So we went live with the messaging sessions in September. And I think we went live in November with the first use cases.

So we went live with the messaging sessions in September. And I think we went live in November with the first use cases.

Definitely, it would have to be Vito. It's a furniture company based in the UK. I was interested in getting a shelving unit from them. And so I contacted them. And instead of pushing me to a product page on their website, they set up a call with me to look into the design options. They wanted to see my space. So it felt really refreshing for a company that is quite big, historical.

Definitely, it would have to be Vito. It's a furniture company based in the UK. I was interested in getting a shelving unit from them. And so I contacted them. And instead of pushing me to a product page on their website, they set up a call with me to look into the design options. They wanted to see my space. So it felt really refreshing for a company that is quite big, historical.

It's kind of an icon of design to take the time and discover the interaction with the customer. They kind of understand that what you are buying from them is a long term investment. It's not cheap and therefore it needs to be the best for you and for your future and for your space. So, yeah, I really appreciated that.

It's kind of an icon of design to take the time and discover the interaction with the customer. They kind of understand that what you are buying from them is a long term investment. It's not cheap and therefore it needs to be the best for you and for your future and for your space. So, yeah, I really appreciated that.

And it reminded me how important sometimes it is not just to be efficient and quick, But taking the time and being intentional and being very human when it comes to the interaction with customers, it really makes a difference. So much so that I've ordered the shelving unit and I'm going to order another one for another room in my flat because I liked it so much. Yeah.

And it reminded me how important sometimes it is not just to be efficient and quick, But taking the time and being intentional and being very human when it comes to the interaction with customers, it really makes a difference. So much so that I've ordered the shelving unit and I'm going to order another one for another room in my flat because I liked it so much. Yeah.

Absolutely. Like people can really feel when they're being sold on rather than actually supported. And that's the key, right? So the key to achieve the best customer experience in general, and it's easier said than done, is to really listen and really try to help customers. as that customer will stay with you for a very long time rather than a quick win.

Absolutely. Like people can really feel when they're being sold on rather than actually supported. And that's the key, right? So the key to achieve the best customer experience in general, and it's easier said than done, is to really listen and really try to help customers. as that customer will stay with you for a very long time rather than a quick win.

They can perceive how genuine you are in the support that you provide.

They can perceive how genuine you are in the support that you provide.

It's the same thing with the AI. It's a long-term investment, I think. And that has to be the mindset when you start this kind of journey. It's learning by doing. It's a completely revolutionary technology. We're at the very beginning of it. And it's something that you need to take as a long-term investment and something that will stay with you for a very long time.

It's the same thing with the AI. It's a long-term investment, I think. And that has to be the mindset when you start this kind of journey. It's learning by doing. It's a completely revolutionary technology. We're at the very beginning of it. And it's something that you need to take as a long-term investment and something that will stay with you for a very long time.

It's start simple. and make sure you have good data. Make sure you have a good foundation of knowledge base. Make sure you lean on the experts out there. Don't DIY this. And make sure you start the journey with AI, with the creation of long-term relationship with your customers in mind. It's not a quick win. deflection journey.

It's start simple. and make sure you have good data. Make sure you have a good foundation of knowledge base. Make sure you lean on the experts out there. Don't DIY this. And make sure you start the journey with AI, with the creation of long-term relationship with your customers in mind. It's not a quick win. deflection journey.

It's actually investing into relationship you have with your customers and tackling everything that you can take away from humans that is not as valuable and that focusing your teams on what humans are best at.

It's actually investing into relationship you have with your customers and tackling everything that you can take away from humans that is not as valuable and that focusing your teams on what humans are best at.

So, yeah, it's really, really fun to be working on this project where the technology is so new and it feels new for everyone. So you don't feel left out in a way. You're not alone. Yeah.

So, yeah, it's really, really fun to be working on this project where the technology is so new and it feels new for everyone. So you don't feel left out in a way. You're not alone. Yeah.

Probably AI is the new internet smartphone technology revolution. So it's like being in the midst of something that is probably life changing. So it's exciting and scary at the same time.

Probably AI is the new internet smartphone technology revolution. So it's like being in the midst of something that is probably life changing. So it's exciting and scary at the same time.

It took longer than expected. And one other challenge was reporting on return on investment. How successful in general was the handling of the conversations with our customers? It's not a quick win deflection journey. It's actually investing into relationship you have with your customers and tackling everything that you can take away from humans.

It took longer than expected. And one other challenge was reporting on return on investment. How successful in general was the handling of the conversations with our customers? It's not a quick win deflection journey. It's actually investing into relationship you have with your customers and tackling everything that you can take away from humans.

So as I said, we started with setting up use cases for the usage of agent force based on the highest reason for contact, which in hindsight, it was the only way we could do this being so unknown and, you know, unknown territory for everyone. But in hindsight, I think we probably chose use cases that were a little too complex for So it took a little longer to build the functionality.

So as I said, we started with setting up use cases for the usage of agent force based on the highest reason for contact, which in hindsight, it was the only way we could do this being so unknown and, you know, unknown territory for everyone. But in hindsight, I think we probably chose use cases that were a little too complex for So it took a little longer to build the functionality.

As it wasn't just answering FAQs questions, we started off by asking the virtual agent to actually perform some actions. So it took longer than expected. And one other challenge was reporting on return on investment, which felt a bit more complicated than what I expected. within the functionality itself.

As it wasn't just answering FAQs questions, we started off by asking the virtual agent to actually perform some actions. So it took longer than expected. And one other challenge was reporting on return on investment, which felt a bit more complicated than what I expected. within the functionality itself.

So it took a little bit of a DIY solution to try to get an understanding of how successful the deflection rate was for the agent and how successful in general was the handling of the conversations with our customers. Yeah, I would say those two. And probably the setup process, it's fairly easy, especially if you're looking into setting up a use case that is an FAQ-based type of use case.

So it took a little bit of a DIY solution to try to get an understanding of how successful the deflection rate was for the agent and how successful in general was the handling of the conversations with our customers. Yeah, I would say those two. And probably the setup process, it's fairly easy, especially if you're looking into setting up a use case that is an FAQ-based type of use case.

But the time that it takes for testing and quality assessment of the interaction, it's quite extensive. And that's something I completely, not completely overlooked, but I kind of did not take into account that much when we started off this journey.

But the time that it takes for testing and quality assessment of the interaction, it's quite extensive. And that's something I completely, not completely overlooked, but I kind of did not take into account that much when we started off this journey.

It was mainly the fact that the deflection rate we were expecting to achieve has not been as high as we expected. Partially, it could have been an unreasonable expectation to have, because, as you said earlier, you know, we thought it would be much more immediate, whereas this is something completely brand new and you kind of have to learn by doing.

It was mainly the fact that the deflection rate we were expecting to achieve has not been as high as we expected. Partially, it could have been an unreasonable expectation to have, because, as you said earlier, you know, we thought it would be much more immediate, whereas this is something completely brand new and you kind of have to learn by doing.

So it's part of the challenge and part of the beauty of this is that you just learn as you go. So that was the challenge. And yeah, because the, um, deflection rate is not what we expected. We kind of have to try different ways of achieving and maximize the power of the technology that we have. Um,

So it's part of the challenge and part of the beauty of this is that you just learn as you go. So that was the challenge. And yeah, because the, um, deflection rate is not what we expected. We kind of have to try different ways of achieving and maximize the power of the technology that we have. Um,

And that's also why this is such an interesting journey, because you have to kind of learn while you're on the ground. There isn't much more we could have planned around this apart from the choice of use cases. So I think it's also about changing the perception on the metrics that you want to look at to measure success. So with technology like this, probably ROI is,

And that's also why this is such an interesting journey, because you have to kind of learn while you're on the ground. There isn't much more we could have planned around this apart from the choice of use cases. So I think it's also about changing the perception on the metrics that you want to look at to measure success. So with technology like this, probably ROI is,

is not the best matrix to give it justice. People could consider also the impact on CISA and satisfaction of customers in the interaction with the virtual agent. The easiness of how It changes from a menu-based bot, which is what we had before, to an LLM that can literally pick up your tone and can have a conversation with you like a human.

is not the best matrix to give it justice. People could consider also the impact on CISA and satisfaction of customers in the interaction with the virtual agent. The easiness of how It changes from a menu-based bot, which is what we had before, to an LLM that can literally pick up your tone and can have a conversation with you like a human.

If you read through some of the transcripts, it is impressive how seamless it is and how the customer didn't even realize they were talking to an AI agent if we weren't declaring it at the beginning of the conversation.

If you read through some of the transcripts, it is impressive how seamless it is and how the customer didn't even realize they were talking to an AI agent if we weren't declaring it at the beginning of the conversation.

So it can pick up the context, it can resend the information. So it's also shifting the focus from ROI to a range of wider metrics. It's possibly something companies have to take into consideration when deciding to embark into an AI journey.

So it can pick up the context, it can resend the information. So it's also shifting the focus from ROI to a range of wider metrics. It's possibly something companies have to take into consideration when deciding to embark into an AI journey.

I was impressed by how easy it is to set up the actual facility, if you're not asking the agent to perform any actions, it's literally like typing a narrative and it's like you're talking to a human explaining a new employee what they're supposed to be doing. and telling them what they should and shouldn't do and how easy it gets on with it in a way, how fast it learns.

I was impressed by how easy it is to set up the actual facility, if you're not asking the agent to perform any actions, it's literally like typing a narrative and it's like you're talking to a human explaining a new employee what they're supposed to be doing. and telling them what they should and shouldn't do and how easy it gets on with it in a way, how fast it learns.

And if you read through some of the transcripts of communications with our customers, it is impressive how fast

And if you read through some of the transcripts of communications with our customers, it is impressive how fast

seamless it is and how the probably the customer didn't even realize they were talking to an ai agent if we were in declaring it at the beginning of the conversation yeah um so that's something that i knew was going to happen but every time i see it i'm like super impressed wow this is happening Yeah. So the easiness of setting it up, I was really impressed with it.

seamless it is and how the probably the customer didn't even realize they were talking to an ai agent if we were in declaring it at the beginning of the conversation yeah um so that's something that i knew was going to happen but every time i see it i'm like super impressed wow this is happening Yeah. So the easiness of setting it up, I was really impressed with it.

Being used to coding and in the backend, you have to be an engineer and developer to set this up or to even test. This was kind of my bread and butter because you actually need someone that knows the business and how the customers are asking questions in order to set this up effectively.

Being used to coding and in the backend, you have to be an engineer and developer to set this up or to even test. This was kind of my bread and butter because you actually need someone that knows the business and how the customers are asking questions in order to set this up effectively.

So I felt it was a very rewarding adventure for someone like me that is not a technical person, but got out of it quite a lot and learned a lot about technology. Amazing.

So I felt it was a very rewarding adventure for someone like me that is not a technical person, but got out of it quite a lot and learned a lot about technology. Amazing.

Well, it takes a lot of time to quality check the interactions. And it takes longer than I expected, I have to say. It's quality checking and testing and troubleshooting and changing the instructions as you go. In terms of training it, it's I feel the key here is to use historical interaction, which is something that is coming up with Salesforce.

Well, it takes a lot of time to quality check the interactions. And it takes longer than I expected, I have to say. It's quality checking and testing and troubleshooting and changing the instructions as you go. In terms of training it, it's I feel the key here is to use historical interaction, which is something that is coming up with Salesforce.

So being able to base the testing and the quality assessment of the interaction with the virtual agent with historical interactions, it will be a game changer. Because now it was a lot of manual work for a human, funnily enough, to be able to be on top of the quality of the interactions and tweaking what needed to be tweaked in the back end to be able to improve the quality of the interactions.

So being able to base the testing and the quality assessment of the interaction with the virtual agent with historical interactions, it will be a game changer. Because now it was a lot of manual work for a human, funnily enough, to be able to be on top of the quality of the interactions and tweaking what needed to be tweaked in the back end to be able to improve the quality of the interactions.

So we wrote up the use cases and we did a lot of testings based on different iteration of the same questions. And we used previous interactions with customers that we entered into the system and tried to test the reaction of the LLM.

So we wrote up the use cases and we did a lot of testings based on different iteration of the same questions. And we used previous interactions with customers that we entered into the system and tried to test the reaction of the LLM.

So at the moment, our QA is human driven, meaning that we review a certain number of interactions per day. And we have put together a form where we score the questions. quality of the interaction, if there were any issues, if the issue needed to be escalated and if it was escalated correctly. And then whenever there is an issue flagged via this QA process, us and the development team that we have

So at the moment, our QA is human driven, meaning that we review a certain number of interactions per day. And we have put together a form where we score the questions. quality of the interaction, if there were any issues, if the issue needed to be escalated and if it was escalated correctly. And then whenever there is an issue flagged via this QA process, us and the development team that we have

work together with to set this up. We'll go in and try to see if it's an actual bug or whether it's just a tweak of an instruction, whether we need an additional KB article, whether we need to change the existing KB article. So there is a lot of trial and error. And at the moment, there is no functionality to bulk test or bulk QA, which I think in the future is coming and it will be very handy.

work together with to set this up. We'll go in and try to see if it's an actual bug or whether it's just a tweak of an instruction, whether we need an additional KB article, whether we need to change the existing KB article. So there is a lot of trial and error. And at the moment, there is no functionality to bulk test or bulk QA, which I think in the future is coming and it will be very handy.

especially for a complicated industry like ours, where even the same theme in terms of type of query can be asked in 20 different ways.

especially for a complicated industry like ours, where even the same theme in terms of type of query can be asked in 20 different ways.

We were, I think, the first customer in Europe to go live with AgentForce. It's learning by doing. It's a completely revolutionary technology. We're in the very beginning of it. And it's something that you need to take as a long-term investment.

It's funny you ask because it's a very much a current discussion that we're having because my role, of course, is I'm mainly responsible for the quality and performance of our external vendor that does the frontline support. And then I got involved into the AI project, but it really takes focus to be able to work on this and improve it in a short amount of time.

So having a pool of experts that is focused on this full time, or at least the majority of their working time, I think it's key to get the best results in the fastest way. way possible.

My take on this is rather than seeing the AI as a way to cut jobs, is I think a way to scale up the company without growing the workforce so much, but not cutting what we have. So it's bringing the company to the next level without having to add more resources, but definitely having what we have focusing on the more interesting side of the interaction with customers.

My goal is to take away all the repetitive and admin-y kind of tasks that agents have to perform. It's not just... in the realm of repetitive queries and easy queries, but it's also workload management and what they have to do to flip through a case. I think AI is key, summarizing a case, case logging, all these things that take up two, three minutes per interaction.

We could be spending more time on the phone with a customer, helping them, upselling, giving an advice of where to go next for their holiday. So yeah, investing more time in building that long-term relationship is the way I look at this. AI should be taking away the redundant tasks and giving the agents more time to do what humans are good at. So building relationship.

Yeah, definitely. And I, I think when it comes to, you know, a, near future and thinking of using AI on voice, for example, the way I would approach this is not saying, oh, now the machines are just going to handle all our conversations and we will deflect everything. It's actually the opposite.

It's kind of understanding actually what is it that the virtual agent can do and filter that out and make whatever the humans are needed for you know, at closer reach so that members and we call our customers members can, you know, be closer to the agents when an agent is actually needed rather than trying to funnel everything through the AI and probably frustrating customers.

But because there is there are certain things that AI will not be able to help our customers with. But the rooting of those questions, picking up the tone

of someone, whether a customer is escalated or angry on the phone and immediately passing it through the right type of agent with the right set of skills, that's how I think we should be looking at AI helping us making good connections between customers and agents. Mm-hmm.

Yeah, so the way we approached this was very efficiency driven. As I said, we looked at the highest reason for contacts and we came up with a set of use cases. What we tried to do in the instructions and scope and description of the use cases was to give it enough boundaries so that we have enough control on what it could and couldn't do.

but there is a fine line between putting boundaries and between putting limits to it. So it took a lot of refinement, but it's basically based on those use cases that we picked at the beginning. In hindsight, probably we could have opted for less complicated use cases, as I said, and start from simply FAQs, question and answer kind of use cases.

However, our industry, unlike, for example, retail, there is much less that you can actually resolve in an FAQ kind of way. So the personalized kind of question about a specific booking, about a specific deal where you actually need to action something, it's much bigger than the simple FAQ environment. So

I think we did the best thing we could do, and we were quite brave in exploring also the actions and making it available to the virtual agent to be able to do something and not just answering based on the knowledge base. And that's something we want to leverage on. That's how we picked it.

But it's a mix of type of queries and type of interaction that you can have with the virtual agent that we got to, but quite unconsciously. Like I wasn't aware of conscious that we were mixing up FAQs and personalized type of questions and access.

Now I realize it in hindsight when I look back and look at the different type of use cases we have and when you go to conferences and you see that we should start from FAQs only and then move forward. I was like, okay, we were quite brave and brave. you know, mixed things up a little bit.

Because of the use cases that we picked, we had to have a knowledge base that could sustain the FAQs type of questions. We were already using a knowledge base and we had articles that belonged to a menu-based bot that we retired when we implemented the agent force agent. So we used a little bit of that.

Of course, it might be in need of a cleanup, but we were already at a very advanced stage when it comes to having the knowledge required for this project inside Salesforce. Some of the use cases required more personalized information, as I said, information about your flights or luggage or your cancellation terms for your specific bookings.

So we had to push that type of data inside Salesforce for us to be able to then have the virtual agent look at the data depending on the question that he was asked and being able to refer it back to the customer. So it took a little bit of preparation and I would say the more data that is in Salesforce, the better it is for you to start the agent force journey.

It's all about the quality of the data you have.

So we worked with a development company that has helped us in the past with Salesforce development. And I strongly recommend companies to assess that possibility. because it was really helpful for us to be able to count on them when it comes to the prep work.

So it took, as I said, longer than we expected, but we were able to go live with the first complex use case in the space of two months since we started the whole process. operation on this. And then it continued for the next three months.

So it took longer than the six weeks everyone goes on about, but it was because we picked quite complex use cases, as I said, and the preparation in terms of data was quite extensive.

Exactly. Especially because within companies, it's kind of a new role. So there isn't anyone that has done this before. Whereas if you start speaking to a development consultancy company, they have worked with our other clients and kind of bring their knowledge and good practice. So they can spare you a lot of time.

Yeah, so the reason why we started the journey with AgentForce was mainly around trying to deflect, or at least the initial thought was around deflecting the contacts for the highest reasons for contact that we were getting through our live chat channel.

And, you know, there are many companies out there where if your values align, then it's an extension of your own team. And it's very easy to work with them because they understand the business and They also understand the technology much better than you and the company.

One of the surprises or like welcome surprise, I was hoping this was going to happen, was that the negative CES score on how easy it is to get in touch with us via live chat has increased massively. Before, around 15% of the negative entries when it comes to negative CES were about how clunky it was to get in touch with us via live chat because mainly of the menu-based bot. That was my baby.

I liked working on it, but it wasn't the best when it comes to smoothness. Whereas now that type of negative feedback is virtually dissolved.

It was great to see that, especially because we were struggling with it so much in the menu-based bot era. Of course, The virtual agent is a little bit more sensitive to escalating the issue to humans because it can pick up the tone. A menu-based bot where you click through the options, it can't really feel your sentiment and tone of voice. So you have to be careful.

careful in tuning the instructions of the agent to avoid escalating as much as it would naturally do or organically do. Our escalation percentage in terms of interactions that are passed to humans has increased. But as I said, overall, I can see a better perception of the interaction with the virtual agent versus the bot.

I have to say that partially one of the reasons why we got on this journey is also because we wanted to be at the forefront of the new technology. So it wasn't just about the efficiency side and the big promises around AI, but also we wanted to be on the AI wagon for everyone else. So we started the pilot with Salesforce. We were, I think, the first company

And the number of repeated contacts from the same customer has decreased, meaning that the customer is not giving up on us on chat and reaching out via phone. But this new journey allows them to stay in the channel of choice and hopefully get to a nice resolution.

Exactly. And also an increase in escalated interactions. And by escalated, I mean passed over to a human, means a decrease in abandoned interactions. So as I said, you don't want your customers to give up on you. You want them to be able to reach a resolution and do it fast. So if the AI can do it, just interacting with the agent, Fine. If not, it's fine to pass it over to a human.

What you don't want is abandonment. So customers giving up on your agent, virtual agent, and choosing to phone us or email us.

I am trying to find different ways to leverage the FAQs side of things, as I said, so to increase and improve our knowledge base for the agent to have additional grounding, but also exploring more of

different type of grounding so that we could expand that side of action and support that the agent can provide to our customers when it comes to helping them about their specific booking, whether they're looking at a specific holiday or whether they have issues with their own specific account.

So, you know, I'm looking at different strategies we could take, but these are the main two avenues that I would like to explore.

And I have to say that I'm also looking at some of the new functionalities that will be coming up in Salesforce when it comes to workload management and co-piloting for agents when it comes to case management and workload management, as I think that would be key for us to free up time from the agents. So case summaries and case logging, analyzing reasons for contacts,

customer in Europe to go live with AgentForce. So it felt really good to be at the forefront of something completely unknown and exciting like AI, but also it was a learning curve and excitement turned into realization that it wasn't exactly what we expected, but also the excitement is not over and there is a long way It's a long-term investment. Of course.

taking that away from humans.

So we went live with the messaging sessions in September. And I think we went live in November with the first use cases.

Definitely, it would have to be Vito. It's a furniture company based in the UK. I was interested in getting a shelving unit from them. And so I contacted them. And instead of pushing me to a product page on their website, they set up a call with me to look into the design options. They wanted to see my space. So it felt really refreshing for a company that is quite big, historical.

It's kind of an icon of design to take the time and discover the interaction with the customer. They kind of understand that what you are buying from them is a long term investment. It's not cheap and therefore it needs to be the best for you and for your future and for your space. So, yeah, I really appreciated that.

And it reminded me how important sometimes it is not just to be efficient and quick, But taking the time and being intentional and being very human when it comes to the interaction with customers, it really makes a difference. So much so that I've ordered the shelving unit and I'm going to order another one for another room in my flat because I liked it so much. Yeah.

Absolutely. Like people can really feel when they're being sold on rather than actually supported. And that's the key, right? So the key to achieve the best customer experience in general, and it's easier said than done, is to really listen and really try to help customers. as that customer will stay with you for a very long time rather than a quick win.

They can perceive how genuine you are in the support that you provide.

It's the same thing with the AI. It's a long-term investment, I think. And that has to be the mindset when you start this kind of journey. It's learning by doing. It's a completely revolutionary technology. We're at the very beginning of it. And it's something that you need to take as a long-term investment and something that will stay with you for a very long time.

It's start simple. and make sure you have good data. Make sure you have a good foundation of knowledge base. Make sure you lean on the experts out there. Don't DIY this. And make sure you start the journey with AI, with the creation of long-term relationship with your customers in mind. It's not a quick win. deflection journey.

It's actually investing into relationship you have with your customers and tackling everything that you can take away from humans that is not as valuable and that focusing your teams on what humans are best at.

So, yeah, it's really, really fun to be working on this project where the technology is so new and it feels new for everyone. So you don't feel left out in a way. You're not alone. Yeah.

Probably AI is the new internet smartphone technology revolution. So it's like being in the midst of something that is probably life changing. So it's exciting and scary at the same time.

It took longer than expected. And one other challenge was reporting on return on investment. How successful in general was the handling of the conversations with our customers? It's not a quick win deflection journey. It's actually investing into relationship you have with your customers and tackling everything that you can take away from humans.

So as I said, we started with setting up use cases for the usage of agent force based on the highest reason for contact, which in hindsight, it was the only way we could do this being so unknown and, you know, unknown territory for everyone. But in hindsight, I think we probably chose use cases that were a little too complex for So it took a little longer to build the functionality.

As it wasn't just answering FAQs questions, we started off by asking the virtual agent to actually perform some actions. So it took longer than expected. And one other challenge was reporting on return on investment, which felt a bit more complicated than what I expected. within the functionality itself.

So it took a little bit of a DIY solution to try to get an understanding of how successful the deflection rate was for the agent and how successful in general was the handling of the conversations with our customers. Yeah, I would say those two. And probably the setup process, it's fairly easy, especially if you're looking into setting up a use case that is an FAQ-based type of use case.

But the time that it takes for testing and quality assessment of the interaction, it's quite extensive. And that's something I completely, not completely overlooked, but I kind of did not take into account that much when we started off this journey.

It was mainly the fact that the deflection rate we were expecting to achieve has not been as high as we expected. Partially, it could have been an unreasonable expectation to have, because, as you said earlier, you know, we thought it would be much more immediate, whereas this is something completely brand new and you kind of have to learn by doing.

So it's part of the challenge and part of the beauty of this is that you just learn as you go. So that was the challenge. And yeah, because the, um, deflection rate is not what we expected. We kind of have to try different ways of achieving and maximize the power of the technology that we have. Um,

And that's also why this is such an interesting journey, because you have to kind of learn while you're on the ground. There isn't much more we could have planned around this apart from the choice of use cases. So I think it's also about changing the perception on the metrics that you want to look at to measure success. So with technology like this, probably ROI is,

is not the best matrix to give it justice. People could consider also the impact on CISA and satisfaction of customers in the interaction with the virtual agent. The easiness of how It changes from a menu-based bot, which is what we had before, to an LLM that can literally pick up your tone and can have a conversation with you like a human.

If you read through some of the transcripts, it is impressive how seamless it is and how the customer didn't even realize they were talking to an AI agent if we weren't declaring it at the beginning of the conversation.

So it can pick up the context, it can resend the information. So it's also shifting the focus from ROI to a range of wider metrics. It's possibly something companies have to take into consideration when deciding to embark into an AI journey.

I was impressed by how easy it is to set up the actual facility, if you're not asking the agent to perform any actions, it's literally like typing a narrative and it's like you're talking to a human explaining a new employee what they're supposed to be doing. and telling them what they should and shouldn't do and how easy it gets on with it in a way, how fast it learns.

And if you read through some of the transcripts of communications with our customers, it is impressive how fast

seamless it is and how the probably the customer didn't even realize they were talking to an ai agent if we were in declaring it at the beginning of the conversation yeah um so that's something that i knew was going to happen but every time i see it i'm like super impressed wow this is happening Yeah. So the easiness of setting it up, I was really impressed with it.

Being used to coding and in the backend, you have to be an engineer and developer to set this up or to even test. This was kind of my bread and butter because you actually need someone that knows the business and how the customers are asking questions in order to set this up effectively.

So I felt it was a very rewarding adventure for someone like me that is not a technical person, but got out of it quite a lot and learned a lot about technology. Amazing.

Well, it takes a lot of time to quality check the interactions. And it takes longer than I expected, I have to say. It's quality checking and testing and troubleshooting and changing the instructions as you go. In terms of training it, it's I feel the key here is to use historical interaction, which is something that is coming up with Salesforce.

So being able to base the testing and the quality assessment of the interaction with the virtual agent with historical interactions, it will be a game changer. Because now it was a lot of manual work for a human, funnily enough, to be able to be on top of the quality of the interactions and tweaking what needed to be tweaked in the back end to be able to improve the quality of the interactions.

So we wrote up the use cases and we did a lot of testings based on different iteration of the same questions. And we used previous interactions with customers that we entered into the system and tried to test the reaction of the LLM.

So at the moment, our QA is human driven, meaning that we review a certain number of interactions per day. And we have put together a form where we score the questions. quality of the interaction, if there were any issues, if the issue needed to be escalated and if it was escalated correctly. And then whenever there is an issue flagged via this QA process, us and the development team that we have

work together with to set this up. We'll go in and try to see if it's an actual bug or whether it's just a tweak of an instruction, whether we need an additional KB article, whether we need to change the existing KB article. So there is a lot of trial and error. And at the moment, there is no functionality to bulk test or bulk QA, which I think in the future is coming and it will be very handy.

especially for a complicated industry like ours, where even the same theme in terms of type of query can be asked in 20 different ways.