Irina Gutman
Appearances
Experts of Experience
The Art of Conversation Design for AI Agents
Give me an example of the sexiest, the most complex, advanced agent that you've ever built. And my answer is, sure, we can do that. But actually, you want the most boring, the most repeatable, the most low-hanging fruit agent to start with.
Experts of Experience
The Art of Conversation Design for AI Agents
It can't take any of the actions or follow any instruction unless it has information, the knowledge to make those decisions and take those steps on. So data is the fourth component. And the last component is called channel. How is it interacting with us? Is it living on the customer's website? Is it an employee agent that lives in Slack?
Experts of Experience
The Art of Conversation Design for AI Agents
Channel is how this agent interacts with a human being or human beings being a customer facing or internal agent. So these five components of an agent really help us understand of how we're starting to make this transition from a technology to almost a digital assistant or digital employee.
Experts of Experience
The Art of Conversation Design for AI Agents
Mm-hmm. Thank you for explaining that. It really helps me to start thinking about how do I start onboarding an agent? Because I just cannot wait. I cannot wait until I have my own AI agent assistant. I'm like, if you know of any tools, please let me know because I'm so ready. We can build one for you. Okay, perfect. I can't wait. I really cannot wait.
Experts of Experience
The Art of Conversation Design for AI Agents
So you, as leading the professional services team, department at Salesforce, you are helping organizations to implement these AI agents. And I'd love to spend some time talking about like, how do you really approach that?
Experts of Experience
The Art of Conversation Design for AI Agents
If you can walk us through a little bit of the practices or the frameworks or the structures that you utilize to help leaders and teams think about where are we bringing these agents in and how are we training them with everything that you just shared?
Experts of Experience
The Art of Conversation Design for AI Agents
Absolutely. And when I meet with customers, sometimes customer would tell me, give me an example of the sexiest, the most complex, advanced agent that you've ever built. And my answer is, sure, we can do that. But actually, you want the most boring, the most repeatable, the most low hanging fruit agent to start with.
Experts of Experience
The Art of Conversation Design for AI Agents
Because what you want to start with is autonomous technology in the area where you want to augment humans. We want to leave creativity to humans. We want humans to address more complex scenarios. And we want to augment humans' capacity in the areas when there is repeatable, there is low risk. And those are the questions that humans may not want to answer.
Experts of Experience
The Art of Conversation Design for AI Agents
Or if we're looking at the assistive agent, the agent that lives inside the company and helps its employees, The same logic applies. You would want it to answer the question and help with those frequently asked problems or frequently occurring problems that humans are facing.
Experts of Experience
The Art of Conversation Design for AI Agents
And really, how can we powerfully leverage this technology safely? So we're going to be talking about all of that today. I am so excited for this episode. I've literally been like giddy for it. Irina, thank you so much for coming back. So great to have you.
Experts of Experience
The Art of Conversation Design for AI Agents
I spoke with the client recently from regulated industry, from financial services, and this particular customer was from the bank. And she said, well, you know how hard it is to implement agentic technology in the bank. And I said, absolutely.
Experts of Experience
The Art of Conversation Design for AI Agents
But if I ask you to make a list of questions that your customers call and ask that always, always, always end up in pretty much the same answer with very minimum variability. What products does this bank offering? How do I open account? I bet you'll be able to provide me a list mile long. She said, absolutely. I said, that's the list that we give to the agent.
Experts of Experience
The Art of Conversation Design for AI Agents
So we start with identifying, it's going back as to any other project, identifying the business objectives, identifying pain points, and saying, can this technology help address those pain points? We also want to start small, incremental, and low risk. In addition to that, we need to identify data that will support that type of a use case.
Experts of Experience
The Art of Conversation Design for AI Agents
But, and it's becoming more and more relevant with AI and agentic technology, turning on tech is the easiest part. The hardest part to tackle, and we talked about it last time, but guess what did not change, is organizational readiness. Yeah. I just had an executive actually awesome event. It's called Empowering Women in AI. And we had a room full of women executives.
Experts of Experience
The Art of Conversation Design for AI Agents
from all over tri-state area. And I'm local to New Jersey. Wanted to say New York, but local to New Jersey. But we did have an event in New York And what we talked about is exactly that. How do we get started? What are some of the blockers within organization?
Experts of Experience
The Art of Conversation Design for AI Agents
And we had absolutely incredible discussion that was focused on organizational readiness, on the new operating model, way more than about the tech. If I could boil it down to the successful approach, I think we'll talk about five phases. So first, just general readiness assessment, right? Kind of checking off all the points. Do we have minimum required criteria? Check, check, check.
Experts of Experience
The Art of Conversation Design for AI Agents
Just enough data, identify use cases, enough people that support this initiative and wouldn't run away. So that is checked. Second, incremental implementation and unlocking of the capability. Okay. You don't want it to be a year-long transformation program where you don't see any outcome until we implement all complex agents in the world. Three, foundational technology.
Experts of Experience
The Art of Conversation Design for AI Agents
Thank you, Lauren. Super excited to be here and share with you what's happening in AI world. And as you mentioned, a lot had happened. So in preparation for this episode, I was actually reflecting and thinking, we didn't talk like that long ago.
Experts of Experience
The Art of Conversation Design for AI Agents
We need to have data and integration capabilities in place in order to unlock agentic capabilities and broader AI capabilities. And that we can also do as an iterative approach, identifying just enough minimum criterias from a data and integration perspective, but also building out that holistic roadmap for the end state.
Experts of Experience
The Art of Conversation Design for AI Agents
Tell me a little bit more about that foundational tech. What are some of the key components that are needed there?
Experts of Experience
The Art of Conversation Design for AI Agents
From a Salesforce infrastructure perspective, and I'm going to talk about Salesforce, we have two key elements. We have data cloud, which is critical for agent force, which is the technology component that Salesforce refers to. That's our version of agentic technology. We call it agent force. Data Cloud, which allows for data syncing and console synchronization across various sources.
Experts of Experience
The Art of Conversation Design for AI Agents
And MuleSoft is our integration layer that allows to connect any data from anywhere. Combination of Data Cloud and MuleSoft unlocks any data needs, any communication needs, being it structured data, unstructured data within Salesforce or outside of Salesforce.
Experts of Experience
The Art of Conversation Design for AI Agents
Okay, great. So it's really having the technology to store and process the data that is then going to be used by AI, by your agents. Okay, great. Thank you for answering that. What's step four?
Experts of Experience
The Art of Conversation Design for AI Agents
Absolutely. Absolutely. Next step is what we talked about, organizational readiness. Do we need to build that new operating model? What are the new roles? And by the way, new roles are required. We're now talking about agent owners and agent monitors. Some people go as far as saying, do we need agent HR? I don't know how many agents who has, but with one, you probably don't need an HR department.
Experts of Experience
The Art of Conversation Design for AI Agents
Totally. I know people who are hiring AI managers, people whose role is just simply to manage the agents, which is a management position, but it's very different. Yeah.
Experts of Experience
The Art of Conversation Design for AI Agents
Exactly. And there are variations of those new roles. Those new roles need to fit within organizational operating model. Standing up center of excellences that help manage that, building out new operating model, defining that process of introducing new roles, enabling new roles, upskilling people is the next absolutely critical component. And we talked about it before.
Experts of Experience
The Art of Conversation Design for AI Agents
And when we first started and talked through different types of AI and evolution of AI, I was mentioning that agentic technology is somewhere there on the horizon, but let's just not talk about it yet. Literally a few months later, not only we're talking about it, it is the primary focus for Salesforce, as well as many tech companies.
Experts of Experience
The Art of Conversation Design for AI Agents
If some of those things were considered optional with more traditional technology, AI and especially agentic technology truly changed it and made it, as we said, as foundational as probably as data. Because if your organization is not ready to take responsibility for this technology, no one's going to use it. Change management is even more and more critical. Last piece is the roadmap.
Experts of Experience
The Art of Conversation Design for AI Agents
We don't want to stop with one agent. Understanding what that ultimate North Star and how organizations are going to progress in maturity of introducing AI, agents, and data is the last piece of the puzzle that I would like to introduce.
Experts of Experience
The Art of Conversation Design for AI Agents
Awesome. I want to go deeper into the new operating model. I think a lot of leaders are thinking about And if they're not, they probably should be thinking about what is my organization going to look like? Like you said, what are the roles, the skills, the governance that we need to implement?
Experts of Experience
The Art of Conversation Design for AI Agents
I know people who are hiring AI managers, people whose role is just simply to manage the agents. The thing about agentic AI that is just so incredible, it can think for itself.
Experts of Experience
The Art of Conversation Design for AI Agents
And I'm curious to know a little bit of, you know, any examples you've seen of this being done really well. And I know it's a lot of this is future thinking. We don't even fully know what it's going to look like in six months from now. But yeah, what have you seen?
Experts of Experience
The Art of Conversation Design for AI Agents
You're right. Absolutely. This is very new and future thinking. And a lot of my more mature customers like, do you have an example? And I'm kind of say, well, you're the first one. So let's figure out together. Totally. But more and more, we have instances when some of the leading customers are being trailblazers in this area.
Experts of Experience
The Art of Conversation Design for AI Agents
But let's talk about kind of some foundational components of that operating model. It is absolutely critical. So when we're talking about bringing in new operating model, it is probably the easiest way to think in the form of standing up some kind of body, meaning a center of excellence or practice focusing on that.
Experts of Experience
The Art of Conversation Design for AI Agents
In fact, Forbes magazine says that the best way to handle emerging technology is to have some kind of a center of excellence around it. There are debates, oh, center of excellence can be obsolete, they're that ivory tower. Not if you design it right. If you design it right, there's a way to make center of excellence really foundational and a way to help control and also drive that innovation.
Experts of Experience
The Art of Conversation Design for AI Agents
Let's discuss some of the key elements. We need to align on strategy and objectives of what it is. We need to get visible and actionable leadership support, understanding that it's not an extracurriculum activity, that it's actually critical for the organization and everyone is on board.
Experts of Experience
The Art of Conversation Design for AI Agents
What's that common vision that we're working towards and what's going to be the charter of this particular center of excellence? For short, I'm going to refer to it as COE. And let's assume it's an agentic or AI-focused COE.
Experts of Experience
The Art of Conversation Design for AI Agents
Do you have any examples of what that vision might be?
Experts of Experience
The Art of Conversation Design for AI Agents
The vision might be is to have agent-first mindset and to prepare organization to intake agentic technology and to elevate, I don't know, our customer service and to the next level with the help of agents. Granted, I made it up on the fly, so.
Experts of Experience
The Art of Conversation Design for AI Agents
No, all good, all good. I just wanted to understand, because it can be so... broad and I, but I also, the vision is so important because it at least starts to give us a little bit more focus in what is the, what is this body of work about? It's the headline. We need to set that headline to kind of guide everyone towards something.
Experts of Experience
The Art of Conversation Design for AI Agents
So very happy to be with you and unpack this very relevant topic.
Experts of Experience
The Art of Conversation Design for AI Agents
What's the headline, but also what is company trying to achieve? Where's the company trying to get to? And then we can articulate how's that particular body within organization is going to help company to get there. That's where it comes. Vision is supported by strategy, by objectives, right? Then we need to define this center of excellence itself.
Experts of Experience
The Art of Conversation Design for AI Agents
That would entail operating model, meaning how is it going to be structured? Is there going to be an overarching leadership committee? How is business and IT are going to work together in that operating model? We talked about it during AI. It is even more critical with agentic technology. IT alone, unfortunately, so sorry everyone who works for IT, but you can make that decision alone.
Experts of Experience
The Art of Conversation Design for AI Agents
Amazing. I want to make sure that everyone is caught up to what agentic AI is, because it is different from the generative AI and the predictive AI. So can you just really quickly define those different types of AI and where we are today?
Experts of Experience
The Art of Conversation Design for AI Agents
It has to be in super strong partnership with the business because guess what? AI is augmenting most likely a business employee. Therefore, IT becomes innovator, capability provider. But business needs to be in the room saying which capabilities they need, which business problems need to be addressing. So critical component of that operating model is that business and IT partnership.
Experts of Experience
The Art of Conversation Design for AI Agents
And then there are kind of discipline going across, such as governance, risk mitigation, and responsible AI usage. Another critical element that I would say is a fun element is innovation.
Experts of Experience
The Art of Conversation Design for AI Agents
You can have, I wouldn't recommend having a genetic COE without explicit focus on innovation to drive that culture of innovation, but also to drive the culture of experimentation, to give a specific focus area where people can freely try new things, try new technology, fail quickly, improve upon, and adjust the product companies need. Another important area is education.
Experts of Experience
The Art of Conversation Design for AI Agents
We just talked about all the new skills that we will need to define that will go into that operating model. You said agent manager. I said agent owner. Some people say agent supervisor. I've heard roles that is a content creator, and we'll talk about that, why it is important. Prom builders, et cetera, et cetera. They're roles ranging from the ones that you need to manage agents
Experts of Experience
The Art of Conversation Design for AI Agents
to the roles that you need to have to build agents, such as I said, content creators or prom building, because guess what? Articles that we human consume need to be written differently for consumption of AI. AI consumes information differently from humans. All of those roles need to be upskilled, trained, and then continuously trained as new innovation or new capabilities come about.
Experts of Experience
The Art of Conversation Design for AI Agents
And with AI, it is faster than we've ever seen before. So that continuous upskilling needs to be a pretty robust process. And that is why we need to have explicit focus on it as well.
Experts of Experience
The Art of Conversation Design for AI Agents
Where can leaders go to upskill their teams? Because like you said, it's happening so fast. And if we try to build our own systems of education, I assume we're going to be a little bit slow. Is there any resources or things that you would suggest people go to look at or learn from?
Experts of Experience
The Art of Conversation Design for AI Agents
No, this is definitely a hard one. And I can tell you where Salesforce are currently addressing that situation as well. As you know, my team is explicitly focused on AI and we're working very closely with product organization to stay ahead of the curve. But Salesforce professional services, 10,000 people. Salesforce partner ecosystem is multiply that by 100,000.
Experts of Experience
The Art of Conversation Design for AI Agents
Absolutely. Yes. Let's get down to the definition and kind of bring everyone up to speed. Let's start with predictive AI, which probably been around for longer than all the other AIs. And the definition is literally in the name for all those technology categories. Predictive AI focuses on making predictions based on data, based on rule, and based on the structure that we provided with.
Experts of Experience
The Art of Conversation Design for AI Agents
All of those experts need to stay up to date on the latest capability and technology. Yes, sure. There are million classes in very respectable institution that will give you foundation and to give you a core knowledge. But then it goes to the next level. So there are multiple upskilling, right?
Experts of Experience
The Art of Conversation Design for AI Agents
Understanding the technology, understanding the essence of this particular, what agents are, how they construct it, how you do prompt engineering. And that skills could be gained from any respectable known institution. There are classes, there are courses there. But then there's a next level of specific to that product. And that I would suggest getting from
Experts of Experience
The Art of Conversation Design for AI Agents
partnering, being at a Salesforce, or if you're working with a different technology provider, working with that particular company to know the nuances of that product. And that's what we're doing. We're working closely with product. We're putting new programs in place that combine propagation of knowledge and propagation of experience across entire professional services.
Experts of Experience
The Art of Conversation Design for AI Agents
And we came to the realization that there's no longer an option to spend three months putting a training program together. Materials and information needs to be available very fast and we need to build into those enablement program constant updating and changes because sometimes the answer I provide you today might be obsolete in a week or two.
Experts of Experience
The Art of Conversation Design for AI Agents
It's happening so fast. I mean, the good news is that AI can help us put those training programs together really quickly, which is great. But what's coming to mind for me right now is the fact that the vendors, for example, Salesforce, you now have another role of education.
Experts of Experience
The Art of Conversation Design for AI Agents
That goes beyond simply this is how you use our tool, but this is how you actually bring your tool into your organization, which is exactly what you do. And I think it's something I tell my clients all the time that if you want to start using AI, first look at the vendors you're already utilizing and go and see what technology they have rolled out. And then
Experts of Experience
The Art of Conversation Design for AI Agents
Ask them to help you learn how to bring it inside. And I just I think that there's this new role almost for SaaS companies to not only say here's technology, but here's actually how you can utilize it in business. the way that's going to be most effective.
Experts of Experience
The Art of Conversation Design for AI Agents
And it goes beyond just simply like click these buttons, but actually like, you know, here's the roles and the education your team needs and all of that, because that's really essential to using AI.
Experts of Experience
The Art of Conversation Design for AI Agents
No, you're, absolutely right. And that's why it folds all into those component of the organizational readiness worksheet, because this is exactly what it will focus on. In addition of what this tool and technology is, is how it's actually going to be used and how companies are going to assume responsibility for it and continuously work with it because AI is ever evolving. It's not once and done.
Experts of Experience
The Art of Conversation Design for AI Agents
And that's what makes it unique and super uncomfortable for a lot of us, Because if you build a system, unless somebody goes and changes the code, it's pretty much going to be operated as expected. With AI, with its learning capability, with its reasoning capability, it is going to evolve. The data that it's grounded on is going to change.
Experts of Experience
The Art of Conversation Design for AI Agents
So there's a lot of variability in this technology, and therefore it requires constant monitoring, updating, and kind of having a pulse on where it is and where it needs to go.
Experts of Experience
The Art of Conversation Design for AI Agents
Yeah. The other thing I'm really curious about, and I'm, I'd love to even hear this from like a personal perspective from you is what are the skills we need to be learning as we look towards the new AI age? Like what are the human, what's the human advantage that we can really like focus on?
Experts of Experience
The Art of Conversation Design for AI Agents
And I'm curious, even just from a personal perspective, what have you been learning and trying to upskill yourself on?
Experts of Experience
The Art of Conversation Design for AI Agents
What's foundational to all AI is how we interact with it. Because AI is grounded or is based on natural language models or natural language processing. Yes, when you are a user of AI, it will respond to you. And as I said, use example of broken English or slang. But in order for it to do it, somebody had to tell it to do that. And that's where we get to prompt engineering.
Experts of Experience
The Art of Conversation Design for AI Agents
A good example in using Salesforce example would be lead sales lead qualifications. will give all the necessary rules, information, and data to predictive AI. Using that information, it would categorize and qualify the leads and put it in the priority category for salesperson to focus on.
Experts of Experience
The Art of Conversation Design for AI Agents
Prompt engineering is absolutely a foundational skill set to have. And when your agent is not actually answering the questions the way you want to, one of the first things we look at is how the prompt is constructed. Sentence structures makes a difference. Punctuation makes a difference. Word choice makes a difference.
Experts of Experience
The Art of Conversation Design for AI Agents
Like you can be free with the words when you interact with the finished product, but when we construct an agent, all those things are critical. Prompt engineering is a new skill with a new requirement. I also mentioned that AI consumes information differently. When we refer to knowledge articles, like, oh yeah, we have million knowledge articles.
Experts of Experience
The Art of Conversation Design for AI Agents
Sometimes we see that those knowledge articles need to be restructured for AI consumption. So there are roles, there are skills to restructure unstructured data. I know it's a mouthful for AI consumption. So AI interprets that information, make it kind of easier for AI to interpret this information that it could provide a better answer.
Experts of Experience
The Art of Conversation Design for AI Agents
And another fun one, and you mentioned in the beginning that sometimes you want to scream at the bot and say, what are you doing to me? Are you trying to torture or deflect me? That talks to customer experience. And we want agents to provide amazing customer experience. Experience designer is a known role.
Experts of Experience
The Art of Conversation Design for AI Agents
Now we want them to elevate their skills to the next level, almost like conversation designers. Designing experience between an agent and a human, is another flavor of a skill that is very new. And that is from a person who is responsible for the team developing the agent. Those are some of the technical skills.
Experts of Experience
The Art of Conversation Design for AI Agents
I don't need to be an expert in those skills, but I do need to have foundational understanding. However, what I was focusing on is what we were talking about is understanding how that organizational ecosystem needs to change. How do we approach change management differently?
Experts of Experience
The Art of Conversation Design for AI Agents
What are all those nuances of things that we've done before from standing up those organizational operating model, from defining all those departments and new roles, from integrating all of that new stuff into organization? What needs to be nuanced? for a genetic technology? And how is that different from more of a traditional change management?
Experts of Experience
The Art of Conversation Design for AI Agents
A lot of it is the same, but still this nuance is important.
Experts of Experience
The Art of Conversation Design for AI Agents
Yeah, we're dealing with a very different technology than other technology rollouts. And it's also the nature of how it changes needs to be built into how we're operating. And the thing I also wonder is like, Now that learning how to use AI, learning how to design our organizations around AI, everything we've been talking about is table stakes.
Experts of Experience
The Art of Conversation Design for AI Agents
Or case qualification, when again, using a set of rules, AI would qualify and classify various cases for the service agent to work on. Now moving on to a generative AI. Again, the definition is in the name. It generates dynamic content.
Experts of Experience
The Art of Conversation Design for AI Agents
But then I think about how are organizations creating a competitive advantage? And the thing I think about, and I love your thoughts on this, is that it's really like in the connections and the way that we're listening to what it is that the customer really wants and needs.
Experts of Experience
The Art of Conversation Design for AI Agents
And it's almost like now that AI is dealing, handling a lot of these tasks that we've typically been bogged down with, what's next? I'd love your thoughts on that. Like now what can we focus our attention on?
Experts of Experience
The Art of Conversation Design for AI Agents
Well, we still need to focus our attention on what's happening now. I'm getting ahead of myself. And what's next? It's really, actually, it's funny you ask because it really depends on people's personality and readiness for change and readiness to embrace this technology. When you're talking about, okay, we're good with AI, we're good with agentic arena, what's next?
Experts of Experience
The Art of Conversation Design for AI Agents
Some of my customers are like, really? I can turn in email generation and no one's going to come after me? So we have a full spectrum of maturity when it comes to AI. But I think what's next and very near or already here is the agent, multi-agent. That is the most logical next evolution is that most of the companies even now have multiple agents.
Experts of Experience
The Art of Conversation Design for AI Agents
When we talk about agentic technology, the natural progression for a lot of companies is internal assistive agent or multiple internal assistive agent, meaning they're not customer facing. They're still autonomous, but they are within the company and they're interacting with employee and they're there to help employees do their job.
Experts of Experience
The Art of Conversation Design for AI Agents
Then you have customer-facing agent, and then you have agent-to-agent collaboration. When you have what we call the multi-agent network, and then you have agents. And it doesn't mean from the same company. It could be Microsoft agent talking to a Salesforce agent. So that is a broad spectrum of multi-agent collaboration, I think, is the next wave. And I can't tell you what's going to happen after.
Experts of Experience
The Art of Conversation Design for AI Agents
It's going to be way too fast even for me.
Experts of Experience
The Art of Conversation Design for AI Agents
I mean, I'm just I just can't wait until I have my own agent who's been doing all my customer service chats for me being like, oh, I need to order a new dress size or can you this thing showed up broken. Can you send me another one? Yeah.
Experts of Experience
The Art of Conversation Design for AI Agents
So Saks actually has an agent like that. Her name is Sophie, and she handles this type of questions for Saks. They were one of our first agentic customers.
Experts of Experience
The Art of Conversation Design for AI Agents
And the demo that was shared at Dreamforce last year, all the forces. Dreamforce. The demo that was shared at Dreamforce last year blew my freaking mind. It's on YouTube. If you want to find it, the sex example, it was great. And I'm really excited for that. My question for you is, what are you the most excited about? You're in such a unique place where you can see what is happening in this space.
Experts of Experience
The Art of Conversation Design for AI Agents
What are you the most excited about?
Experts of Experience
The Art of Conversation Design for AI Agents
So I'm going to use an example of a session, as I said, that we conducted this week for women in AI. And we started the session with the exercise. We asked participants to go to a word cloud space and put one word that comes to mind when they think about agentic technology and digital labor. And the responses that came back were mixed.
Experts of Experience
The Art of Conversation Design for AI Agents
It still uses the data, but it is using natural language models to generate dynamic content based on the request or what we call as prompt or basically what you put in ChatGPT because ChatGPT is the most common example of generative AI.
Experts of Experience
The Art of Conversation Design for AI Agents
Some were optimistic, word opportunity popped up, but some of the words were job replacement, scary, new. Then we conducted this discussion when we talked about how companies get started. What is the incremental and iterative approach? What are some of the foundational elements that you need to put in place? How we partner with the customers to take it through the journey?
Experts of Experience
The Art of Conversation Design for AI Agents
And after only two hours, we did this exercise again. Oh my God, such positive word. I did not see a single negative word on the screen. People were excited. Opportunity still was the hardest word, highest word. But people were excited. They said they use sports as innovation, new roles, new opportunities. So very, very forward looking in just two hours.
Experts of Experience
The Art of Conversation Design for AI Agents
And that is what I'm most excited about is the chance to partner with my customers to help them understand and embrace this technology. and to help them start taking advantage of this technology. Because if it's done right, and I love your attitude, I love that my company can build agents for you to help you with all of that.
Experts of Experience
The Art of Conversation Design for AI Agents
Nobody wants to be a headline for the wrong reason or have a lawsuit. Designing experience between an agent and a human is another flavor of a skill that is very new. Critical component of that operating model is that business and IT partnership. Turning on tech is the easiest part. It requires constant monitoring, updating, and kind of having a pulse on where it is and where it needs to go.
Experts of Experience
The Art of Conversation Design for AI Agents
But those that are a bit more of a skeptics or have a more of like an enterprise or industry where it's harder to introduce, to help them handle those objectives and show them how this technology could be really helpful is very rewarding. The other thing that I'm excited about is when I talk to my innovation team, I'm constantly blown away. I'm like, what? We can do that?
Experts of Experience
The Art of Conversation Design for AI Agents
This is not science fiction. You actually build it? And it is amazing how they take capability of Salesforce AI product and turn it into the solution that elevates customer experience to a new level.
Experts of Experience
The Art of Conversation Design for AI Agents
I mean the opportunity like you said that word came up a lot the opportunity at play is immense but it is almost so big that it is scary of, I mean, it's a lot of change and it's a lot of change happening really quickly. And I could not agree with you more that there's, you know, the, the skeptics in, in the room, I mean, we should all be skeptical to some extent.
Experts of Experience
The Art of Conversation Design for AI Agents
I think it's healthy to have a little bit of that.
Experts of Experience
The Art of Conversation Design for AI Agents
Um, but, but you should talk to my responsible AI team. No skeptics there.
Experts of Experience
The Art of Conversation Design for AI Agents
I love it. I love it. But this is changing whether we like it or not. So let's jump on. Let's jump on the bandwagon. Actually, now that you bring up responsible AI, since we do have a couple minutes left, I do want to talk, just touch on that a little bit around... What does it mean to be responsible with AI? Because it's a little bit feeling like the Wild West out there.
Experts of Experience
The Art of Conversation Design for AI Agents
And how can organizations approach this in a responsible way so they don't have a lawsuit on their door next week?
Experts of Experience
The Art of Conversation Design for AI Agents
I want to stop you right there and just make sure, because this is the LLMs. Generative AI is LLMs. So it's when we can ask a question and it can pull from many different sources where predictive was really, here's the box we want you to operate in.
Experts of Experience
The Art of Conversation Design for AI Agents
No, thank you for bringing it up. It is a very, very important topic. When we're designing agents, we have a dedicated and explicit exercise called guardrails and risk management definition. What we're advising our customers to do as they define the role of the agent and what job it needs to do is
Experts of Experience
The Art of Conversation Design for AI Agents
We asked it to define risk in categories that are very familiar to them, people, process, data, and technology. So we work with the customer to list all the possible risks, all the possible violations that this agent could cause in those four categories. And then we take each of those items and we say, okay, how are we going to mitigate it? Are we going to mitigate it with people?
Experts of Experience
The Art of Conversation Design for AI Agents
Are we going to mitigate it with process, with data, and technology? Yes, the risk is inherent. There's some variability, vulnerability inherited in AI. But it does, as you said, it's not an option. This technology is here to stay. So what we're doing is developing framework, just the one that I explained to you, of how companies can go systematically online.
Experts of Experience
The Art of Conversation Design for AI Agents
from defining an agent to defining risks, to defining risk mitigations, and then turning them into what I referred to earlier in our conversation as guardrails, which is instructions that we give to the agent what it's not allowed to do. The other thing that we do to make sure we do stress testing for bias and toxicity.
Experts of Experience
The Art of Conversation Design for AI Agents
Salesforce already has a trust layer that is very, very robust and pretty much nothing can get in past us, nothing bad from the outside. However, we put agent through additional stress of bias and toxicity testing. When we overload the system, we talk to the agent in bad language, profanities, using racist slurs on purpose.
Experts of Experience
The Art of Conversation Design for AI Agents
It's not a pretty testing, but it's absolutely necessary because as you said, nobody wants to be a headline for the wrong reason. or have a lawsuit. So that type of testing is absolutely critical. And my responsible AI team developed a methodology of how to do that kind of testing. And we set thresholds really low, like 1% or less than 1%. And I'm proud to say that our agents passed.
Experts of Experience
The Art of Conversation Design for AI Agents
Generative can go further than just a simple set of find me, for example, a sales lead that has this revenue in this industry with this number of employees. Okay, great.
Experts of Experience
The Art of Conversation Design for AI Agents
Amazing. Just for anyone who's thinking about utilizing, which we all should be, how are we utilizing agent technology, agentic AI? We have to have those conversations. What are the risks at play? Just put them all out on the table. Let's make sure we're thinking about the edge cases because they are there.
Experts of Experience
The Art of Conversation Design for AI Agents
And when we are not hand in hand with our AI the whole way through, when the AI is operating independently, we need to make sure that we've thought about all the risks at play. So thank you so much for sharing that, Irina. As per usual, we always like to ask this question to end our show. I'd love to hear about a recent experience that you had with a brand that left you impressed.
Experts of Experience
The Art of Conversation Design for AI Agents
So I thought about that. I think I'm going to go with Starbucks. I know it's simple and I know everyone's familiar with it, but you know what? When I'm getting off New Jersey Transit, headed to my, on my way to the office, I click on my Starbucks app. It allows me to set my favorite stores. There are a few, I have one set up for San Francisco because Salesforce headquarters are in San Francisco.
Experts of Experience
The Art of Conversation Design for AI Agents
There is a Starbucks across the street from our office. So it has my different locations. It has my favorite stores per location. I click, it remembers what I've ordered before. It displays my frequent orders. I usually go with caramel macchiato. So it comes up, I click. And when I get to the store, I can see on Tableau saying, hey, Irina's order moved from received to in progress.
Experts of Experience
The Art of Conversation Design for AI Agents
I know next it's going to be on ready. It is simple. It is simple to use, but it has just enough key elements to make my life easier. easy. And also from a customer service perspective, I went to the store that happened to be backed up. I had an urgent meeting. I came to the assistant and I said, I'm so sorry, but this is the situation. Is there any way you can help me?
Experts of Experience
The Art of Conversation Design for AI Agents
Correct. Correct. You're absolutely correct. So now we're going to go one step further and talk about agentic technology, which is the topic of today's conversation. It sees the components of everything that we talked about before, but elevates it to the next level. It still has rules. It follows the rules, but it makes a decision how to act based on those rules.
Experts of Experience
The Art of Conversation Design for AI Agents
She looked and said, in fact, You are next in queue. Let me move you up so they start processing your order. So it's not only visible to me as a customer. There is an option for an assistant to see where I am, to do the overrides and to address my needs. And this is how human plus technology working together provides amazing experience. Yeah.
Experts of Experience
The Art of Conversation Design for AI Agents
I love this example. Speed, efficiency, thinking about what you need. You're in the morning getting off the train. You don't want to stand in line. And the key thing that you shared is that it's also enabling the staff to support you, to jump in and intercept like a team. So thank you so much, Irina. This has been such a wonderful episode. I'm sure I'll see you in a As things keep changing.
Experts of Experience
The Art of Conversation Design for AI Agents
And we'll have something new to talk about, I'm sure.
Experts of Experience
The Art of Conversation Design for AI Agents
I'm sure we will. Well, I wish you a wonderful day and we'll talk soon. Thank you so much.
Experts of Experience
The Art of Conversation Design for AI Agents
It still uses data, but it uses the data to almost reason on what action to take and how to interact. And yes, it interacts to us based on the instruction we're provided with, but it uses natural language processing and converses with us in the natural language.
Experts of Experience
The Art of Conversation Design for AI Agents
So to summarize, we still have rules, we still have data, we still have actions and instructions, but this agent uses this information to reason, to make decision, and to converse with us in natural language.
Experts of Experience
The Art of Conversation Design for AI Agents
One of the examples of an agent, and we will talk further about different type of agent, but the funnest example that I can give is if you've been to Bay Area, they have VAMOS, which is the self-driverless car. This is an example of autonomous technology that operates without a human, makes decision of how to switch lanes.
Experts of Experience
The Art of Conversation Design for AI Agents
It takes the information provided to it, instruction provided to it, as well as the data that it takes from the road and and makes decision how to switch lanes, how to accelerate, when to stop, etc.
Experts of Experience
The Art of Conversation Design for AI Agents
I love that you use Waymo as an example, because if you are in a city that has Waymo and you see them driving around, it's this crazy experience. I mean, I'm still like, I can't believe we are here, but we are. They are driverless cars. They are following the rules of the road. And when you think about it, there are pretty strict guidelines for how to drive. But then there is also this element of
Experts of Experience
The Art of Conversation Design for AI Agents
intuition that is required of, oh, I feel like that car is coming at me really fast. Maybe I should slow down and let it pass or anything like that. And the thing about agentic AI that is just so incredible, which we're going to dive into so much more, but the thing about it that is so incredible is that it can think for itself. All right. Kind of creepy, but here we are.
Experts of Experience
The Art of Conversation Design for AI Agents
And there's so much there's so much opportunity.
Experts of Experience
The Art of Conversation Design for AI Agents
We refer to it as a reasoning engine, which kind of sort of an evolution of thing. One step further, it also has memory. So it can tap in into the previous data and information and make decision not only on the data provided at this time, but also from the learnings of data that it has available to it before. So we refer to it as it has memory in addition to reasoning.
Experts of Experience
The Art of Conversation Design for AI Agents
Great. We love this. I want to make sure that we ultra define this for all the folks in customer experience, because I do hear chatbots and AI agents being intermingled. And can you just now apply this definition to the difference between a chatbot and an AI agent?
Experts of Experience
The Art of Conversation Design for AI Agents
Perfect. So let's keep in mind what we just learned about the genetic AI, right? Components of reasoning. Memory, conversation, being able to infer action based on the information that's available to it and compare it to the chatbot. Chatbot follows a very prescribed process flow. It is pre-programmed how to intake information and how to respond based on that prescribed information.
Experts of Experience
The Art of Conversation Design for AI Agents
When I talk to my innovation team, I'm constantly blown away how they take capability of Salesforce AI product and turn it into the solution that elevates customer experience to a new level. I'm like, what? We can do that? This is not science fiction.
Experts of Experience
The Art of Conversation Design for AI Agents
It cannot deviate. It cannot change a course of action or interaction based on that process flow. And it does not understand natural language. For example, let's say we're using a chatbot, and don't take me wrong, chatbots are super effective and super fast for addressing repeatable processes with very few deviations.
Experts of Experience
The Art of Conversation Design for AI Agents
And in fact, if you have that instance that it's literally a repeatable process that has prescribed process flow with very few deviations, maybe chatbot is an effective technology. However, if we, let's say, ask a chatbot, where is my order? You have to ask it in a certain way and provide an order number. Otherwise, poor chatbot might be lost.
Experts of Experience
The Art of Conversation Design for AI Agents
If you interact with an agent, you say, hey, the thing that I wanted from you last week, what's up with that? An agent literally should be able to interpret the slang, the various variation of the language, and come back to you saying, oh, you're looking to find out the status about your order. Let me get that for you. Would you mind providing an order number?
Experts of Experience
The Art of Conversation Design for AI Agents
Agents should be able to infer, based on the information provided to it, what the next step should be and what response should be to a human. Agent could pick up on the tone and change the way if I speak in slang, maybe in an interaction or two, agent will start responding to me in some slang versus a formal language. Chat cannot deviate from that.
Experts of Experience
The Art of Conversation Design for AI Agents
Every company using agent, they might want that agent to reflect the tone of this company to begin with. More formal, more casual, cool versus some other type of a flow representing company's brand. You have that flexibility with an agent. Chatbot would not be able to do that.
Experts of Experience
The Art of Conversation Design for AI Agents
I think from a consumer standpoint, it is, I'm just so excited as a consumer because we've all had that experience where we're on with a chat bot and it is not understanding what we are trying to say. We are, you know, maybe there was a spelling mistake and it's just like taking us a totally different direction and it's just downright frustrating. But with an agent, it's,
Experts of Experience
The Art of Conversation Design for AI Agents
I feel so much more willing and able to actually solve my issue with that agent. Like before I felt, oh, chatbot, you're just deflecting me because you don't want to talk to me. You don't want to hear what my problem is. And now with an agent, I feel seen and heard and my problems getting solved faster. So it's really like this incredible balance between
Experts of Experience
The Art of Conversation Design for AI Agents
an actual human who can be slower at answering these things at times. And, you know, the chatbot that just wasn't solving my problem at all. So yeah, I'm really excited about this. So last time you were on, we talked a lot about human in the loop. But this is now advancing. How are you working with agents now? If you can kind of give us an overview of that.
Experts of Experience
The Art of Conversation Design for AI Agents
Hello, everyone. Welcome to Experts of Experience. I'm your host, Lauren Wood. Today, we are diving into a very, very big topic, agentic AI. and what it means for businesses, for workforces, for leaders, and really humans at large.
Experts of Experience
The Art of Conversation Design for AI Agents
Absolutely. And by the way, yes, you are allowed to make spelling mistakes with agents, which is awesome.
Experts of Experience
The Art of Conversation Design for AI Agents
Great, because I make them all the time.
Experts of Experience
The Art of Conversation Design for AI Agents
But going back to your question, we did refer to a previous iteration of generative AI as a human in the loop, meaning that human makes the checks and final decisions. Now you have a technology that almost operates agentically. And we change our phraseology from human in the loop to human plus AI.
Experts of Experience
The Art of Conversation Design for AI Agents
We now have AI augmenting and expanding humans' capability with an assistant of this digital, let's say, assistant called agent. Humans still have a decision power, and we provide agents with instructions what to do or not to do.
Experts of Experience
The Art of Conversation Design for AI Agents
But once those instructions are provided, that agent is able to assist us in a way that literally translates to human plus AI versus human is in the loop or in the hell making decisions for AI. Mm-hmm.
Experts of Experience
The Art of Conversation Design for AI Agents
So I think the thing about this is it's much more analogous to an actual human or an employee or a customer experience agent. If we think about here's the guardrails, here's the types of things we want you to answer. Here's how we want you to answer them. Now, instead of saying, yes, answer it, we're saying, okay, you answered this. I'm going to give you feedback so you can do it better next time.
Experts of Experience
The Art of Conversation Design for AI Agents
Because it's... really being like trained through those types of interactions of feedback where it's operating on its own. We don't have to say, yes, go do that. It's doing it, but we now need to work with it in helping it to grow.
Experts of Experience
The Art of Conversation Design for AI Agents
Absolutely. And before we talk about agent as potentially when people refer to it as digital labor, I think it would be helpful to understand main components of an agent. Because that will help us have conversation about agent being that digital employee. Agents have five components. We're going to start with role. Just like a human will have a job description,
Experts of Experience
The Art of Conversation Design for AI Agents
an agent is going to play a specific role. For example, let's say this is going to be a customer service agent. And the job of this customer service agent is to answer a specific set of customer questions. So we just define the role of an agent. Next step is action. What is it actually going to do?
Experts of Experience
The Art of Conversation Design for AI Agents
Well, we want it to answer FAQ, frequently asked questions, and we want it to do certain things based on the instructions that we're going to do, that we're going to give it. And we also don't want it to do certain things. So actions are based on instructions that provide the agent. And it always includes things that we wanted to do and absolutely things that we don't want it to do.
Experts of Experience
The Art of Conversation Design for AI Agents
Which brings me to our next component. It is called guardrails. Guardrails are super, super critical. When you're putting autonomous technology in front of a customer, and by the way, we'll talk about it, agent being assistive internal to the company versus customer facing agent.
Experts of Experience
The Art of Conversation Design for AI Agents
And we are joined by Irina Gutmann, the global leader of AI professional services at Salesforce, who is here for the second time because so much has happened since we talked to her six months ago. And we're really going to dive into what is happening in AI when it comes to the agentic AI layer? How are businesses implementing it? How are organizations adapting?
Experts of Experience
The Art of Conversation Design for AI Agents
But let's say in our example, we're talking about a customer facing agent that is playing a role of the customer service representative. We define its role. We know it needs to take specific action, but now we're going to give it guardrails to tell it what it's not allowed to do. For example, it can only answer questions about the order and whether shipping is free or
Experts of Experience
The Art of Conversation Design for AI Agents
or you have to pay for based on some zip code. But if it goes beyond a certain set of questions that agent knows how to answer, it has to hand off to a human. That's where human plus AI comes in, is that always, always, always, as part of those instructions or guardrails, there is an instruction on when to hand off to a human. The next component that we need to discuss is data.