[MUSIC PLAYING] PRIYANKA VERGADIA: Thank you for– oh my god, I can’t even see at the back, there are so many people Thank you for so many people attending, especially on a Sunday evening It’s exciting And I hope you all had enough tea, coffee And I’ll try and do my best so you don’t sleep So that’s my goal for today All right All right, so topic for today is Conversational AI, which is the new user experience or the new user interface And we’re going to see how conversational AI is becoming the new user interface How many of you all here have heard about conversational AI or have been introduced in some way, shape, or form? OK, it seems about 50% And for those of you who have not, we are going– you’re at least going to learn and get a flavor of it today And I hope you start to think about conversational AI in some of your projects and businesses So with that, just a little introduction about me I’m Priyanka Vergadia I’m a developer advocate at Google For those who don’t know what developer advocates do, I call myself a happy path engineer I build scenarios and demos and sample codes So communities like these can utilize them, how we internally build tools, and then we create samples so that it can be utilized by external communities And then I also do things like this where I interact with communities to learn more from how you’re utilizing our products and then take that feedback back to our product teams so we can improvise and improve based on the feedback that we get from the communities So that’s my role, in short And I can be found on the internet If you have any questions later, if you don’t get a chance to ask, if you’re too shy to ask, you can always reach me on the internet @pvergadia Most of my blogs, Twitter, and GitHub, all the code that you’re going to see today can also be accessed if you search me through @pvergadia in any of those channels With that, we’re going to start with conversational AI A bit of a history– conversation AI is actually not very new We’ve been introduced to it The first conversational experience actually started to exist around 1950s, which is when it was built, the first experience, by some research scientists They were able to identify or recognize 16 words– 1-6 And then Carnegie Mellon University did some more research, and they were able to get to about recognition of about 1,000 words And since then– we didn’t call it conversational AI then, obviously It was just recognizing words or recognizing human language But over time, we’ve been utilizing conversational AI in a lot of different places And one of those is IVRs, or Interactive Voice Response solutions Most of us have gone through that experience when you call into maybe an airline or your bank, and you try to know your account balance or moving a flight, and you get that automated system That’s what an IVR is So we have been familiar with conversational experiences It’s just not been in those terms But most of those experiences lead us here, right? We’ve all been on a phone call where it’s like, can I just get to the agent? I don’t want to speak with you Can I just talk to a human being? And what do you think is the reason for that? Most important reason for that is you are talking to a machine that’s based off of flowcharts So when I say, I want to get some coffee to, say, this phone system, and this phone system responds back, oh, would you like milk in your coffee? And I respond back saying, sure, right? But it was expecting a yes or a no So when I said sure, it had no idea what to do It broke right there So that is what usually happens It’s just a very simple example But when you’re talking to a bank– and you can relate to all the experiences that you’ve had on the phone in the IVR systems And the main reason there is it’s not based on natural-language understanding It’s based on flow charts If the user says this, do this If they say that, do that And if they say anything in the middle, it has no idea what to do And we get this experience, which is quite frustrating So the idea here is– my goal for today in around 25 to 30 minutes is to take you through the journey of what conversational experience is, why it’s becoming very important and mainstream, and why we

should start thinking about it This slide, I’m not going to go through it all, but I wanted it to be here because the previous one, which leads to a very frustrating experience– there are lots of reasons for why a conversational experience can be frustrating One we just talked about, which is very rigid flowcharts or a scripted flow And some of the others could be, you’re not telling the user that they are talking to a bot You’re misleading them and telling them that– you’re basically just keeping that information away from them So they are expecting human-like interaction, which is not going to lead into a better experience And the others could be a bad use case where it cannot be solved through voice or conversation You should actually have a human being there So just utilizing– just taking the right use case and applying conversational experience to it And there are some more, which we are going to see in the later part of the presentation– so not spending more time on it Now, conversational AI is obviously becoming the new user interface And there are multiple reasons for it The biggest reason is internet of things The number of devices that we interact with today has increased so much that it’s going to be nearly impossible for us to interact with our apps with our eyes and hands, right? Think about your smart coffee makers or the lights and the fans in the rooms that are smart And if you get into a car in the morning, and you’re doing all these other things, and you’re like, take me to– you want the directions just to start without looking at the phone Take me to my work, or take me to office, or take me through the way that has less traffic So when we think about these newer devices, it’s nearly going to be impossible to interact with them through other ways So voice is going to be a main channel And we are already starting to see that So as these devices grow, conversation is one of those ways where people are going to resort to to get their work done faster These are some studies and researches that have been done that prove that people actually want to get self-served If you provide a good user experience from the conversational perspective, then 80% of the customer experience can actually be solved by a well-designed bot or a conversational experience So spending resources, energy, and time in learning what those use cases would be is really a good way to start So think about that And then there are lots– I mentioned a few use cases that are common, which is customer service I want to return my shoes You usually don’t have time between 9:00 and 5:00 to call into a retail or an e-commerce store to return your shoes Because you think about that type of stuff after 5:00, right, when you’re not at work? And if the customer experience there can be turned into voice without an agent, you would actually enjoy that You would actually want that So those types of use cases where you’re purchasing things, returning things, trying to get some support on a product that you’ve already purchased– so e-commerce has a lot of those use cases In general, if you’re a software company, another company, then you also look at the product support in a conversational experience manner So that’s one And then the second is the example that I gave around the devices that we use– a washing machine in the house, a refrigerator How do you switch them on and off with voice? Other scenario could be a home entertainment system, auto, when you want to navigate somewhere, when you want to play Spotify or other things, and it needs to be hands-free So we can think about these first two very easily The third one is actually a very interesting one because companies are starting to think about utilizing conversational experiences in their businesses to increase efficiency of the employees I’ll give you an example here When you want to look at how many days of vacation you have left in a year, I’m pretty sure you go through a bunch of clicks, and then you look through a bunch of documents to figure out, oh, I may have five days left, and you spend 15 minutes to actually do that Now, if there was– the conversational experience can actually take all of those knowledge-based articles

and search through those articles It can authenticate you as a user because you could be a part-time employee, a full-time employee, a contractor– and the number of days that you can take as vacation differs depending on who you are and what your position is in the company and how long you’ve worked and all of those things But a well-designed conversational experience can authenticate you, can identify all those parameters, how long you’ve been at the company And based on that, you get 25 days, and out of that, you’ve used 10, and now you have 15 left So there’s a lot that has happened there But that’s what conversational experience can do And companies are starting to think about that Another use case as an example could also be you have customer support agents that need to use some knowledge-based articles And these knowledge-based articles are not at one place So increasing their efficiency would be combining the search base for those knowledge-based articles And then, for the agent itself, they can just search for what they want to get an answer for and then respond back to the user So you just made them a little bit more efficient by utilizing a conversational experience So think about it in those lines The first two are easier to think and start with And as you start to think more about it, you can think about connecting businesses and employees and that as an efficiency factor By now– this is purposefully somewhere in the middle as a slide By now, you might have already understood what a conversational experience is Anything that is not based off of flowcharts, anything that is based off of natural language understanding is what a conversational experience is usually defined as There are lots of different terms that are around– voice bots, chat bots, conversational UX and UI All of that kind of falls under the same umbrella, which is anything that’s powered by natural language understanding And what is natural language understanding? So I know 50% of you all had raised your hands, so may be too basic But for those who did not, I’ll just still go through this for a second Natural language understanding is nothing but a translator that’s translating human language into binary for the computer to understand and then binary from the computer to convert to the human language I’ve oversimplified it, but that’s pretty much what natural language understanding is And there’s lots of other terms that you’re going to come across, which is natural language processing and natural language understanding Natural language understanding is a subset of natural language processing where smart stuff that humans can do– we can paraphrase, we can spell correct, we can understand spellings, we can understand accents– all the smart things that we can do We can brief or summarize something big that we’ve read All of that comes under natural language understanding So once you’ve trained those models, then you can start to utilize them And you’d use the tools to do that Oh, it’s working The other terms that you come across are ASR, which is Automatic Speech Recognition And it kind of goes in full circle So you’ll first understand what the user is saying, which happens through ASR, which is automatic speech recognition I said something Your machine has to understand it, which is ASR Now, it converts from speech that I have said to text so that it can apply all those models And then, from text, it converts it back to speech and says it to the user So you’re going to come across all of these terms in some way, shape, or form when you’re working with conversational experiences That’s why I have them there Don’t necessarily have to know them fully, but just being aware of them is important And all these terms apply to both voice as well as text So if somebody is speaking versus typing something, the natural language understanding technology applies to both voice and text With that background, I think it’s also important to understand that natural language understanding, processing, ASR, all of it kind of falls– it’s not under just one umbrella, which is AI, or ML or deep learning You kind of need a large amount of data to apply machine learning on it And then you need to have the neural networks learn– have the model learn So deep learning would, on its own– so deep learning

is also part of it, and AI is also part of it So it doesn’t quite– those terms don’t quite just fall into one thing I just like to explain it because I do get this question a lot So it’s kind of good to just get that mapping out of the way Now, we’ve all, I’m pretty sure, come across very bad conversational experiences, where it only does one thing, and that too not very well So it’s very important to understand whatever we pick in a conversational experience to build, we go in with a good understanding of what is a factor that builds a good conversational experience Now, there are three factors One is intent, which is, whenever a user is asking your experience for something, it is an intent And there are many ways for a user to ask for the same thing When I ask for coffee, depending on where I am, and what day it is, and how bad a mood I’m in and what I am feeling, I can ask for coffee in multiple ways If I’m happy, I would maybe say oh, give me some coffee If I’m sad, I could be like, oh, coffee would be great right now, right? And that’s just me asking for coffee in multiple ways But if we go across this room– I think we’ve got about 300 people– we can probably come up with about 3,000 ways in which people can ask for coffee And when you think about that, that intent of getting coffee can be asked in 3,000 ways So you need to apply machine learning on all those samples to understand that 3,000– one person who’s going to ask for coffee And that’s not very easy So that’s one challenge that you need to think about when it comes to users intent and understanding it The second challenge is entities There are things that you would have to extract from what the user’s trying to ask So in that intent for, could you get– I would like to get some coffee, coffee is one of the variables If I said, I would like to get some coffee with two creamers and one sugar, then two creamers, one sugar, and coffee– all those three pieces are required in order to actually create coffee, right? So those three pieces become my entities or parameters that I need to extract in order to service the user’s intent Some other examples could be– I’m setting up an appointment for 2:00 PM tomorrow 2:00 PM and tomorrow are critical pieces of information to set that appointment, so you need to grab those Now, it’s easy to think about samples like this where dates and times and numbers and currencies are very common But you still have to build models for those entities because dates can be set in multiple ways It can be tomorrow It could be the date format It could be year first and then days later if they’re typing it So there are different ways in which you’d have to have your model understand the dates as well So you have to need all those examples So again, machine learning would apply on that The entities is another challenge The third is context You and I can talk to each other and understand each other because we can assume– I can assume that what I said two seconds ago is retained somewhere in your brain, somewhere in context, right? We humans interact with each other very well because we can retain context If we expect a similar level of service from a machine, then we need to make sure that the machine can retain context There is an example on the screen there, but you can also assume another example How is the weather like in Chennai today? And the experience responds back with whatever it is– cloudy, sunny And then you want to know the weather for another city, and you say, oh, how about New Delhi? And it still needs to retain– in this case, it still needs to retain the context that we’re talking about weather, and we’re talking about today, but only the place has changed So saving that is also challenging Saving the context and how to utilize it over time in a conversation is challenging So you need to use the right tools All the three things that I mentioned are pretty critical There are some others The fourth one here, which is, you are not just going to provide this experience for one interface It’s not going to be always on the phone It can be over text It can be on your website It can be through a smart speaker So you need to make sure all those integrations to whatever

you’re building as the conversational experience are built And then we talked about the intent, the machine learning, the natural language processing, and the list keeps growing Say today, you’re only building for the region in India, but tomorrow you want to expand in Europe And in that case, you would have to now start to think about machine learning for the language where you’re going to deploy next or go global So all of that combined together is not very easy So you need to use the right tools And then, towards the further right, which is the fulfillment piece, you can have the best possible natural language understanding on the front end to understand what the user’s saying, but if you can’t really have a proper connection, a proper API request and a response mechanism to feed the response back– to first get the response and then feed it back to the user, then all the front end work that you’ve done with amazing natural language processing goes waste because the back end cannot set an appointment, or the back end cannot make that coffee that we were talking about So it’s very important to make sure that the CRM systems, the databases that you’re connecting to have the proper APIs, they have the proper authentication mechanisms so that your front end chat experience or conversation experience can serve that request With that, all the red boxes that you were seeing– without knowing much about machine learning, you can use the tool called Dialogflow in order to create a conversational experience And it will create the intents for you You’ll just give a few examples– and I’ll show you this in a demo You’ll just create a few examples, and it’ll auto-train a machine learning model for you with just those examples Some of the advantages are languages– it supports 20-plus languages If you are going global, you don’t have to think about expanding in some of those other languages And then it utilizes text-to-speech and speech-to-text and natural language processing APIs on the line, which is built on Google’s research and database of large amount of data where we’ve trained the machine learning model for natural language processing And what’s happening is when you provide the examples that, for your business, then the model trains with those subset of examples along with that underlying natural language processing API that is already pre-built. So it makes the conversational experience come to life faster because you’re not training the machine learning models for normal, natural language You’re training it for your specific use case, which is much, much better and easier So I’m going to switch to the demo so you can see the stuff that I’m talking about So what I’ve done is I’ve built a very simple conversational experience in Django Python I’ll show you the architecture in just a little bit, but let’s see what it does We’re using a very bad internet connection, so we’ll see how this goes In any conversational experience, you want to make sure that you tell the user what this experience actually does If you keep it open-ended, it leads into situations where they don’t know what to ask and what your experience can do, and then they’re going to ask things that you can’t support And you obviously have to handle them as well in a very graceful way But it’s always good to tell them what your experience does In my case, since it’s a demo, I’m doing two things, which is setting up appointments and exploring landmarks There are two very random examples because I wanted to show you two very random things– set an appointment for 2:00 PM tomorrow for license And it should, ideally, go back and set an appointment in my calendar OK, so when you come into the calendar, you see the appointment popping up So what’s happening here is– actually, let’s hold that Let’s hold what’s happening Let’s try the other one as well, which is exploring landmarks So what I’m going to do here is I’m going to provide it a demo JPEG image, which is–

because it’s very hard to see in my window there, let me open that demo image here so you can see It is the image of Golden Gate Bridge So when we submit that image, the experience is supposed to take that image and then tell you what’s in that image So there’s a machine learning model in the conversational experience that’s being triggered, which is looking for what’s in the image And it’s specifically looking for– well, hopefully, it comes up– and it’s specifically looking for landmarks in the image And in this case, it was supposed to come back and tell us that there’s Golden Gate Bridge in there And wait, OK, yeah, all right, so we got it I’m not doing any parsing here I’m just looking for labels, and I’m just displaying that here But it gives you an idea of what’s happening So this is what’s actually happening behind the scenes So we had the Dialogflow interface, which is written in Django Python And you could write it in any language you like, and you could connect it to– and I’m basically triggering– whenever the user’s saying set appointment, or uploading a file, I’m triggering detect intent API in Dialogflow And Dialogflow basically has intents, which I’m going to show you in just a second Cool, so I was saying that there’s a detect intent API that gets triggered, which understands what the user said They were trying to set an appointment in the case when I said set an appointment for whatever at whatever time And then it takes that request, parses the entities– date, time, and the appointment type, and then sends it to the Calendar API The Google Calendar API gets triggered if I have some authentication for that in there And then it responds back saying that it was able to set the appointment, or it couldn’t find that time, and tell me another time, and I’ll set it then The other thing that’s happening is, when you upload the image, another intent is detected, which is uploaded a file and I want to apply machine learning on it Once that intent is selected– the file actually, I’m uploading it from the front end to cloud storage, so the front end doesn’t get very cumbersome And then I’m grabbing the file The Vision API grabs the file The Dialogflow triggers the Vision API, grabs the file from cloud storage, and then applies the Vision API on it, and then tells you what’s in that image So I’ll switch back to Dialogflow because that’s the most interesting piece And I have about two minutes, but I’m going to take maybe one more minute extra because of the [? cloud– ?] we’ll see Intent matching– here’s how the interface looks like when you log into Dialogflow To get to Dialogflow, it’s console.dialogflow.com And then, once you get there, you create an account, which is obviously free And then you create what is called an agent And inside an agent, you have intent, entities, and fulfillment, which are the three main things The intent, in this case, was schedule appointments in the demo And I have some examples in there So as soon as you type this example, you get the highlighted versions of your entities that you’ve created– 2:00 PM and the date, the day and the time are system entities because they are things that can be– so days and time are system entities because you don’t have to create dates and time entities because we’ve already created them because you have to apply machine learning and natural language processing on those as well So to make it easier, you don’t have to create any of those They’re already in there because everybody needs dates, times, currencies, and countries, and addresses and things like that, which are common What you would have to create custom is that appointment type And we’ll just keep rolling –is that appointment type, which is for whatever business you are If you are a pizza shop, toppings would be your custom entity In this case, we were doing a driver’s license thing, so driver’s license appointment type is a custom entity So there’s a developer entity, or a custom entity, and then there’s a default system entities You make them required And in the example, you weren’t able to see it But initially, I was trying to show you when I said, set an appointment, that it understood that I’m trying to set an appointment And then it would ask me for the dates and the time, the parameters that I didn’t provide So that is what makes it a very natural experience And if I’m somebody who says everything at the same time– in the second example, when I typed

I did it all at the same time And I said set an appointment for 4:00 PM tomorrow for a driver’s license It knew that I gave it all the three pieces of information in one shot It didn’t ask me for anything else But if I only said 4:00 PM tomorrow, I would like to set an appointment, then it would still know that I have given it two pieces of information, and I need to ask for that third piece as well It keeps the conversation very, very natural And that’s what the beauty of that is You access those variables with a dollar sign So whatever the user had said– you can’t see it here, but the appointment type has a dollar sign So whatever that parameter entity was, you use it with a dollar sign to access it anywhere in your– this is the way you define your entities when they are custom And the parameters– you can tell the user if they have not given you some information Fulfillment is the most important piece You can create– now this is where you write a little bit of code to connect to your back end In my case, I was connecting to the Calendar API and to the Vision API, whatever it is that your use case is, you’re connecting to a database, you’re connecting to a CRM system, to salesforce ticketing, anything that you want to connect to, you connect to the APIs from the webhook The webhook– there are two ways to do it You can have your own webhook that accepts post requests, or you could also use something that’s in Build, which is Cloud Functions And Cloud Functions gives you a template of code already where you can just pop in whatever the function is In my case, it was set appointments and applying Vision API on it So I created two functions, and I just did whatever I needed to do in those two functions, and it takes care of the rest So I don’t have to worry about where it’s deployed and any of that because it’s happening It’s a serverless function I know you’ve learned a lot about serverless today We don’t have to talk about that Integration– so again, this is the most important part as well where you’re not just building this experience for one thing, right? You want to enable it for all the different channels where the users are So you build it once, and then you just click through these one-click buttons for integrating with, say, Twitter, or Facebook Messenger, or you want an action on Google Assistant You can also connect it with Twilio, or you can download the agent and upload it into Amazon as well and make it enabled for Alexa or Microsoft Cortana So you just build it once, and you deploy it anywhere you need it with the one-click button There are lots of other things You can connect it to a phone You can connect it to knowledge bases I gave an example earlier where you can provide FAQs in HTML or text format, and then it’ll create a chat experience for the FAQs so people can just ask instead of reading through your FAQ documents, which is pretty cool You can spell correct You could have built in sentiments in there So you can understand and trigger a human being or a human agent based off of the sentiment that you’re getting in real time in here So you get a sentiment score And say you got a really bad score during the conversation, you can trigger a user– trigger a real agent based off of that score, which is also very cool And then there’s built-in text to speech as well So all of that There’s obviously a lot more that I can’t cover in the time that we have So I’ll leave you with some resources But the very last thing I want to cover is, don’t just jump in I know it’s very easy to get started with building a conversational experience So people just– we, as developers, just want to build something And I think in conversational experience, it’s very important to lay that groundwork, to understand what you’re building, why you’re building, and how you’re going to get there So these steps, I think, are very, very important and crucial to understand Identify those top journeys that actually make sense to convert to a conversational user experience And then once you’ve identified them, build a business plan And I don’t mean like a full-blown business plan What I mean here is, understand what it would entail, understand what those back end systems would be, and then design a persona Is your bot going to be geeky or funny? And how is it going to sound? Because all of that is going to matter And then you start to think about actually

building something So with that, I would like to wrap it up That’s me again @pvergadia is how you can find me on the internet All the demo and the code and everything that you saw today can be found on my media blogs and on Twitter You can also follow my “Deconstructing Chatbots” series, which is on Google Cloud YouTube channel, where I go from 101 if you don’t know anything about conversational experiences to actually building and understanding concepts at a 301, 401 level So feel free to check that out If you are looking to do something very specific and have questions, I’m always available to answer them as well if you reach out to me or LinkedIn or any other social media channels Thank you so much for being here [APPLAUSE] [MUSIC PLAYING]

You Want To Have Your Favorite Car?

We have a big list of modern & classic cars in both used and new categories.