Creating a hosted bot using Microsoft’s botframework couldn’t be easier; hopefully you’ve had a chance to create one already, and if not there’s a great introduction to creating your first bot right here.
In the previous article we saw how to create a QnA (aka FAQ/Knowledge Base) service using a little known QnA Maker service of the botframework.
In this post we’ll start to create a more intelligent bot; one which can appear to understand the intent of the incoming message and extract specific key variables from it.
Understanding the intent of a piece of text is a really tricky problem to solve; totally out of scope for this article, and for most bot projects! Luckily, the botframework has a friend called LUIS – the Language Understanding Intelligence Service.
LUIS exists as a standalone product, and can be called with a query (text input) to give a result (matching “intent” with a “confidence” score, as well as any extracted “entities” – we’ll get on to these later).
The botframework can support LUIS such that the intent response from LUIS can map directly to a class with a matching attribute value, such as
[LuisModel("<LuisAppId>","<LuisModelId>")]It can even map down to a specific method within that class using another attribute:
[LuisIntent("<IntentName>"]Given this magic, it’s no longer obvious what the entry points are for your application which can make developing and debugging the logic within your bot application really tricky..
We’ll get on to the C# and botframework magic later, but first let’s look at getting started with LUIS and how to use it.
Setting up LUIS
Head over to Luis.ai and log in with your Microsoft account. After a brief intro wizard thingy, you’ll see a screen like this
Here is where we can define our apps. Create a new app by clicking either “New App” -> “New Application” or “Build a new language understanding application” – both open the same overlay:
- Give it a name and choose from IoT, Bot, Mobile Application, or Other (honestly have no idea if this info changes anything; AFAIK it’s just data gathering); I just choose “Bot”.
- Choose your “Application domains” – again, this doesn’t alter your experience, so just go with whatever you prefer; I chose “Entertainment”.
- “Application description”? “rposbo’s demo luis bot thingy” is all I put 🙂
- “Culture” might actually be used, but I’ll get on to that in another article.
Now that you’ve got an empty app, you need to set up some intents. You should see a screen that looks a bit like this:
Click the plus sign next to INTENT to create a new intent; in terms of the botframework we can just think of an intent as a category which we’re going to teach LUIS to map sentences to, and extract meaningful terms from.
Give it a name – for example “local.news” – and enter an example “utterance”:
“Utterances” are the short sentences which are used to map the user’s input to an intent. You can define many different ways of saying (uttering) something which you intend to mean the same thing.
For my “local.news” example, I’m going to use the utterance “what’s happening in London today?”
Now I’ve got an intent name – “local. news” – and an example utterance – “what’s happening in London today?” – I can hit “Save” to create this intent.
I’m going to create one more intent – “local.weather” – and seed it with the utterance “what’s the weather like in London today?”
(Notice these utterances have some variables in them – “London” and “today” – which we’ll come back to later on)
The LUIS web service
We’re nearly ready to give this very basic logic an endpoint; to do this we have to TRAIN and PUBLISH the LUIS app. Every time you want to update your LUIS app – if you’ve added more utterances or intents for example – you always need to TRAIN and PUBLISH again.
Firstly, hit TRAIN in the bottom left and wait for the spinny loader thing to finish; the more utterances you have, the longer this process takes.
Uh oh. You do need to define a minimum of two utterances per intent. Add a couple more and hit TRAIN again, then once that’s done hit PUBLISH to get this overlay:
Now hit “publish web service” to create or update your LUIS app’s web service.
Testing LUIS
Now that we’ve got a web service, let’s try it out! In that same overlay you should see a URL for testing and a query input box. If I type in “will it be sunny in Ipswich tomorrow?” and hit TEST, then I should see a new tab open with a json response that looks like this:
Notice the confidence scores are really low – they’re on a scale of 0 to 1. Basically, LUIS was not able to match that input to either intent with any degree of confidence, so the “None” intent got the highest score (0.28 – just above “local.weather” with 0.26, which is promising).
We can easily fix this by submitting the exact phrase into the LUIS app homepage and selecting our prefered intent from the dropdown:
After doing so that same sentence will now match with a confidence of “1” – i.e., super premo number one confidence!
Remember what I said earlier? Before any training changes (i.e., new or recategorised utterances) can take effect, you need to TRAIN and PUBLISH, otherwise the query result won’t change. So hit TRAIN then PUBLISH and “Update Published Application”. Let’s refresh that web service call again:
(Not quite “1”, but it rounds up!)
Checkpoint
We’ve now got a LUIS app with two intents, each with two mapped utterances, and a web service endpoint to send text to and see if LUIS can map that text to an intent.
Training LUIS
Hopping back to the LUIS app’s home screen, you’re presented with a text box where you can enter more utterances to see how well LUIS maps them to intents. You can then override or set the correct intent, hitting “Submit” each time, to help teach LUIS.
Once you’ve done this for a few more utterances, hit TRAIN and PUBLISH to update the web service endpoint.
This is obviously a pretty manual process. If you already have a good set of input data to hand (for example, extracted from chat logs elsewhere) then this method of training your LUIS app is very inefficient.
Luckily there’s the option to upload a text file of utterances directly! Hurrah! This must be in the form of a text file containing one utterance per line, each utterance being unconnected to any intent.
On your LUIS application list page (not the LUIS app homepage itself), hit “import utterances” for the app you want to update (in case you have more than one)
You’ll then be presented with this:
Once the file has been uploaded, LUIS will attempt to map each one to the existing intents.
It takes a bit of clicking around to find these uploaded utterances; you need to head to the SUGGEST tab, where it will gradually show you the uploaded entries along with their mapped utterances and their confidence score.
It’s here that you can recategorise or confirm the terms one by one; each time you submit one the model is retraining and the subsequent utterances have higher confidence scores.
Remember to TRAIN and PUBLISH when you’re done, then try querying the endpoint again with some alternative input; if the intent match is wrong or has a really low score, you can enter the same input on the LUIS app’s home screen – NEW UTTERANCES – to retrain to the correct intent.
Entities
I mentioned variables in the input earlier, so let’s have a look at mapping and extracting those next.
Head back to your LUIS app’s homepage and hit the plus sign next to ENTITIES. That will open the entity overlay where you can create a new entity.
Give your entity a name and hit “Ok”; since my utterances are like “what’s the weather like in London today?” I’m going to create a “where” and a “when” entity.
LUIS already has many built-in entity types which cover location and dates; I’m just creating these for demonstration purposes
Now we can go back to our utterances and improve them by highlighting the entities; go to “Review Labels” and tap “Show all labelled utterances” to start enhancing them.
Choose an utterance and click to highlight the entity, then select the entity type from the pop-up and hit “submit” to confirm
Do this a few times and hit TRAIN and PUBLISH when you’re done.
Now try searching for a new utterance, one that you haven’t explicitly defined the entities for; you should see LUIS has identified the entities for you! If not, highlight and set them yourself.
Submit this new utterance to help improve your LUIS model (…and TRAIN and PUBLISH again..)
Summary
You should now be able to query LUIS via the service url and receive a json response with the most relevant intent and also with the entities identified:
You could now call this endpoint directly – using a webclient for example – and parse the response to determine the code to execute and the variables (entities) that were passed in.
You can also use the built-in support for LUIS in the Botframework.
We’re going to look at both of these options in the next article!