Create your first QnA bot using botframework’s QnA Maker

When talking about the botframework, and chatbots in general, people usually assume that these are all using some clever logic and Natural Language Processing (NLP) to deliver a chunk of business logic via a natural language interface.

With the botframework this is most likely implemented by wiring up the Language Understanding Intelligent Service (LUIS): originally a stand-alone, (optionally) self-training, natural language understanding service, but now part of Microsoft Research’s Cognitive Services – previously “Project Oxford” – a collection of extremely powerful machine learning APIs for processing images, video, text, speech, to extract meaning.

Exceptionally powerful, incredibly clever stuff.

Almost all botframework articles and tutorials you’ll see at the moment will either do very basic pattern matching to extract intent from a message, or they’ll use LUIS (or a combination of the two); how to use LUIS is the subject of another article entirely, since this is no small task (I’ll come back to this in another article).

FAQ bot

There’s also another option; one that’s not well documented, nor widely advertised, but is extremely useful for both getting a very useful chat bot up and running, and also enhancing an existing chat bot by plugging gaps in it’s “conversational knowledge”: the QnA Service.

Using this service you can very easily create your own QnA/FAQ/Knowledge Base bot; the beauty of this type of botframework service is that it requires no coding at all; you simply “seed” your “QnA” bot with predefined content – Questions and Answers – and the QnA Maker service does the rest.

The result is an endpoint which takes a query and returns a json response containing the matching content, if any, and the degree of confidence for that match; you can also embed your QnA bot directly into a web page using the hosted html endpoint.

What’s extremely powerful is how it (presumably) uses LUIS under the hood; since you don’t need to ask the exact question you seeded it with to get the relevant answer, there must be fuzzy logic/pattern matching/NLP going on. You can also “train” your QnA bot with multiple phrasings of the same query for a given response.

Possible Implementations

This is an extremely cheap and easy way of creating your company’s first chat bot; all you need is some seed data in one of many possible formats, and you can even embed the web hosted bot if you don’t want to create your own using your QnA bot’s API endpoint (“service URL”).

If you have more resource (time, knowledge, people), then creating a javascript bot client to call the service URL would enable you to create a bespoke client.

If you’re already using LUIS and the botframework, then you can have your “None” intent (i.e., where LUIS hasn’t matched the incoming message with a high enough confidence score) call the service URL with the same query to see if there’s a relevant FAQ result to display instead, thus plugging a gap in your LUIS bot’s conversational ability.

For example, if you’ve created a chatbot for booking a holiday, and can match all queries about locations and flights and hotels, this FAQ bot could handle questions about the refund policy, or holiday insurance, that obviously wouldn’t be needed by the main logic of the bot.

Create your first QnA bot

Head over to qnamaker.botframework.com and log in with your Microsoft Live account. You’ll initially be redirected to dev.botframework.com before being whisked off to the QnA maker page:

botframework QnA maker - Create new QnA service

Start by clicking that “Create new QnA service” and you’ll be taken to a screen containing a few steps, most of which are “either/or”, but that’s not obvious initially.

Name it

The first item is naming your service (mandatory):

Naming your QnA bot

This is the name that will appear against each response within the hosted web page version of the QnA bot; other than that, it’s not really used.

Seed data

You can now choose from a combination of one or more of the following methods of seeding you QnA bot:

a) FAQ URLs

You can choose to pass in one or more URLs to existing, publicly accessible, FAQ style pages; so long as they have a reasonably consistent “FAQ” type HTML structure, then the QnA Maker service can extract the Qs and As – this is extremely cool, and I’ll get on to it in my 2nd example:

FAQ URL entry for seed data

I’ve tried a few help pages with different HTML structure, and it has been pretty reliable in extracting the data so far.

b) Textbox Q and A pairs

We can enter our Q and A pairs directly into a textarea on the page in the format question:answer, with one pair on each new line – I’ll use this in the first example:
Text entry for seed data

c) Upload a file

Again, this is very cool in that it allows you to take preexisting content, and just import it directly; you may have a FAQ doc or PDF that is used internally, or perhaps was used as the “requirements” for the “help” section of your site.

You could also programatically generate a tsv (tab separated values) file and upload that:

file upload for seed data

QnA Bot v1 – Seed data via text entry

For the first example, I’m going to go with just entering some text in the textarea of section 3:

Who was the plumber always fighting Bowser?:Mario
Which was the first Zelda game on the SNES?:A Link To The Past
Who was the final boss in Street Fighter 2?:M. Bison
Which hedgehog collected rings?:Sonic
Who fought Metroids?:Samus Aran

(And yes, I’m aware that the Japanese version of SF2 named the end bad guys differently, since the boxer – “Balrog” in the non-Japanese version – was meant to be called “Mike Bison”, and Capcom wanted to avoid a lawsuit so shuffled the names around.. but anyway…)

Extract data and Train

Now you’ll have the options to “Cancel” or “Extract”; you have to “Extract” before you can click “Next”.

As such, go ahead and click “Extract”; depending on the data source and data size, this can take a little while. As far as I can tell, this is importing the data, identifying “utterances” for each Question and creating something equivalent to LUIS models for each Answer, then training LUIS.

Once you see the little bit of blue text that says “5 Question-Answer pairs created” you can click the “Next” button to continue…

Test

Now you’ll have a chance to try our new QnA bot out with the embeded web chat.

Testing your QnA bot

If there are multiple possible responses for a question, the QnA service will return the answer with the higher score.

However, in this mode we can see all the potential matches on the left of the chat, and can override the given response, retraining the bot as we go!

On the right of the chat window we can enter alternative phrasing for the same question, to allow the underlying logic (presumably LUIS) to be able to match

Training your QnA bot

There’s even the option to upload a new file here (as you could in stage 4 on the previous screen), so you can keep tweaking the accuracy of the language matching logic.

If you do enter any new info, click “Re-Train” in order to have it apply.

retraining popup

Once you’re happy with your bot, click “Publish” to finalise the process and be given the “Service URL”.

published bot!

Using your QnA bot via the Service URL

The service url is the same endpoint for all QnA bots, just with a different “kbId”. You can just put this in a browser and append your question to get the matching answer along with the score of confidence for that match:

json endpoint response

(That url suggests this service used to be called a “knowledge base” service instead of a “q and a” service, eh?)

Using your QnA bot via a web page

Notice the text that says “You can also chat with rposbo_bot_trivia here” – tap that link and you’ll get your bot in a full browser web page:

webpage hosted bot

Pretty cool stuff, huh?

QnA bot v2 – Seed data via URL

Let’s try that again, but try using a URL to fill the bot’s brain. Head to your qa bot homepage and create a new bot. Give it a name, and this time let’s paste in the URL of a website with a FAQ page.

I’ve used the UCAS FAQ page – and I’ve also tried this with the Thread.com help page, which is a completely different structure but still loads perfectly fine.

The UCAS FAQ page looks like this:

UCAS FAQ page

And an expanded question looks like this:

UCAS FAQ example answer

If you paste the URL https://www.ucas.com/ucas/undergraduate/apply-and-track/frequently-asked-questions into section 2 of the bot creation page, then hit “Extract”, you should see the notification that “14 Question-Answer pairs created”.

Now let’s test it!

testing and training the UCAS FAQ bot

Notice that the question I asked here – “what’s the application process” – isn’t the same as the text on the FAQ page – “What will happen when I’ve sent my application?” – and it appears to match several answers; notice the blue boxes on the left.

Let’s try that again with a different question, and use the service URL instead:

UCAS json endpoint response

Only one response returned, but it’s pretty low confidence score.

Now here it is using the hosted web page version:

UCAS hosted webpage

Again, notice the questions I’ve asked are different between these two tests, and neither are the same as the actual question from the website, yet both match back to the correct answer.

Summary

This is obviously a really powerful and useful chat bot tool, but it’s seriously hidden away in the bot framework portal, lacking in-depth documentation.

You can have your own FAQ bot up and running in just a few minutes.

Hopefully you found this useful, and if you successfully create a QnA bot, please let me know how it goes!

9 thoughts on “Create your first QnA bot using botframework’s QnA Maker

  1. It looks like this is outdated. It no longer automatically generates a web page where you can chat. Would you be so kind as to update it? I’m a newbie and can’t figure it out myself. Thanks.

  2. Robin,
    Very nice article! I too have been playing around with QnA Maker and like you, found the Documentation to be a little lite! Another thing that seems a little lite are the Chat Logs. They only appear to log the Questions, but not the response(s) that were sent back. Overall though, it is really a great tool for quickly implementing a Bot with AI capabilities behind the scenes.

    • Hi Scott.

      If you download the chat logs from the test page, there is a second option to upload the chat logs back into the test bot. This allows you to ‘step through’ each question and receive the answer from the bot.

      Hope this helps

  3. Hi Robin,
    I created a FAQ bot using QnA maker. But I can use it in my company’s bot because its under secure network. So I would like to know that can we use QnA maker to build a standard FAQ bot for enterprise? And if yes then how?

  4. How to send request using html form to connect Qnamaker answer generated how to send subscription key in url to have connection

  5. Hi Robin,
    Huge fan of this post and the rest of your writings.

    Inspired by this I made a Sir David Attenborough Q and A bot that takes his reddit AMAs.

    https://www.messenger.com/t/549022478767012

    PS if the confidence score is really low have you thought of a way around it? Maybe redirect it to another service to see if it can understand the question more accurately?

Leave a Reply

Your email address will not be published. Required fields are marked *