In the last article about bots I covered creating a basic bot using Microsoft’s botframework, setting up Azure, deploying the bot into Azure, and configuring it to work within Skype.

In this article we’re going to investigate the various response types available to us in the botframework to develop a more rich conversational experience.

Markdown

Luckily you’re not limited to plain text in a bot conversation; we’re able to embed images, add attachments, give headers and subheaders, add a button or link, tap events for various areas, as well as use markdown to format the main text content.

If you’re not already familiar with Markdown, then get on the case! It means you can very easily write HTML by using a shorthand syntax which can easily be converted to HTML.

I’ve been using it for many years for blogging and general documentation; using pandoc you can even convert markdown to PDF or a Word Doc. Using remark.js or the more recent Marp you can use it to easily create PowerPoint-like presentations

Botframework messages support using this syntax to make the responses more rich. Of course, for this to work, the attached service needs to know how to render the response (and I’ll get on to this later)

Response Types

If you go and read the first article in my botframework series, you’ll recognise this as the default reply logic within the boilerplate MessagesController:

// return our reply to the user
Activity reply = 
   activity.CreateReply($"You sent {activity.Text} which was {length} characters");

For each of the following response type examples just replace this line from the default MessagesController with the example code.

Let’s make an example response just using some basic markdown to give a heading (a single “#” at the start of a line is an “H1”, two “##” is an “H2”, etc), bold, italic, a link, an image, a blockquote, and a code block:

var markdownContent = "#A heading\n";
markdownContent += "##A subheading\n";
markdownContent += "**something bold**\n\n";
markdownContent += "*something italic*\n\n";
markdownContent += "[a link!](http://robinosborne.co.uk/?s=bot)\n\n";
markdownContent += "![AN IMAGE!](https://res.cloudinary.com/otomotech/image/upload/f_auto,q_auto/robinosborne/2016/07/robinosborne.jpg)\n";
markdownContent += "> A quote of something interesting\n\n";
markdownContent += "```\nvar this = \"code\";\n```\n";

Activity reply = activity.CreateReply(markdownContent);

Now here’s the the rub: as I mentioned at the start, in order to use these rich responses the service to which your bot is talking needs to know how to render the content. The JSON response will just contain a block of markdown, not a pre-rendered chunk of HTML for example. As such, the services (such as Skype or Slack) need to know how to render markdown.

The case in point; here is the above message being rendered in different services:

Markdown: in the Bot Channel Emulator

Alright, that’s pretty awesome! The H1 and H2 came out nicely, as did the bold and italic text. The image was embedded and the alt/title was even rendered underneath, but the image has been stretched to look much bigger than the original; and there’s the link. The blockquote looks great and there’s even a nicely formatted code block! Aces! This is great, right?..

Markdown: in the Skype Desktop App

Uh, okayyy.. H1 and H2 are just underlined, bold and italic are good as is the link; the image isn’t embedded and the quote is certainly rendering as a quote, but needs a cite perhaps. The code block seems to have been ignored.

Markdown: in the Skype Web App

The quote is ignored, and the image doesn’t even link to the image, like it did in the desktop app.

Markdown: in the Skype Android App

Now we’ve got an embedded image (sort of), the quote just has an empty (null) cite and the code block is ignored.

Mixed Results for markdown rendering in Skype vs the emulator, for sure. And these all use the same JSON response:

"text": "#A heading\n##A subheading\n**something bold**\n\n*something italic*\n\n[a link!](http://robinosborne.co.uk/?s=bot)\n\n![AN IMAGE!](https://res.cloudinary.com/otomotech/image/upload/f_auto,q_auto/robinosborne/2016/07/robinosborne.jpg)\n> A quote of something interesting\n\n```\nvar this = \"code\";\n```\n"

Attachments

As well as plain old text and funky markdown messages, we can add simple media attachments – images, audio, video, or generic “files”

Let’s try to send a couple of images:

Activity reply = activity.CreateReply("Here's an **image attachment**");
reply.Attachments = new List<Attachment> {
   new Attachment()
   {
       ContentUrl = "https://res.cloudinary.com/otomotech/image/upload/f_auto,q_auto/robinosborne/2016/07/image-attachment-tachikoma.jpg",
       ContentType = "image/jpg",
       Name = "Tachikoma - jpg"
   },
   new Attachment()
   {
       ContentUrl = "https://res.cloudinary.com/otomotech/image/upload/f_auto,q_auto/robinosborne/2016/07/image-attachment-gypsy-danger.png",
       ContentType = "image/png",
       Name = "Gypsy Danger - png"
   }
};

Image Attachments: in the Emulator

Great stuff; that’s a jpg and a png with their alt appearing as a caption.

Image Attachments: in Skype Desktop

Image Attachments: in Skype Web App

Image Attachments: in Skype Android App

Some subtle differences there, but overall a pretty standard response.

Rich Attachments: Hero Card

As well as plain old media attachments, we can also attach “cards”. These cards can have some extra functionality, like buttons and tap events.

The hero card contains a single large image, a button, and a “tap action”, along with text content.

Activity reply = activity.CreateReply("Here's a **hero**");

var heroCard = new HeroCard()
{
    Title = "I'm a hero card",
    Subtitle = "Robin hearts Tachikoma",
    Images = new List<CardImage> {
        new CardImage(url: "https://res.cloudinary.com/otomotech/image/upload/f_auto,q_auto/robinosborne/2016/07/robinosborne.jpg")
    },
    Buttons = new List<CardAction> {
     new CardAction()
        {
            Value = "https://robinosborne.co.uk/?s=bot",
            Type = "openUrl",
            Title = "Rob's bots"
        } 
    }
};

reply.Attachments = new List<Attachment> {
    heroCard.ToAttachment()
};

Hero: in Emulator

Awesome – like it said in the description; a heading, a button, and an image.

Hero: in Skype

Looking good

Rich Attachments: Thumbnail Card

The thumbnail card can have a single small image, a button, and a “tap action”, along with text content to display on the card.

Activity reply = activity.CreateReply("Here's a **thumbnail**");

var thumbnailCard = new ThumbnailCard()
{
    Title = "I'm a thumbnail card",
    Subtitle = "Tachikoma hearts Robin",
    Images = new List<CardImage> {
        new CardImage(url: "https://res.cloudinary.com/otomotech/image/upload/f_auto,q_auto/robinosborne/2016/07/image-attachment-tachikoma.jpg")
    },
    Buttons = new List<CardAction> {
     new CardAction()
        {
            Value = "https://robinosborne.co.uk/?s=bot",
            Type = "openUrl",
            Title = "Rob's bots"
        }
    }
};

reply.Attachments = new List<Attachment> {
    thumbnailCard.ToAttachment()
};

Thumbnail: in Emulator

Does as expected, but the text content doesn’t look perfect – the justified alignment is odd in the emulator. Maybe Skype can do better?…

Thumbnail: in Skype

The Android app is looking nice.

Rich Attachments: Receipt Card

The concept of this is kinda cool; a card to – presumably – summarise a purchase, which implies it should be possible to complete a transaction all within the scope of a conversation.

The receipt card can contain a title, “facts”, line items, tax, VAT, total value, and one or more buttons. It’s a beefy one – are you ready?..

Activity reply = activity.CreateReply("Here's a **receipt**");
var receiptCard = new ReceiptCard()
{
    Title = "I'm a receipt card",
    Buttons = new List<CardAction> {
        new CardAction()
        {
            Value = "https://robinosborne.co.uk/?s=bot",
            Type = "openUrl",
            Title = "Pay with rposbo bucks",
            Image = "https://res.cloudinary.com/otomotech/image/upload/f_auto,q_auto/robinosborne/2016/07/robinosborne.jpg"
        },
        new CardAction()
        {
            Value = "https://robinosborne.co.uk/?s=bot",
            Type = "openUrl",
            Title = "Paypal"
        }
    },
    Facts = new List<Fact> {
        new Fact("You shopped at", "Epic Shop"),
        new Fact("We love you", "THIS MUCH")
    },
    Items = new List<ReceiptItem> {
        new ReceiptItem
        {
             Title = "Building Great Startup Teams",
             Image = new CardImage("https://images-na.ssl-images-amazon.com/images/I/41xkMlpjRuL._AC_SX60_CR,0,0,60,60_.jpg"),
             Price = "9.99",
             Quantity = "1",
             Subtitle = "A Guide to Attracting, Hiring, and Retaining Employees",
             Text = "Dean Hume"
        },
        new ReceiptItem
        {
             Title = "Fast ASP.NET Websites",
             Image = new CardImage("https://images-na.ssl-images-amazon.com/images/I/51S6zPA2TYL._AC_SX60_CR,0,0,60,60_.jpg"),
             Price = "20.99",
             Quantity = "1",
             Subtitle = "Practical, hands-on guidance to create faster, more efficient ASP.NET sites",
             Text = "Dean Hume"
        }
    },
    Total = "30.98",
    Tax = "9.68",
    Vat = "10"
};

reply.Attachments = new List<Attachment> {
    receiptCard.ToAttachment()
};

Receipt: in Emulator

Woah. Yeah, you can seriously pack a load of information into this one card. It even renders quite nicely in the emulator, except notice that it’s using the main receipt’s title for each line item instead of the item’s title, but the correct title per line item is returned in the JSON, so this is just an issue with the emulator.

Still cool though. Having multiple buttons is interesting, and being able to add an image to a button could be handy for, say, payment method logos.

Receipt: in Skype

This is a really slick looking card.

Rich Attachments: Sign In Card

The sign in card may not look like much (and it doesn’t), but when the card action is set to the type of “signin”, it can initiate an oauth flow; the user can be made to leave the conversation to authenticate with a third party system.

Activity reply = activity.CreateReply("Here's a **signin**");

var signinCard = new SigninCard()
{
    Text = "Authorisation needed",    
    Buttons = new List<CardAction> {
     new CardAction()
        {
            Value = "https://robinosborne.co.uk/?s=bot",
            Type = "signin",
            Title = "Facebook OAuth",
            Image = "https://cdn1.iconfinder.com/data/icons/logotypes/32/square-facebook-128.png"
        },
     new CardAction()
        {
            Value = "https://robinosborne.co.uk/?s=bot",
            Type = "signin",
            Title = "Google",
            Image = "http://images.dailytech.com/nimage/G_is_For_Google_New_Logo_Thumb.png"
        }
    }
};

reply.Attachments = new List<Attachment> {
    signinCard.ToAttachment()
};

SignIn: in Emulator

Ha – the smiley isn’t on the message at all; this is just how the emulator is rendering a “sign in” card!

SignIn: in Skype

Very cute! It hasn’t taken my content, nor my custom buttons; looks like the “sign in” card may use default content only.

Attachment Layouts: Carousel

it’s possible to use the AttachmentLayout property to return a response containing several different rich content cards, and combine them into a carousel display; by default these are displayed as a list, as you can see from the image attachments screenshot.

Currently this only works for hero and thumbnail cards, and the emulator just renders a horizontal scrollbar which flips between each card (i.e., as opposed to a smooth scroll).

Summary

In this article I’ve shown you the various response types available to developers in the botframework with an example implementation for each type.

The possibilities for this are already numerous; you could implement a bot which could have a simple “conversation” with a user, create a carousel of products (hero/thumbnail) with click or tap actions which could add products to a cart (maybe..), give the user a receipt, and have them authenticate via a sign in card.

The concepts are there, and it’s certainly promising.

Have a play, let me know how you get on!