Step by Step guide to make a dumb bot smart
When we hear chatbots getting popular day by day and seeing every company opting for chatbots, it seems ridiculous to believe that any day bots will replace websites and apps.
But, there are more than one reasons to believe so, be it ease of bots usage, no apps needed, no memory space needed, same platform works for a lot of bots and so on.
85% of customer interactions will be managed without a human by 2020.
Last year, Facebook allowed third party developers and Facebook Messenger has become one of the most popular platform for bots. And, seeing the popularity of Messenger, it seems one of the most popular ways to connect to users.
‘Messaging is one of the few things that people do more than social networking’ - Mark Zuckerberg, Facebook CEO
Before we start looking in the ways to develop a smart bot, I would just like to say that the current underlying chatbot technology is incorporating Machine Learning and Artificial Intelligence in the bot.
Let’s begin the smart bot development.
Let’s look at an example scenario and build a smart bot for it: purchasing a house.
Before using a chatbot to buy a house, let’s look at some ways to buy a house.
One way is to go to a broker or agent in person, meet him, go with him to see some houses and then choose one house.
Second option is to go to an online website and then search for the houses, apply some filters and then look for the desired house and make the decision. A good approach but still too much dependency on the available filters which are too mainstream. Suppose I want to have a house near a school as I have kids, I cannot exactly look for it. No such filters available.
The third option is using a chatbot and tell more about yourself to the chatbot directly, save yourself with the hassle of checking all the filters and then selecting all of them. All things can be done in a single or a couple of sentences. But the chatbot must be smart enough to do so.
Now the problem arises: How to make the chatbot smart enough?
To know more about the chatbot, their types and how they work, please have a look at this article.
Rule based bots vs AI botsA breakdown of different approaches to bot buildingmedium.com
There are two ways to make a chatbot interact with the user.
- Sequential Bot
- Conversational Bot
A sequential bot is a bot where the bot asks a series of questions and the user answers them. It works good for some use cases but it fails when there is a change of mind by the user. The bot cannot understands things and the user has to start over.
Let’s see how the House Booking Bot will work in sequential manner.
Here the bot seems to work super good as it asking what needs to be done and once user provides all the information, it works great and searches the results based on the parameters and replies back with all the answers.
Now, let’s look at another scenario. If the user writes more parameters at the start and the bot has to go all the way down to again ask the same questions again in the same sequence to collect all the desired parameters.
As we can clearly see, the bot is following a sequence and is asking the questions in the same manner again irrespective of the number of parameters provided by the user.Conversational bot
A conversational bot behaves more in a human similar way. This works even at those times too when a user changes his mind in between a conversation as we as humans, we do this very often.
The main advantage of this approach is to make the humans more comfortable saying things. But the question is how to implement such a thing. The answers is RAN.
But how RAN works?
In RAN (Random Access Navigation), the major concept is allowing the user to feel independent and converse freely as a human do in real life. Also, the user can modify/change his views anytime. This is important as this is how things work in real life.
In the above we can see user just provides only 2 entities and not the third parameter. Now if we think as a human, a human will ask only one question for the missing parameter and not all the above questions again. Here for example, the question to be asked is What is the budget ? And the bot does the same.
Also, we can see that the user changes it’s location from Copenhagen to New Delhi and the bot updates the parameter of location from Copenhagen to New Delhi in it’s final search.
As we can see above the user has provided all the 3 entities: Copenhagen -> Location, 3 rooms-> Number of rooms and 500000 Euro -> Budget. And as simple as that the bot recognizes all the parameters and then show the outputs. Hassle free thing.
Similarly, if the user provides one parameter or more as above, the bot will only ask relevant missing questions.But what are advantages of RAN ?
Now if we follow the concept of probability here, for 3 parameters the total number of cases will be 3*2 = 6. Using formula (n*(n-1)). Seems feasible to hard code everything.
But suppose we have 5 parameters ? Total possible conditions will be 5*4 = 20. Not possible to hard code everything.
Here is where we want to go with the probability of RAN where we just look for the missing parameters and then ask only the relevant questions and do not hard code everything.
Now what if I write more than one sentence at once?
Now let’s look again at how we write in a conversation. We do not always write all the inputs in one go but type things in different sentences. Let’s look at how our bot will work in such a case:
As we can see clearly from the above case, the bot is taking only the first sentence in consideration and generates it’s reply as it cannot find the budget in the first sentence and asks for it. Later again when the user provides the number of rooms parameter, the budget parameter is still missing for the bot and the bot again asks for the same parameter again.
How we can tackle the problem?
There are two ways to solve the problem:
Put a hard code condition like wait for n seconds (5 secs — 10 secs) depending on the situation before generating the reply. It works for some cases but sometimes it makes the things slow and it might be irritating sometimes.for (user.query()): wait.n_sec()query.process() #find missing parametersbot.reply()
2. Second way is to put a condition in the bot that while the user is typing(can be checked through the typing indicator), just wait before sending out any response and when the user is done typing, append all the sentences and then asks relevant questions or give the response.
while (user.typing()): query = query.concatenate()query.process() #find missing parametersbot.reply()
As we can see above, the bot is waiting till the time user is typing which is evident from the time of the second reply which user typed in another 15 sec and then the bot replied with only the missing parameter (number of rooms in this case) and once user provided the reply, the bot replied with relevant results.Another problem with chatbots is what happens when a user mistypes something or makes a typing mistake ?
There are again two ways in which the problem can be taken into account.
If the user identifies his/her mistake, and retypes the word/sentence again, the parameter value will be overwritten by the new value and the problem can be solved.while (user.query_parameters()): parameters= new_parametersquery.process() #with updated parameters
2. Second way is that the user might not know the mistake but the bot corrects it. So, in any messenger like Facebook messenger, it explicitly provides the correct words for any typos done. But if that is still not enough, we at Botsupply have implemented a concept that we call type-ahead mechanism. In this concept, the bot automatically suggests what the user needs to type when writing a query.
This is beneficial for cases where user can make mistakes while typing words or sometimes when the user is not sure how to proceed. So user can just auto complete the word or sentence based on the suggestions.
Find me a hotel for 500000 Euro bu # bot suggests budgetFind me a hotel for 500000 Euro budget
We will soon come up with an article in detail to explain the concept in more details along with it’s implementation.
Seems quite good till now. But how to make things work in some other language like Danish ?
The answer is to create our own Entity and Intent detector that works in the same manner as in English. It can be done by training a new vocabulary and tokenizer in the desired language.
One very good open source platform that offers such functionality is Rasa-NLU. It supports few languages at the moment but provides an option to add desired languages. To know more, have a look at the language documentation which has step by step guide to add a new language support and feel free to add your new language.
Language Support - rasa NLU 0.9.0a3 documentationWe want to make the process of adding new lanugages as simple as possible to increase the number of supported languages…rasa-nlu.readthedocs.io
We can add a feedback mechanism in the chatbot where we analyze the feedback provided by the user to understand more about the user requirement, and improve the bot. One good way to do that is to analyze the sentiments of the users who are using the bots and work on improving it.Keeping the previous context of the user is good for some use cases, where user does not have to follow the bot again to specify the requirements again and the bot can provide suggestions/recommendations based on that.
For example, a food ordering bot can save the last order of a person and at the start of the conversation, the bot can say:
Hi ! Last time you order a pepperoni pizza with a diet coke. Do you want to go with the same order or want to make any changes ? Even suggestions like we have a new pizza in the house like a pepperoni with cheese. Do you want to try it ?
Lastly, I would like to focus your attention on an article that says majority of the chatbots failing short from delivering par performance.AI powered chatbots failing due to disappointing performance — Tech2Although companies the world over are looking forward to embracing chatbots, most of such artificial intelligence (AI…tech.firstpost.com
So, it is important to understand the objective of a chatbot by both the users and the developers in things that can be achieved from the chatbot.
As well said by Giovanni Toschi :
Our aim is to achieve an objective using chatbots and not to pass the Turing test — Giovanni Toschi, COO Botsupply
However, if you want to know how close we are to pass the Turing test, have a look at this article:
How close are chatbots to pass Turing test?A scientist perspective on chatbots and Turing testmedium.com
Please feel free to add any suggestions or any problems that you faced while using or developing a chatbot.