🖥 Presented by Women Who Code Python 👩💻 Speaker: Poojita Garg ✨ Topic: How to Build a WhatsApp Chatbot Using Python and Twilio
Ever wonder if your WhatsApp was a bot that could perform automatic tasks such as sending birthday messages at exactly midnight or sending you reminders about your day-to-day tasks?
In this talk, you will learn how to create your own WhatsApp chatbot that can help you set and receive daily reminders via WhatsApp. We will make use of the Twilio’s WhatsApp API for communication, Python for development, Google Sheets as database and Heroku for deployment. After this talk, you will be well equipped with a good understanding of a wide range Python packages, such as Flask, gspread, APScheduler, datetime, pytz and more.
For our 💬 slack channel, 🎥 previous event recordings, 🗓 upcoming events, 💻 GitHub repo and more check us out on https://beacons.ai/wwcodepython
___ 💻 WWCode Digital Events: https://www.womenwhocode.com/events 🔎 Job Board: https://www.womenwhocode.com/jobs 💌 Make a Donation: https://www.womenwhocode.com/donate
Okay, so that's all and welcome, everyone. So welcome to our Today's session, which is on how to build a WhatsApp chatbot using Python and Twilio. So Hi, I am Pajitha Girl. Nice to meet you all. And I'll be your speaker for today. And it'll be a really exciting hour discussing about how we can create our own WhatsApp chat bot. So let's dive right into it. There can be many types of WhatsApp chatbots and you might have also come across many for different purposes. For this session, we are going to develop a reminder bot. So this would be a very simple bot which will just ask you a reminder date and reminder message and would eventually remind you about that message at that set date. Okay. So this would be essentially the purpose for the bot which we are building today. So we'll be using Python WhatsApp and trivial a little about me. So I'm Bujet agarg. I'm an engineer, developer, and researcher. Currently, I'm working as a research intern at Georgia Tech and I. D. Roper remotely. And I'm a senior graduate from India. So I graduated this year. I have authored four research publications and a patent in my undergrad for one of my project. And I'm a passionate advocate for women in technology as well. And I serve as chat lead for women who code mobile and Python trucks. So before starting our talk, where can you get the materials? So this is the GitHub repo, which Square has posted all the code for which we are going to use for today. And you can also scan the Qi code given on the screen and there you can access the GitHub people. Okay, so a little sneak peek into what we are going to build today. This is essentially a kind of a flow of how things would be working. And let's say that I am what's up user and I am using that bot. So we'll be using Twilio for communication, essentially connect our WhatsApp with our code, then we'll be hosting all of our code, the Python code on Hiroko so that it can constantly run. So you can see here that there are two directories, basically which we will be hosting. So you can say that there are two apps which we are hosting on Heroku. So we'll start with the WhatsApp bot app. So this app will essentially contain all the logic which will be used to create a conversation between a user. So here it is the what I bought again, after asking the user for a reminder date and message, then it will save all that information to the Google Sheets so that it can be checked later on and the information can be retrieved later. Now the other app, which is Other, which is again hosted on Heroku, we call it Scheduler. So that will constantly check on Google Sheets to whether there's a reminder set for today, let's say. So it will check that it will retrieve data from the Google Sheets and check. And if that's true, if there's some reminder which is set for today then it will send a reminder at the scheduled time and date to the user. So a couple of requirements which you will need to follow along. So it's like Python 3.6 or newer. Secondly smartphone with an active phone number and WhatsApp installed. And thirdly is the Twilio account. So if you're new to Twilio you can create a free account and you can review all the features and limitations at the truly website. So just to get you more excited about today's top couple of Python packages that you're going to use, I'm assuming that everybody is really excited about Python. When I was creating this project I was trying to use many different Python packages so that to get more glimpse of what each package and how it looks like, how to use it. So these are some of the packages which we are going to use including Flask Twilio, Python Helper Library which is essential for using Twilio and then Gspread which is used if you want to read and write anything on Google Sheets, then AP Scheduler which is to schedule your jobs and then few date time packages. Okay, so I have divided not to get things overwhelming here. I have divided the whole talk into five major steps. So the first is to configure the Twilight sandbox. The second is to create a class chatbot service and coding logic. The third is that saving reminders into the Google Sheets. And the fourth is to schedule the reminder messages. And the fifth is to deploying apps on Heroku. So I hope you are all ready. Let's begin with step number one which is to configure the Twilio WhatsApp sandbox. So if you're new to Twilio, Twilio is a US cloud communication platform as a service company and Twilio allows software developers to programmatically, make and receive phone calls, send or receive text messages or WhatsApp messages in our case and perform all the kinds of communication functions using its service API. So it's a very handy tool. Nothing to worry much about. What you can do is you can just create a free account ongoing on Twilio.com and you'll get like a $15 credit for a new login and you can use their services as much as of that for free. Very good. So we'll be using WhatsApp sandbox like their WhatsApp messaging platform. So it's a very interesting platform here. What you have to do is you just have to go to messaging and then search for like send a what's up message or something like WhatsApp messaging platform. And when you go there you will find a Twilio sandbox for WhatsApp. So sandbox is essentially like virtual person. You can say like virtual Player where you can easily develop and test your application. There will be a number associated to your sandbox. Okay. So you have to do is that you have to save that number on your phone and then send a what's up message to that number. So there's nobody behind that number but it is just a service. And here there will be a join code. So you just have to type that code like the join code which will be mentioned on your console there. And once you do that, so there will be a message which you'll receive and it will tell you that. Okay. Now your sandbox is set up like it's connected to your phone. So they're very easy like few steps here which are very straightforward and you just need to follow them as to set up a one way messaging and two way messaging and then you are ready to use your Twilio sandbox for your function. That's all for setting up the Twilio sandbox. And now if we move towards writing Python code for that, you need to first of all create an empty directory and name it what support. So this is after writing all the files which we are going to use which we are going to talk about in the rest of the sections as well. So your WhatsApp bot directory would look like something like this. So it will have these five files. Okay. So now once that you have created the empty directory it's time for step number two which is to create a Flask chatbot service and coding logic. So to always one new to Flask. Python has several web frameworks like Flask and Django etcetera. And these can be used to create a web apps and API's. So these web frameworks provide functionality for building web applications including managing Http requests and rendering the templates. So for this tutorial we'll be using Flask which is one of the popular web frameworks for Python. So to use it let's install it first with the command install Flask. And once that's done you can start with creating empty file app by all of our coding logic behind the chat bot would go in this file. So basically what we're doing is that we're creating a web app and then routing it to our Twilio like we're connecting it with William later on. So this is like a basic Flask app would look like how to make a Flask with that. So this is the basic code for that. Before I start with what the code means, just to note here that we won't be going in super detail about the code which is going to come forward like all the code because that will be very much overwhelming. So what I've done is that I have also created a well documented blog for this stock which I would ask Stephanie to put the link for in the chat. So if you have any doubt there in the blog I have tried to mention the reason for every line of code which I have used and also the link. So I have tried to include all of the links which will be helpful for you if you want to dig deep into any of the let's say package or any of the function. So again later on you can also kind of look into that if that's what you would like to do. So here what we're going to do is the basic like the major crux of why we are using this piece of code and what essentially this code lines are doing. So this is the general how we create a small Flask application. So we start with importing our required modules. The third line of code which is this is basically to create a Flask application object and we're naming it up and which contains all the data about the application and the methods. You can see the object functions that tell the application to do certain actions later on. In the last line we are also running it. So it is one such method of that option. In the fifth line what we are doing is the action is called routing. So basically we are mapping a URL path to a function which is reply, mapping the SMS to the function that is reply inside which, inside the reply function we're going to code all the logic so that's the function of that and what that method is. So the methods is a list which is a keyword argument that let the Flask know what kind of Http requests are allowed. So as you know that there are two kinds of Http requests like the Get request to send the data from your application to the user and the post request which is to receive the data from the user. So for our purpose we are going to use the post request because we want the data to be received from the user. Also we are establishing an end point which is SMS here to the URL which we are going to see later where to use it. Okay, when that's done we are extending our code to include the Twilio library. Okay, so you'll import Twilio and after importing it, basically Twilio expects the web hook needs to be given in twice. So Twilio has its own Twilio markup language which is an XML based language. So the Twilio helper library for Python which we are going to use. So we are using the messaging response. So basically in this code what we are doing is that we want to communicate with our user. Okay? So while we're doing that through Twilio we need everything to be in sync with. What will you expect us our code to give a response. So for that we need to make sure that we are using this video helper functions and we are using messaging responses. So in the later part of the code you can see that majorly. First of all we are accessing our incoming message which is very important. Right? So for doing that we are using the request package, the incoming message which is sent by the user is included in the payload of the post request with the key of body. So here we are kind of accessing that the request form get. So from payload of the key of body. Yes, accessing it. Later on we are converting everything with the string which we are getting into lower case. So that to avoid any discrepancies because later on we'll be like comparing strings for that case. Then later we just setting the response as an object for messaging response. So once that's done, that was the initial set up. Now let's talk about what the logic goes behind our chart. Like, what are we trying to implement? As I talked about earlier, our chart part is fairly simple and we're just trying to create a reminder bot which will ask the user for a date and would ask for a reminder. Right? So it is fairly simple. So what I've done is I've taken four types of user input, so I'm expecting that the user can input like these four things. First of all, I created a trigger. Like let's say I created hello as a trigger. There's a chat bot. Whenever a user wants to set a reminder, he will type something. So inspecting the user would type hello, right? So that's one type of trigger and created to start the conversation with the bot. The second, what user can type when it will say hello? We'll ask that, okay, do you want to set a reminder? So the expected answers could be yes or no. So depending upon that, that's another type of response which we can get. The other type of response, like the user input, would be to set a date and the message right? So for that, what I've done is because later we have to find out whether that state or whether what the user is saying is a reminder. I have created my own custom keyword for that. Like, let's say this is the syntax of how the user should write, which we will mention in the quote as well, so that please type in this format. So what I've done is let's say date, which is kind of custom keyword here at the rate. And then we would ask the user to type the date, whichever they want. So that is how we are expecting our user. That's another type of response which we are trying to get from the user. Similarly, for the reminder message, it's reminder at the rate and then type the message. We'll see later how we are going to use it in our code. These are the major things we were expecting a user to respond with. Now let's talk about the chatbot functions. There will be two major functions because we have created a fairly simple bot and it can perform two functions, which is first of all, it will set the reminder date, which essentially means that whatever the date the user has prompted us with, we are going to save it to the Google sheets to be later retrieved by the schedule. Similarly to set the reminder body. So just saving it onto the Google sheets. So these are the definitions for these two functions basically. Okay, so once that's clear, let's see how we are coding the logic. It's very simple. Again the steps which I just mentioned, the four types of responses. We're just coding it using conditional statements. So the first conditional statement would be that if hello is present in our incoming message, that means that's the initial conversation. So for that figure we are just asking hello, do you want to replying it with hello? Do you want to set it tonight? So that's the first part. Okay, for second part, let me just go back here. You can see on the line 18 we are having words. We are basically splitting our incoming message with at the rate and we are getting the whole list in words in the words variable. So that is the reason why we did that custom thing for fetching the date and reminder message. So later on we're just splitting with Adbirate and we're kind of trying to analyze what kind of reply the user has given to them. So in the first type of conditional statement we are using that if the length of words is equal to one and yes is present and if no is present. So these are two types of conditions. So in this that means that this is the second type of response which we were expecting from the user which is in response to what do you want to set? Right. The length of work means that definitely there is no add the rate. Like it is just one element in the list when we're splitting it and if there's yes then ask for a date. And if there is no then just say okay, have a nice day. Later on if length of words is not equal to one. So that means that there was active rate and it has actually split into more pieces, right? For that I'm checking that for that. Like the first element of our list would contain the input type. So whether it will be date or it can be reminder because as per our context. So if it's date then just ask the reminder as well because we are asking date first and then the reminder. So then ask for the reminder. If that's the reminder then okay then that means then now just the functions need to be performed and the conversation is over. So free or reminder is set. Then in between, we are also calling those two chat bot functions which we defined earlier, like the set reminder date and set reminder body. So we're not like actually defining, we're just calling the functions that are not defining it. And we'll be doing that in another file which will be dedicated to sending and receiving messages from the Google Sheets. So that's a different story there. So we'll be talking about that part in the next slide. Also in continuation to the previous code in the app. So this is just that if nothing is responded then just prompt something to tell the user to reply in a correct format or something. And these are just the functions here. I'm just calling another like save reminder date and save a reminder body to keep it little organized. Okay, so this brings us to step number three which is saving reminders onto the Google Sheets. I hope that everyone has used Google Sheets sometime till now and it's quite handy tool to use. So we have a Python package which is called G Spread which is like particularly dedicated to interacting with Google Sheets for Python code. For that to use it you need to install it with install. Gspread. And I would just like to mention that we'll be taking all the questions at the end so you can type the questions in the chat as well. So I'll be just responding to them in the end because there's a lot to cover and there are many other steps as well. So I just want to finish the talk first and then take the questions with install. Gspad. We'll be just installing it later. For that we're just creating another file. I've named it function. Py and there I'll be defining those functions to save reminder, date and message. And then later you have seen that it can be imported to App Store. But before that there are a few series of steps which you need to follow to get your Google account authenticated to be used in an external piece of code. So that's true for any other GCP, APIs or services if you have used earlier in your code. So that's completely true for all of those. They are like the same steps. So you can find these steps there in the blog as well and you just need to follow them. So at the end of all these steps you will get a credential. Json which is an auto created file. You need to download it. Okay, that will be created for you through your GCP and there that file needs to be present in your directory where you are using it. When you're using the gspread package, this credentials or JSON file should be there in the same directory. That's just the caveat here. So once that's done, once your account is authenticated you are free to use the spread in your code. So let's talk about how we have organized the C sheet function file for us. After importing the libraries and stuff then there are some technical things which we need to authenticate the credentials and link to the credentials with the board. So these are just the typical things. It has to be done this way only. Okay, later on from the line ten you can see that here we have started using the Gspread functions. The first of all we have used Open to open the spreadsheet. So before going in the code, let me just show you how your spreadsheet will look like. Just create a new spreadsheet name. It something I've named it reminders. So this is how after storing some values your spreadsheet will look like. I just created like a very fairly basic sheet. I just created two columns state named date and reminder messages and just saved bunch of reminder dates and messages like after through that conversation for testing purposes. So coming back to the code here, you need to first open your spreadsheet with the reminders, then dot sheet one. Sheet one is the name of the sheet in the spreadsheet which you're using. So I've just kept it for the by default name which is sheet one. When you're using other functions like row values, column values, length. So just to find out the next empty sell for you to add the values right for you to save the values. Then you have created those two functions which we're talking about earlier to save the reminder date and save the reminder body. For that you are using the update cell function to see first the empty cell which is row filled plus one. That would be the empty cell, but you're finding the filled roles and then you're just putting the date and message and just updating the cell with that. Okay, so with this we are like completed with the half of our project which was to establish a conversation like the code, the logic behind the conversation with the bot. Now this is the second part which is to schedule the reminder messages. So for now we have a reminder message, we have our reminder date. So in this form these are some of the reminder message and dates which a user has input and you have saved it onto Google Sheets. Now you want to schedule it as per required. For that I'm creating another directory and I'm naming it scheduler. This is how your type two would look like after we go over all the files which will be needed. So there can be multiple ways to set a reminder. Right? So what I am doing in this case is that I have set a specific time. I have set a specific time of the day, let's say 900 a. M. Which is in the morning. And there you'll get all the reminders for that day so that you can schedule your remaining day for whatever task you have been reminded of for doing for that day. So this is my logic. Like you can create your own logic. So when we're talking this route of scheduling messages, I just want to tell you here that like the WhatsApp bot thing and the scheduler thing. These both are basically different routes with respect to Twilio. So Twilio is perceiving them as two different routes. In the scheduler part we'll be using a Twillo helper function which will basically be communicating our message. But in the what subbot you'll see later on when we will add the URL endpoint to your console which you created in the first step. So there we are directly putting the endpoint that whatever message will you receive from our sandbox, it should direct towards that WhatsApp bot functionality which we have created earlier. But here because here a user is not initiating a conversation. Right? A user is not triggering the bot, it is being triggered by itself so that whenever that day comes when the reminder is set for the bot should automatically send a reminder to you. So that is a different route which we are following here. For that we are basically communicating through Twilio from our code, from our Python code. So we have both the options in Twilio for that purpose. I created a Twilio function. Py okay, this file here we'll be using again from Twilio. Press will import client two things, account. Syd and author. These are two credentials which are there which are associated with your Twilio account and which need to be placed here. So to authenticate that you are the right person who's accessing that account, find those in your profile. Fairly simple process do that. You just copy paste it here in formal string. Then the client is just Authenticating it. Okay, now I created a small function here which we will call later by our scheduler. That okay. We'll call this function from the scheduler which is send reminder parameters are like date and REM which we're going to pass through the scheduler here. This is the format how it really uses, how we can send a message from our code. Okay, so this is the from number which is essentially our sandbox number. So all sandbox numbers are the same. The difference is the code which it gives you which is unique to you. This is the sandbox number. Make sure to write it as it is like type WhatsApp column which this is important. If you don't write WhatsApp Colin then it will treat it as I guess it will treat it as like a simple message. So we have writed WhatsApp calling on the whole because then you'll be writing your number. So just make sure to write it in similar fashion then body and this has kind of formatted it in this form like reminder date and the next line the actual reminder message. So that's all for twiliofunction by let's see how we are going to use the scheduler which we discussed like the AP scheduler. But before that a couple of things. First of all we are going like I discussed about like setting a fixed time. So for the AP scheduler we'll be using a blocking scheduler. So there are many types of schedulers which you can have in your module AP scheduler. Two popular ones are blocking scheduler and background scheduler. So that's all for whether you want your main thread to be blocked or not because the function of my code is exactly to do just one thing which is scheduling and there's no other thread. So I'll be using a blocking scheduler and there are various triggers. I'll be using the date trigger because I essentially have a date to which I want to trigger. That's two things. And also there's one thing when we will be dealing with date time, when we are working with dates and time in Python. There's one thing to take care of is that later on when we will be deploying our apps on Hiroko Hiroko will return calls for the current time in UTC. So I found this package pytz package which is used when we are working with a different time zone. So as I was in ISP Indian time zone. So I tried to convert it into UTC just to keep things like basically to try out this Pietz package naturally. So that was interesting and we'll see how to do that. Okay, one thing more here is that before I go into how the code looks like what logic I have applied here is that I will be traversing the date like the date column of the Google sheet where I've saved the date. I will be traversing that every day like at a six time like let's say in the morning. I'll be traversing once each day and I'll be seeing if first I'll find out what date is today in the quote and then I'll be matching it with the date which is mentioned in that column. So if any of those dates match. So like today's when one reminder needs to be set. So that day would be the day when I want the reminder, right? So this is the logic behind how I'm using the scheduler. So it will be more clear when we go over the code here. Also we will be making use of the Gspread package because here we are not writing anything on the Google sheets but we are kind of accessing the Google sheets to check the data, what is stored in it. So just a couple of packages which we imported and then the typical things using the Gspread package like authentication stuff which we're using again. Then we're just processing our date and collect formats which is a bit of a hassle when working with date and time. So just doing that stuff here and just converting the date, whichever I'll mention from my time zone to UTC later on I just created a scheduler and then again accessing my worksheet, my Google spreadsheet and then again the last part of the code is again the same thing like coding the logic which I just explained to check first traverse all the dates mentioned and then checking if something is equal to the one which is today and then just calling the scheduler a job. Here I'm calling the Sendram function which we like defined here in the Twilio function P by we define the syndrome function. So I'm calling this here by the scheduler. This is the way we do it. Scheduler agile date is the trigger and then my date which is in Utcdt which is the date here and then the argument. So these arguments are basically the date and the reminder values which we will be using in the Sentra and later in the last line we just starting our schedule. Once that's done we come to our step five which is the last of all the steps which is deploying apps on Hiroko. So if you're new to Heroku, Heroku is a platform as a service that enables developers to build, run and operate applications entirely in the cloud. So to keep running our applications constantly we will deploy both of our applications which is the WhatsApp bot and scheduler separately on cloud platform named. So there's a reason why we say it to different directories and why we are separately deploying it on the reason it will be just clear in a minute. So to use Heroku we need to create some special files, some special files which we need for Hiroku only. So one of them is Proc file. So Proc file is a special file which is a mechanism for declaring what commands are run by your application diners on the Hiroko platform. So to run it on Hiroko we need Proc file. It needs to be named as it is without any extension with the capital P. Just make sure that now when we are deploying it. So you need to create two different files for two different directories. But what I bought you will create a profile and just write one line which is Unicorn app. So basically what it is doing and you need to also install unicorn. So to run web apps because we used Flask to create a web app in what I bought and to run that we need this line in prop file. Okay, so this is the process which we need to run which is app here. Okay. And for the scheduler folder. So we created another folder because there's difference in Proc file. So later when we are using Heroku so there is difference. So in the scheduler folder because we are using a scheduler and I need to run that code each day once. Okay. So for that I'm using this clock and I'm initializing it pythonscheduler. Py. So I want the scheduler PY two dot okay at clock. So then it will run like to run it, run the script every time, once each day once. So for that I have used this code in Proc file for scheduler. So this was one file which is Proc file. The other ones are basically requirements. Txt. Okay, so it will contain all the third party libraries required by the app. So we have used a couple of third party libraries. So we need everything in the requirements. Txt versions so simply do Pep trees requirements. Txt and for both the folders and it will be generated then get ignored because we are going to use Roku will be creating a Git repo for both of the folders we'll just add in get ignore so that any of the specific patent files are excluded. Right? So this is how I'm doing that and then runtime PHP just to specify the particular version of Python to use that's a file for that. Now we are all set to create the Roku apps for our folders. So I have listed all these steps like very concisely the steps we need to follow. So we are going to start with creating a Reposit for both of our folders, then creating a Hiroko account and downloading the Hiroko CLI and for those also the steps are linked and this is a fairly simple the usual steps for creating a Git repo, then pushing Heroku master and then if you want to see the logs then again there's a command for that as well. So after creating a local repo and using Hiroko command line to create Heroku apps, what you will get is you'll get a URL for both of your apps. So the URL for the first app which is what the WhatsApp bought, you are going to save that and you're going to put it here. Again. Go to your Twilio console. Twilio sandbox for WhatsApp? After you performed all the configuration steps for sandbox you will find this page like this type of console where you will see that to send and receive messages from sandbox to your application, configure your endpoint URL. So here you will be putting your endpoint URL for WhatsApp? For the conversation you will be putting here when a message comes in. So this will essentially direct whatever message comes to your sandbox to your quote where you're doing all the processing with http post. Also make sure that because while we were configuring our Flask app, while we were routing in there we used SMS as endpoint. So just whatever you use, if you don't use any then you need not change anything. But if you're using something some other endpoint like slash SMS I have used. So I'm just going to append it with my Heroku app URL. So Roku app URLs, I'm just going to append there which I will be saving here like when a message comes here with the URL and just append it. So well done guys. This is all done. Now your chat bot is ready so you can see a demo on your screen. This is the one I created exactly the same thing. And now see I'm typing the date and the reminder message. It will be saved and whichever date you have decided to get reminded that particular message for you will receive that reminder. So that's all any questions? Let me just pull this chat just a second. I hope you enjoyed the talk click. I really enjoyed. Yes, they are recorded. When Twilio asked what do you plan to build with Twilio? Do I click alerts and notifications or something? You can click anything that doesn't really matter. You'll be still be able to access all its services so you can click whatever suits best to your needs. I'll send a link to this presentation yeah you'll get the link to the presentation as well. Also the blog link is already there. So there I have explained each and every line of code which was not there in the presentation just for your knowledge. So you can also go over that and the link to the presentation will also be provided. Thank you so much. Can this project be extended to other apps like hook me? I really don't have any idea for that. You need to research about this thing from Twilio site because obviously you can code for anything right but is there a functionality where your code can communicate with other apps so earlier also in Twilio and there was no functionality of this WhatsApp so around I think 20, 18, 19 they have added this WhatsApp thing because it's a big thing. Now everyone's on WhatsApp so you need to check from the Twilio side like whether that app for which you are building a bot is supported or not because then only you'll get the required helper functions to communicate. So communication is the base that's thank you so much. Thank you so much, Gloria. Okay, so I think there was no more questions. Thank you so much, rishno thank you so much. Thank you so much all for attending. We have a lot of other events as well which are super cool. You can go over our website to get registration links. Thank you so much all.