Video details

Web UI Testing in Python


Unit tests are great, but they don’t catch all bugs because they don’t test features like a user. However, Web UI tests are complicated and notoriously unreliable. So, how can we write tests well? Never fear! Let’s learn how to write robust, scalable Web UI tests using Python, pytest, and Selenium WebDriver that cover the full stack for any Web app. In this talk, we will write one simple test together that covers DuckDuckGo searching. You’ll learn how to write battle-hardened Web UI tests for any Web app, including Django and Flask apps. I’ll also provide plenty of hands-on tutorials and resources to continue learning after this talk.
PUBLICATION PERMISSIONS: Original video was published with the Creative Commons Attribution license (reuse allowed). Link:


As someone who specializes in testing and automation, there is something that really bugs me. When most developers I meet talk about software testing, they only think about unit testing. In fact, many people simply use the term unit testing to refer to all testing. There is more to software testing than writing tests for methods and functions. Unfortunately, I see this misconception fairly prevalently within the Python community. Ever since 2017, the Python Software Foundation has teamed up with JetBrains to conduct the annual Python Developer Survey. The survey asks many questions about who Python developers are and what kinds of work they do. Every year they include questions about test frameworks. The first time I took the survey was in 2018, and I felt super excited to see a question about testing. Here are the results from that year's survey. I was delighted to see Pi test at the top of the list, but then my heart sank when I read the result description. They called all the frameworks unit testing frameworks. So sad. Frameworks like Pytest can be used for more than unit testing. They can be used for testing web apps, mobile apps, and Rest APIs as well. Sadly, that wasn't just a onetime fluke. The 2020 survey used the same naming. Clearly, the notion that unit testing is preeminent is pervasive. I want to make it abundantly clear that there is a difference between testing code and testing features. Code is the text in which software is implemented. It contains the logic for execution. In Python, code has scripts, functions, and classes. Test classes that cover code explicitly are called white box tests because they have direct access to the code's. Callables the tests check. Is the code written to do the expected things? Will this function return a proper result? Will that method update that object as it should? Whitebox tests may be unit tests if they cover individual pieces of functionality, or they may be subcutaneous if they test workflows through the code. Features, on the other hand, are behaviors of built products. Features enable callers to do something. They are all the things a nonengineer could do with your software product. For example, a web app could have features for entering profile information, searching for records, and uploading files. Features are implemented using code, but they themselves are not code. Test cases that cover features are called Blackbox tests because they do not have direct access to the code. Instead, they exercise features like a regular user or caller. The tests check. Does the product meet its requirements? Blackbox tests may be called integration or EndToEnd tests because they make sure all the pieces of a software product come together to produce the correct behavior. I could talk all day about different kinds of testing, but in this talk we'll focus on Web UI testing. Web UI testing is black box testing of a web app through a browser. Its purpose is to verify that a web app works the way a user would expect to that end. It is a type of feature testing because it tests the app like a user. It also is an end test, because all parts of the Web app from the front end pages in the browser to the service calls to the back end, and even to the databases for storing data are all exercised together as one system. Anyone can do Web UI testing by Loading pages in a browser. But test automation makes execution faster and more frequent. Unfortunately, though Web UI testing is expensive, environments have many parts set up for testing, and WebUI tests frequently require multiple steps. It's impossible to test every behavior of a Web app. Therefore, focus on the highest return on investment, write unit and integration tests to cover lowlevel code and calls. Write WebUI tests for the most important behaviors, and watch out for things like race conditions, data collisions, and changing pages. When designing your test steps. It's no lie that WebUI testing can be hard today. Let's try to make that easy. Let's look at an example Web UI test together. The test case is a basic search test for DuckDuck Go, a popular search engine. Duckduckgo is available worldwide. Its UI is simple, and everyone intuitively knows how to use a search engine. Step one in our test is to navigate to the dock. Go [email protected] If you can't find it, I guess you can always Google it. Step two is to enter a search phrase into the search bar. This should trigger a results page to load. Step three is to verify the query on the results page. The phrase we entered on the search page should appear in the search bar on the results page. Step four is to verify that the search phrase also appears in the page title. Sometimes tests need to check multiple things to make sure a feature is behaving correctly. Final step is probably the most important. The test should verify that each answer on the results page somehow relates to the search phrase. The search phrase could appear in the title, the link, or the description for each returned item. We can write this test case in its steps in Gerkin's given when then format for clarity. Notice how this follows the Arrange Act assert pattern. Given an initial state when actions are taken, then verify the outcomes. Even though this Webby test is fairly basic, it still holds a lot of value because in only five lines it determines if search is working or completely broken. We could run this test manually, but we could also automate it using Python. Automation enables us to run tests faster and more frequently. It's incredibly valuable to run automated tests continuously. I'm going to show you how to build a Web UI Test Automation project to automate our Duck Go test using some of the most popular tools and patterns in the industry. As we build this solution together, I want you to keep in mind at all times that test automation is software development full stop. When we automate tests, we need to bring the same care and attention to our code as we would for any other project. We should follow good practices like keeping our code in a source code management tool like Git and also doing things like code reviews. Test automation is a product, and continuous integration is its production environment. Test automation development is software development. Let's sketch out how we would develop our test project. For our programming language, we will use Python, of course, for our core test framework, we will use Pytest. If you recall from the Python Developer Survey results that I showed a few slides before, Pytest is the most popular Python test framework for handling interactions with the UI, we will use the Page object model. And finally, for handling web browser automation, we will use Selenium WebDriver, and we'll mention alternatives later in this talk. So how do all these pieces fit together? Pytest will call Page Objects, which in turn will call Selenium Web driver bindings in Python that connect to either a local web driver executable process or a remote service like Selenium Grid to connect to a live browser session. And wonderfully enough, Selenium WebDriver supports all the major browsers. So let's put those test steps for duckducko searching into Pytest. Pytest is what we call a core test framework. It is Pythonic, and powerful tests are written as functions instead of classes as they would be in Unit test. On this slide, I copied a screenshot from the homepage to show a Pytest test case example. Here, the Ink function takes in a number and adds one to it. The test answer test case calls the Ink function and makes an assertion on its results. In this case, the test would fail because Ink of three should be four, not five. In this talk, I'm not going to teach how to use Pytest, but I will show it in code examples. If you want to learn more about Pytest, check out the [email protected] or pick up either of these two excellent books, Python Testing with Pytest by Brian Hawkins and Pytest Quick Start Guide by Bruno Oliviero. Pytest is not part of Python's standard library, so we need to pick install Pytest to get it. When developing Web UI tests, you can choose to put the tests in the same project as the web app they cover, or you can create a separate project exclusively for test automation. Either way, for a Python project, it is conventional to put all tests in a subdirectory named Tests inside the Tests folder. Add a module named Test underscoresearch PY with the function shown here. I like to start with Test function stubs that have the test case steps written in comments. As I implement each step in Python, I'll leave the original comment in place for documentation. Plain language descriptions for each test case can be quite helpful when rereading test code in the future. To implement these steps, we need Selenium WebDriver for sending Web UI commands like clicking and scraping from the test automation to a real live browser. Web driver can handle every type of Web UI interaction you can imagine. If a user can do it, then so can Web driver. The Web driver API is enormous. I'll admit to you that I don't even know it all. Whenever I want to look up how to do something, I check the docs online. They're very helpful and thorough. For Python, the Selenium WebDriver package for Python is named Selenium. To install it, run Pip install Selenium when running tests from a local machine. Selenium WebDriver also needs a web driver executable for the target browser. In this talk, we will use Chrome, so we need to install Chrome driver on our local machine and also add it to the system path. Chrome driver acts as a proxy between test automation and the browser. Each browser type will have a different WebDriver executable. For example, Firefox uses Gekko driver. Every test case needs its own browser instance and therefore its own Selenium WebDriver object. In a file named conftest. Py, create a Pytest fixture named Browser. The name of this file is important because Pytest uses it to find fixtures like this one. Inside the fixture, construct a new WebDriver instance using a statement like this. B equals Selenium WebDriver Chrome. This line will construct a WebDriver object for the Chrome browser. It will automatically start a Chrome driver process, too. That's why your WebDriver executables like Chrome must be on the system path. If desired, you can construct web drivers for other browsers here. Next, set an implicit weight of 10 seconds for the web driver. Whenever this web driver will try to fetch web elements from a page, it will wait up to 10 seconds for the element to appear before giving up and raising an exception. If you do not provide an explicit wait, then tests will be susceptible to race conditions in which automation will try to access elements before the browser can render them on the page. Implicit weights Keep Tests safe. Other web apps might need more than 10 seconds if they take more time to load. Next, make the fixture return the WebDriver object with Yield B. This return will inject the WebDriver object into any test case that calls this fixture finally quit the browser as the final part of the fixture. Pytest will run anything in a fixture after a Yield statement as cleanup. After a test case is finished. Even if the test case fails, quitting, the web driver will close the window and quit the browser instance. Always quit, or else your machine might leave zombie processes running in the background. As a recommended practice, tests should not call Selenium WebDriver directly. Instead, they should use page objects to model the pages under test. A page object has locators for finding elements on the page, as well as interaction methods. That send commands to located elements. For example, a login page object could have locators for usernames and passwords, and a method for logging in. Our test needs two pages, a dot Ducko search page with methods for food and search and a dot Ducko results page for getting the query title and result picks. Let's look at the search page. I like to put page objects in a package outside the Tests folder so that they can be easily imported into test modules. I also like to put each page object in its own Python module, such as one named search PY. The page object should be a class with an intuitive name. It should have variables for URLs if appropriate, and locators should be stored as tuples with locator type and query. Here, the search input locator can be located using a name locator with the name queue. The class's Dunder and it method should take in a reference to the web driver object, which it will receive from the test case function. The interaction methods here then call the WebDriver to do the needful. The load method loads the DuckDuckGo homepage using the URL and the search method locates the search input element and enters a phrase into it. You can see WebDriver methods like Get, findelement, and send keys. Method names like load and search better describe the actual interaction being done here. So you might be wondering how do we know that the search input element could be located by the name queue? The easiest way to come up with locators is to use a tool like Chrome DevTools to inspect the source code for the page under Test. Locators can use IDs input names, class names, CSS selectors, or XPaths to identify elements. I recommend using the simplest locator possible that uniquely identifies the target element on the DuckDuckGo homepage. The search input element has an attribute named name with a value of Q. Writing good locators could easily be its own talk. If you want to learn more about it, take my free course on Test Automation University entitled Web Element Locator Strategies. Now we have all the pieces in place to automate our test case, revisit the original test function, and add browser as its arguments. Pytest will match this name against the fixture we wrote named Browser and then call it to initialize the web driver object. Using the web driver object, we can construct page objects. We did not show the code for result page, but it would be similar to the code for the search page. The first step calls Search page Load to load the go home page in the browser. The second step performs a search for the phrase Panda. The third step verifies that the results Page's search input value is correct and that it matches the query. The fourth step then verifies that the search phrase is in the title, and the final step makes sure that the search phrase appears in each result link title notice how it uses a for loop to look to iterate over every single title returned for the links and it also formats all titles to lowercase when performing its assertion. And that's our test fully automated. In Python Pytest, Page Objects and Selenium all come together beautifully and the test itself, thanks to our Page objects, is quite readable. That's a lot of information in a short amount of time. There's way more to learn about web UI testing than I could possibly cover in half an hour. If you want to learn more, check out these free courses from Test Automation University. Tau Test Automation University is an incredibly wonderful platform for learning all about testing and automation. I hope these courses can help you become a Python Test Automation champion in no time. Before I close, I want to mention one more thing. Selenium is not the only tool out there for interacting with browsers. Recently a new tool hit the scene. Playwrights Playwright syntax feels more concise and modern than Web driver's API. It also provides features like automatic weighting, more resilient elements, selectors and isolated browser contexts. Things that Selenium WebDriver doesn't provide. Like Selenium, Playwright has bindings in multiple languages including Python. I haven't yet used Playwright myself, but I've heard that it performs a little faster than Selenium. If you automate your tests using Page Objects then theoretically you can easily swap Selenium calls with Playwright calls. The only downside with Playwright right now is it does not have a tool for scaling out execution like Selenium grid. Nevertheless, I believe Playwright will be a major contender against Selenium WebDriver in coming years. It will be fun to watch it happen. So this concludes my talk on Web UI testing. Shishiya Shishiya, thank you very much for joining and thank you especially to the organizers of Python Taiwan for inviting me to speak. Unfortunately, I'm unable to take live questions, but if you have any questions, feel free to contact me through my blog or through Twitter again. My name is Andy Knight and I'm the Automation Panda. I hope to see all around the Python community and I hope to revisit.