Python Automated Testing: Framework and Examples

Automated testing is an essential part of modern software development that helps ensure software reliability, reduces manual testing efforts, and improves the efficiency of the development process. With Python’s simplicity, readability, and vast ecosystem of testing frameworks, it is an excellent choice for automation.

In this blog, we’ll explore the different Python testing frameworks and their features. We will also provide practical examples of how to perform automated testing in Python.

What is Automated Testing?

Automated testing is the process of using scripts, tools, and frameworks to automatically verify that your software works as intended. It’s a crucial part of modern software development because it allows you to test your application quickly and consistently without the need for manual intervention. By automating repetitive testing tasks, you can reduce the chances of human error, speed up your testing cycles, and focus on more complex problems.

Different types of Automated Testing

1. Unit testing involves testing individual components or functions in isolation to verify they work as expected.
2. Integration testing focuses on verifying that different parts of the system, such as databases and APIs, interact correctly with each other.
3. End-to-end testing tests the entire application, simulating real user interactions from start to finish to ensure all components work together properly.

Why Use Python for Automated Testing?

Python has become one of the most popular languages for test automation due to its simplicity, versatility, and the powerful tools it offers for testing. Here are the key benefits of using Python for automated testing:

Simple and Readable Syntax

Python’s syntax is clean and easy to understand, making it an excellent choice for both beginners and experienced developers. This readability helps you write tests that are not only easy to maintain but also easy for others to understand. You don’t need to worry about complex syntax or obscure rules, which makes debugging and updating test scripts much easier.

Rich Ecosystem

Python has a vast ecosystem of testing libraries and frameworks, such as unit test, pytest, Selenium, and Behave, to name a few. These frameworks are highly flexible, well-documented, and widely supported by the Python community. Whether you need to write unit tests, integration tests, or perform web automation, there’s a Python framework tailored to your needs.

Cross-Platform Compatibility

Python is known for being cross-platform, meaning it works seamlessly across various operating systems like Windows, macOS, and Linux. This is particularly valuable when you need to run automated tests on different environments or set up continuous integration/continuous deployment (CI/CD) pipelines that need to support multiple platforms.

Integration with DevOps & CI/CD

Python integrates easily with DevOps tools and CI/CD pipelines. Popular CI/CD tools like Jenkins, GitHub Actions, and GitLab CI can easily trigger Python-based test scripts on code commits or merges, helping you automate the testing process in your development workflow. This ensures that your tests are run automatically, continuously, and consistently, allowing you to catch issues early and maintain a smooth deployment process.

Popular Python Testing Frameworks

Each of these frameworks offers a different approach to testing, and understanding them will help you select the right tool for your project.

1. Unittest (Built-in Python Framework)

unittest is Python’s built-in testing framework, designed to follow the xUnit testing model. It provides a structured approach to writing and executing tests, and it comes pre-installed with Python, so you don’t need to install anything extra to use it.

Writing and Executing Test Cases Using unittest.TestCase

To write tests with unittest, you create test classes that inherit from unittest.TestCase. Each method within the class represents a test, and you can use various assertion methods like assertEqual(), assertTrue(), and assertFalse() to validate expected outcomes.

Running Tests with unittest.main()

After writing your test cases, you can execute them by running the script that contains the test class. Calling unittest.main() triggers the test execution and outputs the results in a readable format. This is a great option if you prefer an integrated and standard solution for testing in Python.

2. Pytest

pytest is one of the most popular and feature-rich testing frameworks in the Python ecosystem. It’s known for its simplicity, flexibility, and rich feature set, making it an ideal choice for both small and large projects.

Writing Simple Test Cases Using Functions Instead of Classes

Unlike unittest, which requires creating test classes, pytest allows you to write tests using simple functions. This reduces boilerplate code and makes test writing faster and easier. Tests are identified by the test_ prefix, and pytest will automatically find them by searching for functions with this prefix.

Using Pytest Fixtures for Reusable Test Setups

pytest introduces the concept of fixtures, which allows you to define setup code that can be reused across multiple tests. This is perfect for scenarios where multiple tests require the same initialization or configuration, such as database connections or web server setups.

Running Multiple Test Cases Efficiently with Pytest Command-Line Options

pytest is designed to run large numbers of tests quickly. It supports parallel test execution with plugins like pytest-xdist and provides detailed output to help diagnose failures. You can also easily filter which tests to run using command-line options like -k to select tests by name.

3. Behave (Behavior-Driven Testing)

Behave is a framework for Behavior-Driven Development (BDD), which focuses on defining the behavior of the system in plain, understandable language. BDD tests are written from the perspective of the user or stakeholder, making it easier for non-developers to understand and contribute to the testing process.

Writing Feature Files in Gherkin Syntax

In Behave, you write tests using Gherkin syntax, which is a domain-specific language for describing the behavior of the system in simple, human-readable format. A typical feature file consists of “Given”, “When”, “Then” statements that describe the preconditions, actions, and expected outcomes of a test scenario.

Implementing Step Definitions in Python

Each step in the feature file is linked to a Python function, called a “step definition”. In these step definitions, you write the actual implementation code that performs the actions described in the feature file. Behave then matches the steps in the feature file to the corresponding Python functions.

Running BDD Tests Using Behave

After writing the feature files and step definitions, you can run your BDD tests using the behave command in the terminal. It will process your feature files, execute the steps defined in Python, and show you the results.

4. Robot Framework

Robot Framework is a generic automation framework that uses keyword-driven testing. It is designed for both technical and non-technical users, allowing testers to create test cases in a natural language format. It is highly extensible and can be used for test automation of various types of applications.

Writing Test Cases in a Human-Readable Format

Test cases in Robot Framework are written in plain text or tabular format, using simple keywords to define actions. For example, you might use keywords like Open Browser, Click Button, and Input Text to describe the test steps. This approach makes the tests more accessible to people without a technical background.

Integrating with Selenium for Web Automation

Robot Framework integrates seamlessly with Selenium for automating web applications. You can use the powerful SeleniumLibrary in Robot Framework to interact with web pages, perform actions like clicking buttons, filling out forms, and verifying page content.

5. Selenium with Python (UI Test Automation)

Selenium is a popular tool for automating web browsers, and using it with Python makes it even more powerful. Selenium WebDriver allows you to control a web browser programmatically—navigating pages, interacting with elements, and verifying content, all from within your Python test scripts.

Finding Elements and Performing Actions Like Clicking Buttons and Filling Forms

Selenium provides a variety of methods for locating elements on a webpage, including find_element_by_id(), find_element_by_xpath(), and find_element_by_css_selector(). Once you’ve located an element, you can perform actions such as clicking buttons, entering text in forms, or selecting dropdown options.

Implementing Waits and Handling Dynamic Elements

Web applications are often dynamic, meaning elements may change or load at different times. Selenium allows you to implement explicit waits using methods like WebDriverWait to ensure elements are loaded before interacting with them. This is essential for handling elements that appear or disappear based on user actions or page loading times.

Setting Up Python for Automated Testing

Getting started with Python for automated testing is a straightforward process. By following these steps, you’ll have a fully functional Python testing environment set up and ready to go for your automated testing needs.

Step 1: Install Python and Pip

First, ensure that Python is installed on your system. You can download the latest version of Python from the official Python website. The installer will automatically include pip, which is Python’s package manager used to install libraries and frameworks.

Once Python is installed, you can use pip to install various testing libraries that you need for automated testing. For example, you can install pytest by running the following command in your terminal:

pip install pytest

Similarly, you can install other testing libraries like unittest, Selenium, or Behave depending on your project requirements.

Step 2: Setting Up Virtual Environments

It’s a good practice to create a virtual environment for your testing projects. A virtual environment isolates your project’s dependencies from your system Python installation, ensuring that you have a clean and conflict-free environment. To create a virtual environment, navigate to your project directory and run the following command:

python -m venv env

This creates a directory named env in your project folder, which contains a separate Python environment.

Installing Dependencies in an Isolated Environment

After creating the virtual environment, activate it by running:

• On Windows:

.\env\Scripts\activate

• On macOS/Linux:

source env/bin/activate

With the virtual environment activated, you can install all the testing libraries specific to your project without affecting your global Python installation.

Step 3: Installing Testing Libraries

With your virtual environment set up, you can install the necessary libraries for automated testing. For example:

To install pytest:

pip install pytest

To install Selenium for browser automation:

pip install selenium

You may also want to install additional libraries like unittest (although it comes built-in with Python) or Behave for behavior-driven testing:

pip install behave

After installing the libraries, you’ll be ready to start writing and running automated tests using Python.

Writing Automated Test Cases in Python

Whether you’re working with unit tests, automating web interactions with Selenium, or applying behavior-driven testing with Behave, Python provides the flexibility and powerful libraries to automate your testing efficiently.

Example 1: Writing a Simple Unit Test with Unittest

To write unit tests with Python’s built-in unittest framework, you need to create a class that inherits from unittest.TestCase. This class will contain methods that define each of your test cases. Here’s a simple example:

import unittest

class TestMathOperations(unittest.TestCase):

    def test_addition(self):

        self.assertEqual(2 + 3, 5)

    def test_subtraction(self):

        self.assertTrue(5 – 3 == 2)

if __name__ == “__main__”:

    unittest.main()

In the example above, we used assertEqual() to check that 2 + 3 equals 5, and assertTrue() to verify that 5 – 3 equals 2. Assertions are critical in automated testing because they check whether the expected output matches the actual result of the code being tested.

To run the tests, you simply execute the script. The unittest.main() method automatically runs all test methods within the class and provides a summary of the results. If any test fails, it will display a failure message, helping you debug the issue.

Example 2: Automating Web Testing with Selenium and Pytest

Selenium is widely used for automating web browser interactions. Let’s use pytest along with Selenium to automate a simple web test. First, install the required libraries:

pip install selenium pytest

Then, write the test case:

from selenium import webdriver

def test_open_google():

    driver = webdriver.Chrome()  # You can use other WebDriver like Firefox

    driver.get(“https://www.google.com”)

    assert “Google” in driver.title

    driver.quit()

Once the browser is open, you can interact with web elements. For example, if you want to search for something on Google, you can locate the search box by ID and send a search term:

def test_google_search():

    driver = webdriver.Chrome()

    driver.get(“https://www.google.com”)

    search_box = driver.find_element_by_name(“q”)

    search_box.send_keys(“Python automated testing”)

    search_box.submit()

    assert “Python automated testing” in driver.page_source

    driver.quit()

Selenium allows you to simulate user actions such as clicking buttons and filling out forms. In the previous example, we used send_keys() to type into the search box, and submit() to trigger the search.

After interacting with the webpage, you can use assertions like assertIn() or assertEqual() to verify that the expected results appear on the page, confirming that the web application is functioning as expected.

Example 3: Running Behavior-Driven Tests with Behave

Behavior-Driven Development (BDD) helps you write tests in plain language. With the Behave framework, you write your tests in feature files using Gherkin syntax. Here’s an example of a simple feature file (search.feature):

Feature: Google Search

  Scenario: Searching for Python automation

    Given I am on the Google search page

    When I search for “Python automated testing”

    Then the page title should contain “Python”

For each step in the feature file, you create a corresponding Python function. These functions define the actions that should be taken in the test. Here’s how you could implement the steps from the feature file:

from behave import given, when, then

from selenium import webdriver

@given(‘I am on the Google search page’)

def step_impl(context):

    context.driver = webdriver.Chrome()

    context.driver.get(“https://www.google.com”)

@when(‘I search for “{search_term}”‘)

def step_impl(context, search_term):

    search_box = context.driver.find_element_by_name(“q”)

    search_box.send_keys(search_term)

    search_box.submit()

@then(‘the page title should contain “{title}”‘)

def step_impl(context, title):

    assert title in context.driver.title

    context.driver.quit()

To run your BDD tests with Behave, simply execute the following command in your terminal:

behave

Behave will read your feature files, execute the steps in Python, and provide a report of the test outcomes. This approach allows you to write automated tests in a more user-friendly format, suitable for collaboration with non-developers.

Integrating Python Tests with CI/CD Pipelines

Integrating Python tests into a Continuous Integration/Continuous Deployment (CI/CD) pipeline is crucial for ensuring that your automated tests run frequently, without requiring manual intervention. This setup allows you to continuously test your code as it gets committed, ensuring that bugs are caught early in the development process.

Step 1: Setting Up a CI/CD Pipeline

The first step in integrating Python tests into a CI/CD pipeline is choosing the right CI/CD tool. Some popular options include:

1. Jenkins: A widely used open-source automation server that supports a wide range of plugins.
2. GitHub Actions: A CI/CD tool that comes integrated with GitHub repositories, making it a great choice for projects hosted on GitHub.
3. GitLab CI: A powerful CI/CD system built into GitLab, which offers easy configuration and integration for GitLab-hosted projects.
4. Once you’ve chosen a CI/CD tool, you’ll need to configure it to run your Python tests automatically each time code is committed.

Creating a YAML Configuration for Test Execution

Most CI/CD systems use YAML configuration files to define how tasks should be executed. For example, with GitHub Actions, you would create a .github/workflows/python.yml file to define the steps for setting up the Python environment and running tests. Here’s a sample configuration:

name: Python CI

on: [push]

jobs:

  test:

    runs-on: ubuntu-latest

    steps:

    – name: Check out the code

      uses: actions/checkout@v2

    – name: Set up Python

      uses: actions/setup-python@v2

      with:

        python-version: ‘3.x’

    – name: Install dependencies

      run: |

        pip install -r requirements.txt

    – name: Run tests

      run: |

        pytest

Step 2: Running Automated Tests in a CI/CD Pipeline

After setting up the CI/CD pipeline configuration, the next step is to make sure that your tests are triggered when the code is pushed to the repository. For example, the YAML configuration above sets up the pytest framework to run automatically when a push is made to the repository. You can configure pytest or unittest to scan for and run all tests in your repository.

1. pytest: If you’re using pytest, it will automatically discover all test functions (methods prefixed with test_ or located in files starting with test_) and execute them.
2. unittest: Similarly, unittest can also be used to automatically discover and run all test cases in your repository by specifying a test suite in the configuration.

Generating Test Reports and Logs

To keep track of test results, you can configure your pipeline to generate reports and logs. For pytest, you can use the –junitxml option to generate a test report in XML format:

pytest –junitxml=report.xml

Step 3: Handling Test Failures in CI/CD

When a test case fails in the CI/CD pipeline, it’s essential to be alerted immediately. Most CI/CD tools support notifications through various channels, including email, Slack, or even through GitHub/GitLab’s built-in notification systems. You can set up notifications to alert your team when a test fails, ensuring that issues are addressed promptly.

1. For GitHub Actions, you can integrate with Slack or email to send failure notifications using third-party services.
2. For Jenkins, you can configure the Email Extension Plugin to notify users if a build fails.

Debugging and Fixing Automation Test Failures

Test failures in a CI/CD pipeline can happen for various reasons, including code changes, environment mismatches, or test flakiness. When a test fails:

1. Review the logs and error messages to identify the root cause.
2. If it’s an issue with the test environment, adjust the configuration or dependencies.
3. If it’s a genuine bug, debug and fix the issue in the code.

Once the issue is fixed, re-run the tests to ensure everything works properly. With the power of CI/CD, you can automate the entire debugging and fixing process by regularly running tests, identifying problems, and keeping your code base stable.

Best Practices for Python Automated Testing

To ensure reliable and maintainable test automation, follow these best practices:

1. Use Descriptive Test Names: Choose meaningful and clear names for your test functions to indicate their purpose. This helps anyone reading the code to understand what the test is verifying.
2. Follow the Arrange-Act-Assert Pattern: Structure your tests in three distinct phases—arranging the test data, performing the action, and asserting the expected outcome. This improves test readability and organization.
3. Use Fixtures for Reusable Setup: To avoid duplicating code across multiple tests, use fixtures to set up common test dependencies. This keeps your tests clean and reduces maintenance efforts.
4. Avoid Hardcoding Data: Instead of hardcoding values directly in test cases, use configuration files or external data sources to supply dynamic test data. This makes tests more flexible and reusable.
5. Run Tests in Parallel: Speed up test execution by running tests in parallel. Tools like pytest-xdist allow you to execute multiple tests simultaneously, reducing the total test runtime.
6. Generate Test Reports: Enhance visibility into test results by generating detailed reports using tools like pytest-html or Allure Reports. These reports can provide insights into test successes, failures, and performance.

Common Challenges in Python Automated Testing and How to Overcome Them

While Python automated testing is powerful, it can come with its own set of challenges. Here are some common issues and practical solutions:

1. Flaky Tests

Challenge: Flaky tests are tests that sometimes pass and sometimes fail due to inconsistent results, often caused by dynamic elements or timing issues in the application. 

Solution: Use explicit waits to ensure that elements are fully loaded before interacting with them. In Selenium, you can use WebDriverWait along with expected conditions to wait for elements to become available or visible before proceeding with actions. This reduces the chances of tests failing due to timing issues.

Example:

from selenium.webdriver.common.by import By

from selenium.webdriver.support.ui import WebDriverWait

from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)

element = wait.until(EC.presence_of_element_located((By.ID, “searchBox”)))

2. Slow Test Execution

Challenge: As your test suite grows, the execution time may increase, leading to slower feedback and longer development cycles. 

Solution: Optimize test scripts by:

1. Running tests in parallel using pytest-xdist to execute multiple tests simultaneously.
2. Reducing unnecessary steps, such as redundant setups or waiting for static elements.
3. Using mocking techniques to simulate parts of your application, speeding up tests that depend on slow external resources.

Example (Running tests in parallel with pytest-xdist):

pytest -n 4  # Runs the tests in 4 parallel processes

3. Cross-Browser Compatibility

Challenge: Ensuring that your web application works across multiple browsers can be time-consuming if you test each browser separately. 

Solution: Use Selenium Grid to run tests on different browsers in parallel. Selenium Grid allows you to execute tests on multiple machines and browsers simultaneously, ensuring cross-browser compatibility without manually switching browsers.

Example (Setting up Selenium Grid with multiple browsers):

from selenium import webdriver

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

capabilities = DesiredCapabilities.CHROME

driver = webdriver.Remote(

    command_executor=’http://selenium-server:4444/wd/hub’,

    desired_capabilities=capabilities

)

4. Maintaining Test Scripts

Challenge: As the application evolves, test scripts can become outdated or harder to maintain due to frequent changes in the UI or business logic. 

Solution: Implement the Page Object Model (POM) design pattern, which helps in organizing test scripts by creating separate classes for different page elements and actions. This makes your tests more scalable and easier to maintain, as any changes in the UI will only require updates to the page object classes, not the test scripts.

Example (Page Object Model in Selenium):

class LoginPage:

    def __init__(self, driver):

        self.driver = driver

        self.username_field = driver.find_element_by_id(“username”)

        self.password_field = driver.find_element_by_id(“password”)

        self.login_button = driver.find_element_by_id(“loginButton”)

    def login(self, username, password):

        self.username_field.send_keys(username)

        self.password_field.send_keys(password)

        self.login_button.click()

Conclusion

Python is a powerful and versatile language for automated testing, thanks to its simplicity and the wide range of testing frameworks available. Whether you’re performing unit tests, UI testing with Selenium, or behavior-driven testing with Behave, Python’s ecosystem has everything you need to automate your tests efficiently.

QA Touch is an efficient test management platform that can help you streamline your testing processes. With our Pytest plugin, you can customize your testing environment based on your needs.

Sign up today. It is free till you upgrade.