In normal testing scenarios, we would run the input through our system and see if the actual result matches what we expected it to be. We would then mark our test as Passed or Failed. When we have a clear idea of the expected return, this method works well, like checking a login response or calculating a checkout total.

What if we don’t have an easy way to determine what the correct result should be?

For example, a search engine within an e-commerce application, a machine learning model, or a route optimisation tool. We have put in a search query and received hundreds of results, or asked a question to an AI model and received a response. How do we determine if the result is correct?

This is where Metamorphic Testing can help us. We can check whether there is a relationship between the outputs of correlated inputs that we know should be there.

What is Metamorphic Testing?

Metamorphic Testing is a form of testing which can be used when it is hard to know what the correct output of the program should be for a certain input. This issue is also known as the Test Oracle Problem.

The idea is that whilst we may not know the exact answer to the input, we do know how the answer should behave.

These relations are known as Metamorphic Relations (MRs).

We may not know what sin(37.4°) will give us without doing any calculations; however, we do know that sin(x) should equal sin(180° − x). If they don’t, then something may be wrong.

This can also be applied to software. Whilst we may not know how many products a search should return, we can check that adding an extra filter removes products which do not fit that filter.

Why Metamorphic Testing is Useful

Metamorphic Testing is useful for systems where determining the expected output is difficult. Search engines, AI models, data analytics and simulations are examples of where it could be useful, as it would be difficult to manually check each output.

Less reliance on hard-coded expected answers: Instead of having an expected answer for each input, we can automate our testing to check that relations stay consistent.

Helps to find issues not found with basic testing: It can help to identify issues with filtering, calculations, data processing, etc., where each output may look fine individually.

A Practical Example: E-commerce Product Search

Imagine you are testing the product search on an online store. You search for “running shoes” and the application returns 1,248 products.

Is 1,248 the correct number? Without an independently established expected result, it may be difficult to say.

However, we know several things about how the search should behave when we change the query, apply filters, or sort the results. These expectations can become metamorphic relations.

These checks do not require us to know the exact correct number of products in advance. Instead, they help us to verify whether the search behaves consistently.

A failed relation can point us towards a defect. For example, if applying a Nike filter retrieves products from other brands, the filter logic may be incorrect. If splitting a price range causes a product to disappear or appear twice, there may be a boundary-condition issue.

One important point: these relationships must match the application’s requirements. Case-insensitive search, for example, should only be expected if the product is designed to support it.

Automating Metamorphic Tests with Python and pytest

Metamorphic relations can also be checked through automated tests. The following examples assume that search() is an API client that returns product records and that the relevant search behaviours are part of the product requirements.

import pytest
from store_api import search


def ids(results):
    return {p["id"] for p in results}


# MR1: Applying a filter should return a subset
def test_filter_narrows_results():
    base = search(q="running shoes")
    filtered = search(q="running shoes", brand="Nike")
    assert ids(filtered) <= ids(base)


# MR2: Filter order should not affect the result set
def test_filter_order_independent():
    a = search(
        q="running shoes",
        filters=[("brand", "Nike"), ("size", "9")]
    )
    b = search(
        q="running shoes",
        filters=[("size", "9"), ("brand", "Nike")]
    )
    assert ids(a) == ids(b)


# MR3: Search should ignore case
@pytest.mark.parametrize(
    "variant",
    ["RUNNING SHOES", "Running Shoes"]
)
def test_query_case(variant):
    assert ids(search(q=variant)) == ids(
        search(q="running shoes")
    )


# MR4: Sorting should preserve product membership
def test_sort_preserves_content():
    relevance = search(q="running shoes", sort="relevance")
    ascending = search(q="running shoes", sort="price_asc")
    assert ids(relevance) == ids(ascending)
    prices = [p["price"] for p in ascending]
    assert prices == sorted(prices)

These tests focus on expected relationships instead of fixed product IDs or a hard-coded result count. They can be run against different search queries and test data to help uncover defects.

For reliable results, the test environment should use a stable dataset or control for changes to the product catalogue. Otherwise, products being added or removed between requests could affect the comparison.

Where Else Can We Use Metamorphic Testing?

The technique is useful in several areas, although the relationship being checked must reflect the actual requirements of the system.

When Should Testers Consider Using It?

Metamorphic testing also works well with automation because the same relationship can be checked across many inputs, including randomly generated test data.

However, there is one limitation to remember: metamorphic testing can only identify violations of the relationships we define. A test may pass even when the output is wrong if the chosen relationship is too weak or does not reflect the actual business requirement. That is why testers should work with developers and product owners to identify the relationships before automating them.

Traditional testing asks, “Does this output match the expected result?”

Metamorphic testing adds another useful question: “When the input changes, does the output behave as expected?”

For testers, that is another way to find defects even when the exact correct answer is not readily available.

Document your metamorphic relations as test cases and track every run in one place with QA Touch. Get started for free.