Skip and xfail

You can mark test functions that cannot be run on certain platforms or that you expect to fail so pytest can deal with them accordingly and present a summary of the test session, while keeping the test suite green.

Skip: A skip means that you expect your test to pass only if some conditions are met, otherwise pytest should skip running the test altogether.

Ex: skipping windows-only tests on non-windows platforms.

xFail: A xfail means that you expect a test to fail for some reason.

Ex: test for a feature not yet implemented. 

Skipping test functions

The simplest way to skip a test function is to mark it with the skip decorator .

@pytest.mark.skip 

Create a test -test_addition.py

import pytest
@pytest.mark.skip
def test_add():
    assert (10+5 ==15)

#@pytest.mark.even
def test_add_even():
    assert(10+5==15)

#@pytest.mark.odd
def test_add_odd():
    assert(5+7==13)

execute using below command

pytest test_addition.py -v

while executing test_add will skip and other two tests(test_add_even and

test_add_odd) will execute.

Output will be 

You can pass option “reason” to the skip function. 

@pytest.mark.skip(reason="not yet implemented")

Below is the script

import pytest

@pytest.mark.skip(reason="method is not yet implemented ")
def test_add():
    assert (10+5 ==15)

#@pytest.mark.even
def test_add_even():
    assert(10+5==15)

#@pytest.mark.odd
def test_add_odd():
    assert(5+7==13)

skipif

If you wish to skip something conditionally then you can use skipif instead.

Ex:marking a test function to be skipped when a> b.

Create a test file called  test_addition.py

import pytest
a=7
b=5
@pytest.mark.skipif(a>b,reason="This method will be skipped when A is greater than B")
def test_add_even():
    assert(a+b < 15)

#@pytest.mark.odd
def test_add_odd():
    assert(5+7==13)

output will be

Test method will be skipped when a is greater that b else it will execute.

Skip all test functions of a class or module

You can use the skipif marker on classes: 

Create a test  called test_classlevel.py

import pytest

@pytest.mark.skip(reason="module not yet ready")
class Testskiplevel:
    def test_skipmodulelevel(self):
        print("will skip")

output will be

XFail: mark test functions as expected to fail 

you can use the xfail marker to indicate that you expect a test to fail:

We can xfail tests using the following marker

@pytest.mark.xfail

Create a test called test_subtraction.py

import  pytest

@pytest.mark.xfail
def test_subtract():
    assert (10-5==6)

def test_subtract_even():
    assert(10-5==5)

This test will be run but no traceback will be reported when it fails.

Output will be

Detailed information about skipped/xfailed tests is not shown by default to avoid cluttering the output. You can use the -r option to see details corresponding to the “short” letters shown in the test progress:

pytest -rx

Example for xFail

import  pytest

@pytest.mark.xfail(reason="issue , bug no - 300")
def test_subtract():
    assert (10-5==6)

#@pytest.mark.even
def test_subtract_even():
    assert(10-5==5)

for executing using below command

pytest -rx test_subtraction.py -v 

Example for skip 

import pytest
@pytest.mark.skip(reason="method is not yet implemented ")
def test_add():
   assert (10+5 ==15)

a,b=7,5
@pytest.mark.skipif(a>b,reason="This method will be skipped when A is greater than B")
def test_add_even():
    assert(a+b < 15)

for executing using below command

pytest -rs test_subtraction.py -v

output will be

Stopping after the first (or N) failures

To stop the testing process after the first (N) failures. The syntax to stop the execution of test suite soon after n number of test fails is as follows

pytest --maxfail = <num>

Create a test file called test_all_failed.py and below is the code.

import pytest

def test_method1():
    assert (2+3==6)

def test_method2():
    assert (3-1==4)

def test_method3():
    assert (4*2==8)

def test_method4():
    assert (4//2==2)

using below commands for executing the test

1.   pytest test_all_failed.py -x

Stop the execution after the first failure.

2.   pytest test_all_failed.py –maxfail=2 -v 

Stop the execution after the two failures.

Detailed summary report 

The -r flag can be used to display a “short test summary info” at the end of the test session, making it easy in large test suites to get a clear picture of all failures, skips, xfails, etc.

Create a test called test_all_failed.py

import pytest

def test_method1():
    assert (2+3==6)

def test_method2():
    assert (3-1==4)

def test_method3():
    assert (4*2==8)

def test_method4():
    assert (4//2==2)

The -r options accepts a number of characters after it.Here is the full list of available characters that can be used:

f - failed

E - error

s - skipped

x - xfailed

X - xpassed

p - passed

P - passed with output

a - all except pP

A - all

Using below command for giving short test summary info only for failed test.

pytest -rf test_all_failed.py -v

Like if you want to display only passed short test summary info, you will use below command

pytest -rp test_all_failed.py -v

More than one character can be used, so for example to only see failed and passed tests, you can execute: 

pytest -rfp test_all_failed.py -v