Testing Framework

As napalm consists of multiple drivers and all of them have to provide similar functionality, we have developed a testing framework to provide a consistent test suite for all the drivers.


The testing framework has the following features:

  1. Same tests across all vendors. Tests defined in napalm_base/test/getters.py are shared across all drivers.
  2. Multiple test cases per test.
  3. Target of the test can be configured with enviromental variables.
  4. Expected output is compared against the actual output of the test result.
  5. NotImplemented methods are skipped automatically.

Using the testing framework

To use the testing framework you have to implement two files in addition to the mocked data:

  • test/unit/test_getters.py - Generic file with the same content as this file test_getters.py
  • test/unit/conftest.py - Code specific to each driver with instructions on how to fake the driver. For example, conftest.py

Multiple test cases

To create test cases for your driver you have to create a folder named test/unit/mocked_data/$name_of_test_function/$name_of_test_case. For example:

  • test/unit/mocked_data/test_get_bgp_neighbors/no_peers/
  • test/unit/mocked_data/test_get_bgp_neighbors/lots_of_peers/

Each folder will have to contain it’s own mocked data and expected result.


By default, the tests are going to be run agains mocked data but you can change that behavior with the following enviromental variables:

  • NAPALM_TEST_MOCK - 1 (default) for mocked data and 0 for connecting to a device.

Mocking the open method

To mock data needed to connect to the device, ie, needed by the open method, just put the data in the folder test/unit/mocked_data/


Multiple test cases:

(napalm) ➜  napalm-eos git:(test_framework) ✗ ls test/unit/mocked_data/test_get_bgp_neighbors
lots_of_peers no_peers      normal
(napalm) ➜  napalm-eos git:(test_framework) ✗ py.test test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[lots_of_peers] <- ../napalm-base/napalm_base/test/getters.py PASSED
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[no_peers] <- ../napalm-base/napalm_base/test/getters.py PASSED
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[normal] <- ../napalm-base/napalm_base/test/getters.py PASSED

Missing test cases:

(napalm) ➜  napalm-eos git:(test_framework) ✗ ls test/unit/mocked_data/test_get_bgp_neighbors
ls: test/unit/mocked_data/test_get_bgp_neighbors: No such file or directory
(napalm) ➜  napalm-eos git:(test_framework) ✗ py.test test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors
test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[no_test_case_found] <- ../napalm-base/napalm_base/test/getters.py FAILED

========================================================= FAILURES ==========================================================
___________________________________ TestGetter.test_get_bgp_neighbors[no_test_case_found] ___________________________________

cls = <test_getters.TestGetter instance at 0x10ed5eb90>, test_case = 'no_test_case_found'

    def wrapper(cls, test_case):
        cls.device.device.current_test = func.__name__
        cls.device.device.current_test_case = test_case

            # This is an ugly, ugly, ugly hack because some python objects don't load
            # as expected. For example, dicts where integers are strings
            result = json.loads(json.dumps(func(cls)))
        except IOError:
            if test_case == "no_test_case_found":
>               pytest.fail("No test case for '{}' found".format(func.__name__))
E               Failed: No test case for 'test_get_bgp_neighbors' found

../napalm-base/napalm_base/test/getters.py:64: Failed
================================================= 1 failed in 0.12 seconds ==================================================

Method not implemented:

(napalm) ➜  napalm-eos git:(test_framework) ✗ py.test test/unit/test_getters.py::TestGetter::test_get_probes_config
test/unit/test_getters.py::TestGetter::test_get_probes_config[no_test_case_found] <- ../napalm-base/napalm_base/test/getters.py SKIPPED

================================================= 1 skipped in 0.09 seconds =================================================