Testing¶
Additional help is needed to develop a proper testing framework of the charts.
Syntax Linting¶
This project is pre-commit enabled, please make sure to run pre-commit tests before committing to the repo.
Linting Templates¶
It is possible to do some basic linting with an existing set of linter_values*.yaml
files. These files have the following purposes to simulate various deployment methods:
linter_values_minimum.yaml
- This file is intended to test the bare-minimum default values.linter_values.yaml
- This file is intended to test as much as possible of the templates, if it can be added/deployed it should be in this file.linter_values_mysql.yaml
- This file is intended to test the templates when deploying with mysql.linter_values_postgresql_ha.yaml
- This file is intended to deploy a redis sentinel cluster with postgresql HA.
These files are intended to be layered on top of each other for additional functionality. In other words to template MySQL values you could run helm template nautobot charts/nautobot -f linter_values_minimum.yaml -f linter_values_mysql.yaml
.
We really should run functional testing with these templates but they require various levels of system resources which makes this impractical on a developer laptop.
Functional Testing¶
Deploy it in minikube and test... yes this needs some work.
Security Testing¶
We use kubescape in the CI pipeline to test for various security best practice patterns in the helm deployment against the above mentioned linter values files. As well as Snyk. You can test kubescape
locally with:
helm template -n testing -f charts/nautobot/linter_values_minimum.yaml -f charts/nautobot/linter_values.yaml charts/nautobot | kubescape scan framework nsa - --fail-threshold 0 --exceptions ./kubescape-exceptions.json
We test with 3 frameworks: nsa
, mitre
, and armobest
We are also working towards implementing checkov (GitHub) tests, these can be run locally with:
checkov --directory charts/nautobot --skip-path "charts/nautobot/linter_values.*.yaml" --var-file charts/nautobot/linter_values_minimum.yaml --var-file charts/nautobot/linter_values.yaml --framework helm
Per-Deployment Testing¶
Helm provides the capability of running tests in each deployment. This is something for us to explorer in much more detail in the future. Today we have a fairly simple curl to ensure the web service is up.