Skip to content

Testing

Pipenv

pip install -r requirements.txt
pip install -r requirements-dev.txt

Invoke

To run and develop Policy Sentry without having to install from PyPi, you can use Invoke.

# List available tasks
invoke -l

# that will show the following options:
Available tasks:

  build.build-package          Build the policy_sentry package from the current
                               directory contents for use with PyPi
  build.install-package        Install the policy_sentry package built from the
                               current directory contents (not PyPi)
  build.uninstall-package      Uninstall the policy_sentry package
  build.upload-prod            Upload the package to the PyPi production server
                               (requires credentials)
  build.upload-test            Upload the package to the TestPyPi server
                               (requires credentials)
  docs.make-html               Make the HTML docs locally
  docs.open-html-docs          Open HTML docs in Google Chrome locally on your
                               computer
  docs.remove-html-files       Remove the html files
  integration.analyze-policy   Integration testing: Tests the `analyze`
                               functionality
  integration.clean            Runs `rm -rf $HOME/.policy_sentry`
  integration.initialize       Integration testing: Initialize the
                               policy_sentry database
  integration.query            Integration testing: Tests the `query`
                               functionality (querying the IAM database)
  integration.query-yaml       Integration testing: Tests the `query`
                               functionality (querying the IAM database) - but
                               with yaml
  integration.version          Print the version
  integration.write-policy     Integration testing: Tests the `write-policy`
                               function.
  test.lint                    Linting with `pylint` and `autopep8`
  test.security                Runs `bandit` and `safety check`
  unit.pytest                  Unit testing: Runs unit tests using `pytest`


# To run them, specify `invoke` plus the options:
invoke build.build-package

invoke integration.clean
invoke integration.initialize
invoke integration.analyze-policy
invoke integration.query
invoke integration.write-policy

invoke test.security

Local Unit Testing and Integration Testing:

Strategy to write new unit tests

See the writeup here: https://github.com/salesforce/policy_sentry/pull/254#issuecomment-710098269

Quick and Easy way to run tests

Just run this from the root of the repository:

We highly suggest that you run all the tests before pushing a significant commit. It would be painful to copy/paste all of those lines above - so we've compiled a test script in the utils folder.

./utils/run_tests.sh

It will execute all of the tests that would normally be run during the build. If you want to see if it will pass GitHub actions, you can just run that quick command on your machine.

Running the Test Suite

We use pytest for unit testing. All tests are placed in the test folder.

  • Just run the following:
pytest -v

# This will output the print() statements in your test code
pytest -v --show-capture=no

# This will include the debug logging statements in the test output
pytest -v --log-level=DEBUG
  • Alternatively, you can use invoke, as mentioned above:
invoke unit.pytest

Output:

test/analysis/test_analyze.py::AnalysisExpandWildcardActionsTestCase::test_a_determine_actions_to_expand_not_upper_camelcase PASSED  [  0%]
test/analysis/test_analyze.py::AnalysisExpandWildcardActionsTestCase::test_analyze_by_access_level PASSED                            [  1%]
test/analysis/test_analyze.py::AnalysisExpandWildcardActionsTestCase::test_analyze_statement_by_access_level PASSED                  [  2%]
test/analysis/test_analyze.py::AnalysisExpandWildcardActionsTestCase::test_determine_actions_to_expand PASSED                        [  2%]
test/analysis/test_analyze.py::AnalysisExpandWildcardActionsTestCase::test_gh_162 PASSED                                             [  3%]
test/analysis/test_expand.py::PolicyExpansionTestCase::test_policy_expansion PASSED                                                  [  4%]
...

========================================================= 134 passed in 51.04s ============================================================