1. Home
  2. CLI
  3. Templates
  4. Tests

Template Tests

How the Tests Work

Each test can be run locally, remotely, or both ways. See the CLI command for more information.

Local running of the test prepares a directory with a fake empty project and applies the template to it using the inputs.json file for the template inputs. Then the state of the directory is compared to the expected-out directory of the test.

Remote running of the test again applies the template to a project using the inputs.json file for the template inputs like the local one but uses a real project in a public Keboola stack that you have to provide (see the CLI command for details). Then it performs a push to the project and runs the main config of the template. It waits for the result of the job and evaluates if it ended successfully or not. Any other error during the whole process is considered as the test fail and the command exits with an error.

A template directory contains the directory tests that should gather one or more subdirectories with tests for the template. The directory for each test has to contain the directory expected-out and the file inputs.json.

You can have a test generated by a CLI command.

Expected Out

The expected-out directory looks like a regular project directory after applying the template. Some values in the project manifest cannot be reused freely and are dependent on the project (project id, host, and main branch id) or the template instance (instance id). For these values the template test command supports placeholders in the directory:

  • __PROJECT_ID__ is replaced by a current project id during the test run.
  • __STORAGE_API_HOST__ is replaced by a current Keboola stack host of the project during the test run.
  • __MAIN_BRANCH_ID__ is replaced by a current main branch id of the project during the test run.

Then you can use wildcards for comparison of dynamic values:

  • %e: Represents a directory separator, for example, / on Linux.
  • %s: One or more of anything (character or white space) except the end-of-line character.
  • %S: Zero or more of anything (character or white space) except the end-of-line character.
  • %a: One or more of anything (character or white space) including the end-of-line character.
  • %A: Zero or more of anything (character or white space) including the end-of-line character.
  • %w: Zero or more white space characters.
  • %i: A signed integer value, for example, +3142, -3142.
  • %d: An unsigned integer value, for example, 123456.
  • %x: One or more hexadecimal character. That is, characters in the range 0-9, a-f, and A-F.
  • %f: A floating point number, for example, 3.142, -3.142, 3.142E-10, 3.142e+10.
  • %c: A single character of any sort.
  • %%: A literal percent character: %.

Example of .keboola/manifest.json:

{
  "version": 2,
  "project": {
    "id": __PROJECT_ID__,
    "apiHost": "__STORAGE_API_HOST__"
  },
  "sortBy": "id",
  "naming": {},
  "allowedBranches": ["*"],
  "ignoredComponents": [],
  "templates": {
    "repositories": [
      {
        "type": "dir",
        "name": "keboola",
        "url": "../repository",
        "ref": "main"
      }
    ]
  },
  "branches": [
    {
      "id": __MAIN_BRANCH_ID__,
      "path": "main",
      "metadata": {
        "KBC.KAC.templates.instances": "%s"
      }
    }
  ],
  "configurations": [
    {
      "branchId": __MAIN_BRANCH_ID__,
      "componentId": "ex-generic-v2",
      "id": "%s",
      "path": "extractor/ex-generic-v2/empty",
      "metadata": {
        "KBC.KAC.templates.configId": "{\"idInTemplate\":\"empty\"}",
        "KBC.KAC.templates.configInputs": "[{\"input\":\"ex-generic-v2-oauth\",\"key\":\"authorization.oauth_api\"},{\"input\":\"ex-generic-v2-api-base-url\",\"key\":\"parameters.api.baseUrl\"}]",
        "KBC.KAC.templates.instanceId": "%s",
        "KBC.KAC.templates.repository": "keboola",
        "KBC.KAC.templates.templateId": "my-template"
      },
      "rows": []
    }
  ]
}

Inputs.json

The inputs.json file has to contain values for inputs that the template expects during its application to the project. It is a one-level key-value store:

{
  "input1": "value 1",
  "input2": 2
}

The file supports complex values and can even refer to an OAuth configuration:

{
  "oauthInput": {
    "id": "123",
    "version": 3
  }
}

Environment Placeholders

Environment variables can be referred to using a placeholder encapsulated by ##.

This snippet means that the configuration incrementalDays will get its value from the environmental variable INC_DAYS:

{
  "incrementalDays": ##INC_DAYS##
}

See GitHub’s documentation on how to define environment variables in the Actions workflow.

Sensitive Values

Sensitive values can be stored in GitHub Secrets and will be passed to the test environmental variables. They need to be prefixed by KBC_SECRET_.

This snippet means that the input inputToken will get its value from the GitHub secret KBC_SECRET_MY_TEMPLATE_TOKEN:

{
  "inputToken": "##KBC_SECRET_MY_TEMPLATE_TOKEN##"
}

See GitHub’s documentation on how to define GitHub secrets.