Configuration

In this part of the tutorial, you will see how to pass arbitrary configuration parameters to your component. By this time, you probably noticed that your component has a configuration field:

Screenshot -- Configuration Empty

You can use this field to pass arbitrary configuration parameters to your component. The parameters will be available in the /data/config.json file provided to the component when it is run.

Let’s assume you want to make the sample component add a given sound to each row a given number of times. For that you’ll need two parameters: sound and repeat.

Modifying Source Code

To implement the above, you can change the sample component to:

import csv
# Load the KBC library to process the config file
from keboola import docker
cfg = docker.Config('/data/')
params = cfg.get_parameters()

print("Hello world from python")

csvlt = '\n'
csvdel = ','
csvquo = '"'
with open('/data/in/tables/source.csv', mode='rt', encoding='utf-8') as in_file, \
        open('/data/out/tables/odd.csv', mode='wt', encoding='utf-8') as odd_file, \
        open('/data/out/tables/even.csv', mode='wt', encoding='utf-8') as even_file:
    lazy_lines = (line.replace('\0', '') for line in in_file)
    reader = csv.DictReader(lazy_lines, lineterminator=csvlt, delimiter=csvdel,
                            quotechar=csvquo)

    odd_writer = csv.DictWriter(odd_file, fieldnames=reader.fieldnames,
                                lineterminator=csvlt, delimiter=csvdel,
                                quotechar=csvquo)
    odd_writer.writeheader()

    even_writer = csv.DictWriter(even_file, fieldnames=reader.fieldnames,
                                 lineterminator=csvlt, delimiter=csvdel,
                                 quotechar=csvquo)
    even_writer.writeheader()
    i = 0
    for row in reader:
        if i % 2 == 0:
            even_writer.writerow(row)
        else:
            newRow = {}
            for key in reader.fieldnames:
                newRow[key] = row[key] + ''.join([params['sound']] * params['repeat'])
            odd_writer.writerow(newRow)
        i = i + 1

At the beginning, the KBC Docker library is imported and initialized by reading the data directory (docker.Config('/data/')). Its method get_parameters will provide the configuration parameters as a dictionary. The library is currently available for the R language and Python. It does no magic or rocket science, so you can read the config file directly if you wish.

Commit and push the code in your repository and tag it with a normal version tag. This will trigger a build on Travis CI and automatically deploy the new version into KBC. Keep in mind that after the deployment, it may take up to 5 minutes for the update to propagate to all KBC instances.

Verifying

To verify that the parameters work, simply edit the component configuration in KBC and paste in, for example:

{
    "sound": "Moo",
    "repeat": 2
}

Screenshot -- Configuration Filled

Run the component and examine the job results. In the odd result table, you should see that Moo was added twice to every value.

Screenshot -- Table Results

Creating UI

Entering configuration parameters using JSON data is quite low-level. Therefore you should provide a UI for the end user. The easiest option is to take advantage of the JSON editor based on the configuration schema. For the above configuration, the following schema can be created:

{
    "title": "Person",
    "type": "object",
    "properties": {
        "sound": {
            "type": "string",
            "title": "Sound:",
            "default": "Boo",
            "description": "The sound to make."
        },
        "repeat": {
            "type": "integer",
            "title": "Repeat sound:",
            "description": "Number of times to repeat the sound.",
            "default": 2,
            "minimum": 0,
            "maximum": 10
        }
    },
    "required": ["sound", "repeat"]
}

In the schema the two properties sound and repeat are declared along with the specification of their form input fields. You can test the above schema online and verify that the form generated from it produces the desired JSON structure. Once satisfied with the result, simply paste the schema into the Configuration schema in your component properties in the Developer Portal.

Once the change propagates to your KBC instance, you should see the form in the UI:

Screenshot -- Configuration Form

The end user can now configure your component without writing the JSON with parameters. For a very complex UI, the JSON schema editor is not really suitable; contact us about available options.

Summary

Your component can now successfully read configuration parameters provided by the end user. You can read more about all the features of the configuration file. Keep in mind that the code presented above is simplified as it does not use any validation of end user parameters. The next part of the tutorial will show you how to configure processors.