Input Mapping

In this part of the tutorial, you will see how to use input mapping. Input mapping defines what data your component receives from the end-users’ project. A component never operates on data in project Storage, it always receives a copy of the selected data.

Input mapping is therefore used when your application needs to read data from the customer project – these are typically writers and applications. Implementing the input mapping requires three steps:

  • adding input mapping to the component UI
  • adding input mapping to the component source code
  • verifying

Adding Input Mapping to UI

Adding the input mapping to the component UI is very simple. In the Developer Portal, edit the component and add genericDockerUI-tableInput or genericDockerUI-fileInput flags (or both). You can read more about all available UI options.

Screenshot -- Input Mapping Configuration

Save the changes, and the corresponding UI elements will appear in your component configuration. Keep in mind that the changes take up to 5 minutes to propagate to all KBC instances.

Modifying the source code

You also need to modify the source code so that it works with the input data. The data from input mapping will be available in the /data/in/tables/ and /data/in/files folders when the component is run. This is almost identical to writing Transformation code – the only difference is that you need to use absolute paths (or KBC_DATADIR environment variable).

Therefore you can modify the example component code to, for example, the following one, which works with table input mapping.

import csv

print("Hello, World! from python")

csvlt = '\n'
csvdel = ','
csvquo = '"'
with open('/data/in/tables/source.csv', mode='rt', encoding='utf-8') as in_file:
    lazy_lines = (line.replace('\0', '') for line in in_file)
    reader = csv.DictReader(lazy_lines, lineterminator=csvlt, delimiter=csvdel, quotechar=csvquo)
    for row in reader:
        # do something
        print("The first row is ", row)
        # we don't want to print the entire file to the output
        exit()

Commit and push the code in your repository and tag it with normal version tag. This will trigger a build on Travis CI and automatically deploy the new version into KBC. Keep in mind that after the deploy it may take up to 5 minutes for the update to propagate to all KBC instances.

Verifying

If you configured the UI correctly, you should see the corresponding control in the component configuration page.

Screenshot -- Component Configuration

Add a New Table Input, select an arbitrary table from the project and make sure to set Destination to source.csv so that the final path matches /data/in/tables/source.csv which is what you are expecting in the code.

Screenshot -- Input Mapping Detail

Now run the component and you should see the proper message in job events.

Screenshot -- Input Mapping Events

Summary

Your component can now successfully read a provided table. You can read more about other input mapping options in the data folder specification. Also, the manifest files contain metadata which might be useful in your component. The next part of the tutorial will show you how to create output mapping.