In this tutorial, you will create a simple “Hello, World!” component which runs in Keboola Connection (KBC).
You need to have a computer with working Docker to develop the KBC component code. To be able to create new components, you also need to have an account in the Keboola Developer Portal, which manages the list of components available in KBC.
The Developer portal uses different credentials than KBC. Creating an account is free; it requires a working email address (to which a confirmation email will be sent) and a mobile phone for a mandatory two-factor authorization.
When you log in to the developer portal, you have to join a vendor — an organization of developers. Every KBC application has to have a vendor assigned. If you join an existing vendor, an administrator of that vendor has to approve your request. If you do not work for a company, create a vendor with your name (even a single developer has to be assigned to a vendor).
In order to create a new vendor, a Keboola Administrator has to approve your request, and you will receive a development project in KBC. Also, you need to provide us with a channel for receiving internal errors from your components. Basically anything supported by Papertrail notifications is available, though e-mail or a Slack channel is most commonly used.
When you are confirmed as a member of a vendor, you may proceed to creating your own component. The example component is written in Python language, but no knowledge of Python is required. Before you continue with this tutorial, make sure you:
To add a component, use the Create Component button on the main page, and fill in the component name and type:
Important: Do not use the words ‘extractor’, ‘writer’ or ‘application’ in the component name.
To choose the appropriate type:
extractor– brings data into KBC
writer– sends data out of KBC
application– does some transformation of the data, or does something completely different.
The above does not mean technically that for example an extractor can’t send data out of KBC, or that an application cannot bring new data into KBC. It is a matter of user perception, so use your judgement to select the correct type.
When you fill the form, you will obtain a Component ID (in the
vendor-id.component-name, for instance,
keboola-test.ex-docs-tutorial). Take note of the Component ID.
To be able to deploy the application to KBC, you will need Service credentials. For security reasons we strongly advice against using your own credentials in any deployment service. To create new deployment credentials, click the Create a service account button on the Service accounts page.
Fill in name (e.g.
ex_docs_tutorial_travis) and description (e.g.
Travis deployment credentials) and confirm:
Take note of the Username and Password.
Once you have the Component Id and service account Username and Password, you can user our Component generator tool to create a component skeleton for you in your favorite programming language.
Create an empty Github repository. The name of the repository is arbitrary, but using the component is probably a good idea to avoid confusion.
Checkout the repository on your local computer and execute the following from command line:
docker run -i -t --volume=/path/to/repository/:/code/ quay.io/keboola/component-generator
/path/to/repository/ with an absolute local path to your empty repository. Follow
the on-screen instructions:
When done, you will have an initialized repository with a “Hello, World!” application.
In the above example, I choose the
simple-python template which contains:
You will also obtain a path to your Travis CI configuration (in the above example https://travis-ci.org/keboola/ex-docs-tutorial).
When done exploring, push to the repository.
This will automatically trigger a build on the Travis service, you can view the build
progress by visiting the provided link. In fact, two builds will be triggered, one
master branch, and one for the
We are more interested in the latter, because that is going to trigger the deployment to KBC.
If the deployment passed without errors, the component will become available in KBC. You can verify that in the component details in the Developer Portal:
This means that the application deployment is fully automated. If you change the component source code, all you need to do is push the changes to the git repository and tag them with normal version tag.
Once the application is deployed, it becomes available in KBC. Note that it takes up to 5 minutes to propagate the changes to all KBC instances. Once propagated, you can configure the application by visiting the following URL:
You can then run the configuration without any settings.
And you should see the “Hello, World” message in the events:
You have just created your own KBC component. Although it does not do much, it show the easiest path to bring your own application logic to KBC. You can now continue with other parts of the tutorial:
Although you usually don’t need everything (e.g. you don’t need input mapping when building an extractor), we suggest you go through all of the above to gain a general overview of the available options. You can also read all the details in the respective parts of the documentation: