Skip to main content

Pipelines

A pipeline denotes flow of data from source to destination. It can have multiple transformation or decision blocks in between. Pipelines can be used to move data from one system to another, or to enrich data with additional information, or to remove sensitive information from the event payload. Pipelines can be scheduled to run at a specific time or can be triggered to run for individual events.

How to create a new Pipeline​

To create a Pipeline:

  1. Login to your cnips instance
  2. Click on Pipelines in the left navbar
  3. Click New Pipeline.

Pipeline Creation

  1. Provide a name for the Pipeline.
  2. Select a source and destination.
  3. Click on sumbit.

How to work with the pipeline editor​

  1. Click on Editor after creating a pipeline.

Pipeline Editor

  1. Click on the plus symbol associated with the Transformations/Decision Blocks/Destinations as per your flow requirement to add them to your editor.
  2. Make sure to connect the new nodes/blocks into your pipeline.
  3. Click on submit.

Things to keep in mind while configuring the flow:

  1. Except for source, every other type of block can have multiple inputs.
  2. Transformations only have one output.
  3. Decision Blocks have two outputs, one for true and one for false, these can be configured to a single Transformation/Destination/Decision block individually.
  4. All flows must end at a Destination block.