Pipe Concepts

This page contains additional information about pipes.

Pipe types

Single provider pipes

Many pipes use a single provider for processing. Providers can be a third-party service like LeadMagic  or ZeroBounce . Native pipes use Pipe0 as a provider.

Waterfall pipes

Waterfall pipes use many providers in a cascading sequence. We try provider A. If the data is found, it is returned. If the data is not found we try provider B. If the data is not found again, we try provider C and so on.

A billing event occurs for the provider that returns a result. Providers that return no_result are not billed.

Waterfall enrichments can drastically increase data coverage. But keep in mind that trying many providers can slow down processing speed.

Choosing waterfall providers

Waterfall pipes have a config.providers option. The option is typically of the type Array<{ provider: <PROVIDER_NAME> }>. If no config is provided, the default config is applied. If you remove a provider from the array, the provider is removed from your processing sequence.

Providers are processed according to their position in the config.providers array. The provider with the index 0 is processed first. You are encouraged to change the order.

Field mode

The field mode determines the shape of the config object a pipe takes.

Field mode: static

Most pipes have static input and output fields. When field_mode is set to static, the pipe’s in- and outputs are known in advance.

You can point the input fields to different fields or rename the outputs, but the pipe’s API is clearly defined in advance.

Example:

When using the pipe people:workemail:waterfall@1, we use the input fields name and company_website_url to generate the output field work_email.

Rename static pipe outputs

All pipes with static outputs allow renaming of output fields. To rename output fields, use the config.output_fields[<FIELD_NAME>].alias configuration.

fetch('https://api.pipe0.com/v1/pipes/run', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json'
    },
    body: JSON.stringify({
        pipes: [
            {
                pipe_id: 'people:name:split@1',
                config: {
                    output_fields: {
                        first_name: {
                            alias: 'special_first_name' // <- RENAME FIELD
                        }
                    }
                }
            }
        ],
        input: [
            {
                id: '1',
                name: 'John Doe'
            }
        ]
    })
})

Disable static pipe outputs

All pipes with static outputs allow the enabling/disabling of output fields. To enable/disable output fields, use the config.output_fields[<FIELD_NAME>].enabled option.

fetch('https://api.pipe0.com/v1/pipes/run', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json'
    },
    body: JSON.stringify({
        pipes: [
            {
                pipe_id: 'people:name:split@1',
                config: {
                    output_fields: {
                        first_name: {
                            enabled: false // <- THIS OPTION
                        }
                    }
                }
            }
        ],
        input: [
            {
                id: '1',
                name: 'John Doe'
            }
        ]
    })
})

Field mode: config

The inputs and outputs of this pipe are not known beforehand. This is the case when the user can define input and output fields as part of the pipe configuration.

Templates

A common pattern that uses user-defined in- and outputs is templates. With templates you spcify in- and outputs as text. This is useful when working with large language models (LLM).

Not all teamplates support the definition of output tags.

Example: The template for the prompt:run@1 pipe requires you to specify in- and output fields as template tags. In contrast, the email:write@1 pipe only supports the definition of input fields via template tags.

Template example

Templates have the following structure:

{
  "prompt": {
    "template": "Determine if company {{ input company_name type=\"string\" }} is a good customer for my magic wand 🪄 business. {{ output reason type=\"json\" schema=\"reason\" description=\"Explain why the customer is a good prospect\" }} {{ output is_good_customer type=\"boolean\" format=\"int\" description=\"a flag that summarized the overall assessment of the customers fit\" }}",
    "json_schemas": {
      "reason": {
        "type": "object",
        "properties": {
          "reason": {
            "type": "string",
            "description": "A summary of why the customer is or isn't a good prospect for my business."
          },
        }
      }
    }
  }
}

Template tags

Input field tags support the following properties:

Output field tags support the following properties:

Triggers

Triggers allow you to run pipes conditionally. You can specify conditions like:

Run pipe people:name:split@1 if the field name is completed and the resolved value contains the value Tom.

The request for the example above would look like this.

{
	"pipes": [
		{
			"pipe_id": "people:name:split@1",
			"trigger": {
				"action": "run",
				"when": {
					"logic": "and",
					"conditions": [
						{
							"field_name": "name",
							"property": "value",
							"operator": "contains",
							"value": "Tom"
						}
					]
				}
			}
		}
	],
  "input": [...]
}

Trigger Structure

action

The action that will be performed if all conditions are evaluated to true.

Supported values: run.

Triggers don’t bypass other mechanisms of pipe evaluation. Example: If your trigger conditions are met, but required input fields are missing pipe execution will still be skipped.

when.logic

A logic operator that connects all conditions.
Supported values: and, or.

when.conditions[]

A list of conditions. If all conditions evaluate to true, the action will be performed.

condition.field_name

A field in your pipeline that will be used for evaluation. This field can be provided by your input values or another pipe as an output_field.

Conditions targeting JSON fields are not supported. To act conditionally on JSON properties, expand them into their own fields.

condition.property

The property of the target field (field_name) that will be evaluated by the operator and value.

Supported values: value, status.

condition.operator

The operator used for comparison. Allowed operators are determined by property and the field.type.

Operators by type

Supported values: value, status.

condition.value

The value to compare against.