Inputs

Input data is rarely clean. pipe0 sanitizes every input object before processing: it fixes common format errors, regenerates invalid values where it can, and handles missing fields per record instead of failing the whole task.

Input sanitation

Enrichment input usually comes from CRMs, ATSs, or web forms, and carries the mistakes users typed into them.

Cleanup

The following request payload contains common errors but will be processed successfully.

{
  "pipes": [
    {
      "pipe_id": "company:identity@3"
    }
  ],
  "input": [
    {
      "id": 1,
      "name": "Susi Jui",
      "company_name": "Pipe0",
      "email": "mailto:Susi@pipe0.com", // becomes "susi@pipe0.com"
      "website_url": "wwww.pipe0.com/" // becomes "https://www.pipe0.com"
    }
  ]
}

What gets cleaned depends on the field's format in the field catalog:

1. URL fields (url, website_url, profile_url): add a missing https://, upgrade http:// to https://, fix the wwww. typo, remove trailing slashes
2. Email fields: strip mailto:, lowercase, remove characters that are invalid in an email address
3. Date fields: parse common date formats (MM/dd/yyyy, yyyyMMdd, and others) into ISO
4. Number fields: convert between int, float, and string on demand

Fields without a format, like company_domain (plain text) or custom fields pipe0 doesn't know, are trimmed and copied as-is.

Regeneration

When an input value fails its format check and a pipe in your request claims that field as an output field, pipe0 doesn't fail the field. It queues it and lets the pipe resolve it fresh.

{
  "pipes": [
    {
      "pipe_id": "person:workemail:waterfall@1"
    }
  ],
  "input": [
    {
      "id": 2,
      "name": "Tom Schmidt",
      "company_domain": "pipe0.com",
      "work_email": "tom@" // fails the email format check
    }
  ]
}

work_email is an output field of person:workemail:waterfall@1, so the pipe replaces the broken value:

{
    "id": 2,
    "name": "Tom Schmidt",
    "company_domain": "pipe0.com",
    "work_email": "tom@pipe0.com" // healed
}

Regeneration only applies to fields with a format. If an invalid value belongs to a field no pipe claims, the field is marked failed with reason InvalidValue instead.

Incomplete data

Input data is often incomplete. Failing the entire task because one input object cannot be processed would be impractical, so validation works per record.

Partially missing input fields

If at least one input object can be processed, pipeline validation passes.

Take the following request payload:

{
  "pipes": [
    {
      "pipe_id": "company:identity@3"
    }
  ],
  "input": [
    {
      "id": 1,
      "name": "Susi Jui",
      "company_name": "Pipe0"
    },
    { // CANNOT be processed by "company:identity@3"
      "id": 2
      // required `company_name` missing
    }
  ]
}

The pipe company:identity@3 requires the input field company_name, which is not present in record id=2. In this case:

• Pipeline validation passes. Validation checks each pipe's requirement against the union of field names across all input objects, so one record providing company_name is enough.
• Record id=1 is processed in full.
• Record id=2's output fields are marked skipped with reason RequirementUnmet.

No input object has the required input fields

Another example:

{
  "pipes": [
    {
      "pipe_id": "company:identity@3"
    }
  ],
  "input": [
    { // CANNOT be processed by "company:identity@3"
      "id": 1,
      "name": "Susi Jui"
    },
    { // CANNOT be processed by "company:identity@3"
      "id": 2,
      "name": "Tom Schmidt"
    }
  ]
}

No input object has the required field company_name. The request fails during request validation, before processing starts.

Never fail a task

If you don't want to handle failing tasks, there's an escape hatch: define the expected input fields and set them to null. Pipeline validation passes, the task never fails, and only individual fields fail instead.

Input expansion

When you enrich data with pipe0 you transform your input objects into output records. An input object may look like this:

{
    "id": 2,
    "name": "Tom Schmidt"
}

Some interactions require you to reprocess previously processed fields. For this, it is common to transform your output records back to input objects. By doing so, previous processing information is lost. This includes metadata like the result of a waterfall or UI widgets.

If you pass a plain value to the API, it is always marked as resolved_by: input.

Input expansion is the alternative: pass your inputs fully or partially expanded, as the field value shape of the response object.

{
  "id": 2,
  "name": {
      "value": "Tom Schmidt",
      "status": "completed",
      "type": "string",
      "format": "text",
      "reason": null,
      "claimed_by": null,
      "resolved_by": {
        "ref": "input",
        "config_hash": null,
        "input_hash": null,
        "environment": null
      }
  }
}

Expanded fields carry their resolved_by fingerprints (config_hash, input_hash). When you resubmit a previous response as input, pipe0 compares those fingerprints against the current request and reprocesses only the fields whose pipe config or inputs changed; completed fields with matching fingerprints are copied, not recomputed. That makes re-running a partially failed request cheap.

Expanding inputs gives you control but shifts the responsibility of providing valid input states to you.