Pagination and output

A source API collects records within its validated plan and requested limits. A bounded run is not a promise to enumerate an entire website.

Set the collection scope#

Start with max_pages: 1 and detail_limit: 0, then inspect the records and summary. Increase limits only within the generated schema and the scope you intend to pay for. Different sources may use links, buttons, scrolling, or an underlying API; the accepted definition determines what can be collected.

max_pages is a collection budget, not a universal page-number or cursor parameter. Do not add page, offset, or cursor inputs unless your generated contract explicitly supports them. Repeating the same run is a new collection attempt, not an instruction to fetch the next page.

Interpret completeness#

Check summary.within_scope_passed alongside summary.status. A result may be completed within its selected scope, bounded by the chosen limits, or partial because some expected work did not complete.

A successful bounded run can still omit later pages and unselected detail records. Review summary.limits, details_selected, details_complete, and details_not_selected where present. Do not describe the response as the full catalog when catalog_complete is false.

The fictional source-change example shows why returned records must be read together with a partial execution summary.

Use the schema#

Field names and types belong to the accepted source definition. Validate against its schema and distinguish a missing value from an empty string or zero. Detail fields may be unavailable when detail enrichment is disabled or incomplete.

Treat example records as examples of the build, not current data. Executing a versioned definition applies its rules to source content available at run time; pinning the version does not freeze the source website.

Handle partial output#

Preserve the summary with the records, identify missing fields or incomplete details, and decide whether the result is usable for your workflow. Do not quietly replace an execution error with an empty dataset. If complete coverage is required, confirm that the source contract supports that scope before relying on it.

Read errors and billing before retrying, and include the source, version, and trace identifier in a support request.