Import from API

Import data from an external service.

This tool is currently in Beta and is still being tested. Want to learn more? Like to provide feedback? Please reach out to [e-mail]

Import from API allows data to be extracted from third-party services via REST APIs.

At this time, Import from API can only make GET requests and accepts only JSON responses.

Interpreting API Documentation

Before configuring the Import from API tool, it is important to understand how the API is intended to be used. To find out, you may need to search for documentation or reference material provided by the third party. Often this will be labeled "API Documentation" or "API Reference" or something similar under a "Developers" page. These pages often contain code snippets, sometimes in multiple programming languages.

Main Configuration

Option	Description
Endpoint URL	The URL to request data from.
Location of data from response (optional)	The JSONPath that contains the tabular data to extract. When blank, interprets the entire response as data. The possible JSONPaths to be selected are determined by inspecting the shape of the response. Learn more about JSONPaths here.

Authentication

In many cases, an API request will need to be authenticated in order to validate authorization of the requested data. Cascade supports two different authentication methods: username / password (or basic authentication) and bearer token authentication.

Username / Password (Basic)

Option	Description
Username	The username that the request should authenticate with. Typically this is the username a user will use to sign into the third-party service.
Password	The password that the request should authenticate with. Typically this is the password a user will use to sign into the third-party service.

Bearer Token

Option	Description
Token	The token that should be used in the Authorization header of the request for authentication. Typically this is an API token or API key.

Pagination

In many cases, the full results from an API request will be provided over multiple requests (or pages). Cascade supports three different pagination methods: page, offset / limit, and cursor pagination.

Page pagination

Option	Description
Page key	The name of the parameter used to reference which page number is being requested. Typically, this is page.
First page number	The starting page number. Typically, this is either 1 or 0.

Offset / limit pagination

Option	Description
Offset key	The name of the parameter used to reference how many items should be skipped when requesting a given page. Often, this is offset, since, after, or a similar name.
Initial offset	The starting index of results returned by the API. Typically, this is 0.
Limit key (optional)	The name of the parameter used to limit the number of items returned by the API per page. Often, this is limit, max, or a similar name. When omitted, requests will not specify an explicit limit.
Limit size (optional)	The number of items to request per page. When omitted (or when limit key is omitted), the size will be inferred from the number of items returned by the first request.

Cursor pagination

Option	Description
Cursor key (optional)	The name of the parameter used to reference the cursor in pagination. Typically, this is cursor. When omitted, assumes the cursor path references a full URL rather than a token.
Cursor path	The JSONPath to the token or URL for the next page. The possible JSONPaths to be selected are determined by inspecting the shape of the response. Learn more about JSONPaths here.

All pagination

Regardless of the pagination used, the following parameters may also be specified:

Option	Description
Max pages to request (optional)	The maximum number of pages to request when paginating. Note that fewer requests may be made if the API returns no items before the maximum number of pages are requested. If omitted, requests will be made until no items are returned by the API.
Max pages per minute (optional)	The maximum number of pages to request per minute when paginating. This option can be used to rate limit requests sent to the API. Each request will be spaced out evenly, so even if the max pages to request is less than the max pages per minute, this option will still have an effect. If omitted, requests will be made immediately after the previous one completes.

Note: Each time your workflow is re-run, the Import from API tool will re-run as well, incurring any charges that may be imposed from the API service. To avoid this, you can utilize the Lock Output feature which will retain the data from the latest Import from API run.