Amazon S3
Last updated
Last updated
Integrate with Amazon S3 to gain publish-access from Cascade to Amazon S3 buckets. Any team member(s) with access to the workspace will be able to publish data to the S3 bucket.
In the top-left corner, confirm the workspace you are about to integrate with Amazon S3. Any team members with access to this workspace will gain access to the integration.
From the left-nav of your workspace, select the "Integrations" tab
Select the "New Integration" button in the top-right corner
Select "Amazon S3" to create a new (or additional) integration with Amazon S3
In the provided field, name your integration
Note: Any name may be provided. However, it may be simplest to label it with the Amazon S3 account name.
Next, fill out your AWS Access Key ID and AWS Secret Access Key
See Amazon S3 Documentation for how to get your AWS credentials
Click "Test & Save" - if connection is successful, status should update to "Connected".
At the moment, Cascade only provides publish access to Amazon S3. Open a new or existing workflow from the now integrated workspace. You'll be able to add the "Publish to Amazon S3" tool to your canvas.
From the tool side panel, select "Publish to Integration" to drill in
Drag "Publish to Amazon S3" from the tool side panel to your canvas
Connect it to the upstream dataset you wish to publish to Amazon S3
Select the Publish to Amazon S3 tool to configure it, options include:
Select Integration
Select the integration, named in an earlier step, for this Amazon S3 account
Bucket
Enter the bucket container name within your S3 account you would like to publish your objects to
Key
Enter the key (the file path) of the object you would like to publish to within your S3 bucket
This can be an already existing key that you're overwriting or a new key that will be written to
Output Options
Select "Overwrite" to publish over any existing data in Amazon S3
Select "Disable Publishing" to disable workflow from publishing new data to the Amazon S3 bucket
Use this option during workflow building and testing so non-finalized datasets are not pushed to S3
Publish File Type
Select the file type you would like to publish to S3:
CSV
Parquet
Excel
Hyper
Don't get blocked by a data integration. If you need help configuring and integration or would like to recommend an integration, join the Cascade Slack Community, tell us what you're after :)
