Configure Athena destination for ingestion pipeline

After your first connection you will need to setup your Athena default destination for your ingestion pipeline.

1. Login with your Administrator’s credential

2. Go to “Your Ingestion pipelines” interface

3. Create a new ingestion pipeline

4. Configure your ingestion Source. This example use a csv file. Load Data into the Data Lake from CSV

5. Configure your ingestion Destination.

6. Fill the destination details.

Destination Name: any string

• Username

• Role Name

To retrieve the above two values you will need to go to the CloudFormation for Data Pipes deployment Output.

a. Visit the AWS CloudFormation Console.

b. Ensure that you are in the correct region.

c. Navigate to Stacks.

d. Click on the Data Pipes stack (if you followed the docs, the name is datapipes).

e. Click on the Output tab.

f. Look for AthenaUsername and AthenaRoleName

• Access Key

• Secret Key

To retrieve the above two values you will need to go to the Amazon Secret Manager.

a. Visit the AWS CloudFormation Console.

b. Ensure that you are in the correct region.

c. Search the Secrets /iam_admin_creds.

d. Click on the Data Pipes stack (if you followed the docs, the name is datapipes).

e. Open it and scroll down and click on the Retrieve Secret value.

f. Look for ACCESS_KEY and SECRET_ACCESS_KEY

e. Use them in the Data Pipes ingestion form

7. Athena destination is now configured and ready to be used