Connecting Informatica PC Or Cloud To Amazon S3
Amazon S3 is currently one of the best cloud storage options. We can connect to Amazon S3 bucket with various ETL tools. We can transform the data into files and place in Amazon S3. This document helps us to know, how the data transforms from different databases or files to Amazon S3 bucket through Informatica PowerCenter (PC) or Informatica Cloud.
1. Informatica PC – Amazon S3
We can connect Informatica PC and Amazon S3, which is a cloud based store and can store objects and delimited files which are in string 256 format on buckets.
Steps for loading Data to Amazon S3:
- Pull the data from various sources and load it as files into the AWS S3 bucket.
- Configure the AWS S3 bucket and use the script to load the files into AWS S3 bucket.
- Make the scripts reusable and reuse them to load as many times as required without having to modifying them.
- This process enables reduction of costs.
- Enables script reusability across various environments as the S3 locations are not hard-coded.
Fig: Flow – Informatica PC connecting to Amazon S3
2. Informatica Cloud – Amazon S3
When the data is available to be pulled from either database or files,
- Text file or table gets moved to source directory.
- Files are uploaded to S3 bucket.
- This process can directly connect to Amazon S3 bucket.
- Once the connector is created, it can be reused it in several jobs.
The below AWS signature settings are applicable to all the event sources and event targets in the connection:
Data format of Amazon S3 Files
- Text Qualifier : Double quote (“”)
- Delimiter: Colon (:)
- Escape Character : Back Slash (\)
We can optimize the performance while uploading data to Amazon S3. The key point to be remembered here is, before uploading the data to S3 bucket, it needs to be compressed and encrypted.
Contact for further details
Praveen Kumar Peruca
Technical Architect – Analytics Data Engineering