cloud:aws:big_data [Tohuwabohu excorcism]

cloud:aws:big_data

This is an old revision of the document!

Table of Contents

Big Data
- Analytics
  - Video about Big Data
  - Structure of Data pipeline

Big Data

Analytics

https://docs.aws.amazon.com/whitepapers/latest/derive-insights-from-aws-modern-data/why-use-aws-for-modern-data-analytics.html

Video about Big Data

A nice overview of UI for available BigData tools

https://www.youtube.com/watch?v=HeJ15SpR66w

Here the title picture again

Query S3 data with Athena
Glue Job to transform data
Query S3 data with Redshift Spectrum
Query Postgres data with Redshift Federated Query
Query data in S3, RDS (Postgres) and Redshift together
Create Materialized View with Federated Query in Redshift

Structure of Data pipeline

Even better overview of tools for data-pipelines

https://www.youtube.com/watch?v=tykcCf-Zz1M

Data Source >
Data Ingestion >
Raw Storage >
Business rules transformation, consolidation (Glue, EMR)
Processed Zone

cloud/aws/big_data.1697634425.txt.gz · Last modified: 2023/10/18 13:07 by skipidar