GoLang Data Warehouse
Designed and built an end-to-end ETL pipeline in Go that extracts real-time hourly weather data from a public REST API for 5 global cities, transforms raw JSON into structured records, and loads data into a three-layer PostgreSQL warehouse (Raw → Staging → Warehouse) following the medallion architecture.
Automated daily execution and monitoring using an Apache Airflow DAG with database health checks and row-count verification. Implemented idempotent upserts, transactional batch inserts, and connection pooling, and containerised the stack with Docker Compose.
Apache AirflowDockerPostgreSQLPythonGo
