Summary
Engaged in comprehensive data engineering coursework and project development, building a strong foundation in data systems and architecture.
Highlights
Architected and implemented data ingestion pipelines utilizing Docker and Docker Compose, facilitating efficient transfer of diverse flat file data into PostgreSQL databases.
Developed robust and idempotent class-based interactions for PostgreSQL and Google Cloud Storage (GCS), ensuring data integrity and consistency.
Engineered a comprehensive end-to-end ELT (Extract, Load, Transform) data pipeline, integrating PostgreSQL, Apache Airflow, dbt, and Metabase to enhance data transformation and visualization capabilities.