Back to Projects

LinkedIn - Job Postings Data Pipeline

Python Python
Apache Airflow Apache Airflow
dbt dbt
PostgreSQL PostgreSQL
AWS AWS
Tableau Tableau

Overview

A comprehensive data engineering project that automates the collection and processing of job postings from LinkedIn. The pipeline uses a modular architecture with microservices for each stage of the ELT process. Key Features: - Modular ELT system using Python and Airflow - Data enrichment using AI for skills extraction and summary generation - Automated cover letter generation based on job descriptions - Staging and Marts layers implemented with dbt - Interactive visualization in Tableau

View Source Code