Role Overview

Builds and maintains the infrastructure and systems required for data collection, storage, and processing at scale. They create data pipelines and architectures that enable data scientists and analysts to access clean, reliable data for analysis. Responsibilities include designing and implementing data pipelines, building data warehouses and data lakes, developing ETL (Extract, Transform, Load) processes, ensuring data quality and consistency, and optimizing data storage and retrieval systems. They work with big data technologies like Apache Spark, Hadoop, cloud platforms (AWS, Azure, GCP), and database systems. Daily tasks involve building data ingestion systems, optimizing data processing workflows, monitoring data pipeline performance, troubleshooting data quality issues, implementing data governance practices, and collaborating with data scientists to ensure data accessibility.