Job Description
Purpose Statement
The Data Engineer II designs, builds, and maintains robust and scalable data pipelines and infrastructure to support the organization's data needs, particularly related to cloud-based technologies. This includes acquiring, transforming, and optimizing large datasets to ensure high-quality, reliable, and accessible data for analytics, reporting, machine learning and AI initiatives.
Key Job Accountabilities
- Lead Data Pipeline and Infrastructure Development: Design, develop, and optimize end-to-end ETL/ELT pipelines(on-premises and cloud) using Python, SQL, Spark, etc. to ingest data into scalable cloud infrastructure, including Data Lakes and Data Warehouses (AWS, Azure, or GCP).
- Collaborate on data modeling and schema design: Collaborate with stakeholders to design and implement efficient data models and schema structures. Enforce data quality through robust validation, monitoring, and alerting mechanisms to ensure accu...