Job Description
Key Responsibilities
- Design and implement scalable data architectures including data lakehouse and real-time streaming solutions.
- Develop and maintain robust data pipelines using Spark, EMR, Glue, and other big data technologies.
- Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements.
- Ensure data quality, integrity, and security across all data platforms.
- Lead data modeling efforts using Kimball, Inmon, Data Vault, and Medallion architectures.
- Establish and enforce best practices for data governance, metadata management, and data lineage.
- Mentor junior engineers and contribute to team development and knowledge sharing.
- Implement CI/CD pipelines and infrastructure-as-code for data workflows using tools like Terraform and CloudFormation.
- Monitor and optimize data workflows and implement alerting mechanisms for pipeline ...