Job Description
Key Responsibilities
- Create and maintain large-scale data processing systems and infrastructure.
- Build robust, performant, and scalable data pipelines to ingest, transform, and store complex data from multiple sources.
- Collaborate with data science and software engineering team members to provide required data in an accessible, timely, and accurate manner.
- Optimize and refine processes, algorithms, and systems to enhance data quality and reliability.
- Ensure data privacy and security compliance.
- Implement monitoring, logging, and alert systems to ensure data pipeline health and performance.
- Collaborate with infrastructure and IT teams to ensure optimal data storage and retrieval mechanisms.
- Drive the optimization, testing, and tooling to improve data quality.
- Mentor junior data engineers, imparting knowledge and promoting best practices.
- Stay updated with emerging technologies and int...