Manager Data Engineer - AWS Databricks

Key Responsibilities:

• Lead the design, development, and optimization of scalable and secure data pipelines using AWS services such as Glue, S3, Lambda, EMR , and Databricks Notebooks , Jobs, and Workflows.
• Oversee the development and maintenance of data lakes on AWS Databricks , ensuring performance and scalability.
• Build and manage robust ETL/ELT workflows using Python and SQL , handling both structured and semi-structured data.
• Implement distributed data processing solutions using Apache Spark / PySpark for large-scale data transformation.
• Collaborate with cross-functional teams including data scientists, analysts, and product managers to ensure data is accurate, accessible, and well-structured.
• Enforce best practices for data quality, gov...