Job Description
Role: Senior Data Engineer
Experience Range: 3 to 9 years
Location: Bangalore
As a Senior Data Engineer, you will be responsible for building and delivering a Lakehouse-based data pipeline. This role is hands-on and focuses on implementing real-time and batch data ingestion, processing, and delivery workflows, while also ensuring monitoring, observability, and data quality across the pipeline.
Must have skills:
3+ years of hands-on experience in building large-scale data pipelines.
Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads
Proficiency in Pyspark/Python, including building Kafka producers for data ingestion
Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.
Solid understanding of AWS services including S3, Redshift, Glue, Cloud Watch, and Secrets Manager
Experience working with Medallion architecture and hybrid data destinations (e.g., Red...
Experience Range: 3 to 9 years
Location: Bangalore
As a Senior Data Engineer, you will be responsible for building and delivering a Lakehouse-based data pipeline. This role is hands-on and focuses on implementing real-time and batch data ingestion, processing, and delivery workflows, while also ensuring monitoring, observability, and data quality across the pipeline.
Must have skills:
3+ years of hands-on experience in building large-scale data pipelines.
Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads
Proficiency in Pyspark/Python, including building Kafka producers for data ingestion
Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.
Solid understanding of AWS services including S3, Redshift, Glue, Cloud Watch, and Secrets Manager
Experience working with Medallion architecture and hybrid data destinations (e.g., Red...