Job Description
Main Tasks and Responsibilities:
- Design and implement scalable, resilient, and secure Big Data architectures.
- Configure and manage distributed environments (on‑premises or cloud) for large‑scale data processing.
- Develop and maintain data pipelines using Apache Airflow, including complex DAG design, monitoring, and CI/CD best practices.
- Build and optimize batch and streaming applications using Apache Spark.
- Drive technical decisions related to data storage, processing, governance, and system integration.
- Collaborate with engineering, analytics, and business teams to ensure data quality and availability.
- Produce clear documentation of architectures, data flows, and technical components.
Technical Skills:
- Proven experience as a Big Data Architect or similar senior technical role.
- Strong expertise in Apache Spark (PySpark, SQL, performance tuning).
- Solid han...