Azure Databricks Data Engineer (Castilla Y León)
B
Bluetab, an IBM
📍 león, kingdom of spain, Spain
Job Description
Responsibilities
- Develop and optimize data pipelines in Scala, Python or PySpark by building and executing Spark processes or on Databricks, ensuring performance, quality and scalability.
- Design and construct Data Lake and Data Warehouse architectures primarily on Azure, with possible use of AWS or GCP.
- Transform large-scale data using Spark, applying best practices for distributed processing.
- Optimize queries and storage in Databricks to improve data consumption efficiency and reduce costs.
- Integrate multiple data sources, ensuring data quality, consistency and traceability.
- Collaborate with multidisciplinary teams to drive continuous improvement in ingestion, transformation and exploitation processes.
- Minimum of 4–5 years of consolidated experience.
- Solid experience developing with Scala, Python or PySpark on Spark for distributed processing.
- Experience with Databricks and knowledge...