Job Description
Position Description:
The Azure Databricks Engineer will design, build, and optimize scalable data pipelines and lakehouse solutions using Azure Databricks. This role is critical for enabling advanced analytics and supporting machine learning initiatives. This role requires US Citizenship.
The preferred location for this role is Lafayette, LA, however this position can also be filled out of Belton, TX, Lebanon, VA or Knoxville TN. (Hybrid role)
Your future duties and responsibilities:
. Develop and maintain ETL/ELT pipelines using Databricks workflows, Delta Lake, and Apache Spark.
. Architect and optimize data lakehouse solutions for efficient storage and retrieval.
. Configure and manage Databricks workspaces, clusters, and implement security and cost optimization strategies.
. Build real-time streaming and batch processing solutions for diverse data sources.
. Collaborate with data scientists to d...