Data Engineer (Senior) 1053
O
Open Source (Pty) Ltd
📍 menlyn, menlyn, South-Africa
Job Description
Essential Skills
- Strong hands-on experience with Python (3.x) and PySpark for large-scale data processing
- Minimum 3+ years working with AWS services such as Athena, Glue, Lambda, S3, and ECS
- Experience with NoSQL (DynamoDB) and relational databases (Oracle/PostgreSQL) , including advanced Oracle SQL
- Proven experience with Oracle Cloud Infrastructure (OCI) services
- Expertise in data formats and schema design including Parquet, AVRO, JSON, XML and CSV
- Solid experience building ETL pipelines using AWS Glue or similar tools
- Experience with Docker and containerisation (Kubernetes/OpenShift advantageous)
- Strong scripting skills (Bash, PowerShell ) and familiarity with Linux/Unix environments
- Hands-on experience with