AWS Data Engineer (Expert) - Midrand
ESSENTIAL SKILLS REQUIREMENTS:
- Technical Skills / Technology (in order of importance):
- Terraform
- Python 3.x
- SQL - Oracle/PostgreSQL
- PySpark
- Boto3
- ETL
- Docker
- Linux / Unix
- Big Data
- Powershell / Bash
- Cloud Data Hub (CDH)
- CDEC Blueprint
- Any additional responsibilities assigned in the Agile Working Model (AWM) Charter
ADVANTAGEOUS SKILLS REQUIREMENTS:
- Expertise in data modelling with Oracle SQL.
- Exceptional analytical skills for analyzing large and complex data sets.
- Thorough testing and data validation to ensure the accuracy of data transformations.
- Strong written and verbal communication skills, with precise documentation.
- Self-driven team player with the ability to work independently and multi-task.
- Experience with Enterprise Collaboration tools such as Confluence, JIRA, etc.
- Experience developing technical documentation and artefacts.
- Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV, etc.
- Experience with Data Quality Tools such as Great Expectations.
- Experience developing and working with REST APIs.
- Experience building data pipelines using AWS Glue or Data Pipeline, or similar platforms.
- Familiarity with data stores such as AWS S3, and AWS RDS or DynamoDB.
- Understanding of various software design patterns.
- Experience preparing specifications for programs to be written, designed, coded, tested, and debugged.
- Strong organizational skills.
BASIC EXPERIENCE/UNDERSTANDING OF AWS COMPONENTS (in order of importance):
- Glue
- CloudWatch
- SNS
- Athena
- S3
- Kinesis Streams (Kinesis, Kinesis Firehose)
- Lambda
- DynamoDB
- Step Function
- Param Store
- Secrets Manager
- Code Build/Pipeline
- CloudFormation
- Business Intelligence (BI) Experience
- Technical data modelling and schema design (not drag and drop)
- Kafka
- AWS EMR
- Redshift
- Basic experience in Networking and troubleshooting network issues.
- Knowledge of the Agile Working Model.
QUALIFICATIONS/EXPERIENCE REQUIRED:
- Relevant IT / Business / Engineering Degree
- Preferred candidates with one or more of the following certifications:
- AWS Certified Cloud Practitioner
- AWS Certified SysOps Associate
- AWS Certified Developer Associate
- AWS Certified Architect Associate
- AWS Certified Architect Professional
- Hashicorp Certified Terraform Associate
ROLE AND RESPONSIBILITIES:
Data Engineers are responsible for building and maintaining Big Data Pipelines using Data Platforms. They are custodians of data and must ensure that data is shared in line with the information classification requirements on a need-to-know basis. Apply Now
- Technical Skills / Technology (in order of importance):
- Terraform
- Python 3.x
- SQL - Oracle/PostgreSQL
- PySpark
- Boto3
- ETL
- Docker
- Linux / Unix
- Big Data
- Powershell / Bash
- Cloud Data Hub (CDH)
- CDEC Blueprint
- Any additional responsibilities assigned in the Agile Working Model (AWM) Charter
ADVANTAGEOUS SKILLS REQUIREMENTS:
- Expertise in data modelling with Oracle SQL.
- Exceptional analytical skills for analyzing large and complex data sets.
- Thorough testing and data validation to ensure the accuracy of data transformations.
- Strong written and verbal communication skills, with precise documentation.
- Self-driven team player with the ability to work independently and multi-task.
- Experience with Enterprise Collaboration tools such as Confluence, JIRA, etc.
- Experience developing technical documentation and artefacts.
- Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV, etc.
- Experience with Data Quality Tools such as Great Expectations.
- Experience developing and working with REST APIs.
- Experience building data pipelines using AWS Glue or Data Pipeline, or similar platforms.
- Familiarity with data stores such as AWS S3, and AWS RDS or DynamoDB.
- Understanding of various software design patterns.
- Experience preparing specifications for programs to be written, designed, coded, tested, and debugged.
- Strong organizational skills.
BASIC EXPERIENCE/UNDERSTANDING OF AWS COMPONENTS (in order of importance):
- Glue
- CloudWatch
- SNS
- Athena
- S3
- Kinesis Streams (Kinesis, Kinesis Firehose)
- Lambda
- DynamoDB
- Step Function
- Param Store
- Secrets Manager
- Code Build/Pipeline
- CloudFormation
- Business Intelligence (BI) Experience
- Technical data modelling and schema design (not drag and drop)
- Kafka
- AWS EMR
- Redshift
- Basic experience in Networking and troubleshooting network issues.
- Knowledge of the Agile Working Model.
QUALIFICATIONS/EXPERIENCE REQUIRED:
- Relevant IT / Business / Engineering Degree
- Preferred candidates with one or more of the following certifications:
- AWS Certified Cloud Practitioner
- AWS Certified SysOps Associate
- AWS Certified Developer Associate
- AWS Certified Architect Associate
- AWS Certified Architect Professional
- Hashicorp Certified Terraform Associate
ROLE AND RESPONSIBILITIES:
Data Engineers are responsible for building and maintaining Big Data Pipelines using Data Platforms. They are custodians of data and must ensure that data is shared in line with the information classification requirements on a need-to-know basis. Apply Now
Facebook
WhatsappRelated Jobs
Aws Expert Midrand - Midrand
Liyema Consulting
...
Data Engineer Midrand - South Africa
iSanqa
...
Senior Aws Engineer Midrand - Midrand
E-Merge IT Recruitment
...
AWS Data Engineer (Expert) - Centurion
...
Aws Expert Centurion - Centurion
Liyema Consulting
...