Data Engineer - North Johannesburg
EQPlus
Responsible for maintaining the data warehouse through design and implementation of ETL/ELT methodologies and technologies, as well as providing maintenance and support to our ETL and ML environments. To ensure optimal performance, he conducted root cause analysis on production issues and provided technical leadership throughout the entire information management process of both structured and unstructured data. Duties: Responsible for solution design and development of Various Functionality in AWS for the project flow Develop and maintain automated ETL pipelines (with monitoring) using scripting languages such as Python, SQL and AWS services such as S3, Glue, Lambda, SNS, Redshift, SQS, KMS Develop Glue Jobs for batch data processing and created Glue Catalog for metadata sync up Develop Data Pipelines using AWS lambda, Step Functions for data processing Manage data coming from different sources Experienced with AWS services to smoothly manage application in the cloud and creating or modifying instances. Implement with Scaled Agile Framework (SAFe) Be involved with the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark Be involved in creating Hive Tables, loading with data and writing Hive queries Requirements: Degree (Information Technology) 6 -7 Experience in the same role Talend SSIS, SSAS, and SSRS Clover ETL Strong SQL Background AWS Jupiter Notebook Data Warehousing Cloud Warehousing
Apply Now
Facebook
WhatsappRelated Jobs
AWS Data Engineer EXP 0675 - Midrand
...
Data Engineer Midrand - Midrand
E-Merge IT Recruitment
...
Senior Aws Engineer Midrand - Midrand
E-Merge IT Recruitment
...
Data Engineer Midrand - Midrand
Liyema Consulting
...
Data Engineer Midrand - Midrand
Cruze Control Technologies
...