Data Engineer Umhlanga - Umhlanga Rocks
Careerbox
The data engineer is responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The Data Engineer will support software developers, database architects, data warehouse managers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. Key Responsibilities: Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet functional / non-functional business requirements. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure ‘big data' technologies. Work with stakeholders including the Executive, Service, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. Create data tools for analytics and data scientist team Work with data and analytics experts to strive for greater functionality in our data systems. Map data between source systems, data warehouses, and data marts. Keep our data separated and secure across national boundaries through multiple data centers and Cloud regions. Implement business rules via stored procedures, middleware, or other technologies. Develop and implement data extraction procedures from other systems, such as callcentres, HR, or external systems. Create supporting documentation, such as metadata and diagrams of entity relationships, business processes, and process flow Create plans, test files, and scripts for database testing, ranging from unit to integration testing. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Select methods, techniques, or criteria for database evaluative procedures. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics. Design and execute reporting systems for processes. Analyze system requirements and gather all business information. Research and implement appropriate technologies to ensure that the data warehouse is always operating optimally Experience, Knowledge and Qualifications The ideal candidate is currently a Database Administrator with 3 years of experience in writing queries, extracting data, data scrubbing and large volumes of records. Previous experience with Relational and/or NoSQL databases essential Previous experience with ETL processes essential A qualification in Data Informatics or at least 3 years relevant experience in large data environments Experience managing a team of analysts, ETL procedures, demonstratable technical expertise in writing complex database queries
Apply Now
Facebook
WhatsappRelated Jobs
Bi Tech Lead eThekwini - Durban
Xcede Group
...
Senior Data Scientist - Durban
...
Data Scientist Durban - Durban
Datafin IT Recruitment
...
Business Intelligence (BI) Analyst - Durban
...
Senior Applications Developer Umhlanga - Umhlanga Rocks
Careerbox
...