Boto3 ETL Docker Linux / Unix Big Data Powershell / Bash BMW Cloud Data Hub (CDH) BMW CDEC Blueprint · Any expertise in data modelling Oracle SQL. - Exceptional analytical skills analysing large and complex data sets Perform thorough testing and data validation to ensure the accuracy of data transformations. - Strong written Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV etc. · Experience working with Data Quality building data pipeline using AWS Glue or Data Pipeline, or similar platforms. · Familiar with data store
Linux / Unix
- Big Data
- Powershell / Bash
- Cloud Data Hub (CDH)
- CDEC Blueprint
- Expertise in data modelling with Oracle SQL.
- Exceptional analytical
complex data sets.
- Thorough testing and data validation to ensure the accuracy of data transformations
Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV, etc.
- Experience with Data Quality
APIs.
- Experience building data pipelines using AWS Glue or Data Pipeline, or similar platforms.
space and has embarked on an exciting, strategic, data modernization program that will equip them to better looking for someone who can help to ensure that the Data Services deliveries to both internal and external functional, non-functional, and business aspects of the data pipeline and analytics solutions. Identify areas to validate the functionality, performance, and data integrity of the Azure Databricks environment. Utilize reliability of the data pipeline. Data Validation and Quality Assurance: Develop automated data validation scripts
An amazing opportunity for a Senior Data Engineer to join a multinational organization that produces work with feature teams to design and implement data extraction, cleaning, analysis, visualization, and database types to target data containers for the purpose of migrating data to cloud technologies If you Assist in creating data pipelines from source to target platforms. Experience in Data validation and verification to transform data from source to target for analytical purposes (Business Objects/Data Lakes) and migration
Our client requires the services of a Data Engineer/Scientist ( Expert) - Midrand/Menlyn/Rosslyn/Home as possible ROLE: Data Engineers are responsible for building and maintaining Big Data Pipelines using using GROUP Data Platforms. Data Engineers are custodians of data and must ensure that data is shared in line citizens/residents are preferred. Relevant IT / Business / Engineering Degree Certifications: Candidates with one or Certified SysOps Associate AWS Certified Developer Associate AWS Certified Architect Associate AWS Certified
some Python and AWS data engineering experience, as everything will be related to data ingestion, storage
strong leadership of a highly technical Senior Data Engineer to join its team. You will be responsible for infrastructure and systems that enable efficient data processing, storage, and retrieval. You will also help ensure the availability and reliability of data for analysis and decision-making. The successful Science/Software Engineering/Information Systems or similar discipline with 4-5years' work experience in Data Engineering or Informatica. DUTIES: Data Architecture and Modeling - Design and develop data architectures that support
Role : AWS Data Engineer Location : Johannesburg [ 3 Days in a week from office] Duration : 12 Months seeking a highly motivated and experienced AWS Data Engineer to join our Global Markets division. In this quantitative analysts, and data scientists to design, develop, and maintain data infrastructure and pipelines background in C# or Python programming and building data pipelines on AWS, especially with AWS Glue Jobs contributing to the design and implementation of data pipelines to support global markets business. It
initiative aimed at pooling highly skilled Data Engineers for upcoming opportunities in Africa. By registering requirements of the data & analytics environment Analyse business or system requirements and data to enhance years in a data environment Experience with ETL processes and SDLC Database system and cloud data warehousing experience (relational databases, star schemas) Data Analysis, Design, and Architecture experience Ideally
Salt Senior Data Engineer | Innovative Tech Firm | Cape Town Model: Hybrid (3 days in office) Location: a Senior Data Engineer with a knack for AWS and Terraform to build and maintain their data infrastructure go-to expert for all data-related tasks. From architecting and managing the AWS Data Lake and Warehouse and ensuring data quality, you'll have the opportunity to influence every stage of the data journey. Requirements: (Terraform preferred). - Solid understanding of data security best practices and privacy laws. - Agile