The business operates in the Fintech space and has embarked on an exciting, strategic, data modernization program that will equip them to better serve their clients in the core financial markets within which they operate. They are a dynamic team of driven individuals who strive to perform and deliver optimal solutions that are innovative and future proof. They are looking for someone who can help to ensure that the Data Services deliveries to both internal and external stakeholders are of the highest possible quality. This means that all our releases have been fully tested and are fully compliant with the business's expected governance processes, and that these are automated wherever possible to ensure continuous excellence in their product delivery. A key part of this role will be to ensure mandated documentation and controls are put in place, whilst remaining true to agile process practices. They spend a lot of time focusing on automation, including how they test through the CI/CD process. Test Planning and Design: Collaborate with the development team, product owners, and stakeholders to understand project requirements and define test strategies. Create and maintain test plans, test cases, and test scenarios covering functional, non-functional, and business aspects of the data pipeline and analytics solutions. Identify areas suitable for test automation and prioritize test cases based on their impact and criticality. Test Automation Development: Develop and implement automated test scripts using Python to validate the functionality, performance, and data integrity of the Azure Databricks environment. Utilize Python libraries and frameworks such as pytest or unittest to create robust and maintainable test automation scripts. Integrate test automation with CI/CD pipelines to enable continuous testing and ensure the reliability of the data pipeline. Data Validation and Quality Assurance: Develop automated data validation scripts to ensure the accuracy, completeness, and consistency of data processed in the Azure Databricks environment. Perform data profiling, data quality checks, and data reconciliation to identify and report data anomalies or discrepancies. Collaborate with data engineers and analysts to establish data quality standards and implement data validation rules. Power BI and Plotly Testing: Create and execute test cases to validate the functionality, usability, and performance of Power BI reports and dashboards. Ensure the accuracy and consistency of data visualizations created using Plotly. Test the integration between Azure Databricks, Power BI, and Plotly to ensure seamless data flow and visualization. Non-Functional Testing: Conduct performance testing to evaluate the scalability, responsiveness, and efficiency of the data pipeline and analytics solutions. Perform load testing to assess the system's behaviour under various workload conditions. Execute security testing to identify potential vulnerabilities and ensure data privacy and compliance. Agile Collaboration and Communication: Actively participate in agile ceremonies such as sprint planning, daily stand-ups, and retrospectives. Collaborate closely with data engineers, data analysts, and other stakeholders to align testing efforts with project goals and timelines. Provide regular updates on test progress, identified issues, and test metrics to the agile team and stakeholders. Test Reporting and Documentation: Document test cases, test scenarios, and test results in a clear and comprehensive manner. Generate test reports and dashboards to provide visibility into the quality and stability of the data pipeline and analytics solutions. Maintain traceability between requirements, test cases, and defects to ensure comprehensive test coverage. Continuous Improvement: Continuously review and optimize the test automation framework and scripts to improve efficiency and maintainability. Stay up-to-date with the latest testing methodologies, tools, and best practices in the data engineering and analytics domain. Ideally a Bachelor's or master's degree in computer science, Information Technology, Software Engineering, or a related field. Strong foundation in data structures, algorithms, and software testing principles. Proficiency in Python programming language and its testing frameworks (e.g., pytest, unittest). Knowledge of data engineering concepts, data warehousing, and ETL processes. Familiarity with big data technologies, such as Apache Spark. Microsoft certification Apply Now

Share this job with someone you think should apply!

Facebook

Data Engineer Johannesburg - Johannesburg

Related Jobs

Devops Engineer Contract Johannesburg - Johannesburg

Data Engineer Johannesburg - Johannesburg

Automation Tester Johannesburg - Johannesburg

Automation Tester Johannesburg, South Africa - Johannesburg

Data Engineer Johannesburg - Johannesburg

Want to do another search?