Senior Data Engineer – Data Analytics Platform r-19553

Design, develop, and maintain data pipelines and backend services for real-time decisioning, reporting, data collecting, and related functions

Data Requirements and Modeling

Data Management and Transformation

Produce high-quality, well-tested, and secure code

Develop and maintain software designed to improve data governance and security

Develop processes designed to ensure Data Security and Data Quality

Bachelor’s or Master’s degree in Computer Science, Data Analytics, Information Systems, or a related technical field; or equivalent training and professional experience

5+ years of experience in ETL/ELT, data warehousing, Business Intelligence, and integrating data processing/workflow management tools into pipeline design

5+ years building and maintaining end-to-end data systems using Python, Scala, or similar programming languages

Solid experience in utilising SQL for data analysis, investigating data issues, diagnosing root causes, and designing effective solutions

Experience in working with cloud-based data technologies, preferably on Microsoft Azure, within continuous integration and delivery (CI/CD) environments

Demonstrated experience working with large-scale datasets using Databricks, PySpark, Spark Streaming, and Delta Lake

Hands-on experience with structured, semi-structured, and unstructured data across various storage systems, including relational databases (RDBMS), data warehouses, in-memory caches, and document databases

Familiarity with cloud storage solutions such as Azure Data Lake and Blob Storage

Solid understanding of data modelling principles (experience with Data Vault is a plus) and strong skills in system design, implementation, and testing

Experience with event-driven architectures (e.g., Kafka, Event Hubs, Apache Flink) and containerised microservices platforms (e.g., Kubernetes, Docker, Helm Charts)

Knowledge and practical use of dbt (data build tool), experience with BI tools

Excellent communication and collaboration skills, ability to provide technical leadership to other developers

12 months with option for extension

Fluent English