Data Platform Engineer

Salary: £110,000 - 120,000 per year

Requirements

• Strong AWS cloud platform migration experience
• Strong Terraform and Infrastructure as Code (IaC) skills, including developing and maintaining Terraform code
• In-depth data engineering experience
• Experience with Python, PySpark, SQL, data pipeline development, ETL/ELT design, data modelling, data quality, and data validation
• Experience with AWS cloud platform services such as S3 and IAM
• Knowledge of CI/CD concepts, DevOps principles, and Agile delivery
• Government, GOV.UK, or public sector domain experience strongly preferred
• Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE), and NHS/healthcare data experience are advantageous
• Must be active SC Cleared or fully eligible to undergo the full SC Security Clearance process

Responsibilities

• We will support the delivery of large-scale data processing solutions
• We will drive data engineering and cloud platform migration work
• We will help modernise a major data platform programme by migrating data pipelines, notebooks, and data workloads from a legacy Databricks environment to modern AWS cloud-based data platforms
• We will develop and maintain Terraform code to support data platform deployments
• We will support deployment of data platform components, jobs, permissions, and environment configurations
• We will analyse current data engineering solutions and identify data migration requirements
• We will onboard datasets, users, and workloads onto the target platform

Technologies

• AWS
• CI/CD
• Cloud
• Databricks
• DevOps
• ETL
• IAM
• Support
• Python
• PySpark
• SQL
• Security
• Terraform

More

We are a leading global IT transformation business specialising in large-scale Government projects. This is a 6-month rolling contract for a Data Platform Engineer based in London on a hybrid working basis, with 3-4 days per week remote and 1-2 days per week in the office, plus occasional travel to our Leeds office. The role offers the opportunity to work on a major data platform modernisation programme within a secure public sector environment.

last updated 26 week of 2026