Data Engineer

Salary: £? - ? per year

Requirements

• Proven experience leading data engineering teams, with a strong track record of mentoring and developing junior engineers.
• Expert knowledge of Python and Airbyte CDK, with experience building custom data connectors.
• Strong experience developing ELT pipelines using PySpark in Databricks within a Medallion Architecture.
• Experience with Terraform and deploying Airbyte connections and Databricks assets using GitHub Actions.
• Confident working with JIRA, GitHub, and VS Code.
• Experience with Databricks Delta Live Tables (DLT) is desirable.
• Knowledge of Databricks Asset Bundles is desirable.

Responsibilities

• Lead a high-profile data engineering project.
• Mentor and develop junior engineers.
• Build custom data connectors using Airbyte CDK.
• Develop ELT pipelines using PySpark in Databricks within a Medallion Architecture.
• Deploy Airbyte connections and Databricks assets using Terraform and GitHub Actions.
• Work with JIRA, GitHub, and VS Code in day-to-day delivery.

Technologies

• Databricks
• GitHub
• JIRA
• Python
• PySpark
• Terraform
• VS Code

More

We are hiring an experienced Lead Data Engineer for a 3-month initial contract on an outside IR35 basis at £500–£525 per day. This is a hybrid role based in London, requiring two days per week onsite with the remainder worked remotely. You will join a high-profile project and play a key role in leading engineering delivery.

last updated 29 week of 2026