Lead Python Architect

Salary: £100,000 - 150,000 per year

Requirements

• We are looking for a high-caliber Lead Python Architect with expert-level PySpark and strong Python fundamentals.
• We require proven experience translating high-level strategy into detailed HLD and LLD design documents.
• We need extensive experience with legacy Azure Synapse environments and migration to Spark-native platforms.
• We expect deep knowledge of Spark internals, including the Catalyst Optimizer and memory management.
• We require experience with Delta Lake and open standards to support long-term portability.
• We need a strong background in backend software engineering with modular, testable, and idempotent pipeline design.
• We expect experience planning and executing multi-phase enterprise migrations.
• We require the ability to balance quick wins with architectural integrity in complex environments.
• We are looking for someone who can lead code reviews and provide deep technical guidance.
• We value a hands-on technical leader who can act as a playing coach and final engineering escalation point.

Responsibilities

• We will rely on you to own the technical design specifications for our Lakehouse estate.
• You will define the architectural blueprint from schema enforcement to Delta Lake optimization.
• You will design metadata and security layers that protect the enterprise while maintaining flexibility.
• You will help define the migration roadmap and sequence the transition from Synapse to Fabric Lakehouse.
• You will partner with leadership to decide where to refactor legacy logic and where to re-platform.
• You will select companion tooling for CI/CD, testing, and observability.
• You will personally refactor complex legacy T-SQL and Synapse utilities into optimized PySpark modules.
• You will build foundational ingestion and transformation templates for global engineering teams to scale.
• You will lead code reviews and perform deep Spark performance tuning.
• You will mentor engineers and set a standard of engineering excellence across the function.

Technologies

• Architect
• Azure
• Backend
• CI/CD
• Fabric
• Support
• Python
• PySpark
• SQL
• Security
• Spark
• Cloud
• Databricks

More

We are leading an enterprise-wide data modernization from a legacy Azure Synapse environment to a modern Fabric Lakehouse, with a strong focus on data portability and platform-agnostic engineering built on Delta Lake standards. This is a hands-on, strategic role reporting to the Overall Architecture Lead, where you will act as the technical engine-room lead and help shape the long-term engineering roadmap. We are offering a full-time, temp-to-perm opportunity on a 12-month contract, with a salary range of £100,000 to £150,000 per year. The role is hybrid remote based in London, and you will work with global engineering teams across the UK, EU, and offshore locations.

last updated 21 week of 2026