Senior Data Engineer

Salary: £35,000 - 46,000 per year

Requirements

• Experience building or contributing to large-scale scraping or data ingestion systems
• Ownership of architecture and implementation
• Strong understanding of systems, trade-offs, and long-term maintainability
• Ability to communicate clearly with non-technical stakeholders
• Proficiency in Python (Playwright, Scrapy, Selenium)
• Experience with APIs (FastAPI, Django)
• Familiarity with queues/orchestration (Celery, Kafka, RabbitMQ)
• Knowledge of cloud infrastructure (AWS, GCP, etc.)

Responsibilities

• Design a system to extract data from thousands of heterogeneous websites
• Build pipelines for continuous, reliable data updates
• Deliver structured data via a low-latency API layer
• Implement efficient caching strategies with minimal long-term storage
• Handle site variability at scale
• Work around anti-bot protections and rate limiting
• Design for resilience, monitoring, and failure recovery
• Keep infrastructure lean and cost-efficient

Technologies

• API
• AWS
• Cloud
• Celery
• Django
• FastAPI
• GCP
• Kafka
• Playwright
• Python
• RabbitMQ
• Selenium

More

We are a UK-based startup building a platform focused on international property. Our goal is to use large-scale data aggregation to enhance user experience while directing users back to original sources. We are currently at an advanced stage and are looking for a Senior Data Engineer to take ownership of our core data platform. In this role, you will have a high degree of ownership and autonomy, working closely with decision-makers. We offer competitive compensation for a senior hire, including equity. This position is hybrid remote in Lancaster (Lancashire).

last updated 16 week of 2026