Head of Data Engineering (Azure Databricks)

Posted 21 August 2023
Salary £100000 - £115000 per annum, Benefits: 10% Bonus
Job type Permanent
Discipline DevelopmentData
Contact NameMatt Trought

Job description

Head of Data Engineering (Azure Databricks)

Salary: £100,000 - £115,000 doe + 10% Bonus
Location: Remote (Occasional Travel to Nottingham and Romania)


A fantastic opportunity has come up for a Head of Data Engineering to join a truly innovative and creative organisation that has some exciting and huge plans to grow the Engineering team.

They are looking for a highly skilled and experienced Head of Data Engineering to lead a Data strategy and also a talented team of Data Engineers to help build and maintain their near real-time Data Platform in Microsoft Azure. This is a great opportunity to help build, evangelise and execute a data strategy which helps the business make data-driven decisions to improve operational execution and customer experience.

In addition to steering their data strategy, you will be accountable for delivering their data projects and ensuring the reliability, scalability, and security of the data infrastructure, pipelines, and storage architecture while balancing cost and scale. In addition to the technical aspects of the role, you will be comfortable presenting data insights to technical and non-technical stakeholders to help articulate how using the data captured can improve our business performance.


  • Leading the Data Community of Practice and collaborating with cross-functional teams to ensure that our data warehouse aligns with business objectives and delivers internal and customer value
  • Developing and implementing data engineering strategies to improve data quality, reliability, and performance across our platforms
  • Collaborating with Product to design the data roadmap and build the team capacity plan to mit and deliver projects on time and to a high standard
  • Identifying areas of improvement for our data infrastructure and building a plan to implement solutions and remediating technical debt
  • Ensuring that our data storage and processing infrastructure is scalable, secure, and highly available whilst balancing costs
  • Proactively managing data quality issues with system owners to improve data integrity within our data platform
  • Overseeing the design, implementation, and maintenance of our data platform architecture, warehouse, pipelines, and data storage in the cloud
  • Providing leadership and line management to the data engineering team, guiding and coaching them to develop technical solutions aligned with business objectives
  • Building career paths for team members, investing time and experience in developing their skills and technical capabilities across the data stack
  • Operating as a senior leader within the business and a key part of the Technology Leadership Team
  • An ambition to research in new technologies that could help the future road map of data such as Machine Learning
  • Researching and suggesting new services, technologies, and methodologies to promote the evolution of our data platform and features available to our end-users
  • Defining and managing SLAs and SLOs for all data sets and processes running in production in collaboration with the Heads of Engineering, Architecture & TechOps
  • Providing appropriate out-of-hours escalation cover for data platform incident management



  • Strong Databricks experience and supporting adoption within a cloud environment, ideally using the Microsoft Azure data stack
  • Excellent knowledge of SQL, NoSQL, and API-based data pipelines, delivering data engineering solutions into production with quality and automation at the forefront
  • Experience of working with event driven eventually consistent architectures
  • Strong experience in testing and deploying data pipelines into production with quality gates to measure data accuracy and integrity
  • Strong experience delivering to agile practices, ideally SCRUM or Kanban, and leading teams to deliver high-quality solutions and iteratively deliver value to the business
  • Excellent written and verbal communication skills to articulate complex technical concepts to technical and non-technical stakeholders
  • Proven ability to manage stakeholders at executive and senior management levels, build relationships, and influence outcomes
  • Extensive Data Engineering background along with experience of leading and coaching high-performing Data Engineering teams that deliver business-critical reporting solutions
  • Previous experience in designing and implementing a cloud-based data platform with real-time and batch ETL processing for large volumes of data
  • Experience working with Product Managers and Product Owners to translate complex business requirements into data-driven solutions and roadmaps
  • Ability to manage multiple priorities in a fast-paced environment across multiple teams and technologies


  • Warehouse/Azure Synapse and Azure Stream Analytics
  • Exposure to Big Data analysis
  • Experience managing data sovereignty and PII
  • Experience using Tableau Report Design or Power BI
  • Experience of Kimball Star Schema data modelling
  • Knowledge of IaC technologies such as Microsoft ARM templates or Terraform
  • Strong domain experience in messaging or business communications
  • Knowledge of Microsoft Azure Data Stack e.g. CosmoDB, DataFactory, Azure SQL Data
  • Experience of data streaming technologies such as Kafka or Red Panda

If you feel you have the required skills and experience and want to be considered for this opportunity, please forward an up-to-date version of your CV, and someone will contact you back within 48 hours.