Data Engineer
Advert Closes: Monday 2nd December
Are you a data enthusiast ready to make an impact?
Join us in building the new Crown Estate Azure data platform! As a Data Engineer, you’ll deliver technical solutions and create end-to-end analytical systems that are highly available, scalable, stable, secure, and cost-effective. Your main goal will be to build, deploy and automate data pipelines using the Azure cloud platform, including IAC (Bicep), Data Factory, Python/PySpark and Databricks. You’ll support the Senior Data Engineer in making data usable, monitoring its quality, and delivering data services that drive The Crown Estate’s strategy.
Key Responsibilities:
- Design, implement, and operate large-scale, high-performance data pipelines for analytics and data science.
- Apply best practices in development, test design/automation, CI/CD, and documentation.
- Develop, monitor, and maintain data pipelines, ensuring consistent availability and quality.
- Analyse data structures, identify performance changes, and suggest improvements.
- Understand and meet stakeholder requirements.
- Ensure information security at all times.
Experience and Skills:
- Degree in Computer Science, Engineering, Mathematics, or related field, or equivalent industry experience.
- Proven experience in developing data ingestion, processing, and analytical pipelines for big data, relational databases, NoSQL, and data warehouse solutions.
- Extensive hands-on experience implementing data pipelines & services using Azure: Data Factory, Functions, Synapse, Azure SQL DB, Event Hubs/Grids, Databricks, Unity Data Catalog, Cosmos Db, etc.
- Hands-on experience in Big Data technologies such as PySpark, Python, SQL, Databricks, ADLS/Storage options, Lakehouse design, File formats (JSON/Delta/Parquet), Powershell, and streaming technologies.
- Well versed in DevOps, IaC (Bicep/Terraform) and CI/CD deployments, experience working with Developer tools such as Azure DevOps, VS Code, GitHub, etc.
- Experience in building scalable and performant API data services.
- Awareness with Azure AI technologies and tools: openAI, ML, Cognitive Services, AI Studio
- Excellent stakeholder management skills at all levels.
Nice-to-Have Certifications:
- DP-200 Implementing an Azure Data Solution / DP-201 Designing an Azure Data Solution is desirable but not essential
Key Business Capabilities:
- Continuous Improvement: Lead in adopting industry best practices and staying updated with technical and compliance requirements.
- Communication: Strong interpersonal skills, especially under pressure. Ability to explain technical topics in simple terms.
- Technology and Data: Expertise in developing and implementing high-quality technology solutions and processes in a matrixed organisation.
- Customer Excellence: Commitment to continuous improvement and challenging the status quo to enhance business growth and customer experience.
- Collaboration: Team player with the ability to influence and align stakeholders and partners, build consensus, and support major initiatives.
Join us and play a pivotal role in driving our mission forward with cutting-edge data solutions!
We are proud supporters of Women in Data®. Connect, engage and belong to the largest free female data community in the UK – visit: www.womenindata.co.uk to join our community.
“Stay connected! Follow us on LinkedIn for updates on career opportunities and more.”