Data Engineer
WEG Technology Recruitment
- Coventry
- £45,395-54,585 per year
- Permanent
- Full-time
- Technical work to model data and Design and Develop Data Pipelines
- Undertake and manage the development of data models
- Contribute to the design of data products and organise the backlog of work in planning tools like Jira
- Utilise Azure Data Factory to design and develop scalable and efficient data pipelines
- Utilise DBT (Data Build Tool) to create and manage data transformation processes, ensuring consistent and reliable data output
- Design, implement and maintain DataVault and Kimball-style data models to ensure efficient storage, retrieval, and analysis of data by different user communities across the university.
- Monitor and optimise the performance of data pipelines and data warehouses, identifying and resolving bottlenecks and inefficiencies.
- Design and implement data security measures
- Design and implement data quality checks and validation processes within the pipelines using expectations with the pipeline, or dedicated testing tools (SODA)
- Collaborate with data governance, architects, system/product owners and end users to ensure quality issues are addressed as needed.
- Create and maintain technical documentation, including data pipeline specifications, data models, and transformation logic
- Contribute to data modelling and schema design standards and related documentation.
- Degree in Computer Science, Data Engineering, or a related field or equivalent experience
- Strong experience working as a Data Engineer, preferably in a cloud-based environment.
- Experience with cloud-based data storage platforms, preferably Snowflake, Azure SQL Data Warehouse or AWS Redshift.
- Solid practical experience and understanding of DataVault and Kimball-style data warehousing methodologies.
- Proficient in SQL and data querying languages for data manipulation and analysis. Proficiency in Azure Data Factory and DBT, with a demonstrated ability to build scalable and reliable data pipelines and transformation processes.
- Familiarity with data modelling concepts and techniques, including dimensional modelling.
- Strong problem-solving and analytical skills, with the ability to analyse complex data-related issues and provide effective solutions.
- Excellent communication and collaboration skills, with the ability to work effectively in a team environment and engage with stakeholders at various levels.
Jobs.ac.uk