Data Engineer II (R-16185)
Dun & Bradstreet
- Hyderabad, Telangana
- Permanent
- Full-time
- The data engineer will report directly to the Data & Operations leader for Canada.
- Responsibilities will be
- Work with our data and operations team to improve the overall quality of data delivered to customers
- Develop data ingestion and mastering processes on new structured and unstructured data sources to synthesize new insight and enhance production database
- Identification, design and implementation of process improvements including redesigning current infrastructure/process for greater scalability
- Automation of manual processes
- Building required infrastructure to support development and production environments using AWS, S3, GCP, No Sql and SQL technologies
- Building new analytical processes providing insight to data quality issues and implementation of data improvement solutions
- Build processes to fetch data from open data sources, eg web sites
- Analysis, development, and implementation: Work independently and with the team to identify data quality and supply chain issues. Determine and implement solution
- Operations: Ensure that DIME (automated data ingestion and mastering engine) operates optimally on a daily basis
- Data ingestion: Map new data sources into DIME and build automated ingestion process
- Infrastructure optimization and enhancements: Identify and implement efficiency improvements in our current data maintenance processes; migrate current DIME installation to AWS.
- Bachelor's degree in computer science, data science, information systems, or other related field or equivalent work experience.
- 3+ years work experience on large data projects - defining and implementing data analysis and improvement processes in a MS-SQL Server environment
- Ability to perform root cause analysis on external and internal processes and data to identify data quality improvement opportunities
- Analytical skills associated with working on large structured and unstructured datasets
- Required technical proficiency - Microsoft SSIS and Sql Server, Python, C#, writing and tuning SQL queries, stored procs and functions
- Additional beneficial experience/skills - Graph database technology, AWS, GCP, No Sql, Sql Server administration, R, machine learning, web technologies, data mapping from multiple data sources
- Strong organization skills with attention to detail
- Ability to clearly communicate problem statement and solution Able to work independently and willing to learn new skills through independent learning