Senior DataBricks Developer
DEW Softech
- McConnellsburg, PA
- Permanent
- Full-time
- Work closely with Software Designers to ensure best practices are being followed and offer suggestions for improvements to code proficiency and maintainability
- Occasionally interface with customers to assist with analyzing user needs to determine technical requirements
- Design, build, and maintain scalable and reliable data pipelines using DataBricks
- Develop high-quality code with an emphasis on performance, scalability, and security
- Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs
- Implement data transformations and complex algorithms within the DataBricks environment
- Optimize data processing and refine data architecture to improve system efficiency and data quality
- Mentor junior engineers and contribute to the development of best practices within the team
- Stay up-to-date with emerging trends and technologies in data engineering and cloud computing
- Required:
- 7+ years of experience in data engineering or a related field
- Strong hands-on experience with DataBricks, including development of code, pipelines, and data transformations
- Proficient in at least one programming language (e.g., Python, Scala, Java).
- In-depth knowledge of Apache Spark and its integration within DataBricks
- Experience with cloud services (AWS, Azure, or GCP) and their data-related products
- Familiarity with CI/CD practices, version control (Git), and automated testing.
- Excellent problem-solving skills and the ability to work independently or as part of a team
- Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field
- Ability to safely and successfully perform the essential job functions consistent with the ADA and other federal, state and local standards
- Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc.
- Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor