Senior Data Engineer (AWS) IRC223874
GlobalLogic
- Kraków, małopolskie
- Stała
- Pełny etat
- Solid experience as a Data Engineer;
- Good understanding of AWS components e.g. Sagemaker, SecretsManager, Dynamo, Glue, Redshift, Hive, GIT, Elasticbeanstalk, ECR / ECS, EC2, SQS, Lambda, S3, Cloudwatch, ELB, Elasticache, Step functions, Kinesis, Kafka.
- Solid knowledge of Python language;
- Knowledge and experience of writing ETLs in Scala;
- Expertise in Spark coding and Functional Programming;
- Advanced SQL coding;
- Advanced knowledge and working experience of MySQL, Postgres, MS SQL 2016;
- Advanced knowledge of Linux tools;
- Good understanding of SDLC life-cycle (Dev, test, preprod, production and the CI/CD pipeline);
- A good level of English (written/verbal) for daily interaction with the team and the customer.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and cloud-based ‘big data’ technologies from AWS.
- Create tool-chains for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Create and maintain optimal data and model data Ops pipeline architecture
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.