Senior Data Engineer
ICONMA
- Richmond, VA
- Permanent
- Full-time
Duration: 12 months with possible contract to hire/ with possible extensionDescription:
Software Engineer responsible for developing and maintaining data processing application which will handle data from small to large volumes in terabytes of business critical enterprise data.
Must have 4+ years of developing data centric application using Spark/Scala or Python running on EMR or AWS Glue.
Must have data pipeline orchestration experience using self created pipelines or using AWS step functions handling pipeline using Python based AWS Lambda.
Must have 3+ years experience in utilizing AWS cloud service.
4+ years of experience in CICD pipelineCall notes:
Team Info:
EDML Enterprise data - Overall Data Space
An application doing large volume of data transformation
Tech stack is on AWS utilizing Glue, S3, stepfunctions, lambdas, ECS/Fargate, SQL, Scala, Python & Spark
Automation Test suits, Spark based SDKs & Python based Lambdas
Stepfunctions for orchestrations
Lambdas executed by stepfunctions
ECS for elastic services & server hosted
This app is a data centric and software utilizing is for Automation
Team of 4 EngineersRole Info:
Hands on Sr. Data Engineer roles with strong Spark, Scala, Python, SQL & AWS Background
80% Data Engineering & 20% is on automation side (Python based Behave & Batch side Scala)Top Skills:
Spark exp is a must
Scala & Python exp is a must
SQL exp is a must