Senior Data Engineer
Algolia
- Paris
- CDI
- Temps-plein
- Developing data pipelines and ETL workflows mainly with Python, Airflow (AWS MWAA), Spark (AWS Glue, EMR), other AWS services managed with Terraform.
- Improving and maintaining our data warehouse (AWS Redshift).
- Support the Analytics Engineering team to build data models with dbt to be used by Analysts on business reports.
- Interacting with Engineering and Business teams to understand requirements.
- Design, build, enrich, scale up our data pipelines.
- Work with engineers, data analysts, business analysts to capture and model data.
- Monitor data integrity and growth.
- Help improve our data lake and data warehouse architecture, to increase performance, simplicity and autonomy of the users.
- Help analysts industrialise reports and dashboards to improve company productivity.
- Ensure every product is released with a data-driven approach.
- Strong experience designing and building data pipelines.
- Experience orchestrating pipelines with Airflow.
- Professional knowledge of Python.
- Strong experience working with cloud platforms and architecting them.
- Strong experience with data warehouses.
- Experience about infrastructure topics specific to data engineering.
- Interest in Big Data challenges (~600Tb of data, +14Tb per week, ~700M files, +1,2M per week, ~300 dbt models, ~70 Airflow DAGs)
- Interest in understanding the data and business requirements.
- Excellent spoken and written English skills.
- You are humble, curious, proactive, and strike a balance between creativity, resourcefulness, and pragmatism.
- Experience using and managing AWS Redshift.
- Experience with a parallel data processing framework such as Apache Spark.
- Experience analysing data quality using dbt.
- Experience at our current stage and beyond ($50-200M ARR range, high growth, lots of change and building internal infrastructure).
- Required