Senior Data Engineer - Data Insights (Remote)

• BS or MS in computer science or a related technical field.
• Proven experience as a big data engineer or a similar role, with a deep understanding of big data technologies, frameworks, and best practices.
• Designing and implementing large-scale, distributed data processing systems using technologies such as Apache Hadoop, Apache Spark, or Apache Flink.
• Developing and optimizing data pipelines and workflows for ingesting, storing, processing, and analyzing large volumes of structured and unstructured data.

• Building and maintaining data infrastructure, including data lakes, data warehouses, and real-time streaming platforms.
• Designing and implementing data models and schemas for efficient data storage and retrieval.
• Ensuring the scalability, availability, and fault-tolerance of big data systems through proper configuration, monitoring, and performance tuning.
• Identifying and evaluating new technologies, tools, and frameworks to improve the efficiency and effectiveness of big data processing.
• Implementing data security and privacy measures to protect sensitive information throughout the data lifecycle.
• Knowledge of distributed computing principles and technologies, such as HDFS, YARN, and containerization (e.g., Docker, Kubernetes).
• Understanding of real-time streaming technologies and frameworks, such as Kafka (Confluent) or Apache Flink.
• Strong problem-solving skills and ability to optimize and tune big data processing systems for performance and scalability.
• Excellent communication and teamwork skills to collaborate with cross-functional teams and stakeholders.

• Leadership skills: guiding and mentoring junior members, coordinating projects and collaborating with other teams.
• Problem solving: Dealing with complex challenges, optimizing data pipelines and ensuring data quality and troubleshooting issues.
• Domain Knowledge:Having understanding of the hospitality industry can greatly enhance your ability to design and build effective data pipelines.
• Data Architecture Expertise: Contributing to designing data architecture and system and having deep understanding of data storage and ability to design a robust, scalable and maintainable architecture is a valuable skill.
• Performance Optimization: Being able to optimize the performance of data pipelines is a crucial skill such as query optimization, indexing, caching and data partitioning and sharding.
• Data Governance and compliance: having an understanding of different data governance principles and compliance regulations such as GDPR and HIPAA is very important.
• Programming knowledge: Knowledge of Python and SQL is essential, familiarity with Java and other languages is a plus.
• Experience working with a remote-first and globally distributed team
• Experience with CI/CD tooling, including GitHub Actions and Build Workflows
• Knowledge of Confluent and AWS
• Knowledge of Spotify orgs and terminology

Cloudbeds