Specific Requirements at a Glance 15 years of experience having architected multiple data solutions on Google Cloud primarily, like Data Warehouse, Data Lake, Data Lakehouse, etc. (experience in designing Data Mesh would be a huge plus) Good data modeling skills, having modelled Common Data Platforms for enterprises Understanding of data integration / ETL / ELT pipelines having architected complex scenarios Good understanding of Data Governance concepts as they apply to a Data Lakehouse and BI architecture Architecting solutions using Databricks (Unity Catalog, Delta Lake experience is a big plus ) or Snowflake Architecting solutions using AWS stack or GCP - BigQuery, GCS, Dataproc, Dataplex, Pub-Sub, etc. (Dataplex experience would be a huge plus) Experience on other clouds would be appreciated; however, Google Cloud experience has to be primary Job brief We are looking for a Data Architect to build, optimize and maintain data lake architectures. The work includes Data Transformation using Cloud platforms and Data Lakehouse design and enhancement to better enable downstream BI applications using emerging technologies and standard Data Models. We are looking for an experienced leader who herself / himself has utilized optimal methods to transform raw data into useful decision support systems to align with the business goals. A detail-oriented person, with excellent organizational skills, who has strong analytical skills, and the ability to combine data from different sources. Familiarity with several programming languages like Python, Scala, having worked on multiple cloud platforms and possessing knowledge of AI/ML technologies & algorithms and their applications. An architect who has specifically designed Data Lakes on GCP while ensuring Metadata Management, operational cost reduction, data standardization, Data Governance Security, Scalability and Performance. Primary Expectations data systems and pipelines upon evaluate business needs and objectives complex data analysis on raw source data and design data lake / model / storage to house it in a standardized manner with data scientists and architects on several projects and design solutions for use by the Data Science teams lead a team of 15+ developers, engineers, scientists to design complex data lake solutions on experience in developing ETL workflows and pipelines using Apache Airflow , SAP BODS or similar tools in working on cloud technologies GCP / AWS / Azure especially related to data & analytics on Big Data technologies especially, Spark, NoSQL, HDFS expertise with data models and architecture, data migration and managing data quality on and efficient in writing complex SQL queries for all test cases (Unit / System / functional / Data reconciliation / etc.) understanding of BI & DWH development methodologies Experience with Data governance (Data Quality, Metadata Management, Security etc.) good knowledge of SAP ERP data , e.g., HR / Payroll / SuccessFactors, Finance, Procurement, etc. Data Lake skills 8 to 14 years of experience in the field of Data & Analytics, Cloud Data Management, Data Warehousing, etc. at least 3 years of experience working as a lead on a large-scale Data Lake implementation organizational and time management with ability to manage multiple priorities to accomplish the objectives and goals anticipating and adjusting for problems and interruption / roadblocks understanding and experience practicing Agile and iterative software development methodologies numerical and analytical skills Technology Experience (AWS or GCP) AWS specific - Experience with AWS is a must and should have strong development experience in working with Lambda's/Cloud Functions, Cloud Storage / S3 Buckets, Compute Engine, EMR, RDS, Redshift / Athena, Glue / DataBrew / Step functions etc. Data transfer between S3 / Redshift Data transfer between Data Factory / Synapse / SQL Airflow and / or other data orchestration platforms using Python / PySpark, Scala, Java, etc. GCP Specific - Big Query (must have) Excellent knowledge on RDBMS SQL / No SQL & BQ SQL concepts Hands on knowledge on writing complex SQL joins and operations Hands on knowledge on Stored procedure / Functions / Triggers etc. Hands on knowledge on Dynamic SQL and code conversion. Experience on writing cost effective Big Query SQL code Other GCP Tools (should have worked on at least 2 from the list) Hands on knowledge using Database migration tasks Hands on knowledge using Cloud function Hands on knowledge using Data Transfer Services Hands on knowledge using Dataproc Hands on knowledge using Pub/Sub Hands on knowledge using Data Flow Hands on knowledge managing GCS Buckets Hands on knowledge working with Cloud Composer Cloud Data Fusion (good to have) Data Integration/Migration (JDBC) drivers with Cloud Data Fusion pipelines. Multi-cloud integration (AWS / GCP / Azure) Knowledge of CDAP 6.8.0 Data fusion governance Realtime/Batch data integration Hands on REST APIs, time-based schedules, pipeline state-based triggers, logs, metrics. Educational Qualifications in Computer Science, IT, or similar field; a Master's is a plus Architect / engineering certification (e.g., AWS / GCP certification) is a plus
