Solution Architect

This role requires strong technical knowledge of enterprise data platforms, excellent stakeholder management skills, and proven experience delivering complex projects within the banking and financial services industry.

• 10+ years of experience.

• 5+ years managing Enterprise Data Lake, Data Warehouse or Big Data delivery initiatives.

• Proven experience in Hadoop/Cloudera environments.

• Experience managing multiple concurrent project deliveries and BAU portfolios.

• Banking and Financial Services experience preferred.

Data Engineering & Integration

· Design, develop, and maintain scalable data pipelines within the Enterprise Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.

· Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.

· Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.

· Design and implement data ingestion patterns supporting structured, semi-structured, and unstructured datasets.

· Build and maintain batch and near real-time data integration solutions.

· Implement source-to-target mappings and ingestion specifications.

Hadoop & Cloudera Platform Development

· Develop and support solutions within the Cloudera Data Platform ecosystem.

· Build and maintain data pipelines leveraging:

· Hadoop HDFS

· Hive

· Impala

· Spark

· Kafka

· YARN, Oozie

· Perform deployment, troubleshooting, and performance tuning of applications running on the Hadoop ecosystem.

· Partner with platform teams to ensure optimal cluster utilization and workload management.

Data Quality & Governance

· Ensure adherence to Data Lake onboarding standards and governance requirements.

· Investigate and resolve data quality issues and production incidents.

· Ensure compliance with enterprise data governance standards.

DevOps & Platform Automation

· Integrate data engineering solutions into CI/CD pipelines using Git, Jenkins, Azure DevOps, or equivalent tools.

· Automate deployment, testing, monitoring, and operational support activities.

· Develop reusable frameworks and utilities to improve engineering productivity and platform scalability.

Production Support & Incident Management

· Provide L3 support for Data Lake application.

· Perform root cause analysis for production incidents and implement permanent fixes.

· Monitor pipeline execution, cluster health, and workload performance.

· Participate in release implementation, deployment validation, and post-production support activities.

· Strong hands on experience with Hadoop ecosystem and data pipeline orchestration.

· Hadoop/HDFS

· Hive

· Spark

· Unix/Linux scripting

· Shell scripting

· ETL/ELT development experience.

· Data modelling fundamentals.

· Performance tuning and optimization.

· Experience in Data Warehousing and Data Lake environments.

· Strong Java / Python competency with experiences.

· Strong SQL development skills.