Big Data Administrator – L3

We are looking for an experienced Big Data Administrator to join the IT infrastructure and production support team. The candidate will be responsible for the administration, maintenance, monitoring and optimisation of Big Data platforms and associated infrastructure in a production environment.

This is an L3 technical role that will work closely with infrastructure, development, architecture and project teams to maintain system availability, stability, performance and operational readiness.

Key Responsibilities

• Administer, maintain and support Hadoop-based Big Data infrastructure in production environments.

• Build, configure and maintain Hadoop clusters and associated components.

• Perform Hadoop updates, patches, upgrades and cluster deployments.

• Manage cluster nodes, including node addition, removal and configuration.

• Support HDFS backup, restore, file-system management and monitoring activities.

• Perform Hadoop user administration, including Linux user setup, Kerberos authentication and access management.

• Monitor and troubleshoot Hadoop services using system logs, Hadoop logs and monitoring/alerting tools.

• Support and administer Kafka environments and troubleshoot related operational issues.

• Manage and monitor OpenSearch / Elasticsearch clusters.

• Perform capacity planning, performance tuning and operational optimisation of Big Data platforms.

• Monitor cluster connectivity, security and overall system health.

• Review technology changes and identify potential operational risks.

• Participate in the resolution of complex and high-severity incidents and drive issues through to resolution.

• Support production readiness activities for new technologies, projects and solutions.

• Work closely with development, architecture and project teams on technology roadmaps, implementations and transformation initiatives.

• Develop and maintain technical documentation, operational procedures and knowledge-base materials.

• Identify opportunities for service improvement, automation and operational efficiency.

• Develop and maintain shell, Perl or Python scripts for system administration and automation.

• Provide technical guidance and knowledge sharing to team members where required.

• Work collaboratively with cross-functional teams to maintain data availability, reliability and service quality.

Technical Requirements

Essential Skills

• Strong experience in Linux system administration.

• Hands-on experience with Hadoop administration, including HDFS, YARN and related ecosystem components.

• Experience with Redis cluster administration.

• Experience with MongoDB administration.

• Hands-on experience with Kafka administration and understanding of Kafka internals.

• Experience managing OpenSearch / Elasticsearch clusters.

• Strong scripting skills in at least one scripting language, preferably Shell, Perl or Python.

• Experience troubleshooting complex Big Data platform issues using system and application logs.

• Knowledge of system monitoring, performance tuning and capacity management.

• Knowledge of OS administration, including authentication, disk quotas, system limits and security patching.

• Ability to automate software installation, configuration and operational tasks on Linux servers.

• Experience working in large-scale infrastructure or data centre environments.

Good to Have

• Experience with Hadoop ecosystem components such as Hive, Pig, Oozie, HBase, Flume and Sqoop.

• Experience administering Hadoop clusters using YARN-based applications such as Spark.

• Experience with Apache Ambari or similar Hadoop management tools.

• Knowledge of PAM and Kerberos security mechanisms.

• Knowledge of hardware configuration, disk topology and RAID.

• Experience with virtual machine deployment and configuration.

• Knowledge of JVM administration and troubleshooting.

• Proficiency in Python, Java or Scala.

• Proficiency in SQL, Hive or other SQL-on-Hadoop technologies.

• Experience with ETL processes or related technologies.

• AWS cloud certification or relevant cloud experience.

Experience & Qualifications

• Minimum 8 years of relevant experience in system administration, infrastructure support or a related technical environment.

• At least 1–2 years of hands-on experience deploying and administering large-scale Hadoop clusters.

• Bachelor's degree in Computer Science, Information Technology, Engineering or a related discipline.

• Experience in financial services or other highly regulated environments would be an advantage.

• Experience in software development using Java, Python, Perl, PHP, CSS or HTML would be an advantage.

• Experience with Spark development would be an advantage.

Other Requirements

• Strong analytical, troubleshooting and problem-solving abilities.

• Ability to work independently as well as collaboratively within a technical team.

• Ability to manage and resolve moderate to highly complex technical issues.

• Comfortable working in a production environment where timely incident resolution is important.

• Willingness to participate in a rotational on-call support arrangement, where required.
  

  If you believe you fit the requirements for the role, please submit your application below or drop us an email directly quoting the job title.

  Due to an anticipated high volume of applicants, we regret that only shortlisted candidates will be notified. The information provided is for recruitment purposes only.

  Cornerstone Global Partners (EA License Number: 19C9859) is an affirmative equal-opportunity employer and recruitment firm. We evaluate qualified applicants without regard to race, colour, religion, creed, gender, sexual orientation, gender identity, marital status, national origin, age, veteran status, disability, or any other protected class.

  (Scarlett) Zhang Jifan
  [email protected]

  EA Registration Number: R26159541.

  Cornerstone Global Partners Pte Ltd (EA License: 19C9859)