Data Engineer

🎯 Job Summary

We are looking for an experienced Data Engineer with 7+ years of experience in designing, developing, and maintaining scalable data pipelines and data infrastructure. The candidate will be responsible for building ETL/ELT processes, managing large datasets, optimising data workflows, and supporting data analytics and business intelligence initiatives.

🔧 Key Responsibilities

• Design, develop, and maintain scalable data pipelines and data integration solutions.

• Build and manage ETL/ELT workflows for structured and unstructured data.

• Develop data models and optimise data warehouse solutions.

• Integrate data from multiple sources, including databases, APIs, and cloud platforms.

• Implement data processing solutions using batch and real-time streaming technologies.

• Ensure data quality, accuracy, consistency, and reliability.

• Work with data scientists, analysts, architects, and business stakeholders.

• Develop and optimise SQL queries and database performance.

• Implement data governance, security, and access control practices.

• Monitor, troubleshoot, and resolve data pipeline failures and performance issues.

• Work with cloud-based data platforms and distributed processing frameworks.

• Maintain technical documentation and follow software development best practices.

💻 Technical Skills & Requirements

• Strong proficiency in SQL and Python.

• Hands-on experience with ETL/ELT tools and data pipeline development.

• Experience with Apache Spark, PySpark, and Hadoop.

• Knowledge of Apache Airflow or similar workflow orchestration tools.

• Experience with data warehouses such as Snowflake, Amazon Redshift, Google BigQuery, or Azure Synapse.

• Hands-on experience with AWS, Microsoft Azure, or Google Cloud Platform.

• Knowledge of relational and NoSQL databases.

• Experience with data integration, data modelling, and data transformation.

• Familiarity with Kafka or other real-time data streaming technologies.

• Experience with Git, CI/CD, and automated deployment processes.

• Understanding of data governance, data security, and data quality frameworks.

🎓 Education & Experience

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related discipline.

• Minimum 7 years of relevant experience in Data Engineering.

• Proven experience developing and managing enterprise-scale data pipelines.

• Experience working with cloud data platforms and big data technologies.

🤝 Required Skills

• Strong analytical and problem-solving abilities.

• Excellent SQL and programming skills.

• Ability to handle large and complex datasets.

• Strong communication and collaboration skills.

• Ability to work independently and manage multiple priorities.

⭐ Preferred Certifications

• AWS Certified Data Engineer.

• Microsoft Certified: Azure Data Engineer Associate.

• Google Cloud Professional Data Engineer.

• Databricks Data Engineer Certification.