WE ARE HIRING: Engineer – Data Engineering
Location: Sri Lanka
Employment Type: Contract
Company Industry: Software
Role Overview
We are seeking a motivated and technically skilled Engineer – Data Engineering to design, develop, and maintain scalable enterprise data solutions and cloud-based data-processing applications.
The selected candidate will be responsible for building reliable data pipelines, integrating multiple source systems, processing large volumes of data, and supporting solutions throughout the complete software-development lifecycle. The role requires strong programming skills, hands-on cloud experience, and the ability to work effectively within an Agile and DevOps environment.
Key Responsibilities
- Design and develop scalable enterprise-grade data-processing solutions.
- Build distributed and highly available data applications that support large-scale business requirements.
- Design, develop, test, and maintain data pipelines connecting multiple source systems and target platforms.
- Develop ETL and ELT workflows for batch and near-real-time data processing.
- Process, transform, and validate large volumes of structured and unstructured data.
- Develop cloud-native data solutions using Google Cloud Platform services.
- Build data-processing and orchestration workflows using tools such as Airflow and Cloud Composer.
- Work with services such as BigQuery, Dataflow, Dataproc, Datastream, Pub/Sub, Cloud Functions, and Cloud Run.
- Support data-processing solutions using AWS services such as Glue, Lambda, EMR, and Data Pipeline when required.
- Develop data-processing applications using Python, Shell scripting, and SQL.
- Design and optimize solutions using relational databases, NoSQL databases, and distributed storage engines.
- Support streaming-data applications using technologies such as Kafka, Pub/Sub, Spark, or similar tools.
- Contribute to the development of data warehouses, data marts, data lakes, and data-mesh solutions.
- Apply software-engineering best practices to ensure clean, reusable, maintainable, and high-quality code.
- Participate in code reviews, technical design discussions, and solution-architecture activities.
- Implement and maintain Continuous Integration and Continuous Delivery pipelines.
- Use code-management and automation tools such as GitHub, GitLab, Jenkins, or equivalent platforms.
- Follow DevOps principles throughout development, testing, deployment, and production support.
- Monitor application health, data-pipeline performance, and system reliability.
- Investigate and resolve data-processing failures, performance issues, and production incidents.
- Use monitoring platforms such as Datadog or equivalent tools when required.
- Collaborate with software engineers, data engineers, quality engineers, analysts, architects, and product stakeholders.
- Participate in Agile activities, including sprint planning, daily stand-ups, reviews, and retrospectives.
- Support technical alignment across team members and contribute to implementation planning.
- Maintain technical documentation, data mappings, solution designs, and operational procedures.
- Continuously evaluate new data-engineering tools, cloud services, and development practices.
Candidate Profile
- Bachelor's degree in Computer Science, Software Engineering, Information Technology, or an equivalent field.
- Minimum 1 to 2 years of experience developing enterprise-grade data-processing applications.
- Strong programming skills in Python, Shell scripting, and SQL.
- Hands-on experience processing large volumes of data.
- Experience designing and developing ETL or ELT data pipelines.
- Practical experience with relational databases, NoSQL databases, and distributed storage engines.
- Hands-on experience with Google Cloud Platform.
- Experience with GCP services such as BigQuery, Dataflow, Dataproc, Datastream, Pub/Sub, Cloud Functions, Cloud Run, and Cloud Composer.
- Familiarity with ETL and orchestration tools such as Airflow.
- Exposure to AWS data services such as Glue, Lambda, EMR, Spark, Hive, or Data Pipeline will be an advantage.
- Experience working with streaming technologies such as Kafka, Pub/Sub, Storm, or Spark Streaming will be an advantage.
- Good understanding of distributed systems, scalable architectures, and cloud-based data processing.
- Experience working within Scrum and Agile delivery environments.
- Familiarity with DevOps practices and software-development lifecycle processes.
- Experience with source-control and CI/CD tools such as GitHub, GitLab, and Jenkins.
- Exposure to application-monitoring tools such as Datadog or an equivalent platform will be an advantage.
- Understanding of data warehousing, data marts, data lakes, or data-mesh concepts will be preferred.
- Good knowledge of software-engineering standards, testing practices, and code-quality principles.
- Strong analytical, troubleshooting, and problem-solving skills.
- Strong written and verbal communication skills.
- Ability to communicate technical ideas clearly and influence implementation decisions.
- Ability to work collaboratively within cross-functional technical teams.
- Proactive, adaptable, and committed to continuous learning and professional development.
- Strong interest in building and maintaining reliable, high-performance data solutions.
Ready to take your career to new heights?
We're InTalent Asia, your recruitment partner in Sri Lanka, and we've got an exciting opportunity for you! Our client is looking for a dynamic individual to fill the role of Engineer – Data Engineering.
Apply now and see how you can be the perfect fit for this exclusive position!
#InTalentAsia #CareerOpportunity #JobVacancy #DataEngineer #DataEngineering #GoogleCloudPlatform #PythonJobs #CloudData #ETL #ElevateYourCareer
Originally posted on