Data Engineer
We are looking for a Data Engineer to join our dynamic team.
Key Responsibilities
- Set up and manage Kafka clusters and Kafka Connect to handle real-time data flows.
- Monitor and support streaming data pipelines, ensuring reliability, scalability, and performance.
- Build and maintain data models in Snowflake to support reporting and analytics needs.
- Optimize Snowflake usage for scalability and performance across teams.
- Design and develop ETL workflows using AWS Glue to process both structured and unstructured data.
- Utilize the AWS Glue Data Catalog to organize metadata and support querying through Amazon Athena.
- Implement best practices for organizing and securing data stored in AWS S3.
- Create and manage data workflows in Apache Airflow, ensuring timely and reliable ETL execution.
- Troubleshoot failures and continuously improve orchestration processes.
- Support data quality, governance, and access control initiatives to maintain trust in data assets.
- Collaborate with data analysts, data scientists, and business stakeholders to deliver reliable and usable data.
- Stay up to date with evolving tools and best practices in cloud data engineering.
Candidate Profile
- Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.
- 2–3 years of hands-on experience in data engineering, with a focus on cloud platforms.
- Practical experience with:
- Kafka for real-time data processing.
- Snowflake for data warehousing.
- AWS Glue, Athena, and S3 for data processing and storage.
- Apache Airflow for workflow orchestration.
- Proficiency in SQL, Python, and shell scripting.
- Solid understanding of data modeling and cloud security principles.
Preferred Qualifications
- Exposure to Microsoft Azure or Google Cloud Platform (GCP).
- Experience with CI/CD tools such as Jenkins or Git.
- Familiarity with Docker or basic containerization concepts.