Senior Data EngineerHybridHyderabad, Telangana, India

Overview

Application

Senior Data Engineer with AWS

Key Responsibilities:

· Design and implement robust ETL pipelines using AWS Glue, Lambda, and S3.

· Monitor and optimize the performance of data workflows and batch processing jobs.

· Troubleshoot and resolve issues related to data pipeline failures, inconsistencies, and performance bottlenecks.

· Collaborate with cross-functional teams to define data requirements and ensure data quality and accuracy.

· Develop and maintain automated solutions for data transformation, migration, and integration tasks.

· Implement best practices for data security, data governance, and compliance within AWS environments.

· Continuously improve and optimize AWS Glue jobs, Lambda functions, and S3 storage management.

· Maintain comprehensive documentation for data pipeline architecture, job schedules, and issue resolution processes.

Required Skills and Experience:

· Strong experience with Data Engineering practices.

· Experience in AWS services, particularly AWS Glue, Lambda, S3, and other AWS data tools.

· Proficiency in SQL, python , Pyspark, numpy etc and experience in working with large-scale data sets.

· Experience in designing and implementing ETL pipelines in cloud environments.

· Expertise in troubleshooting and optimizing data processing workflows.

· Familiarity with data warehousing concepts and cloud-native data architecture.

· Knowledge of automation and orchestration tools in a cloud-based environment.

· Strong problem-solving skills and the ability to debug and improve the performance of data jobs.

· Excellent communication skills and the ability to work collaboratively with cross-functional teams.

· Good to have knowledge of DBT & Snowflake

Preferred Qualifications:

Location: Candidates should be based in Gurgaon/Hyderabad