Job Summary:
We are looking for a skilled Data Engineer with strong hands-on experience in AWS to design, build, and maintain scalable data pipelines and cloud-based data platforms. The ideal candidate will have expertise in modern data warehousing, ETL/ELT development, and distributed data processing while ensuring data quality, performance, and security.
Key Responsibilities:
· Design, develop, and maintain scalable data pipelines using AWS services.
· Build and optimize ETL/ELT workflows for structured and unstructured data.
· Implement data lakes and data warehouses on AWS.
· Work with large datasets to ensure high performance, reliability, and data integrity.
· Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
· Perform data modeling for analytical and operational use cases.
· Ensure data quality, governance, and security best practices.
· Monitor and troubleshoot data workflows and production issues.
· Support CI/CD and automation for data platform deployments.
Required Skills & Experience:
· Strong hands-on experience with AWS services such as S3, Glue, Redshift, Lambda, EMR, Athena, and RDS.
· Proficiency in Python and SQL for data processing and analysis.
· Experience in building and optimizing ETL/ELT pipelines.
· Solid understanding of data warehousing and data lake architecture.
· Experience with Apache Spark / PySpark.
· Knowledge of workflow orchestration tools (Airflow or similar).
· Familiarity with streaming frameworks (Kinesis/Kafka) is a plus.
· Experience with data modeling and performance tuning.
· Understanding of DevOps, CI/CD, and infrastructure as code (Terraform/CloudFormation).
· Experience working in Agile/Scrum environments.
Good to Have:
· Experience with Snowflake on AWS.
· Exposure to real-time data processing.
· AWS certification (e.g., AWS Certified Data Analytics / Solutions Architect).
Soft Skills:
· Strong problem-solving and analytical skills.
· Excellent communication and stakeholder management.
· Ability to work in a fast-paced, collaborative environment.