Senior Data Engineer

AWS Data Engineer

<div><strong style="color: rgb(239, 72, 21);"><u>Role Description:</u></strong></div><div><br></div><div class="ql-align-justify"><span style="color: rgb(64, 64, 64);">We are seeking a skilled professional to maintain and support batch jobs in a legacy environment. The role involves managing and monitoring ETL processes, addressing issues, and enhancing existing PL/SQL scripts. The ideal candidate will have strong expertise in Informatica, SQL Server, and data warehousing concepts, along with experience in troubleshooting and improving batch job performance.</span></div><div><strong style="color: rgb(239, 72, 21);"><u>&nbsp;</u></strong></div><div><strong style="color: rgb(239, 72, 21);"><u>Key Responsibilities:</u></strong></div><div><br></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Design and&nbsp;implement robust ETL pipelines using AWS Glue, Lambda, Redshift and S3.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Monitor and optimize the performance of data workflows and batch processing jobs.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Troubleshoot and resolve issues related to data pipeline failures, inconsistencies, and performance bottlenecks.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Collaborate with cross-functional teams to define data requirements and ensure data quality and accuracy.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Develop and maintain automated solutions for data transformation, migration, and integration tasks.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implement best practices for data security, data governance, and compliance within AWS environments.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Continuously improve and optimize AWS Glue jobs, Lambda functions, and S3 storage management.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Maintain comprehensive documentation for data pipeline architecture, job schedules, and issue resolution processes.</span></div><div><span style="color: black;">&nbsp;</span></div><div><strong style="color: rgb(239, 72, 21);"><u>Required Skills and Experience:</u></strong></div><div><br></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Strong experience with Data Engineering practices.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Experience in AWS services, particularly AWS Glue, Lambda, S3, and other AWS data tools.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Proficiency in SQL, python , Pyspark, numpy etc and experience in working with large-scale data sets.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Experience in designing and implementing ETL pipelines in cloud environments.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Expertise in troubleshooting and optimizing data processing workflows.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Familiarity with data warehousing concepts and cloud-native data architecture.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Knowledge of automation and orchestration tools in a cloud-based environment.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Strong problem-solving skills and the ability to debug and improve the performance of data jobs.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Excellent communication skills and the ability to work collaboratively with cross-functional teams.</span></div><div><span style="color: black;">·&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Good to have knowledge of DBT &amp; Snowflake</span></div><div class="ql-align-justify"><br></div><div class="ql-align-justify"><strong style="color: rgb(239, 72, 21);"><u>We are an Equal Opportunity Employer:</u></strong></div><div class="ql-align-justify"><span style="color: black;">We value diversity at Incedo. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.</span></div>