isCompletedInterview

jobTitle

location

productDomainEngineering

skills

yearsOfExperience

Tech Lead- Data Engineering

Technical Architect/Manager- Data Engineering

Job Title: Technical Architect/Manager- Data Engineering Job Summary:-We are looking for a highly skilled and motivated Technical Architect/Manager to spearhead the development of scalable and efficient data engineering solutions. The ideal candidate will possess deep expertise in Python, PySpark, AWS services, and streaming data platforms, with a proven ability to integrate complex data sources and develop distributed data processing frameworks. This role requires a strong technical leader who can guide the team, solve complex challenges, and deliver optimal solutions that align with client requirements. Wealth management experience is an added benefit. Key Responsibilities:<ol><li>Technical Leadership:</li></ol><ul><li class="ql-indent-1">Provide hands-on technical leadership to the team in designing and implementing data engineering solutions.</li><li class="ql-indent-1">Lead by example in adopting best practices for coding, testing, and deployment.</li></ul><ol><li>ETL Development:</li></ol><ul><li class="ql-indent-1">Design and develop robust ETL pipelines using AWS Glue, Lambda, and other AWS services to process large volumes of data efficiently.</li><li class="ql-indent-1">Implement complex data transformations and integrate data from multiple sources such as APIs, databases, and streaming platforms.</li></ul><ol><li>Streaming Data Processing:</li></ol><ul><li class="ql-indent-1">Design and implement streaming data pipelines using Kafka, AWS Kinesis, or similar technologies.</li><li class="ql-indent-1">Build scalable frameworks to handle real-time data ingestion and processing.</li></ul><ol><li>Distributed Data Processing:</li></ol><ul><li class="ql-indent-1">Develop distributed data processing frameworks to ensure performance and scalability in handling large datasets.</li><li class="ql-indent-1">Optimize the performance of data processing jobs for both batch and real-time workloads.</li></ul><ol><li>Solutioning &amp; Architecture:</li></ol><ul><li class="ql-indent-1">Provide optimal data engineering solutions aligned with client requirements and business objectives.</li><li class="ql-indent-1">Collaborate with architects to design scalable and secure data solutions leveraging AWS cloud services.</li></ul><ol><li>AWS Expertise:</li></ol><ul><li class="ql-indent-1">Utilize AWS services (e.g., S3, Glue, Lambda, Kinesis, DynamoDB) to build efficient and scalable cloud-based solutions.</li><li class="ql-indent-1">Stay updated with the latest AWS services and features to continuously improve system performance and cost efficiency.</li></ul><ol><li>Stakeholder Collaboration:</li></ol><ul><li class="ql-indent-1">Work closely with clients, business analysts, and other stakeholders to understand requirements and translate them into technical solutions.</li><li class="ql-indent-1">Communicate progress, challenges, and solutions effectively to both technical and non-technical stakeholders.</li></ul> Qualifications:<ul><li>Education:</li><li class="ql-indent-1">Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.</li><li>Experience:</li><li class="ql-indent-1">10+ years of experience in data engineering, with at least 3+ years in an Architect role.</li><li class="ql-indent-1">Strong hands-on expertise in Python, PySpark, and AWS services for data processing and integration.</li><li class="ql-indent-1">Proven experience designing and developing streaming data solutions using Kafka, AWS Kinesis, or similar technologies.</li><li class="ql-indent-1">Solid experience working with APIs and integrating data from diverse sources.</li><li class="ql-indent-1">Extensive knowledge of distributed data processing frameworks and best practices.</li><li>Skills:</li><li class="ql-indent-1">Strong problem-solving and solution-oriented mindset to deliver optimal results.</li><li class="ql-indent-1">Excellent knowledge of data integration techniques and cloud-based architecture.</li><li class="ql-indent-1">Proficient in implementing complex data transformations and scalable data workflows.</li><li class="ql-indent-1">Exceptional team leadership and mentoring abilities.</li><li class="ql-indent-1">Strong communication skills for effective stakeholder collaboration.</li></ul> Preferred Qualifications:<ul><li>AWS Certified Solutions Architect certification or equivalent.</li><li>Familiarity with Terraform or CloudFormation for AWS infrastructure as code.</li><li>Wealth Management domain experience.</li></ul> Why Join Us?<ul><li>Be a key player in building cutting-edge data engineering solutions for large-scale projects.</li><li>Work with a talented and collaborative team in a dynamic environment.</li><li>Competitive salary, benefits, and opportunities for career growth.</li></ul>&nbsp;