Land the Job Find the Person

Senior Data Engineer

We are seeking a Senior Data Engineer for a long term consulting engagement to lead the development and maintenance of cross-cloud provider integrations between our legacy and future-state applications. The ideal candidate will be responsible for building, maintaining, and future-proofing our data pipeline, with a focus on seamlessly integrating our core AWS-based applications with our new Dynamics 365 CRM system in Azure.

What You’ll Do:
• Design and implement data integration solutions between AWS and Azure, with a focus on connecting our core AWS applications to Dynamics 365
• Develop and maintain ETL processes using tools like Glue and Lambda functions
• Create and optimize data pipelines for ongoing replication and maintenance between cloud providers
• Work closely with the product manager, business analyst, and solutions architect to develop technical designs and solutions
• Automate and optimize processes across both AWS and Azure environments
• Assess, diagnose & remediate development issues and problems
• Participate in code reviews and implement best practices for cross-cloud development
• Write well-designed, testable, and efficient code, with a focus on Python
• Build reusable components and libraries for data integration
• Implement and adhere to industry best practices and development trends
• Promote collaboration and teamwork in a geographically distributed development environment

Who You Are (Required Skills and Experience):
• 3-5 years of experience in cross-platform data migration, specifically between AWS and Azure
• Strong understanding of both AWS and Azure architectures and services
• Expertise in Python programming language
• Experience with ETL tools such as Glue and Lambda functions
• Proficiency in SQL and database technologies, particularly PostgreSQL and Redshift
• Understanding of Microsoft Dynamics 365 data structures and integration patterns
• Experience in creating and maintaining pipelines to support data replication across multiple applications and platforms
• Ability to create and maintain technical documentation, including architectural diagrams, data flows, database mapping, data dictionaries, and ERDs
• Knowledge of big data technologies and data warehousing solutions

Nice to Have:
• Experience with Kafka and Airflow
• Familiarity with NoSQL databases
• Understanding of designing and building high-volume serverless solutions in AWS
• Knowledge of the software development lifecycle and concepts such as Agile, SAFe, Scrum, CI/CD, and DevOps
• Experience with AWS Database Migration Service (DMS)
• Familiarity with CRM/Finance/Accounting systems
• AWS Certification

Key Responsibilities:
• Lead the integration efforts between our AWS-based core applications and Dynamics 365 CRM
• Ensure seamless data flow and synchronization between AWS and Azure environments
• Develop and maintain robust data validation processes for cross-cloud integrations
• Continuously optimize and improve data replication and integration processes
• Stay up-to-date with the latest developments in both AWS and Azure technologies
• Collaborate with cross-functional teams to understand and implement business requirements in data integration solutions

This role offers an exciting opportunity to work at the forefront of cross-cloud integration, bridging the gap between AWS and Azure environments while supporting our transition to Dynamics 365 as our future CRM solution. The ideal candidate will have a strong background in both AWS and Azure, with the ability to navigate the complexities of data integration across these platforms.

**No c2c



Apply now

"*" indicates required fields

Accepted file types: pdf, doc, docx, txt, Max. file size: 5 MB.