Lead Cloud Data Engineer

We are looking to add a highly skilled and motivated **Lead Cloud Data Engineer** with strong backend development skills and Cloud platform experience (AWS/GCP) to our team.As a vital team member, you will be charged with designing, implementing, and maintaining our data-centric, Cloud-hosted backend infrastructure and services. You will be pivotal in expanding and optimizing our data processing pipelines and APIs, ensuring they are highly available, secure, and perform efficiently.RESPONSIBILITIES- Optimize data pipelines using ELT frameworks like Fivetran, dbt, Airflow, and PySpark, ensuring data moves and transforms efficiently; schema design and data modeling expertise are crucial- Develop and maintain high-throughput, low-latency backend services for data ingestion, processing, and transformation using languages such as Python and Java- Build and support robust, scalable, and secure RESTful APIs using frameworks like Spring Boot (Java) or Flask/FastAPI (Python); familiarity with API gateway services such as Apigee, Kong beneficial- Provide advice on and design OData APIs- Establish and enforce security protocols including OAuth, SSO, and SSL- Design and execute event-driven architectures utilizing technologies such as Kafka or Pub/Sub- Develop and maintain CI/CD pipelines using tools such as Jenkins, GitLab CI to automate backend service deployment and scaling- Craft and maintain infrastructure-as-code (IAC) utilizing tools like Pulumi, Terraform or CloudFormation- Work collaboratively with Architects and Security teams to ensure Cloud infrastructure and services meet industry standards like SOC 2, HIPAA, GDPR- Diagnose and resolve service-related problems, delivering solutions in a timely and efficient manner- Keep abreast of the latest Cloud technology, backend development methodologies, and security vulnerabilities- Mentor and offer leadership to other team members as required**REQUIREMENTS**:- Bachelor’s degree in Computer Science, Computer Engineering, or a related field- Minimum 6 years of backend development experience in a Cloud environment (AWS or GCP preferred)- Minimum 5 years of Data Engineering experience in a Cloud environment- Proficiency in Python and Java; SQL competence required- In-depth experience with RESTful API design, development, and testing using relevant frameworks; familiarity with OpenAPI/Swagger specifications beneficial- Background in event-driven architectures and message queues like Kafka, RabbitMQ, Azure Event Hub, Amazon MSK- Skill in ELT data processing with frameworks such as Fivetran, dbt, Airflow, and/or PySpark- Proven expertise in containerization (Docker) and orchestration (Kubernetes)- Knowledge of CI/CD pipelines and infrastructure-as-code (Pulumi/Terraform)- Familiarity with Linux, shell scripting, and Git- Deep understanding of cloud security best practices, including IAM and data encryption- Strong problem-solving and analytical capabilities- Excellent written and verbal communication skills- Ability to work effectively within a team environmentWE OFFER- Career plan and real growth opportunities- Unlimited access to LinkedIn learning solutions- International Mobility Plan within 25 countries- Constant training, mentoring, online corporate courses, eLearning and more- English classes with a certified teacher- Support for employee’s initiatives (Algorithms club, toastmasters, agile club and more)- Enjoyable working environment (Gaming room, napping area, amenities, events, sport teams and more)- Flexible work schedule and dress code- Collaborate in a multicultural environment and share best practices from around the globe- Hired directly by EPAM & 100% under payroll- Law benefits (IMSS, INFONAVIT, 25% vacation bonus)- Major medical expenses insurance: Life, Major medical expenses with dental & visual coverage (for the employee and direct family members)- 13 % employee savings fund, capped to the law limit- Grocery coupons- 30 days December bonus- Employee Stock Purchase Plan- 12 vacations days plus 4 floating days- Official Mexican holidays, plus 5 extra holidays (Maundry Thursday and Friday, November 2nd, December 24th & 31st)- Monthly non-taxable amount for the electricity and internet bills