Lead Big Data Engineer with Databricks

hace 4 semanas


Desde casa, México EPAM Systems, Inc. A tiempo completo

We are looking for a skilled** remote Lead Big Data Engineer with Databricks** experience to join our team.Your primary responsibility will be to lead the development of our ETL pipelines on Databricks using Spark/Scala, Databricks, and AWS. You must have extremely strong experience with Databricks, AWS, Python, and Elastic.**Responsibilities**- Lead the development of ETL pipelines on Databricks using Spark/Scala, Databricks, and AWS- Work closely with cross-functional teams to define data requirements- Develop and maintain scalable data infrastructure- Ensure the reliability, efficiency, and scalability of data pipelines- Coach and mentor other members of the team**Requirements**:- At least 5 years of experience in Big Data Engineering- 1+ years of relevant leadership experience- Expertise in Databricks, AWS, Python, and Elastic- Deep understanding of Spark/Scala and data architecture principles- Knowledge of data models, ETL design, implementation, and maintenance- Strong communication skills and ability to collaborate with cross-functional teams- B2+ English level**Nice to have**- Experience with Apache Airflow- Familiarity with Agile methodologies and SDLC practices**We offer**- International projects with top brands- Work with global teams of highly skilled, diverse peers- Healthcare benefits- Employee financial programs- Paid time off and sick leave- Upskilling, reskilling and certification courses- Unlimited access to the LinkedIn Learning library and 22,000+ courses- Global career opportunities- Volunteer and community involvement opportunities- EPAM Employee Groups- Award-winning culture recognized by Glassdoor, Newsweek and LinkedInEPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.



  • Desde casa, México EPAM Systems, Inc. A tiempo completo

    **Responsibilities** - Develop and maintain reliable data pipelines for ingesting, transforming, and storing data from multiple sources - Manipulate complex datasets and preprocess large-scale logs and time-series data using state-of-the-art tools and frameworks - Leverage your knowledge of Python (Pandas, NumPy, etc.) to perform exploratory data analysis...


  • Desde casa, México EPAM Systems A tiempo completo

    **DESCRIPTION**: Join EPAM as a **Senior Data Engineer specializing in Azure Databricks.** In this role, you'll focus on building a metadata-driven framework for the transformation layer in Azure Databricks and migrating existing jobs to it. If you have hands-on coding experience with Databricks Jobs using Python and PySpark, experience in Azure cloud data...


  • Desde casa, México EPAM Systems, Inc. A tiempo completo

    We are looking for a Lead Data DevOps Engineer to drive the migration of Spark, Flink, and additional services to Databricks.You will be responsible for creating and managing robust cloud infrastructure to enhance data processing pipelines. Join us to leverage your AWS, Terraform, and Databricks expertise and help deliver cutting-edge data...

  • Sr Data Engineer

    hace 20 horas


    Desde casa, México Framework Science A tiempo completo

    Framework Science is on a MISSION that focuses on Exploring new technologies and building tomorrow’s Applications. This means we hire TOP Engineers and Designers by providing great benefits and pay so they can focus on solving what’s never been solved before. Our aim is to push the needle of innovation while enabling Technical staff to impact code or...

  • Lead Data Engineer

    hace 3 semanas


    Desde casa, México Luxoft A tiempo completo

    **Project** Description**:Luxoft DXC Technology Company is an established company focusing on consulting and implementation of complex projects in the financial industry. At the interface between technology and business, we convince with our know-how, well-founded methodology and pleasure in success. As a reliable partner to our renowned customers, we...

  • Databricks Developer

    hace 6 días


    Desde casa, México U hi A tiempo completo

    **Databricks Developer****Location: Remote**We are seeking a talented Databricks Developer to join our team and contribute to the design, development, and implementation of Databricks solutions. As a Databricks Developer, you will play a crucial role in building and optimizing data pipelines, creating scalable data models, and enabling advanced analytics and...

  • Api Testing

    hace 2 semanas


    Desde casa, México U-Hi A tiempo completo

    **Experience Required**: 6+ yearsAbout the RoleWe are looking for an experienced **API Testing / Databricks Engineer** with strong expertise in functional testing, database validation, and API testing. The role requires solid hands-on experience with Agile projects, automation, and modern data tools such as Databricks, Microsoft Fabric, or Power BI.Key...

  • Databricks Developer

    hace 1 semana


    Desde casa, México U hi A tiempo completo

    **Databricks Developer** **Location: Remote** We are seeking a talented Databricks Developer to join our team and contribute to the design, development, and implementation of Databricks solutions. As a Databricks Developer, you will play a crucial role in building and optimizing data pipelines, creating scalable data models, and enabling advanced analytics...

  • Azure Data Engineer

    hace 4 semanas


    Desde casa, México MSNS HR Technologies A tiempo completo

    **Role: Azure Data Engineer****Location: GDL or Mexico City, Remote/Hybrid****Duration: Fulltime/Permanent****Languages: English Mandatory****Openings: 5+ positions****Responsibilities**:Strong technical SME in Azure components - Azure Functions, ADF, Databricks, EventHub, Synapse, Datawarehouse, Cosmos, Azure SQL, ADLS, SPARK and SCALA programming (ALL BIG...


  • Desde casa, México EPAM Systems, Inc. A tiempo completo

    If you possess advanced expertise in Python-based data workflows, have operated extensively with cutting-edge tools like Databricks, and excel in addressing the complexities of large-scale datasets, we would like to connect with you! **Responsibilities** - Drive the design and governance of enterprise-level data pipelines for scalable ingestion,...