Databricks Developer

hace 6 días

Ciudad de México, Ciudad de México Quadrant Technologies A tiempo completo

Job Description: Senior Databricks Developer

Minimum 5+ Years Experience on Cloud Platforms (AWS, Azure)

Position Overview

We are seeking a highly skilled and experienced Databricks Developer / Data Engineer to join our team. The ideal candidate will have a minimum of 5 years of hands-on experience working with Databricks on leading cloud platforms such as AWS and Azure. You will be responsible for designing, building, and optimizing scalable data pipelines, collaborating with cross-functional teams, and driving data-driven decision making across the organization.

Key Responsibilities

Design, develop, and maintain scalable data pipelines and ETL processes using Databricks on AWS and Azure.

Optimize Spark jobs for performance, reliability, and cost-efficiency.

Integrate Databricks workflows with cloud-native storage solutions (S3, ADLS).

Implement data quality, validation, and governance best practices.

Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver solutions.

Troubleshoot and resolve issues in production data pipelines.

Document technical solutions and create knowledge-sharing materials for the team.

Must Have Skills

Databricks Platform Expertise: 5+ years of hands-on experience developing solutions on Databricks, including workspace management, notebook development, job scheduling, and cluster configuration.

Cloud Platform Experience: Proven experience in deploying and managing Databricks environments on AWS and Azure.

Apache Spark: Advanced knowledge of Spark (PySpark/Scala/SQL) for data processing and analytics.

ETL & Data Pipeline Development: Expertise in building robust ETL processes and data pipelines.

Programming Languages: Proficiency in Python and/or Scala.

Cloud Storage Integration: Experience integrating with S3, ADLS, or similar cloud storage services.

SQL Skills: Strong ability to write and optimize complex SQL queries.

Data Modeling: Solid understanding of data modeling concepts and best practices.

Version Control: Familiarity with Git or similar version control systems.

Problem Solving: Excellent troubleshooting and analytical skills.

Good to Have Skills

Experience with CI/CD pipelines and DevOps practices for data engineering.

Knowledge of Delta Lake and advanced Databricks features (e.g., MLflow, Databricks SQL).

Experience with other cloud platforms (Google Cloud Platform, etc.).

Exposure to data visualization tools (Power BI, Tableau, etc.).

Understanding of data security, compliance, and privacy standards.

Hands-on experience with streaming data (Kafka, Spark Streaming).

Familiarity with REST APIs and data integration patterns.

Ability to mentor junior team members and conduct code reviews.

Experience with infrastructure as code (Terraform, CloudFormation).

Qualifications

Bachelor's or Master's degree in Computer Science, Engineering, or related field.

Minimum 5 years of experience in Databricks development on AWS and/or Azure.

Strong communication skills and ability to work in a collaborative environment.

Relevant certifications (Databricks, AWS, Azure) are a plus.

  • MuleSoft Developer

    hace 6 días

    Ciudad de México, Ciudad de México Argano A tiempo completo

    IntroductionJOB DESCRIPTIONWe are seeking a skilled and motivatedMuleSoft Developerto join our dynamic IT team. In this role, you will be responsible for designing, developing, and deploying integration solutions that enable seamless communication between applications and systems. You'll work on creating new integration use cases, while also supporting and...

  • etl developer

    hace 6 días

    Ciudad de México, Ciudad de México Zurich Insurance A tiempo completo

    ZCCZurich Capability Center is currently hiring a:ETL Developer / LeadWe are looking for aSenior ETL Developer / Leadwith extensive experience inInformatica PowerCenterto join our data engineering team. In this role, you will not onlybuild, optimize, and integrate robust data pipelinesbut alsoguide technical decisions and mentor junior team members,...

  • Senior Business Intelligence Developer

    hace 7 días

    Ciudad de México, Ciudad de México Allstate A tiempo completo

    Integon is a part of The Allstate Corporation, which means we have the same innovative drive that keeps us a step ahead of our customers' evolving needs. It's how we've become industry leaders in property and casualty insurance servicing. We support our insurance company affiliates by offering property and casualty insurance products inclusive of personal...

  • Senior Web UI Developer

    hace 24 horas

    Ciudad de México, Ciudad de México ION A tiempo completo

    Lab49 is an award winning specialist consultancy that creates bespoke technology in partnership with the most important companies in finance.We were founded in 2002 to bring Silicon Valley solutions to Wall Street's door. Since then, we have worked on successive waves of technological change, including distributed computing, high-speed automation, enterprise...

  • Data Engineer

    hace 4 días

    Ciudad de México, Ciudad de México Astellas Pharma A tiempo completo

    Job description:Are you driven to make a real difference in the lives of patients?We're seeking passionate individuals who thrive in dynamic environments, embrace new ideas and aren't afraid to take intelligent risks. People who act with unwavering integrity and are deeply committed to making a tangible impact.Purpose and ScopeAs an Information X Data...

  • Fullstack () AI Ex Software Dev

    hace 3 días

    Ciudad de México, Ciudad de México Softtek A tiempo completo

    About the CompanyFounded in 1982, Softtek is a global provider of process-driven IT solutions with 30 offices in North America, Latin America, Europe and Asia. With 15 Global Delivery Centers in the U.S., Mexico, China, Brazil, Argentina, Costa Rica, Spain, Hungary and India. Softtek helps improve and produce predictable outcomes for top-tier corporations in...

  • Business Intelligence

    hace 7 días

    Ciudad de México, Ciudad de México Rockwell Automation A tiempo completo

    Rockwell Automation is a global technology leader focused on helping the world's manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale,...

  • Data Engineer

    hace 6 días

    Ciudad de México, Ciudad de México Sproule ERCE A tiempo completo

    About UsWe empower energy companies and financial stakeholders to make confident, strategic decisions. Through technical, commercial, and operational expertise, we deliver authoritative analyses for asset valuation, regulatory reporting, portfolio optimization, and capital allocation, helping clients around the world unlock value, manage risk, and drive...

  • Principal Software Engineer

    hace 1 semana

    Naucalpan de Juárez, México AT&T A tiempo completo

    Join AT&T and help shape the future of communications and technology that connect the world. We value innovators who seek to explore the unknown and challenge the status quo. Bring your bold ideas and fearless spirit to redefine connectivity and transform how people share stories and experiences. At AT&T, you won't just imagine the future—you'll build...

  • Job announcement for Roster

    hace 1 semana

    Texcoco de Mora, México CIMMYT A tiempo completo

    CIMMYT is a cutting edge, non-profit, international organization dedicated to solving tomorrow's problems today. It is entrusted with fostering improved quantity, quality, and dependability of production systems and basic cereals such as maize, wheat, triticale, sorghum, millets, and associated crops through applied agricultural science, particularly in the...