Senior Data Platform Engineer | Python, Gcp, Airflow

Hace 20 horas

méxico Confiz Limited Jornada completa

IMPORTANT: This position is ONLY for candidates currently based in Mexico and legally authorized to work in Mexico. Applications from outside Mexico will be automatically REJECTED

CONSULTFIZ LATAM, a fully owned subsidiary of Confiz LLC, is looking for a Data Platform Engineer with strong experience in Python, GCP, Airflow, and Terraform to design, develop, and maintain scalable data orchestration platforms and self-service solutions. The ideal candidate will have hands-on experience with ETL/ELT pipelines, cloud data infrastructure, data orchestration, streaming technologies, and Infrastructure as Code (Ia C) . This role is for a Fortune 500 client .

Responsibilities
  • Design, evolve, and maintain scalable data orchestration platforms that simplify ETL/ELT operations and abstract the complexities of big data technologies.
  • Build and maintain self-service platform capabilities for engineers, data scientists, and analysts.
  • Develop and maintain integrations with diverse data sources and destinations, including Kafka, GCS, Big Query, Postgres, and S3, supporting both batch and real-time processing.
  • Provision and manage cloud data infrastructure using Terraform, leveraging automated and version-controlled Infrastructure as Code pipelines.
  • Build, maintain, and orchestrate reliable data pipelines using Airflow and Kubernetes.
  • Develop platform tooling, automation, services, and CLIs using Python.
  • Build and operate metadata management, data cataloging, governance, and data-lineage capabilities to make enterprise data discoverable and trustworthy.
  • Design scalable and resilient solutions capable of handling large data volumes, high velocity, and diverse data formats.
  • Apply modern CI/CD and Git-based workflows to platform and data pipeline delivery.
  • Evaluate and incorporate emerging big data, ETL, and AI-assisted development technologies where appropriate.
  • Create and maintain comprehensive technical and user documentation for platform capabilities and workflows.
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Engineering , or equivalent practical experience.
  • 5–7 years of professional experience in Data Engineering, Data Platform Engineering, or related software engineering roles.
  • Strong professional programming experience with Python , including building and maintaining automation, services, and/or command-line tools.
  • Strong proficiency in SQL , particularly within modern cloud data environments such as Big Query .
  • Hands-on experience designing and maintaining ETL/ELT pipelines and a solid understanding of modern data pipeline architecture.
  • Experience with modern data transformation frameworks such as dbt .
  • Strong hands-on experience with Google Cloud Platform (GCP) data technologies, particularly Big Query, GCS, and Dataproc Serverless .
  • Strong experience with Infrastructure as Code (Ia C), particularly Terraform , for provisioning and managing cloud data infrastructure.
  • Hands-on experience building and orchestrating data pipelines using Apache Airflow .
  • Experience working with Kubernetes in data platform or cloud environments.
  • Experience with streaming and event-driven data technologies such as Kafka / Confluent Cloud .
  • Working knowledge of data governance, metadata management, data cataloging, and data lineage concepts.
  • Familiarity with CI/CD and Git-based development workflows , including tools such as Git Hub Actions.
  • Experience using AI-assisted developer tooling , including LLMs, Model Context Protocol (MCP), and/or copilot-style productivity tools.
  • Excellent problem-solving and communication skills, with the ability to collaborate effectively across engineering and business teams.
Nice to Have
  • Experience with AWS data and cloud technologies.
  • Hands-on experience with metadata, data cataloging, or data-lineage platforms such as Data Hub, Collibra, Amundsen , or similar tools.
  • Experience with BI or semantic-layer technologies such as Looker / Look ML .
  • Experience with data quality, observability, SLOs, error budgets, and production/on-call operations for data platforms.
  • Familiarity with distributed data processing frameworks such as Apache Spark .
  • Experience with Java and/or Scala is a plus.

We have a global team of amazing individuals working on highly innovative enterprise projects & products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast growth fintech, and multiple Silicon Valley startu