Data Engineer Senior

hace 3 días

Zapopan, México Derevo A tiempo completo

We are looking for your talent ✋

*** Data Engineer Senior**

**‍**

**El perfil deseado debe tener al menos 5 años de experiência práctica en el diseño, establecimiento y mantenimiento de sistemas de gestión y almacenamiento de datos. Hábil en la recopilación, procesamiento, limpieza y despliegue de grandes conjuntos de datos, la comprensión de los modelos de datos ER, y la integración con múltiples fuentes de datos. Eficaz en el análisis, la comunicación y la propuesta de diferentes formas de crear almacenes de datos, lagos de datos, conductos de extremo a extremo y soluciones de Big Data para los clientes, ya sea en estrategias por lotes o en streaming.**

Será muy importante que tengas los siguientes conocimientos/experiência:

- ** Inglés B2+ o más** (llevarás proyectos 100% con el idioma, por lo que será indispensable el dominio hablado y escrito)

**Technical Proficiencies**:

- **SQL**:
Data Definition Language, Data Manipulation Language, Intermediate/advanced queries for analytical purpose, Subqueries, CTEs, Data types, Joins with business rules applied, Grouping and Aggregates for business metrics, Indexing and optimizing queries for efficient ETL process, Stored Procedures for transforming and preparing data, SSMS, DBeaver
- **Python**:
Experience in object-oriented programming, Management and processing datasets, Use of variables, lists, dictionaries and tuples, Conditional and iterating functions, Optimization of memory consumption, Structures and data types, Data ingestion through various structured and semi-structured data sources, Knowledge of libraries such as pandas, numpy, sqlalchemy, Must have good practices when writing code
- **Databricks / Pyspark**:
Intermediate knowledge in

Understanding of narrow and wide transformations, actions, and lazy evaluations

How DataFrames are transformed, executed, and optimized in Spark

Use DataFrame API to explore, preprocess, join, and ingest data in Spark

Use Delta Lake to improve the quality and performance of data pipelines

Use SQL and Python to write production data pipelines to extract, transform, and load data into

tables and views in the Lakehouse

Understand the most common performance problems associated with data ingestion and how to

mitigate them

Monitor Spark UI: Jobs, Stages, Tasks, Storage, Environment, Executors, and Execution Plans

Configure a Spark cluster for maximum performance given specific job requirements

Configure Databricks to access Blob, ADL, SAS, user tokens, Secret Scopes and Azure Key Vault

Configure governance solutions through Unity Catalog and Delta Sharing

Use Delta Live Tables to manage an end-to-end pipeline with unit and integrations test
- **Azure**:
Intermediate/Advanced knowledge in

**Azure Storage Account**:
Provision Azure Blob Storage or Azure Data Lake instances

Build efficient file systems for storing data into folders with static or parametrized names, considering possible security rules and risks

Experience identifying use cases for open-source file formats like parquet, AVRO, ORC

Understanding optimized column-oriented file formats vs optimized row-oriented file formats

Implementing security configurations through Access Keys, SAS, AAD, RBAC, ACLs

**Azure Data Factory**:
Provision Azure Data Factory instances

Use Azure IR, Self-Hosted IR, Azure-SSIS to establish connections to distinct data sources

Use of Copy or Polybase activities for loading data

Build efficient and optimized ADF Pipelines using linked services, datasets, parameters, triggers, data movement activities, data transformation activities, control flow activities and mapping data flows

Build Incremental and Re-Processing Loads
- **Apache Kafka, Azure Event Hubs or AWS Kinesis**

Intermediate/Advanced knowledge in

Architecture and fundamental concepts of event streaming platforms, including producers, consumers, topics, partitions, and consumer groups

Configuration, deployment, and management of event streaming clusters/services for high availability, scalability, and fault tolerance

Performance tuning and optimization of event streaming clusters, including message retention, partition sizing, and data replication

Implementing common usage patterns such as asynchronous messaging, real-time stream processing, and end-to-end data pipelines for real-time data ingestion and processing

Security best practices for event streaming platforms, including encryption, authentication, and access control mechanisms

**Además, valoramos mucho es que a nível personal encajes con la cultura de Derevo**:

- Capacidad de adaptación y superación. Buscamos personas que se quieran comer el mundo, proactivas y flexibles, a las que no les importe adaptarse a los cambios tecnológicos y metodologías existentes.
- Capacidad analítica y capaz de transmitir confianza en entornos de incertidumbre: debes tener capacidad para gestionar los problemas y verlos como punto de partida para la mejora. Tener y generar

  • Senior Data Engineer

    hace 2 días

    Zapopan, Jalisco, México Koch Global Services A tiempo completo

    Drive Business Growth with Cutting-Edge BI Solutions">We're seeking a results-driven Senior Data Engineer to lead the development of innovative Business Intelligence (BI) solutions that drive business growth and decision-making. As a Sr. Bi Developer, you will collaborate with cross-functional teams to design, develop, and maintain cutting-edge BI solutions...

  • Sr. Data Engineer

    hace 1 semana

    Zapopan, México TECNOCONTROL VEHICULAR A tiempo completo

    ¿Quiénes somos?Somos un holding de empresas líder en soluciones para la gestión de flotas. Nuestro propósito es transformar la experiencia de nuestros clientes mediante tecnología innovadora y un diseño centrado en el usuario. Creemos firmemente en el poder de ofrecer experiencias excepcionales, y por ello estamos en la búsqueda de un Data Engineer...

  • Sr. Data Engineer

    hace 5 días

    Zapopan, Jalisco, México TECNOCONTROL VEHICULAR A tiempo completo

    ¿Quiénes somos?Somos un holding de empresas líder en soluciones para la gestión de flotas. Nuestro propósito es transformar la experiencia de nuestros clientes mediante tecnología innovadora y un diseño centrado en el usuario. Creemos firmemente en el poder de ofrecer experiencias excepcionales, y por ello estamos en la búsqueda de un Data Engineer...

  • Sr. Data Engineer

    hace 5 días

    Zapopan, Jalisco, México TECNOCONTROL VEHICULAR A tiempo completo

    ¿Quiénes somos?Somos un holding de empresas líder en soluciones para la gestión de flotas. Nuestro propósito es transformar la experiencia de nuestros clientes mediante tecnología innovadora y un diseño centrado en el usuario. Creemos firmemente en el poder de ofrecer experiencias excepcionales, y por ello estamos en la búsqueda de un Data Engineer...

  • Sr. Data Engineer

    hace 2 semanas

    Zapopan, Jal., México TECNOCONTROL VEHICULAR A tiempo completo EUR55,000

    ¿Quiénes somos? Somos un holding de empresas líder en soluciones para la gestión de flotas. Nuestro propósito es transformar la experiencia de nuestros clientes mediante tecnología innovadora y un diseño centrado en el usuario. Creemos firmemente en el poder de ofrecer experiencias excepcionales, y por ello estamos en la búsqueda de un Data Engineer...

  • Data Engineer Jr

    hace 4 semanas

    Zapopan, México Derevo A tiempo completo

    ¡¡Buscamos tu talento!!?**DATA ENGINEER JR**En **Derevo** buscamos empoderar a las empresas y a las personas para liberar el valor de los datos en las organizaciones, a través de la implementación de procesos y plataformas de analítica con un enfoque que cubre el ciclo completo que necesitan llevar a cabo para lograrlo.Derevo inicio en 2010 con una idea...

  • Data Engineer- Onsite

    hace 4 semanas

    Zapopan, México Gsb Solutions A tiempo completo

    Important IT company At the Latin American level, growth requires:**Data Engineer****Job description**:- Advanced Google Cloud development skills.- Experience with Google Cloud FHIR data stores.**Schedule**: 9:00 am a 6:00 pm.**Job type**: Onsite.**Location**:Av.Acueducto 4851 Piso 14, Puerta de Hierro, 45116 Zapopan, Jal.**Salary**: Open- Salary according...

  • Senior Data Scientist

    hace 5 días

    Zapopan, Jalisco, México HCLTech A tiempo completo

    HCLTech is a leading global technology company that delivers industry-leading capabilities in digital, engineering, cloud and AI. Our diverse team of experts serves clients across major verticals, providing tailored solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and...

  • Sr. Data Analytics Engineer

    hace 2 semanas

    Zapopan, México Oracle A tiempo completo

    **Data Analytics Engineer**Why Oracle NetSuite Global Business Unit (NSGBU)?One word — transformation.At Oracle NetSuite Global Business Unit (NSGBU) we believe the cloud is here to stay and so do our 20,000+ customers.We believe businesses should not be bogged down by the overhead of bulky data centers and expensive personnel to run it all.Businesses need...

  • Etl + Data Engineer

    hace 3 días

    Zapopan, Jalisco, México Bairesdev A tiempo completo

    At BairesDev, we've been leading the way in technology projects for over 15 years.We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley.Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide.ETL + Data Engineer at BairesDevWe...

  • Senior Software Engineer For Goldengate Big Data

    hace 2 semanas

    Zapopan, México Oracle A tiempo completo

    **Our Team : Oracle GoldenGate for BigData**Oracle GoldenGate (OGG) is a comprehensive software package for real-time data integration and replication in heterogeneous IT environments.Oracle GoldenGate for Big Data streams transactional data into big data systems in real time, raising the quality and timeliness of business insights.For more information,...

  • Senior Platform Engineer

    hace 3 semanas

    Zapopan, México UST A tiempo completo

    **Senior Platform Engineer** **Lead I - DevOps Engineering **Who we are**: Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world. UST is a...

  • Data Modeling Engineer

    hace 4 semanas

    Zapopan, México AP Consulting A tiempo completo

    ¡Descubre el potencial de tu carrera con nosotros! Explora nuestras oportunidades laborales y da el primer paso hacia un futuro profesional emocionante. **Data Modeling Engineer** El Data Modeling Engineer apoyará a diseñar, implementar y mantener modelos de datos que respalden nuestros objetivos comerciales. Este puesto está centrado en el uso de...

  • Senior Platform Engineer

    hace 2 días

    Zapopan, Jalisco, México Ust A tiempo completo

    **Senior Platform Engineer****Lead I - DevOps Engineering**Who we are**:Born digital, UST transforms lives through the power of technology.We walk alongside our clients and partners, embedding innovation and agility into everything they do.We help them create transformative experiences and human-centered solutions for a better world.UST is a mission-driven...

  • Cloud Data Engineer

    hace 2 días

    Zapopan, Jalisco, México Oracle A tiempo completo

    Job DescriptionCompany OverviewWe are a leading provider of cloud-based solutions, and we're seeking a highly skilled Cloud Data Engineer to join our team.Job SummaryWe're looking for a seasoned Cloud Data Engineer with expertise in finance domain to design, build, and maintain large-scale data processing systems. The ideal candidate will have a strong...

  • Data Modeling Engineer

    hace 4 semanas

    Zapopan, México Ap Consulting A tiempo completo

    ¡Descubre el potencial de tu carrera con nosotros!Explora nuestras oportunidades laborales y da el primer paso hacia un futuro profesional emocionante.**Data Modeling Engineer**El Data Modeling Engineer apoyará a diseñar, implementar y mantener modelos de datos que respalden nuestros objetivos comerciales.Este puesto está centrado en el uso de...

  • Big Data Engineer

    hace 4 semanas

    Zapopan, México Intugo A tiempo completo

    **Position Summary**:Our team is looking for a **Data Engineer** with a diverse background in data integration to join the Data Services team.Some data are small, some data are very large (1 trillion+ rows), some data is structured, some data is not.Our data comes in all kinds of sizes, shapes and formats.Traditional RDBMS like PostgreSQL, Oracle, SQL...

  • Data Engineer Position: Expertise in Cloud Development

    hace 1 día

    Zapopan, Jalisco, México Gsb Solutions A tiempo completo

    Job SummaryGsb Solutions is looking for a Data Engineer with expertise in Google Cloud development to join our team. This onsite role will focus on advancing our data engineering capabilities using Google Cloud FHIR data stores.About the RoleThis is an excellent opportunity for a skilled Data Engineer to contribute to the growth of our organization. As a key...

  • Etl + Data Engineer

    hace 5 días

    Zapopan, México BairesDev A tiempo completo

    At BairesDev®, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide. ETL + Data Engineer at...

  • Data Engineer, Science

    hace 2 semanas

    Zapopan, México Servicios Comerciales Amazon Mexico S. De R.L. De C.V. - D44 A tiempo completo

    3+ years of data engineering experience- Experience with data modeling, warehousing and building ETL pipelinesThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products.We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you...