Data Engineer Mid

hace 4 semanas

Zapopan, México Derevo A tiempo completo

We are looking for your talent ✋

**Data Engineer Mid**

**‍ **The desired profile should have at least 3 years hands-on experience in designing, establishing, and maintaining data management and storing systems. Skilled in collecting, processing, cleaning, and deploying large datasets, understanding ER data models, and integrating with multiple data sources. Efficient in analyzing, communicating, and proposing different ways of building Data Warehouses, Data Lakes, End-to-End Pipelines, and Big Data solutions to clients, either in batch or streaming strategies.

It will be very important that you have the following skills/experience:
**English B2 or higher**

**Technical Proficiencies**:

- SQL:
Data Definition Language, Data Manipulation Language, Intermediate/advanced queries for analytical purpose, Subqueries, CTEs, Data types, Joins with business rules applied, Grouping and Aggregates for business metrics, Indexing and optimizing queries for efficient ETL process, Stored Procedures for transforming and preparing data, SSMS, DBeaver
- Python:
Experience in object-oriented programming, Management and processing datasets, Use of variables, lists, dictionaries and tuples, Conditional and iterating functions, Optimization of memory consumption, Structures and data types, Data ingestion through various structured and semi-structured data sources, Knowledge of libraries such as pandas, numpy, sqlalchemy, Must have good practices when writing code
- Databricks / Pyspark:
Intermediate knowledge in

Understanding of narrow and wide transformations, actions, and lazy evaluations

How DataFrames are transformed, executed, and optimized in Spark

Use DataFrame API to explore, preprocess, join, and ingest data in Spark

Use Delta Lake to improve the quality and performance of data pipelines

Use SQL and Python to write production data pipelines to extract, transform, and load data into

tables and views in the Lakehouse

Understand the most common performance problems associated with data ingestion and how to

mitigate them

Monitor Spark UI: Jobs, Stages, Tasks, Storage, Environment, Executors, and Execution Plans

Configure a Spark cluster for maximum performance given specific job requirements

Configure Databricks to access Blob, ADL, SAS, user tokens, Secret Scopes and Azure Key Vault

Configure governance solutions through Unity Catalog and Delta Sharing

Use Delta Live Tables to manage an end-to-end pipeline with unit and integrations test
- Azure:
Intermediate knowledge in

Azure Storage Account:
Provision Azure Blob Storage or Azure Data Lake instances

Build efficient file systems for storing data into folders with static or parametrized names, considering possible security rules and risks

Experience identifying use cases for open-source file formats like parquet, AVRO, ORC

Understanding optimized column-oriented file formats vs optimized row-oriented file formats

Implementing security configurations through Access Keys, SAS, AAD, RBAC, ACLs

Azure Data Factory:
Provision Azure Data Factory instances

Use Azure IR, Self-Hosted IR, Azure-SSIS to establish connections to distinct data sources

Use of Copy or Polybase activities for loading data

Build efficient and optimized ADF Pipelines using linked services, datasets, parameters, triggers, data movement activities, data transformation activities, control flow activities and mapping data flows

Build Incremental and Re-Processing Loads

** What benefits will you have?**

WELLNESS: We will promote your integral wellbeing through personal, professional and economic balance. Our legal and additional benefits will help you achieve it.
LET'S RELEASE YOUR POWER: You will have the opportunity to specialize in a comprehensive manner in different areas and technologies, thus achieving an interdisciplinary development. We will push you to take on new challenges and surpass yourself.
WE CREATE NEW THINGS: We like to think outside the box. You will have the space, confidence and freedom to create and the training required to achieve it.
WE GROW TOGETHER: You will participate in cutting-edge, multinational technology projects with foreign teams.

**Where will you do it?**

We are a great team working in a remote scheme, we are flexible and structured; providing the necessary equipment to work with and internal communication tools that facilitate our operation and that of our clients.

If you meet most of the requirements and you are interested in the profile do not hesitate to apply, our Talent team will contact you

Become derevian & develop your superpower

  • Data Scientist Mid

    hace 4 semanas

    Zapopan, México Derevo A tiempo completo

    ¿Tienes experiência en la plataforma Azure, Python y Azure Functions? ¿Has trabajado con el uso de APIs y bases de datos tanto en entornos en la nube como locales? Además, ¿cuentas con experiência en el manejo de OpenAI y en la ingeniería de prompts? ¡Entonces, esta vacante está diseñada para ti! Continúa leyendo para conocer más detalles. En...

  • Consultor Data Science Mid

    hace 4 semanas

    Zapopan, México Derevo A tiempo completo

    ¿Tienes experiência en la creación de modelos de Machine Learning en Azure Machine Learning Designer? ¡Entonces, esta vacante está diseñada para ti! Sigue leyendo para conocer más detalles. En Derevo buscamos empoderar a las empresas y a las personas para liberar el valor de los datos en las organizaciones, a través de la implementación de procesos...

  • Data Engineer, Science

    hace 1 mes

    Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    3+ years of data engineering experience - Experience with data modeling, warehousing and building ETL pipelines The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What...

  • Data Engineer Senior

    hace 1 mes

    Zapopan, México Derevo A tiempo completo

    ¡¡We are looking for your talent!! ✋ *** Data Engineer Senior** **‍** **El perfil deseado debe tener al menos 5 años de experiência práctica en el diseño, establecimiento y mantenimiento de sistemas de gestión y almacenamiento de datos. Hábil en la recopilación, procesamiento, limpieza y despliegue de grandes conjuntos de datos, la...

  • Data Engineer, Science

    hace 4 semanas

    Zapopan, Jal., México Amazon A tiempo completo

    Data Engineer, Science & Data Technology team The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What will you help us create? The Team: How often have you had an...

  • Data Engineer, DS2-Science

    hace 22 horas

    Zapopan, Jalisco, México myGwork - LGBTQ+ Business Community A tiempo completo

    This inclusive employer is a member of myGwork – the largest global platform for the LGBTQ+ business community. Description: The Amazon Devices team is behind popular consumer electronics like the Kindle, Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What You'll Contribute: Be part of a pioneering team leveraging innovative technology to solve...

  • Data Engineer, Science

    hace 1 mes

    Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...

  • Data Engineer, Ds2-science

    hace 4 días

    Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    5+ years of experience with data warehouse technical architectures, ETL/ ELT, reporting/analytic tools and, scripting. - 5+ years of demonstrated quantitative and qualitative data experience with data modeling, ETL development - Knowledge of data modeling and experience SQL with Redshift, Oracle, MySQL, and Columnar Databases - Experience managing competing...

  • Fresher / Data Engineer & Analytics Developer

    hace 1 semana

    Zapopan, México Oracle A tiempo completo

    Analyze, design develop, troubleshoot and debug software programs for commercial or end user applications. Writes code, completes programming and performs testing and debugging of applications. We are seeking a professional to fill the combined role of Data Engineer and Analytics. This position requires a candidate who is willing to navigate between...

  • Data Engineer, Ds2-science

    hace 4 días

    Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    5+ years of experience with data warehouse technical architectures, ETL/ ELT, reporting/analytic tools and, scripting. - 5+ years of demonstrated quantitative and qualitative data experience with data modeling, ETL development - Knowledge of data modeling and experience SQL with Redshift, Oracle, MySQL, and Columnar Databases - Experience managing competing...

  • Data Engineer, DS2-Science

    hace 4 días

    Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...

  • Data Engineer, DS2-Science

    hace 5 días

    Zapopan, Jalisco, México Amazon A tiempo completo

    The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...

  • Data Engineer, DS2-Science

    hace 5 días

    Zapopan, Jalisco, México Amazon A tiempo completo

    The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...

  • Data Engineer, DS2-Science

    hace 1 día

    Zapopan, México myGwork - LGBTQ+ Business Community A tiempo completo

    This inclusive employer is a member of myGwork – the largest global platform for the LGBTQ+ business community. DescriptionThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon...

  • Data Engineer, DS2-Science

    hace 4 días

    Zapopan, México Amazon A tiempo completo

    DescriptionThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team...

  • Data Engineer, DS2-Science

    hace 4 días

    Zapopan, México Amazon A tiempo completo

    DescriptionThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team...

  • Data Engineer

    hace 4 semanas

    Zapopan, México Azka IT Consulting SA de CV A tiempo completo

    **Data Engineer - ETL Developer** En Azka IT Consulting estamos en busca de tu talento para integrarte a una Empresa Multinacional de Consutltoria y Servicios de Tecnologías de la Información **Requisitos**: - Licenciatura o ingeniería - **Inglés Intermedio-Avanzado Conversacional**: - Experiência de 3 años en adelante - IBM Datastage o...

  • Fresher / Data Engineer & Analytics Developer

    hace 1 semana

    Zapopan, México Oracle A tiempo completo

    We are looking for Big Data developers and Analytics skills. Preferred experience in Cloud development, Spark, SQL, Python, Data Bases and scripting. Career Level - IC2 **Responsibilities**: Data Engineering: 1. Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse...

  • Big Data

    hace 2 semanas

    Zapopan, México Oracle A tiempo completo

    Analyze, design develop, troubleshoot and debug software programs for commercial or end user applications. Writes code, completes programming and performs testing and debugging of applications. We are seeking a versatile and experienced professional to fill the combined role of Data Engineer with analytics experience. This position requires a candidate who...

  • Business Intelligence Engineer, DS2-Science

    hace 1 semana

    Zapopan, Jal., México Amazon A tiempo completo

    Business Intelligence Engineer, DS2-Science & Data Technology team Are you a data enthusiast? Does the world’s most complex logistic systems inspire your curiosity? Is your passion to navigate through hundreds of systems, processes, and data sources to solve the puzzles and identify the next big opportunity? Are you a creative big thinker who is...