Data Engineer Senior
hace 4 semanas
We are looking for your talent ✋
*** Data Engineer Senior**
****
**El perfil deseado debe tener al menos 5 años de experiência práctica en el diseño, establecimiento y mantenimiento de sistemas de gestión y almacenamiento de datos. Hábil en la recopilación, procesamiento, limpieza y despliegue de grandes conjuntos de datos, la comprensión de los modelos de datos ER, y la integración con múltiples fuentes de datos. Eficaz en el análisis, la comunicación y la propuesta de diferentes formas de crear almacenes de datos, lagos de datos, conductos de extremo a extremo y soluciones de Big Data para los clientes, ya sea en estrategias por lotes o en streaming.**
Será muy importante que tengas los siguientes conocimientos/experiência:
- ** Inglés B2+ o más** (llevarás proyectos 100% con el idioma, por lo que será indispensable el dominio hablado y escrito)
**Technical Proficiencies**:
- **SQL**:
Data Definition Language, Data Manipulation Language, Intermediate/advanced queries for analytical purpose, Subqueries, CTEs, Data types, Joins with business rules applied, Grouping and Aggregates for business metrics, Indexing and optimizing queries for efficient ETL process, Stored Procedures for transforming and preparing data, SSMS, DBeaver
- **Python**:
Experience in object-oriented programming, Management and processing datasets, Use of variables, lists, dictionaries and tuples, Conditional and iterating functions, Optimization of memory consumption, Structures and data types, Data ingestion through various structured and semi-structured data sources, Knowledge of libraries such as pandas, numpy, sqlalchemy, Must have good practices when writing code
- **Databricks / Pyspark**:
Intermediate knowledge in
Understanding of narrow and wide transformations, actions, and lazy evaluations
How DataFrames are transformed, executed, and optimized in Spark
Use DataFrame API to explore, preprocess, join, and ingest data in Spark
Use Delta Lake to improve the quality and performance of data pipelines
Use SQL and Python to write production data pipelines to extract, transform, and load data into
tables and views in the Lakehouse
Understand the most common performance problems associated with data ingestion and how to
mitigate them
Monitor Spark UI: Jobs, Stages, Tasks, Storage, Environment, Executors, and Execution Plans
Configure a Spark cluster for maximum performance given specific job requirements
Configure Databricks to access Blob, ADL, SAS, user tokens, Secret Scopes and Azure Key Vault
Configure governance solutions through Unity Catalog and Delta Sharing
Use Delta Live Tables to manage an end-to-end pipeline with unit and integrations test
- **Azure**:
Intermediate/Advanced knowledge in
**Azure Storage Account**:
Provision Azure Blob Storage or Azure Data Lake instances
Build efficient file systems for storing data into folders with static or parametrized names, considering possible security rules and risks
Experience identifying use cases for open-source file formats like parquet, AVRO, ORC
Understanding optimized column-oriented file formats vs optimized row-oriented file formats
Implementing security configurations through Access Keys, SAS, AAD, RBAC, ACLs
**Azure Data Factory**:
Provision Azure Data Factory instances
Use Azure IR, Self-Hosted IR, Azure-SSIS to establish connections to distinct data sources
Use of Copy or Polybase activities for loading data
Build efficient and optimized ADF Pipelines using linked services, datasets, parameters, triggers, data movement activities, data transformation activities, control flow activities and mapping data flows
Build Incremental and Re-Processing Loads
- **Apache Kafka, Azure Event Hubs or AWS Kinesis**
Intermediate/Advanced knowledge in
Architecture and fundamental concepts of event streaming platforms, including producers, consumers, topics, partitions, and consumer groups
Configuration, deployment, and management of event streaming clusters/services for high availability, scalability, and fault tolerance
Performance tuning and optimization of event streaming clusters, including message retention, partition sizing, and data replication
Implementing common usage patterns such as asynchronous messaging, real-time stream processing, and end-to-end data pipelines for real-time data ingestion and processing
Security best practices for event streaming platforms, including encryption, authentication, and access control mechanisms
**Además, valoramos mucho es que a nível personal encajes con la cultura de Derevo**:
- Capacidad de adaptación y superación. Buscamos personas que se quieran comer el mundo, proactivas y flexibles, a las que no les importe adaptarse a los cambios tecnológicos y metodologías existentes.
- Capacidad analítica y capaz de transmitir confianza en entornos de incertidumbre: debes tener capacidad para gestionar los problemas y verlos como punto de partida para la mejora. Tener y generar
-
Senior Software Engineer for Goldengate Big Data
hace 4 semanas
Zapopan, México Oracle A tiempo completo**Our Team : Oracle GoldenGate for BigData** Oracle GoldenGate (OGG) is a comprehensive software package for real-time data integration and replication in heterogeneous IT environments. Oracle GoldenGate for Big Data streams transactional data into big data systems in real time, raising the quality and timeliness of business insights. For more information,...
-
Senior Data Analyst
hace 4 semanas
Zapopan, México Bizee A tiempo completoWe are seeking a highly skilled and experienced Senior Data Analyst to join our growing team. As a Senior Data Analyst, you will play a pivotal role in driving data-driven decision-making across our organization. The primary focus of this position will be on analyzing marketing efficacy, product efficiency, Google Analytics, and other first-party data to...
-
Data Engineer Mid
hace 4 semanas
Zapopan, México Derevo A tiempo completo¡¡We are looking for your talent!! ✋ **Data Engineer Mid** ** **The desired profile should have at least 3 years hands-on experience in designing, establishing, and maintaining data management and storing systems. Skilled in collecting, processing, cleaning, and deploying large datasets, understanding ER data models, and integrating with multiple...
-
Data Engineer, Science
hace 4 semanas
Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo3+ years of data engineering experience - Experience with data modeling, warehousing and building ETL pipelines The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What...
-
Data Engineer
hace 7 días
Zapopan, México STAND 8 A tiempo completoSTAND 8 is a global leader providing end-to-end IT Solutions. We solve business problems through PEOPLE, PROCESS, and TECHNOLOGY and are looking for individuals to help us scale software projects designed to change the world! **Responsibilities** - Augment and maintain the existing repositories and data structures within AWS (used to process and store large...
-
Data Engineer, Science
hace 4 semanas
Zapopan, Jal., México Amazon A tiempo completoData Engineer, Science & Data Technology team The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What will you help us create? The Team: How often have you had an...
-
Data Engineer, Science
hace 4 semanas
Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completoThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...
-
Data Engineer, Ds2-science
hace 1 día
Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo5+ years of experience with data warehouse technical architectures, ETL/ ELT, reporting/analytic tools and, scripting. - 5+ years of demonstrated quantitative and qualitative data experience with data modeling, ETL development - Knowledge of data modeling and experience SQL with Redshift, Oracle, MySQL, and Columnar Databases - Experience managing competing...
-
Fresher / Data Engineer & Analytics Developer
hace 6 días
Zapopan, México Oracle A tiempo completoAnalyze, design develop, troubleshoot and debug software programs for commercial or end user applications. Writes code, completes programming and performs testing and debugging of applications. We are seeking a professional to fill the combined role of Data Engineer and Analytics. This position requires a candidate who is willing to navigate between...
-
Data Engineer, Ds2-science
hace 1 día
Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo5+ years of experience with data warehouse technical architectures, ETL/ ELT, reporting/analytic tools and, scripting. - 5+ years of demonstrated quantitative and qualitative data experience with data modeling, ETL development - Knowledge of data modeling and experience SQL with Redshift, Oracle, MySQL, and Columnar Databases - Experience managing competing...
-
Data Engineer, DS2-Science
hace 19 horas
Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completoThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...
-
Data Engineer, DS2-Science
hace 2 días
Zapopan, Jalisco, México Amazon A tiempo completoThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...
-
Data Engineer, DS2-Science
hace 2 días
Zapopan, Jalisco, México Amazon A tiempo completoThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team that is...
-
Data Engineer, DS2-Science
hace 1 día
Zapopan, México Amazon A tiempo completoDescriptionThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team...
-
Data Engineer, DS2-Science
hace 1 día
Zapopan, México Amazon A tiempo completoDescriptionThe Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.What will you help us create?The Team: How often have you had an opportunity to be a founding member of a team...
-
Fresher / Data Engineer & Analytics Developer
hace 5 días
Zapopan, México Oracle A tiempo completoWe are looking for Big Data developers and Analytics skills. Preferred experience in Cloud development, Spark, SQL, Python, Data Bases and scripting. Career Level - IC2 **Responsibilities**: Data Engineering: 1. Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse...
-
Senior Software Developer
hace 2 semanas
Zapopan, Jal., México Ll Oefentherapie A tiempo completoOracle is a leading technology company dedicated to innovation and creating ground breaking products. We are seeking a dedicated and motivated Software Engineer to join and contribute to our dynamic team. Some of the key areas you will be involved with are: Collaborating with multiple teams including developers, support and support leadership Applying...
-
Big Data
hace 1 semana
Zapopan, México Oracle A tiempo completoAnalyze, design develop, troubleshoot and debug software programs for commercial or end user applications. Writes code, completes programming and performs testing and debugging of applications. We are seeking a versatile and experienced professional to fill the combined role of Data Engineer with analytics experience. This position requires a candidate who...
-
Business Intelligence Engineer, Ds2-science
hace 7 días
Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completoExperience using SQL to pull data from a database or data warehouse and scripting experience (Python) to process data for modeling - Experience in data mining, ETL, etc. and using databases in a business environment with large-scale, complex datasets - Experience with data visualization using Tableau, Quicksight, or similar tools - Bachelor's degree in...
-
Senior Civil Engineer
hace 4 semanas
Zapopan, México AMCG A tiempo completo**Vacante para la empresa AMCG en Zapopan -Zapopan, Jalisco**: We are an international company looking for a Civil Engineer III to support our efforts in our Guadalajara office. If you are looking to join a dynamic, innovative, and collaborative team, this is the perfect place for you! Currently we are growing and offer exciting career opportunities for...