Data Engineer Senior

hace 3 semanas

Zapopan, Jalisco, México Derevo A tiempo completo

We are looking for a talented professional to join our team at Derevo.

**About the role:**

We are seeking a highly skilled Data Engineer Senior to lead our data engineering efforts. The ideal candidate will have at least 5 years of experience in designing, implementing, and maintaining data management and storage systems.

**Key responsibilities:**

Collect, process, and deploy large datasets, understand ER data models, and integrate with multiple data sources. Analyze, communicate, and propose different data storage solutions, data lakes, and Big Data solutions for clients.

**Requirements:**

English B2+ or higher (you will lead projects 100% in the language, so it is essential to have a high level of spoken and written proficiency).

**Technical Proficiencies:**

**SQL:** Data Definition Language, Data Manipulation Language, Intermediate/advanced queries for analytical purposes, Subqueries, CTEs, Data types, Joins with business rules applied, Grouping and Aggregates for business metrics, Indexing and optimizing queries for efficient ETL processes, Stored Procedures for transforming and preparing data, SSMS, DBeaver

**Python:** Experience in object-oriented programming, Management and processing datasets, Use of variables, lists, dictionaries, and tuples, Conditional and iterating functions, Optimization of memory consumption, Structures and data types, Data ingestion through various structured and semi-structured data sources, Knowledge of libraries such as pandas, numpy, sqlalchemy, Good practices when writing code

**Databricks / Pyspark:** Intermediate knowledge in Understanding narrow and wide transformations, actions, and lazy evaluations, How DataFrames are transformed, executed, and optimized in Spark, Use DataFrame API to explore, preprocess, join, and ingest data in Spark, Use Delta Lake to improve the quality and performance of data pipelines, Use SQL and Python to write production data pipelines to extract, transform, and load data into tables and views in the Lakehouse, Understand the most common performance problems associated with data ingestion and how to mitigate them, Monitor Spark UI: Jobs, Stages, Tasks, Storage, Environment, Executors, and Execution Plans, Configure a Spark cluster for maximum performance given specific job requirements, Configure Databricks to access Blob, ADL, SAS, user tokens, Secret Scopes, and Azure Key Vault, Configure governance solutions through Unity Catalog and Delta Sharing, Use Delta Live Tables to manage an end-to-end pipeline with unit and integration tests

**Azure:** Intermediate/Advanced knowledge in Azure Storage Account: Provision Azure Blob Storage or Azure Data Lake instances, Build efficient file systems for storing data into folders with static or parametrized names, considering possible security rules and risks, Experience identifying use cases for open-source file formats like parquet, AVRO, ORC, Understanding optimized column-oriented file formats vs optimized row-oriented file formats, Implementing security configurations through Access Keys, SAS, AAD, RBAC, ACLs, Azure Data Factory: Provision Azure Data Factory instances, Use Azure IR, Self-Hosted IR, Azure-SSIS to establish connections to distinct data sources, Use of Copy or Polybase activities for loading data, Build efficient and optimized ADF Pipelines using linked services, datasets, parameters, triggers, data movement activities, data transformation activities, control flow activities, and mapping data flows, Build Incremental and Re-Processing Loads, Apache Kafka, Azure Event Hubs, or AWS Kinesis: Intermediate/Advanced knowledge in Architecture and fundamental concepts of event streaming platforms, including producers, consumers, topics, partitions, and consumer groups, Configuration, deployment, and management of event streaming clusters/services for high availability, scalability, and fault tolerance, Performance tuning and optimization of event streaming clusters, including message retention, partition sizing, and data replication, Implementing common usage patterns such as asynchronous messaging, real-time stream processing, and end-to-end data pipelines for real-time data ingestion and processing, Security best practices for event streaming platforms, including encryption, authentication, and access control mechanisms

**Personal qualities:**

We value people who are adaptable and proactive, who are not afraid to learn and adapt to new technologies and methodologies. We are looking for people who can analyze problems and communicate effectively in uncertain environments.

  • Senior Data Engineer

    hace 2 semanas

    Zapopan, Jalisco, México Servicios Comerciales Amazon Mexico S. De R.L. De C.V. - D44 A tiempo completo

    As a Senior Data Engineer at Servicios Comerciales Amazon Mexico S. De R.L. De C.V. - D44, you will play a key role in designing and building state-of-the-art data solutions for Amazon Devices. With a focus on data modeling, warehousing, and building ETL pipelines, you will help us provide customer-centric products and keep data secure.The Amazon Devices...

  • Senior Big Data Engineer

    hace 2 semanas

    Zapopan, Jalisco, México Zillow A tiempo completo

    About the RoleWe are seeking a highly skilled Senior Big Data Engineer to join our Analytics Tooling team at Zillow. As a key member of our team, you will play a critical role in building and maintaining innovative tools for data-driven decision making.Key Responsibilities:Design and develop scalable data pipelines and architectures to support business...

  • Senior Data Engineer

    hace 1 mes

    Zapopan, Jalisco, México Oracle A tiempo completo

    Oracle Database Engineering - Tools and Application TeamThe Oracle Database Engineering - Tools and Application team is building the next-generation critical development tools, applications, and analytics on Oracle Cloud Infrastructure. We use state-of-the-art technologies - Oracle Cloud Infrastructure, Oracle Analytics Cloud, Machine Learning, AI,...

  • Data Engineer

    hace 2 meses

    Zapopan, Jalisco, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Amazon Devices. As a Data Engineer, you will play a critical role in designing, building, and maintaining large-scale data systems that support our business operations.Key ResponsibilitiesDesign and implement data architectures that meet business requirementsDevelop and maintain...

  • Data Engineer

    hace 1 mes

    Zapopan, Jalisco, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Amazon Devices. As a Data Engineer, you will play a key role in designing, building, and maintaining large-scale data systems that support our business operations.Key ResponsibilitiesDesign and implement data warehouse technical architectures, ETL/ELT, and reporting/analytic...

  • Data Engineer

    hace 1 mes

    Zapopan, Jalisco, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Amazon Devices. As a Data Engineer, you will play a critical role in designing, building, and maintaining large-scale data systems that support our business operations.Key ResponsibilitiesDesign and implement data architectures that meet the needs of our business...

  • Senior Software Engineer

    hace 7 días

    Zapopan, Jalisco, México Ragnar - Seguridad Privada A tiempo completo

    About the RoleAs a senior software engineer in our data science team, you will be responsible for designing and developing AI/ML models to drive business growth.Key ResponsibilitiesDesign and develop AI/ML models to drive business growthCollaborate with cross-functional teams to integrate AI/ML models into our software development pipelineDevelop and...

  • Senior Software Engineer

    hace 6 días

    Zapopan, Jalisco, México Gelden Cosméticos A tiempo completo

    As a Data Platforms Engineer at our company, you will be responsible for designing and developing scalable and secure data platforms using cloud-based technologies. With a focus on software development, you will work closely with cross-functional teams to deliver high-quality solutions that meet our customers' needs.Key responsibilities will...

  • Data Engineer Mid

    hace 4 semanas

    Zapopan, Jalisco, México Derevo A tiempo completo

    Join Our Team as a Data Engineer MidWe are seeking a highly skilled Data Engineer Mid to join our team at Derevo. As a key member of our data engineering team, you will be responsible for designing, establishing, and maintaining data management and storing systems.Key Responsibilities:Design and implement data management and storing systemsCollect, process,...

  • Data Engineer

    hace 3 semanas

    Zapopan, Jalisco, México Oracle A tiempo completo

    Data Engineer - Analytics and Big DataWe are seeking a highly skilled Data Engineer to join our team at Oracle. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources.Key Responsibilities:Design and implement...

  • Data Engineer

    hace 1 mes

    Zapopan, Jalisco, México Oracle A tiempo completo

    Job Title: Data Engineer - Analytics and Big DataWe are seeking a highly skilled Data Engineer to join our team at Oracle. As a Data Engineer, you will play a critical role in designing, developing, and maintaining scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources.Responsibilities:Design and...

  • Senior Database Engineer

    hace 3 semanas

    Zapopan, Jalisco, México Oracle A tiempo completo

    Job Title: Senior Database EngineerWe are seeking a highly skilled Senior Database Engineer to join our Oracle Retail Science R&D group. As a key member of our team, you will be responsible for designing, developing, and deploying scalable and performing database solutions that meet the needs of our retail customers.Responsibilities:Design and implement data...

  • Senior Software Engineer

    hace 3 semanas

    Zapopan, Jalisco, México Ll Oefentherapie A tiempo completo

    Job Title: Senior Software EngineerWe are seeking a highly skilled Senior Software Engineer to join our team at Ll Oefentherapie. As a key member of our infrastructure team, you will be responsible for designing, developing, and maintaining Oracle Linux 8+ infrastructure.Key Responsibilities:Design and develop Oracle Linux 8+ infrastructure, including image...

  • Data Engineer

    hace 2 semanas

    Zapopan, Jalisco, México Oracle A tiempo completo

    Data Engineer - Analytics and Big DataWe are seeking a highly skilled Data Engineer to join our team at Oracle. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources.Key Responsibilities:Design and develop data...

  • Data Engineer, Science

    hace 1 mes

    Zapopan, Jalisco, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    About the RoleWe are seeking a highly skilled Data Engineer to join our Amazon Devices team in Guadalajara, Mexico. As a key member of our Science and Data technology team, you will design, implement, and evangelize state-of-the-art solutions for complex data problems.Key ResponsibilitiesDesign and implement data pipelines, architectures, and solutions for...

  • Senior Data Engineer, DS2-Science

    hace 1 semana

    Zapopan, Jalisco, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Amazon Devices. As a key member of our team, you will design, implement, and maintain data warehouses and systems to support our business needs.Key ResponsibilitiesDesign and implement data warehouse architectures using ETL/ELT tools and scripting languages.Develop and maintain...

  • Senior Data Science Engineer

    hace 3 semanas

    Zapopan, Jalisco, México Oracle A tiempo completo

    We are seeking a highly skilled Data Science Engineer to join our team at Oracle. As a key member of our agile environment, you will work closely with cross-functional teams to design and develop innovative solutions for product design, integration, and testing.**Key Responsibilities:** Design and develop RESTful Web Services to meet business requirements...

  • Senior Oracle DBA Engineer

    hace 2 meses

    Zapopan, Jalisco, México Ll Oefentherapie A tiempo completo

    Job Title: Senior Oracle DBA EngineerWe are seeking a highly skilled and experienced Senior Oracle DBA Engineer to join our team at Ll Oefentherapie. As a key member of our Oracle Retail Science R&D group, you will be responsible for designing, implementing, and maintaining large-scale Oracle databases that support our data-driven applications.Key...

  • Data Engineer

    hace 3 semanas

    Zapopan, Jalisco, México Oracle A tiempo completo

    Job SummaryWe are seeking a highly skilled Data Engineer to join our team at Oracle. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources.Key ResponsibilitiesDesign and implement distributed computing...

  • Data Engineer

    hace 3 semanas

    Zapopan, Jalisco, México Oracle A tiempo completo

    We are seeking a skilled Data Engineer to design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources.Career Level - IC4**Responsibilities**:Data Engineering:1. Collaborate with cross-functional teams to understand data requirements and deliver innovative solutions for data...