Sr. Big Data Engoneer

hace 7 meses


Zapopan, México Intugo A tiempo completo

**Position Summary**:
Our team is looking for a **Sr. Data Engineer** with a diverse background in data integration to join the Data Services team.
Some data are small, some data are very large (1 trillion+ rows), some data is structured, some data is not.
Our data comes in all kinds of sizes, shapes and formats.
Traditional RDBMS like PostgreSQL, Oracle, SQL Server, MPPs like StarRocks, Vertica, Snowflake, Google BigQuery, and unstructured, key-value like MongoDB, Elasticsearch, to name a few.

We are looking for individuals who can design and solve any data problems using different types of databases and technologies supported within our team.
We use MPP databases to analyze billions of rows in seconds.
We use Spark and Iceberg, batch or streaming to process whatever the data needs are.
We also use Trino to connect all different types of data without moving them around.

Besides a competitive compensation package, you’ll be working with a great group of technologists interested in finding the right database to use, the right technology for the job in a culture that encourages innovation.
If you’re ready to step up and take on some new technical challenges at a well-respected company, this is a unique opportunity for you.

**Responsibilities**:

- Implement ETL/ELT processes using various tools and programming languages (Scala, Python) against our MPP databases StarRocks, Vertica and Snowflake
- As Data Engineers, not only are we developers, but we also maintain and administrate our MPP ecosystem. We will tune and maximize our hardware potential from OS, network and storage levels
- Work with the Hadoop team and optimize Hive and Iceberg tables
- Running POC between different table formats
- Contribute to the existing Data Lake and Data Warehouse imitative using Hive, Spark, Iceberg, Presto/Trino
- Analyze business requirements, design and implement required data models

**Qualifications: (must have)**
- BA/BS in Computer Science or in related field
- 2+ years of experience with MPP databases such as StarRocks, Vertica, Snowflake
- 5+ years of experience with RDBMS databases such as Oracle, MSSQL or PostgreSQL
- 2+ years of experience managing or developing in the Hadoop ecosystem
- Programming background with Scala, Python, Java or C/C++
- Experience with Elasticsearch or ELK stack
- Working knowledge of streaming technologies such as Kafka
- Strong in any of the Linux distributions, RHEL,CentOS or Fedora
- Deep knowledge shell scripting, scheduling, and monitoring processes on Linux
- Experience working in both OLAP and OLTP environments
- Experience working on-prem, not just cloud environments

**Desired: (nice to have)**
- Working knowledge of data unification and setup using Presto/Trino
- Working knowledge of orchestration tools such Oozie and Airflow
- Experience with Spark. PySpark, SparkSQL, Spark Streaming, etc
- Experience using ETL tools such as Informatica, Talend and/or Pentaho
- Understanding of Healthcare data
- Data Analyst or Business Intelligence would be a plus

Tipo de puesto: Tiempo completo, Por tiempo indeterminado

Salario: $55,000.00 - $60,000.00 al mes

Horario:

- Lunes a viernes

Experiência:

- Oracle: 5 años (Obligatorio)
- MSSQL: 5 años (Obligatorio)
- PostgreSQL: 5 años (Obligatorio)
- Fedora: 5 años (Obligatorio)
- Python: 5 años (Obligatorio)
- CentOS: 5 años (Obligatorio)

Idioma:

- Inglés (Obligatorio)

Lugar de trabajo: remoto híbrido en 45050, Zapopan, Jal.



  • Zapopan, Jalisco, México Oracle A tiempo completo

    About Oracle Cloud InfrastructureOracle Cloud Infrastructure is a comprehensive cloud service offering spanning IaaS, PaaS, and SaaS. It is a strategic growth area for Oracle, focusing on building a future-ready Gen2 cloud Data Science service platform.The core of this platform lies in the Big Data Service, a fully automated and handled cloud service...

  • Big Data

    hace 7 meses


    Zapopan, México Oracle A tiempo completo

    Career Level - IC4 **Responsibilities**: Data Engineering: 1. Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources. 2. Implement and optimize distributed computing frameworks such as Apache Hadoop, Apache Spark, or Apache Flink to handle big data processing tasks...


  • Zapopan, México Oracle A tiempo completo

    **Our Team : Oracle GoldenGate for BigData** Oracle GoldenGate (OGG) is a comprehensive software package for real-time data integration and replication in heterogeneous IT environments. Oracle GoldenGate for Big Data streams transactional data into big data systems in real time, raising the quality and timeliness of business insights. For more information,...

  • Sr Data Engineer

    hace 4 meses


    Zapopan, México Ryscode A tiempo completo

    Description Looking for a Sr. Data Engineer located in GUADALAJARA, México or willing to relocate for a full-time on-site work scheme. Required Technologies Python, AWS, Kafka, snowflake Good English communication GDL Position: ON-SITE Tipo de puesto: Tiempo completo Sueldo: $60,000.00 - $68,400.00 al mes Beneficios: - Caja de ahorro - Seguro de...

  • Big Data Engineer

    hace 7 meses


    Zapopan, México Intugo A tiempo completo

    **Position Summary**: Our team is looking for a **Data Engineer** with a diverse background in data integration to join the Data Services team. Some data are small, some data are very large (1 trillion+ rows), some data is structured, some data is not. Our data comes in all kinds of sizes, shapes and formats. Traditional RDBMS like PostgreSQL, Oracle, SQL...

  • Big Data Engineer

    hace 3 semanas


    Zapopan, Jalisco, México Intugo A tiempo completo

    Data Engineering RoleAt Intugo, we are looking for a skilled Data Engineer to join our Data Services team. This position requires a diverse background in data integration, with experience in various database systems and technologies.As a Data Engineer, you will be responsible for designing and solving complex data problems using different types of databases...

  • Sr Data Scientist

    hace 4 meses


    Zapopan, México Ryscode A tiempo completo

    Description Looking for a Sr. Data Scientist with knowledge of Core ML located in **GUADALAJARA, Mexico or willing to relocate for a full-time on-site work scheme.** **Requirements**: Python: Proficiency in Python is essential as it is the primary programming language used.AWS: Experience with AWS services is required for deploying and managing machine...

  • SAP Pp Sr

    hace 6 meses


    Zapopan, México NTT DATA A tiempo completo

    We are currently seeking a SAP PP Sr to join our team in Zapopan, Jalisco (MX-JAL), Mexico (MX). **Role**: SAP PP Consultant - At NTT DATA, we know that with the right people on board, anything is possible. The quality, integrity, and commitment of our employees are key factors in our company's growth, market presence and our ability to help our clients...


  • Zapopan, México Oracle A tiempo completo

    **Data Analytics Engineer** Why Oracle NetSuite Global Business Unit (NSGBU)? One word — transformation. At Oracle NetSuite Global Business Unit (NSGBU) we believe the cloud is here to stay and so do our 20,000+ customers. We believe businesses should not be bogged down by the overhead of bulky data centers and expensive personnel to run it all....


  • Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    5+ years of experience with data warehouse technical architectures, ETL/ ELT, reporting/analytic tools and, scripting. - 5+ years of demonstrated quantitative and qualitative data experience with data modeling, ETL development - Knowledge of data modeling and experience SQL with Redshift, Oracle, MySQL, and Columnar Databases - Experience managing competing...


  • Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    5+ years of experience with data warehouse technical architectures, ETL/ ELT, reporting/analytic tools and, scripting. - 5+ years of demonstrated quantitative and qualitative data experience with data modeling, ETL development - Knowledge of data modeling and experience SQL with Redshift, Oracle, MySQL, and Columnar Databases - Experience managing competing...

  • Data Engineer Leader

    hace 3 semanas


    Zapopan, Jalisco, México Derevo A tiempo completo

    Job Title: Data Engineer LeaderWe are seeking a highly skilled Data Engineer Leader to join our team at Derevo. As a key member of our engineering team, you will be responsible for designing, establishing, and maintaining data management and storing systems.About the Role:Design, develop, and maintain large-scale data systems, including data warehouses, data...

  • Data Engineer Jr

    hace 7 meses


    Zapopan, México Derevo A tiempo completo

    ¡¡Buscamos tu talento!! ✋ **DATA ENGINEER JR** En **Derevo** buscamos empoderar a las empresas y a las personas para liberar el valor de los datos en las organizaciones, a través de la implementación de procesos y plataformas de analítica con un enfoque que cubre el ciclo completo que necesitan llevar a cabo para lograrlo. Derevo inicio en 2010 con...


  • Zapopan, Jalisco, México Oracle A tiempo completo

    We are seeking an experienced professional to join our team as a Data Engineering and Data Warehousing Specialist.OverviewAs a key member of our team, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources.You will implement and...


  • Zapopan, México Oracle A tiempo completo

    We are looking for Big Data developers and Analytics skills. Preferred experience in Cloud development, Spark, SQL, Python, Data Bases and scripting. Career Level - IC2 **Responsibilities**: Data Engineering: 1. Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse...


  • Zapopan, Jalisco, México Oracle A tiempo completo

    Oracle Cloud Infrastructure OverviewOracle Cloud Infrastructure (OCI) is a strategic growth area for Oracle, offering a comprehensive cloud service in the enterprise software industry. It spans Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).We are building a future-ready Gen2 cloud Data Science service...


  • Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    Experience using SQL to pull data from a database or data warehouse and scripting experience (Python) to process data for modeling - Experience in data mining, ETL, etc. and using databases in a business environment with large-scale, complex datasets - Experience with data visualization using Tableau, Quicksight, or similar tools - Bachelor's degree in...


  • Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    Experience using SQL to pull data from a database or data warehouse and scripting experience (Python) to process data for modeling - Experience in data mining, ETL, etc. and using databases in a business environment with large-scale, complex datasets - Experience with data visualization using Tableau, Quicksight, or similar tools - Bachelor's degree in...

  • Data Engineer, Science

    hace 7 meses


    Zapopan, México Servicios Comerciales Amazon Mexico S. de R.L. de C.V. - D44 A tiempo completo

    3+ years of data engineering experience - Experience with data modeling, warehousing and building ETL pipelines The Amazon Devices team designs and engineers high-profile consumer electronics, including the best-selling Kindle family of products. We have also produced groundbreaking devices like Fire tablets, Fire TV, Amazon Dash, and Amazon Echo. What...


  • Zapopan, México Oracle A tiempo completo

    **About the Company and Team **The Infrastructure Industries Global Industry Unit’s (IGIU) mission is to build and deliver technology and solutions that improve the lives of global citizens. For Energy & Water, that’s striving to ensure that every global citizen has access to clean and affordable energy and water. For Construction & Engineering, it’s...