Data Ingestion Engineer
hace 4 semanas
- Years of Experience : 3+ years of experience in data engineering, ingestion pipelining, and ETL/ELT.
- Job Mode : Full-time.
- Work Mode : Remote.
The Data Ingestion Engineer will be responsible for managing data ingestion processes, platform management, and optimizing ETL/ELT pipelines on the Databricks platform. The role involves ensuring compliance with data quality, security, and PII management standards while utilizing system integration tools and CI/CD methodologies. This position also focuses on documenting workflows, managing metadata, and supporting the overall data pipeline infrastructure.
Responsibilities and Duties:- Ingest data from various source systems, adapting ingestion strategies as needed.
- Manage and oversee ETL/ELT pipelines on the Databricks platform.
- Optimize data pipelines for scalability, speed, and performance.
- Document ingestion, integration flows, and pipelines for clarity and future reference.
- Schedule and automate ingestion jobs using Apache Airflow.
- Manage metadata and master data within the technical data catalog.
- Ensure adherence to security, compliance guidelines, and PII management during data ingestion.
- Maintain pipeline infrastructure and implement automated monitoring strategies.
- Follow SDLC best practices to ensure quality and consistency.
- Bachelor's degree in Computer Science, Engineering, Statistics, or a related field.
- 3+ years of experience in data engineering, ingestion pipelines, and ETL/ELT processes.
- Strong knowledge of data lake ingestion processes and best practices.
- Expertise in CI/CD pipelines and version control systems like Git.
- Proficient in agile methodologies.
- Hands-on experience with Spark/Scala, SQL, and Python/PySpark.
- Proficiency in working with Databricks and Unity Catalog.
- Experience with ETL/ELT development, monitoring, and pipelining tools such as Apache Airflow.
- Knowledge of ingestion tools such as Dell Boomi.
- Strong understanding of data quality guidelines and best practices for managing large data sets.
- Databricks, Spark/Scala, SQL, Python/PySpark.
- Apache Airflow, Unity Catalog, Dell Boomi.
- CI/CD tools, Git, and version control.
- Data pipeline management and optimization.
- ETL/ELT pipeline development and monitoring.
- System integration and metadata management.
- Compliance with security and PII guidelines.
- Automated monitoring and CI/CD implementation.
Encora is the preferred digital engineering and modernization partner of some of the world's leading enterprises and digital-native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora's technology practices include Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering.
At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.
#J-18808-Ljbffr-
Data Platform Engineer
hace 4 semanas
distrito federal, México Infostretch Corporation A tiempo completoCompany Description: Apexon is a digital-first technology services firm backed by Goldman Sachs Asset Management and Everstone Capital. We specialize in accelerating business transformation and delivering human-centric digital experiences. For over 17 years, Apexon has been meeting customers wherever they are in the digital lifecycle and helping them...
-
Cloud Data Engineer Lead
hace 4 semanas
distrito federal, México MX003 Marsh And Mclennan Servicios S.A. De Cv A tiempo completoDescription : MMC is seeking candidates for the following position based in the Mexico City office and be onsite 3 days a week : Cloud Data Engineer Lead / Data Engineering Lead What can you expect? The Cloud Data Engineer Lead will lead a small team, and be part of a very talented large team building data ingestion and transformation pipelines in the...
-
Senior Data Engineer/Data Architect
hace 4 semanas
distrito federal, México LAAgencia A tiempo completoSenior Data Engineer/Data Architect (Azure) Mexico Tasks: Implement data ingestion pipelines from multiple data sources using Azure Data Factory, Azure Databricks and other ETL tools. Develop scalable and re-usable, self-service frameworks for data ingestion and processing. Design, build, and manage SQL Server databases in the Azure cloud. Perform data...
-
QA Engineer
hace 4 semanas
distrito federal, México Data Privacy A tiempo completoYou will be part of the QA team that is focused on quality on front end embedded client/TV as well as backend cloud services that reports high quality data to customers. The client QA team is responsible to test various embedded clients and firmware for a variety of Vizio TVs. The backend QA team is responsible to validate our award-winning Inscape data set...
-
Senior Data Scientist
hace 3 semanas
distrito federal, México Blue Orange Digital A tiempo completoCompany overview: Blue Orange Digital is a cloud-based data transformation and predictive analytics development firm with offices in NYC and Washington, DC. From startups to Fortune 500s, we help companies make sense of their business challenges by applying modern data analytics techniques, visualizations, and AI/ML. Founded by engineers, we love passionate...
-
Senior Data Scientist
hace 2 semanas
distrito federal, México Blue Orange Digital A tiempo completoCompany overview: Blue Orange Digital is a cloud-based data transformation and predictive analytics development firm with offices in NYC and Washington, DC. From startups to Fortune 500s, we help companies make sense of their business challenges by applying modern data analytics techniques, visualizations, and AI/ML. Founded by engineers, we love passionate...
-
Lead Data Engineer
hace 4 semanas
distrito federal, México Thomson Reuters A tiempo completoAre you passionate about the chance to bring your experience to a world-class company that is market-leading for both content and technology? If yes, we are looking for you! Join our team! We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, developing, and...
-
DevOps Engineer
hace 2 semanas
distrito federal, México NTT DATA, Inc. A tiempo completoNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a DevOps Engineer to join our team in Guadalajara, Jalisco (MX-JAL), Mexico (MX). Responsibilities: DevOps Engineer with over 5 years...
-
Data Analyst
hace 4 semanas
distrito federal, México Wood Mackenzie Inc. A tiempo completoIn the middle of the energy transition, businesses and governments are faced with significant challenges. But the pace and scale of change mean every decision is made under mounting pressure. Now, more than ever, companies need reliable data, analytics and actionable insight. Wood Mackenzie is the leading global provider of data and analytics solutions for...
-
Data Platform Engineer Databricks Senior
hace 4 semanas
distrito federal, México Marsh McLennan A tiempo completoMercerTech DnAi Platform engineering team is seeking candidates for the following position based in the CDMX office and be onsite 3 days a week: Sr. Data Platform Engineer MercerTech DnAi Platform engineering team is seeking an experienced Mid-Level Data Platform Engineer with experience on data with AWS cloud who has expertise in the administration,...
-
Senior Data Integration Engineer
hace 1 semana
distrito federal, México EPAM Systems A tiempo completoWe are in search of a Senior Data Integration Engineer with expertise in ETL/ELT solutions, SQL, data visualization, Google Cloud Platform, and Pentaho BI. Your primary role will involve supporting the entire data pipeline, identifying and resolving issues with data sources, understanding the logic behind queries, and communicating effectively with key...
-
Data platform engineer
hace 3 semanas
distrito federal, México MX003 Marsh And Mclennan Servicios S.A. De Cv A tiempo completoDescription : MercerTech DnAi Platform engineering team is seeking candidates for the following position based in the CDMX office and be onsite 3 days a week : Sr. Data Platform Engineer MercerTech DnAi Platform engineering team is seeking an experienced Mid-Level Data Platform Engineer with expertise in data and AWS cloud who has experience in the...
-
Data Analytics Consultant
hace 4 semanas
distrito federal, México Coca-Cola HBC A tiempo completoThe Coca-Cola Company is looking for a hands-on Data Analytics Consultant to analyze, interpret, and document business processes, products, services, and technology. This role will support data product teams to design, build, and implement data and analytics products and solutions for various stakeholders. Will work with cross-functional teams across...
-
Data Analytics Consultant
hace 3 semanas
distrito federal, México Coca-Cola A tiempo completoThe Coca-Cola Company is looking for a hands-on Data Analytics Consultant to analyze, interpret, and document business processes, products, services, and technology. This role will support data product teams to design, build, and implement data and analytics products and solutions for various stakeholders. Will work with cross-functional teams across...
-
Data Analytics Consultant
hace 4 semanas
distrito federal, México Coca-Cola A tiempo completoThe Coca-Cola Company is looking for a hands-on Data Analytics Consultant to analyze, interpret, and document business processes, products, services, and technology. This role will support data product teams to design, build, and implement data and analytics products and solutions for various stakeholders. Will work with cross-functional teams across...
-
Data Engineer/Integration Lead
hace 4 semanas
distrito federal, México Infosys Limited A tiempo completoCompany Requisition ID: 122859BR Required Qualifications: 5+ years of experience in data engineering using Python with a focus on AWS S3, EMR, Glue, Step Functions, Apache NiFi and Spark. Proven track record of building scalable data pipelines in cloud environments. Proficiency in flow design, processors, and data provenance in Apache NiFi. Strong expertise...
-
Data Engineer/Integration Lead
hace 4 semanas
distrito federal, México Infosys A tiempo completoRequired Qualifications: 5+ years of experience in data engineering using Python with a focus on AWS S3, EMR, Glue, Step Functions, Apache NiFi and Spark. Proven track record of building scalable data pipelines in cloud environments. Proficiency in flow design, processors, and data provenance in Apache NiFi. Strong expertise in Spark, Hadoop, and...
-
Head of Data Engineering
hace 3 semanas
distrito federal, México Chubb Ltd. A tiempo completoTeam Leadership and Management: Lead and oversee a medium team, ensuring their professional development, motivation, and high performance. Provide coaching, guidance, and technical expertise to team members, promoting a culture of continuous learning and growth. Establish clear goals and expectations, monitor progress, and provide regular feedback to...
-
Data Transformation Engineer
hace 4 semanas
distrito federal, México S&P Global A tiempo completoAbout the Role: Grade Level (for internal use): 09 The Team: The Forecast Ops team within the Analytics Enablement group drives innovation and efficiency gains for economic forecasting processes. The team creates data pipelines, designs automated processes, and assists forecast product teams with improving their procedures. Responsibilities and Impact: We...
-
Data Analytics Consultant
hace 3 semanas
distrito federal, México The Coca-Cola Company A tiempo completoLocation(s): Mexico City/Cities: Mexico Travel Required: 00% - 25% Relocation Provided: No Job Posting End Date: October 19, 2024 Job Description Summary: The Coca-Cola Company is looking for a hands-on Data Analytics Consultant to analyze, interpret, and document business processes, products, services, and technology. This role will support data...
