Lead Data Engineer

**Project** Description**:Luxoft DXC Technology Company is an established company focusing on consulting and implementation of complex projects in the financial industry. At the interface between technology and business, we convince with our know-how, well-founded methodology and pleasure in success. As a reliable partner to our renowned customers, we support them in planning, designing and implementing the desired innovations. Together with the customer, we deliver top performanceFor one of our Client in the Insurance Segment, we are searching for a Lead Data Engineer.**Responsibilities**:- Demonstrates ownership and teamwork. Leads data product design, code development, production deployment, and production support efforts.- Coordinates Data product development efforts with other Data Engineers in the squad and helps Product Owner in work assignments. Represents other Data Engineers and provides updates on overall development efforts- Represents Data product changes in CAB meetings.- Leads resolution and implementation of all production defects and changes- Reviews the code developed by other Data Engineers and ensures data engineering best practices are followed- Actively supports Data products production runs, validates the data products for completeness & accuracy and coordinates with Production support teams- Ensures implementation of data and IT controls in Data products- Actively involved in creation of data products including development, testing, deployment, and production support. Required to perform following activities hands-on- Explore and understand data sets. o Building data pipelines, integrating, and scheduling ETL jobs using CI/CD framework. o Implement required data transformation in the data lake. o Visualize the data set; determine whether the data set has enough information to answer the data product requirements. o Work with IT support to create and visualize the data and data products on the data lake. o Configure required security and data masking to a data set. o Support testing of data acquisition, data set correlation, and / or product development. Investigate; resolve data and interface issues o Work with IT to harden and productionize data products, and business procedures.**Skills**:Must have- Master's/ Bachelor's Degree in one of the following: Engineering, Computer Science, Statistical Analytics, Data Science, or Actuarial Science.- Big Data/Hadoop Technical Lead with extensive experience on the Hadoop platform and related Big Data tools/technologies.- 5+ Years of experience in Big Data Technologies, 10+ years' experience in Data Warehouse, Business Intelligence, and analytics area. o 3 + Years of experience in Hadoop Eco Systems o 2+ Years of experience in Spark o 1+ years' experience with Microsoft Azure and/or AWS o Any exposure to Mainframe as a source is preferred- Good knowledge of architecture, design patterns, Source target mappings, ETL Architecture in Hadoop space, data modeling techniques, performance tuning in Hadoop environment.- Experience with big data tools: Hadoop (CDH), Spark, Kafka, PySpark, python, Impala, Hive, Hue, HDFS and related tools- Experience in Azure DevOps and CI/CD tools such as Maven, Git/Stash, Jenkins, Docker etc.- Experience in Tableau data visualization tool is plus- Advance level knowledge and experience with SQL queries including performance optimization techniques- Working experience in Shell Scripts, Oozie workflows, scheduling tools (Stone branch or CA7).- Able to analyze huge volume of data and understand patterns, data quality issues and design solutions to manage volume.- Experience working in Agile environment- Ability to modularize the project components, architect the complete project and provide solution architecture.- Excellent written, verbal, and interpersonal skills, a must as there will be significant collaboration with the business and IT- Experience with collaborative development workflows (e.g., Microsoft DevOps Tools).Nice to have**Languages**:English: C1 Advanced**Seniority**:Lead**Relocation package**:If needed, we can help you with relocation process.Vacancy SpecializationBigData DevelopmentRef NumberVR-95718