Big Data Engineer

Company DescriptionVisa is a world leader in digital payments, facilitating more than 215 billion payments transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories each year.Our mission is to connect the world through the most innovative, convenient, reliable and secure payments network, enabling individuals, businesses and economies to thrive.When you join Visa, you join a culture of purpose and belonging - where your growth is priority, your identity is embraced, and the work you do matters.We believe that economies that include everyone everywhere, uplift everyone everywhere.Your work will have a direct impact on billions of people around the world - helping unlock financial access to enable the future of money movement.**Join Visa: A Network Working for Everyone.****Job Description**:Visa Consulting and Analytics (VCA), the consulting arm of Visa, is a global team of industry experts in strategy, marketing, operations, risk and economics consulting, with decades of experience in the payments industry.Our VCA teams offers:- Consulting services customized to the needs of Visa client's business objectives and strategy- Business and economic insights and perspectives that impact business and investment decisions- Self-service digital solutions Visa clients can leverage to improve performance in product, marketing and operations- Proven data-driven marketing strategies to increase clients' ROIHe/She must have experience using a variety of data mining/data analysis methods, using a variety of distributed data platforms, and leveraging the latest open-source technologies.He/She must have a proven ability to drive business results with their data-based insights.Adept at creative and critical thinking, be able to deconstruct problems and transform insights into large scale, state-of-the-art solutions.**Responsibilities**- Automate and standardize data processes developed by team members.- Leverage DevOps to create end-to-end streamline CI/CD data and ML pipelines.- Review and manage data pipelines, branching, and deployment process.- Work with partners on requirements and implementation designs of data solutions.- Implement data quality framework at scale using open-source technologies.- Create data monitoring dashboards with real-time notifications.- Unify data engineering and machine learning engineering pipelines.- Document process, designs, test results, and analysis.- Ability to articulate complex architectures to non-technical audiences, management, and leadership.- Continuously research industry best practices and technologies.- Evangelize end to end automation and standardization across the organization.- Partner with functional areas, and regional and global teams to leverage the breadth and depth of Visa's resources.- This is a hybrid position.Hybrid employees can alternate time between both remote and office.Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs._**Qualifications**:Basic Qualifications- BA/BS required, MBA or other relevant Master's degree preferred (e.g. engineering, computer science, computer engineering, applied mathematics, or other related fields)Preferred Qualifications- At least 5 years of experience as data engineer or data scientist with open-source tools.- Experience in retail banking, payments, financial services, and/or technology industries is a plus.Strong interest in the future of payments is a must.- Strong technical competency and experience with shell-scripting and Linux systems.- Experience with CI/CD pipeline using Azure DevOps, GitHub actions, Jenkins, or Airflow.- Strong coding skills in Spark, Python and SQL to manipulate big data in distributed platforms.- Good to have experience in navigating in Linux/Unix/Container based apps such as Docker, Kubernetes, or Microservices environments.- Knowledge in how to leverage AI assistance tools like chatGPT for creating and debugging code.- Ability to interact with big data clusters using Jupiter Notebooks, terminal, or GUI.- Demonstrate experience leveraging open-source tools, libraries, and platforms.- Experience with data visualization and business intelligence tools like Tableau, PowerBI, Microstrategy, or Excel.- Problem solving ability and process creator with strategic focus on replicability, scalability, innovation, and governance.- Proficient with git for version control and code collaboration using branches and pull requests.- Must be passionate about automation and data and able to deliver high quality work.- Experience developing as part of Agile/Scrum team.- Fluency in English (spoken/written).Portuguese or Spanish is a plus.Additional InformationVisa is an EEO Employer.Qualified applicants will receive c