Data Engineer

Department Overview

The world is changing rapidly, and our customers are inventing new ways to meet that change confidently - with our help. We transform engineering and technical workflows with solutions that provide the right data at the right time. Our unparalleled combination of expansive technical knowledge and AI-based technology helps leaders build the right connections across their teams and workflows, so they can focus on designing for a world that runs faster, cleaner, safer, and smarter for everyone.

Our development team architect and design high-availability, scalable, and fault tolerant systems that are decoupled and easy-to-maintain. A core part of our development philosophy revolves around Microservices and the DevOps model. All our new products are developed using a microservice architecture, are containerized, and are then deployed on container management systems such as Kubernetes. The developers on our teams subscribe to a DevOps model where time-to-market functions as a vital measure of our performance, productivity, and success. We are committed to stay ahead of the curve and we are always looking at new technologies that can enhance our product offerings.

Position Summary

Are you passionate for the latest technologies in Data Engineering critical for success of Data Science, Machine Learning projects in Natural Language Processing domain? Come and be part of the S&P Global’s Artificial Intelligence team We are building deep-learning based natural language processing, information retrieval, document understanding, data mining and knowledge engineering solutions into S&P Global intelligent products that serve all major industries and markets.

S&P Global is looking for a Data Engineer/Python (NLP, Knowledge Management and Information Retrieval domain) to join our AI Research & Development department. In this role, you will be responsible for data engineering aspects of data-driven projects, building robust data processing pipelines and curating all questions related to data lifecycle.

Job

**Responsibilities**:

- Own data engineering in projects with Machine Learning, Natural Language Processing, Information Retrieval
- Work in the team with data scientists, ML engineers and developers on building the intelligent capabilities into company products
- Ensure dataset quality and suitability for ML projects (automation of labeling, inspection and cleaning, normalization, augmentation)
- Discover new data (finding and obtaining raw data necessary for experimental setups, e.g. find and download available data from internet with focused crawling)
- Develop data processing and transformation pipelines (designing ETL system for ML/DL projects, designing online leaning loops, embedding active learning algorithms into data annotation toolset, etc.)
- Organize data warehousing, storage and versioning (make sure ML experiments are repeatable and keep track from data state)

Required Qualifications & Experience
- Strong coding and software engineering skills
- Ability to make good design decisions related to data
- Python programming experience 2+ years
- Experience with textual data engineering (encoding, formats, tools)
- Developed skills in algorithms and data structures
- Experience with data processing automation, schedulers and pipeline tools (Airflow, Beam, make, etc.)
- Experience with big data tools (Hadoop, Hive or Spark, etc.)
- Advanced Linux experience (Bash, CL tools)
- English language (B1+)
- Experience with SQL, NoSQL and Graph databases
- MS or PhD degree related to computer science, data science or statistics
- Publications in related domain
- Experience on projects with deep learning, natural language processing or information retrieval
- C++ programming
- Experience in Machine Learning

Our projects

We are building next generation knowledge discovery platform empowered with high quality intelligent functions built on top of world class natural language processing and document understanding engine for many languages.
- Enterprise and industry-wide knowledge management,
- Semantic search,
- Natural language query understanding,
- Question answering and semantic facets,
- Cross-lingual information retrieval,
- Automatic summarization,
- Machine translation,
- Document logical layout understanding,
- Metadata synthesis,
- Subject domain categorization,
- Named entity recognition,
- Key phrase extraction,
- Similarity detection,
- Dynamic data linking,
- Data capturing,
- and much more.

Methods that we are using
- development of own AI/ML architectures based on deep learning,
- training on powerful private GPU cloud,
- domain-specific language representation,
- word-, sentence
- and document embedding,
- feature engineering,
- learning to rank,
- dataset development from any sources: own annotators team, raw data bootstrapping, click-data analysis, controlled expert annotations.