Data Engineer

**Who We Are**:
**Build. Scale. Sustain.**:
**PALO IT** is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We do the right thing, and we do it right. We're proud to be a World Economic Forum New Champion, and a B Corp-certified company.

- We are small enough to care locally, big enough to deliver globally (5 continents, 18 offices, +650 experts from +50 nationalities)

- We are robust and resilient (100% independent and 0 debt)

- We are entrepreneurs and passionate experts: we invest in what we believe genuinely and work as a collective intelligence

- We are positive, courageous, caring, doers and committed to excellence

**About Gen-e2**:
While the market is still largely AI-augmenting delivery, we have reinvented the SDLC to be AI First. Our approach is a game-changer in productivity and quality, with a strong collaboration between generative AI and our best Talents:

- We now generate 95% of the entire product — code, documentation, infrastructure as code, and even design — with GitHub Copilot.

- The quality consistently exceeds the output of our best traditional engineering teams.

- A product repository houses all product artefacts, giving AI full project context for higher-quality generation.

- A library of rules and prompts defines coding standards, design principles, and security guidelines, ensuring enterprise-grade quality and scalability.

With **Gen-e2**, we deliver end-to-end products 2-3× faster than traditional approaches, while raising the bar for engineering excellence.

**Your Role**:
We're looking for a **Data Engineer** to support our partner in managing the influx of weather-related datasets that are critical for their business transformation in the insurance domain. You'll be the go-to expert for ingesting, organizing, and preparing weather-based data pipelines that drive insights and automation.

**Key Responsibilities**:

- Design, develop and manage scalable and secure data pipelines to ingest and process large volumes of weather data from various external sources.
- Leverage APIs, batch, and streaming ingestion mechanisms for real-time and historical weather datasets.
- Collaborate with data scientists, solution architects, and engineering teams to integrate weather data into products and machine learning workflows.
- Implement data quality checks, validation routines, and anomaly detection specific to weather information (e.g., temperature, precipitation, wind patterns).
- Optimize storage and compute costs for high-volume weather datasets, ensuring data pipelines are cost-effective and resilient.
- Maintain documentation and metadata to provide transparency and governance over data usage.

**Who You Are**:

- Proven experience working as a **Data Engineer**, ideally within insurance, climate tech, or industries where weather data plays a pivotal role.
- Strong proficiency in **Python**, **SQL**, and at least one major cloud platform (AWS preferred; GCP or Azure also welcome).
- Experience working with **weather data APIs**, such as NOAA, OpenWeatherMap, Weather Company, Meteomatics, or similar.
- Proficient in **ETL tools**, **data orchestration frameworks** (Airflow, Prefect, Dagster), and streaming platforms (Kafka, Kinesis).
- Familiar with geospatial data formats and processing tools (GeoJSON, raster data, GDAL, PostGIS).
- Solid understanding of **data lakes**, **data warehouses** (Snowflake, BigQuery, Redshift), and data modeling best practices.
- Strong problem-solving skills with attention to detail and a proactive mindset.
- B2-C1 level in **English**

**:
**More About PALO IT**:
We're eager to adapt to change, learn from our experiences and move to meet our planet's urgent needs. We are continuously taking action to:

- Become a climate net-zero company

- Deliver projects with a positive impact

- Train 100% of our workforce on impact

- Achieve B Corp certification among all our offices across the globe

- Continuously measure & improve employee happiness

Our clients are amongst the world's most successful companies. We innovate with both established Fortune 1000s, SMEs and start-ups who aim to make an impact, become global leaders and address the world's most complex challenges.

**What We Offer**:

- Stimulating working environments

- Unique career path

- International mobility

- Internal R&D projects

- Knowledge sharing

- Personalized training

- Entrepreneurship & intrapreneurship opportunities

For more on our team culture and benefits,
Check out our careers page
️ And our PALOCast with direct testimonies from our Palowans