Data Engineer Spark

**Who We Are****Build. Scale. Sustain.**PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. **We do the right thing, and we do it right.**We're proud to be a World Economic Forum New Champion, and a B Corp-certified company.- We are **small enough to care locally, big enough to deliver globally** (5 continents, 18 offices, +650 experts from +50 nationalities)- We are **robust and resilient** (100% independent and 0 debt)- We are **entrepreneurs and passionate experts**: We invest in what we believe genuinely and work as a collective intelligence- We are **positive, courageous, caring, doers and committed to excellence****Your**R**ole**As a Data Engineer Spark, you will be responsible for designing and implementing robust, scalable data pipelines and ensuring data is ready for analytics, machine learning, and operational purposes. Working with stakeholders across teams, you will build solutions that leverage big data technologies to process and manage complex datasets.**Who**Y**ou**A**re**- Design, develop, and optimize large-scale data pipelines using Apache Spark.- Implement ETL (Extract, Transform, Load) processes to support data transformation and integration.- Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver actionable insights.- Develop data models, schemas, and storage solutions for structured and unstructured data.- Optimize data processing for performance, scalability, and cost-effectiveness.- Work with cloud platforms (AWS, Azure, or Google Cloud) to manage and deploy data workflows.- Ensure data security, quality, and governance are integrated into all data workflows.- Monitor and troubleshoot production pipelines, ensuring reliability and uptime.- Stay updated on advancements in big data technologies and recommend innovations to improve the data ecosystem.**What You Need for This Role**- 4+ years of experience in data engineering, with a strong focus on big data technologies.- Proficiency in Apache Spark and distributed data processing frameworks.- Experience in programming languages such as Python, Scala, or Java.- Strong understanding of data lakes, warehouses, and databases (e.g., Snowflake, Redshift, BigQuery).- Hands-on experience with cloud platforms such as AWS (Glue, EMR, S3), Azure (Databricks, Data Factory), or Google Cloud (BigQuery, Dataflow).- Familiarity with workflow orchestration tools like Apache Airflow or similar.- Deep knowledge of data formats such as Parquet, Avro, and ORC.- Experience with CI/CD pipelines and version control (e.g., Git).- Solid understanding of data governance and security principles.- Excellent problem-solving skills and the ability to work in an Agile environment.**You're aligned with our value by**:- Your willingness to do the right thing even when facing adversity- You care about the well-being of others and the world at large- You strive to approach things in a optimistic way- You nail the fundamentals, sweat the details- You understand the whole is more than the sum of its parts and actively work towards continuous improvement of the group**More About PALO IT**We're eager to adapt to change, learn from our experiences and move to meet our planet's urgent needs. We are continuously taking action to:- Become a climate net-zero company- Deliver projects with a positive impact- Train 100% of our workforce on impact- Achieve B Corp certification among all our offices across the globe- Continuously measure & improve employee happinessOur clients are amongst the world's most successful companies. We innovate with both established Fortune 1000s, SMEs and start-ups who aim to make an impact, become global leaders and address the world's most complex challenges.**What We Offer**- Stimulating working environments- Unique career path- International mobility- Internal R&D projects- Knowledge sharing- Personalized training- Entrepreneurship & intrapreneurshipFor more on our team culture and benefits, check out our careers page.