About the Company
ATLANSE is a digital services company focused on sustainable and responsible IT transformation. The company supports clients in optimizing the performance of their information systems while valuing human capital as its most important resource. Operating in the Services & Information Systems sector, ATLANSE provides expertise in data engineering, analytics, and digital solutions for organizations seeking scalable, efficient, and sustainable technology.
About the Role
The Senior PySpark Data Engineer role is responsible for ensuring the performance, reliability, and scalability of large-scale data pipelines. The position focuses on optimizing existing pipelines, implementing industrialization best practices, and supporting RUN and stabilization activities in an Agile environment. The role also involves collaborating closely with Data Scientists and ML Engineers to enhance architecture and performance for global deployments.
Key Responsibilities
- Audit, refactor, and evolve data engineering modules
- Optimize PySpark pipelines for large-scale data processing (partitioning, joins, caching, skew, volume)
- Improve modularity, readability, scalability, and maintainability of pipelines
- Reduce technical debt and modernize legacy systems
- Structure a flexible framework for adding new features and industrializing developments
- Implement unit, integration, and functional tests
- Formalize development standards and best practices
- Monitor pipelines and enhance stability through CI/CD practices
- Structure and prioritize RUN and stabilization actions
- Define and execute prioritized optimization plans with measurable gains
- Adapt pipelines for global and multi-region deployment
- Optimize cost/performance ratio of data processing
- Collaborate with Data Scientists and ML Engineers to improve architecture and processes
Requirements
- 7+ years of experience in data engineering
- Strong expertise in Apache Spark, PySpark, Scala, Databricks, Python
- Experience with AWS services: S3, Glue, Lambda, Redshift, ECR
- Knowledge of Docker, uv, and Poetry
- Experience with Airflow and GitHub
- Familiarity with Agile methodologies and Scrum
- Professional proficiency in French and English
Personal Qualities
- Strong analytical and problem-solving skills
- Rigor and ability to prioritize and structure work
- Autonomy and proactive mindset
- Team spirit and collaboration across functions
- Commitment to continuous improvement
Benefits
- Full-time permanent contract (CDI)
- Partial remote work option
- Opportunity to work on large-scale data projects in retail
- Professional development in modern data engineering technologies
- Collaborative and impact-driven work environment