Senior Python Developer with PySpark and AWS

Job summary

Integrate large datasets into the research ecosystem through the resource platform. Help rationalize and integrate the Analytical Work Bench with another internal application, including development of a JupyterLab-based IDE using client-developed custom code.

Job details

Project Description: The project involves the data engineering organization’s Analytical Work Bench , which is an integrated development environment. The objective is to rationalize and integrate AWB with another internally developed application. Additionally, the project includes creating an IDE based on the Jupyter Lab environment using custom code developed by Client. Responsibilities: - Integrate large datasets into the research ecosystem via the resource platform. Mandatory Skills Description: • A university degree in Engineering or Computer Science is preferred. Hard skills: • Proven experience working with cloud-based technologies, specifically AWS. • Solid expertise in Big Data solutions utilizing AWS EMR. • Advanced proficiency in Python Core and PySpark, with over 8 years of hands-on experience. • Practical knowledge of Pylint and Pytest for code quality and testing, with at least two years of experience. • Extensive experience with package management and automation tools such as PyPI and Gulp for over five years. • Familiarity with continuous integration and continuous delivery pipelines using Jenkins and Urban Code, with more than two years of experience. Soft skills: • Excellent communication skills demonstrated through collaboration with end users to gather and discuss complex business requirements effectively. Nice-to-Have Skills Description: - UI development with React: Familiarity with building user interfaces using React is advantageous. - JupiterLab: Experience working with JupiterLab is a beneficial asset. - Data Engineering and/or Data Science experience: Having skills or experience in data engineering or data science is a plus. Languages: English: C1 Advanced

What you’ll do

Integrate large datasets into the research ecosystem through the resource platform. Help rationalize and integrate the Analytical Work Bench with another internal application, including development of a JupyterLab-based IDE using client-developed custom code.

Requirements

Requires over eight years of hands-on experience with Python and PySpark, cloud technologies including AWS and EMR, and experience with code quality, testing, package management, automation, and CI/CD tools. Strong communication skills are needed to collaborate with end users on complex business requirements; an engineering or computer science degree is preferred.

Listed skills

• React · Preferred
• Communication · Preferred
• Amazon Web Services · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Python
• PySpark
• AWS
• AWS EMR
• Big Data
• Pylint
• Pytest
• PyPI
• Gulp
• Jenkins
• UrbanCode
• JupyterLab
• React
• Data Engineering
• Data Science
• Communication

Job areas

• Technology
• Software
• Data & Analytics
• Engineering

More jobs from Luxoft