Specialist - Data Engineering

Job summary

Lead the design, development, deployment, and optimization of scalable big data solutions using Scala, Apache Spark, and Spark SQL. Collaborate with stakeholders and cross-functional teams, guide junior developers, review code, monitor production systems, and ensure data governance and security compliance.

Job details

Role Description Job role : Spark and scala developer Location : Mississauga ,Ontario Job Summary Seeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design develop and optimize scalable big data processing solutions within the Hadoop Spark SparkSQL ecosystem Job Description Design develop and optimize scalable data processing applications using Scala and Apache Spark Collaborate with crossfunctional teams including data scientists analysts and architects to build and maintain robust data pipelines Analyze large and complex datasets to extract actionable insights supporting business objectives Develop highperformance applications in distributed computing environments leveraging SparkSQL Ensure code quality maintainability and adherence to Scala programming best practices Continuously evaluate and integrate emerging big data technologies and tools to enhance platform capabilities Troubleshoot and resolve complex data processing issues to maintain system reliability Document technical designs and implementation details for effective knowledge sharing Roles and Responsibilities Lead the development and deployment of big data solutions leveraging Scala Apache Spark and SparkSQL Collaborate with stakeholders to gather requirements and deliver effective data solutions Optimize Spark jobs for performance and scalability in production environments Mentor junior team members and provide technical guidance on best practices Conduct code reviews to ensure adherence to coding standards and promote continuous improvement Monitor system performance proactively and address bottlenecks or failures Participate in sprint planning estimation and agile ceremonies to deliver highquality software Ensure compliance with data governance and security policies

What you’ll do

Lead the design, development, deployment, and optimization of scalable big data solutions using Scala, Apache Spark, and Spark SQL. Collaborate with stakeholders and cross-functional teams, guide junior developers, review code, monitor production systems, and ensure data governance and security compliance.

Requirements

The role seeks 5 to 7 years of experience with Scala and Apache Spark in big data processing environments. Candidates should be able to build and optimize distributed data applications, maintain code quality, troubleshoot complex issues, and provide technical guidance.

Listed skills

• Data analysis · Preferred
• Troubleshooting · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Scala
• Apache Spark
• Spark SQL
• Big Data Processing
• Data Pipeline Development
• Distributed Computing
• Performance Optimization
• Data Analysis
• Troubleshooting
• Code Review
• Technical Mentoring
• Data Governance
• Data Security
• Agile Methodologies

Job areas

• Technology
• Data & Analytics
• Software
• Engineering

More jobs from LTM