GenAI Prompt/Automation Engineer

Job summary

Design and refine prompts, validate AI-generated responses, and develop and execute test cases to assess quality, accuracy, robustness, and user experience. Provide structured feedback and collaborate with data scientists, engineers, and product managers to improve AI model behavior and functionality.

Job details

We are looking to hire GenAI Prompt/Automation Engineer at Brampton/Toronto, ON. Key Responsibilities: -Prompt Writing: Design and refine creative, effective, and contextually relevant prompts for generative Al systems to maximize performance and accuracy across various use cases. -Al Response Validation: Analyze and validate Al-generated outputs to ensure they meet quality standards, are contextually appropriate, and align with user needs. -Test Case Development: Create and execute comprehensive test cases for Al generated responses, assessing robustness, accuracy, and user experience. -Iteration and Feedback: Provide structured feedback to improve Al model behavior, performance, and overall functionality. -Collaboration: Work closely with cross-functional teams including data scientists, engineers, and product managers to implement and test improvements. Experience Required: -5-8 years of professional experience in software development or automation testing. -Minimum 1 year of hands-on experience with generative Al technologies, large language models (LLMs), or GPT-based systems. -Strong understanding of natural language processing (NLP) principles. -Proficiency in programming languages such as Python, JavaScript, or similar tools. -Experience with automation frameworks and test scripting. -Familiarity with Al and ML workflows is a plus. -Strong analytical and problem-solving skills.

What you’ll do

Design and refine prompts, validate AI-generated responses, and develop and execute test cases to assess quality, accuracy, robustness, and user experience. Provide structured feedback and collaborate with data scientists, engineers, and product managers to improve AI model behavior and functionality.

Requirements

Requires 5–8 years of experience in software development or automation testing, including at least one year of hands-on experience with generative AI, LLMs, or GPT-based systems. Candidates should understand NLP principles and have programming and automation testing skills; familiarity with AI and machine learning workflows is a plus.

Listed skills

• Problem solving · Preferred
• JavaScript · Preferred
• Python · Preferred
• Cross-Functional Collaboration · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Prompt Writing
• Generative AI
• Large Language Models
• GPT-Based Systems
• Natural Language Processing
• Python
• JavaScript
• Automation Testing
• Test Case Development
• AI Response Validation
• Automation Frameworks
• Test Scripting
• AI and Machine Learning Workflows
• Analytical Skills
• Problem-Solving
• Cross-Functional Collaboration

Job areas

• Technology
• Software
• Engineering
• Data & Analytics

More jobs from COGNITO INC