AI QA Engineer (m/f) at Abysalto
Worldwide
<p><strong>Dive deeper. Aim higher.</strong></p><p>At Abysalto, that’s not just a motto — it’s how we work.<br/>We build serious AI-powered systems used in real production environments, often with high expectations around reliability, security, and correctness. As our AI products and teams grow, quality becomes just as critical as innovation.</p><p>That’s why we’re looking for an <strong>AI QA Engineer</strong> who understands how AI systems fail, how to test them properly, and how to help teams ship AI features with confidence.</p><p>Ready to dive in?</p><p></p><p><strong>🔷 What will you do?</strong></p><ul><li>Design and implement <strong>quality assurance strategies for AI-powered systems</strong>, including LLM-based applications, ML models, and data pipelines</li><li>Define <strong>test plans and acceptance criteria</strong> for AI features (functional, non-functional, and edge-case behavior)</li><li>Test AI systems end-to-end: APIs, model inference, data flows, prompts, and integrations</li><li>Develop and maintain <strong>automated tests</strong> for backend services and AI workflows (API tests, regression tests, smoke tests)</li><li>Validate <strong>model outputs</strong> for correctness, robustness, bias, hallucinations, and failure modes</li><li>Collaborate closely with AI Backend Developers, ML Engineers, and Data Engineers to catch issues early</li><li>Help define <strong>quality metrics for AI systems</strong> (accuracy thresholds, latency limits, stability, reproducibility)</li><li>Participate in incident analysis: reproduce issues, identify root causes, and help prevent regressions</li><li>Continuously improve QA processes, tooling, and documentation as products scale</li></ul><p><strong>🔷What do we expect from you?</strong> </p><ul><li>Experience in <strong>software testing or quality engineering</strong>, preferably in backend or data-heavy systems</li><li>Solid understanding of <strong>REST APIs</strong>, HTTP, JSON, and backend architectures</li><li>Hands-on experience with <strong>test automation</strong> (e.g. pytest, unittest, Postman/Newman, or similar tools)</li><li>Basic programming skills in <strong>Python</strong> (reading code, writing tests, simple scripts)</li><li>Ability to reason about <strong>AI/ML system behavior</strong>, even if you haven’t trained models yourself</li><li>Understanding of common AI failure modes: non-determinism, hallucinations, data drift, edge cases</li><li>Experience working with <strong>Git</strong>, CI/CD pipelines, and Linux environments</li><li>Strong attention to detail, analytical thinking, and a structured approach to problem solving</li><li>Good communication skills in English — you can explain issues clearly and constructively</li></ul><p><strong>🔶</strong> <strong>Nice to have</strong></p><ul><li>Experience testing <strong>LLM-based systems</strong> (RAG pipelines, prompt-based applications, chatbots)</li><li>Familiarity with <strong>ML concepts</strong> (classification, embeddings, inference vs training, evaluation metrics)</li><li>Experience with <strong>performance and load testing</strong> (latency, throughput, resource usage)</li><li>Knowledge of <strong>data validation</strong> and testing data pipelines</li><li>Exposure to <strong>MLOps practices</strong> (model versioning, monitoring, rollback strategies)</li><li>Experience in regulated or enterprise environments where quality really matters</li></ul><p><strong>🔷What do we offer? </strong></p><ul><li>A key role in shaping <strong>quality standards for AI products</strong> across the company</li><li>Close collaboration with experienced AI engineers and tech leads</li><li>Supportive environment for growth, learning, and specialization in AI quality engineering</li><li>Modern tech setup — high-performance hardware, ergonomic workstations (Aeron chairs), GPU access</li><li>Flexible working hours with a hybrid work option</li><li>Agile workflows with SCRUM methodology (flexible) and a strong engineering culture</li><li>All perks and benefits available on our <a href="https://abysalto.eu/careers/" rel="noopener noreferrer" target="_blank">career page</a></li><li>A chance to help build AI systems that millions of people rely on</li></ul><p>We solve complex technological challenges in order to simplify and improve everyday lives of millions of people. Our goal is to become a leader in the software industry, recognized for excellence and quality.</p><p>If you're ready to shape what’s yet to be — send us your CV.<br/><strong>Apply via the link below. </strong></p><p>We look forward to meeting you!</p>
Apply Now