PAC launches its first PAC RADAR on Quality Engineering, with LLMs in mind

PAC is launching its first PAC RADAR on quality engineering (also known as software testing or quality assurance). Quality Engineering (QE), along with other software development lifecycle (SDLC) activities, is undergoing a significant transformation with the advent of Large Language Models (LLMs).

Within QE, functional testing accounts for ~75% of expenditures (the rest consisting of non-functional testing and UX testing). Functional testing has relied, for almost 30 years, on the requirement-test scenarios, cases, and scripts process. The activity remains largely manual, despite many attempts to automate this core process.

LLMs enable the automation of this process, from requirements to test script generation, and also generate synthetic data used for testing, all within minutes. A revolution is underway. And beyond functional automation, other use cases are emerging.

One of the most promising LLM use cases is reverse engineering: the current LLM approach is “greenfield,” where automation does not consider existing client QE artifacts. The reality, however, is that large companies often have several thousand test artifacts, which are often poorly understood and sometimes misused. The idea is therefore to convert these artifacts, such as test case scripts, into “user stories.” The benefits are numerous, including improved test coverage.

With its first QE RADAR, PAC will assess LLM use cases across GenAI and Agentic AI/AI agents used for software testing. PAC will also evaluate consulting offerings related to the usage of LLMs, for instance, on how to orchestrate the co-existence of ML and LLMs, and monitor the cost of LLMs (with FinOps).

In detail, the scope of the study includes:

• The LLM use case offering for test automation, across custom applications and SAP platforms.
• The portfolio of non-deterministic AI model testing
• Approaches to orchestrating ML, NLP, and LLM models.
• Consulting offerings for test function transformation.

Publication of the QE PAC RADAR is planned for April/May 2026.

For more information on this study, please contact Dominique Raviart.