Say goodbye to costly methods like LLM-as-a-Judge and outdated metrics like ROUGE or BLEURT. We simplify RAG evaluation using GRACE (Grounded Retrieval-Augmented Citation Evaluation).

GRACE helps us evaluate LLMs effectively and affordably using simple classification metrics like accuracy, just by comparing the LLM-selected citations with the annotated citations of an LLM-generated answer. To enable it, we follow a specific RAG architecture with specific Knowledge Base preprocessing, which we call Grounded RAG with Citations (Grounded RAG+C).

The main issue: how to evaluate RAG?

Retrieval-Augmented Generation (RAG) is the go-to approach for enabling LLMs to answer questions on proprietary documents, such as business knowledge bases. Behind every RAG system, one question matters most: did we provide the correct answer to the user?

One approach is an LLM-as-a-Judge, asking a different LLM to score responses on relevance or groundedness. It is promising, but expensive to scale. Traditional NLG metrics like ROUGE or BLEURT are limited too: two answers with the same meaning but different lengths score poorly.

If the correct article is cited, the LLM has likely given the correct answer.

Our approach: Grounded RAG with Citations

Grounded RAG+C builds on standard RAG with one key improvement: the LLM is prompted to ground its response on a single retrieved article and include an inline citation. We then check whether the cited article matches the predefined correct article for the query: a hit or a miss.

All we need to do is:

1. 1Split the Knowledge Base into self-contained articles to embed in the vector database, instead of smaller chunks.
2. 2Instruct the LLM to select exactly one article and report it in the generated response.

Results on a real-world dataset

For our internal HR support chatbot we compiled over ~700 hard real-world user queries, each linked to the self-contained article it should match. Leading proprietary models scored highest, while smaller open-weight models failed to answer around half of the questions.

{{ post.excerpt }}

Mockup: only the featured write-up has body copy. Read this one on helvia.ai

In a wide-ranging interview with Insider.gr, Dimitris Balaouras, co-founder of helvia.ai, argues that 2026 is the year AI stops being a side experiment and becomes part of how a company actually runs. His one-line answer, if you only keep one: the quality of your AI implementation is directly tied to the quality of your data.

The original interview, conducted by Niki Papazoglou, was published in Greek on Insider.gr. We’ve translated the full conversation into English below.

Where are we really today in terms of business maturity?

The Greek market has moved past the phase of AI demos. What’s needed now are companies that already operate as AI-native — and that’s the real threshold of 2026.

Over the last two years we saw thousands of pilots and experimental chatbots. That phase was necessary, so we could all understand what the technology can and can’t do. At helvia.ai we see the shift in the conversation with leadership clearly: today they don’t ask us “what can AI do.” They ask us “how do we govern it, how do we control it, how do we put it into production.”

The difference between experimentation and business value isn’t technological. It’s architectural.

When you treat AI as one more project, you stay in experimentation. When you treat it as a new operating layer for the business, you gain a real competitive advantage.

Small pilots, or a full transformation rolled out in phases?

It’s not a dilemma. The implementation has to be incremental. The design, though, has to be holistic.

For us, the right approach has three stages:

1. 1Design the central AI platform.
2. 2Put governance and observability in place from the start — not as an afterthought.
3. 3Implement use cases with clear prioritization based on business value.

AI isn’t a project roadmap — it’s a transformation roadmap.

If a CEO could keep just one piece of AI advice for 2026, what would it be?

In most cases where an AI implementation didn’t deliver what was expected, the cause wasn’t the technology. It was the data — scattered across different systems, inconsistent, locked in silos. So before you go hunting for the perfect AI tool, it’s worth spending time on the foundations: getting your data organized, reliable, and accessible to your teams.

{{ post.excerpt }}

Mockup: only the featured article has full body copy. Read this one on helvia.ai