inferwire
/
AI·3 min read

Researchers Adapt NVIDIA's Nemotron Retrieval Stack to Greek

A preprint adapts NVIDIA's Nemotron retrieval models to Modern Greek and finds that plain BM25 search beats several off-the-shelf multilingual embedders on specialist Greek documents.

TL;DR

  • A preprint adapts NVIDIA's Nemotron retrieval and answer models to Modern Greek, which the models and major retrieval benchmarks had left out [^1].
  • Plain BM25 keyword search beat several off-the-shelf multilingual embedders on specialist Greek corpora; fine-tuning raised Nemotron 1B's nDCG@10 from 0.362 to 0.835 [^1].

Background

Retrieval-augmented generation, or RAG, lets a language model answer from documents. An embedding model turns questions and passages into vectors and finds the closest matches, a reranker reorders the candidates, and a reader model writes the answer. BM25 is an older keyword-based ranking method that needs no training. nDCG@10 scores how well the top ten results are ordered; higher is better, and a perfect ranking scores one. All of these parts have to work in the user's language.

What happened

Ayoub Kirouane and Christos Petrocheilos posted the paper on arXiv on August 5 [^1]. They start from a gap: Modern Greek is missing from NVIDIA's Nemotron retrieval models and from major multilingual retrieval benchmarks, even though RAG in Greek matters for legal, energy, financial and medical applications [^1]. Their answer is an end-to-end adaptation of the Nemotron retrieval stack. It covers mining a corpus, generating synthetic training data, training the retrieval model, adapting the reranker and fine-tuning the reader, plus a new benchmark called HERA [^1].

The first finding is about the starting point. On specialist Greek corpora, the parameter-free BM25 baseline outperformed several off-the-shelf multilingual dense retrieval models [^1]. After fine-tuning on 65,773 Greek retrieval pairs, a Nemotron embedding model with 1 billion parameters improved its nDCG@10 from 0.362 to 0.835 and clearly beat its unadapted version [^1]. The authors report that the learned Greek also helps on general-domain text, but that whether the tuned model beats BM25 depends on the domain [^1].

For the later stages, an adapted cross-encoder reranker improved results across the specialist domains [^1]. The authors then used LoRA, a lightweight fine-tuning method, on Nemotron 30B-A3B, a mixture-of-experts model, as the reader. Judged answer correctness rose from 29.4 to 66.9 percent, and the authors report better faithfulness to the sources and better citations [^1]. They describe HERA as the first large-scale Greek benchmark for RAG and release it together with the adapted models [^1].

Why it matters

The BM25 result is the practical takeaway for anyone building search in a language the big models cover poorly. A multilingual embedder isn't automatically better than keyword search on specialist documents, so it pays to measure both before choosing [^1]. The jump after fine-tuning shows that 65,773 Greek training pairs were enough to change that picture for one model, although the authors themselves say the advantage over BM25 varies by domain [^1].

The released benchmark may matter as much as the models. Without a Greek RAG benchmark, teams have had little shared ground to compare systems on, and HERA gives them one [^1]. The figures so far come from the authors' own evaluation in a preprint, so other groups testing the released models on their own Greek data will show how well the gains hold.

Practical example

Say you're building a search tool over Greek energy regulations for your company. Before you pick an off-the-shelf multilingual embedding model, you run plain BM25 on the same documents, since the paper found it beat several such models on specialist Greek corpora [^1]. If you can collect enough question and passage pairs from your own files, fine-tuning the embedder is the step that lifted Nemotron 1B in the paper [^1]. You then compare both setups on your own test questions and keep whichever ranks the right regulation higher.

Related gear

We recommend this book because it explains the basics of search systems, from keyword ranking to how result lists are evaluated.

AdvertisementAmazon

Introduction to Information Retrieval

Sources

  1. [1]arXiv — Teaching Nemotron Greek: Mining a Corpus, Adapting Retrieval, and Grounding Generation for Modern Greek across Specialist Domains