Translate a Website with a Local AI Model

A multilingual website has a subtle problem: it becomes outdated in three quarters of its languages. A sentence on the German page is quickly corrected — until the same correction reaches English, French and Italian, weeks pass. So it remains unchanged. After a year, the foreign-language pages contain something different from the German one, and no one knows exactly what.

The usual solution is a translation office or a translation API. Both work, but both have the same drawback: they cost per operation. And what a single half-sentence change costs, you don’t pay for.

The Goal

Translation should be as cheap and casual as a spell checker. That is exactly what a model running on your own machine delivers: A call costs nothing except electricity. This changes what is practically possible — not the quality of a single translation, but how often it can be done.

Today that looks like this for us: we maintain only German. One command, and English, French and Italian follow. Whoever changes a single sentence gets the three translations in about twenty seconds — because only what has changed is retranslated.

The Gain, concretely

The last point is often underestimated. If a run costs nothing, you can discard it. We rebuilt our prompts a dozen times, discarded entire runs, and compared three models against each other — with per-operation costs we would have settled for the result after the second attempt.

What you need

Nothing exotic, but also nothing trivial — memory is the limiting factor:

Two things you should know beforehand: A model that doesn’t fit into memory never loads — LM Studio aborts with a clear message instead of forcing the computer to its knees. And translation is not a matter of seconds: for our entire website in three languages we expect one to three hours. That is a one‑time cost; every subsequent run only concerns the changed parts.

What emerged was less a question of model choice than a question of testing. The interesting errors were not the ones where something visibly broke.

The Build in Three Sentences

LM Studio starts a local server that speaks the same interface as the OpenAI API. A Node script slices the content into blocks, sends them there, and stitches the responses back together. An intermediate cache remembers every translation by the hash of the German original, so that a second run only touches changed sentences.

The decisive design step lies in block size.

Context beats Granularity

The first attempt split everything into individual text pieces — 2 279 pieces for our surface texts. Each went to the model separately. The result was mediocre, and systematically:

A model that sees “Zur Übersicht” without any context simply lacks the information that it is a button. It guesses — and for short fragments happily provides several suggestions in a row, which breaks the layout in a table cell.

So the opposite: large blocks instead of small pieces. We slice the surface texts along their tree structure into 109 blocks averaging about 2 300 characters; blog posts go through in one piece. From 2 279 requests we get 125. We always cut at whole keys, never in the middle of a value — that way nothing is lost, as verified by a test that reassembles all blocks and compares them to the original.

Three Models, Three Characters

Tested on a Mac with Apple Silicon, all models via LM Studio, each the same block of 2 701 characters.

With Qwen, look at the decimal place: qwen3.5-9b and qwen3.8-27b are two generations of the same family, and the differences below concern exactly these two versions — not “Qwen3” in general. In production we run Qwen3.8 27B.

The sizes are the actually occupied ones — models are delivered quantized, i.e. with reduced numerical precision. A 27‑billion‑parameter model occupies 15 GB instead of over 50 GB, with barely noticeable quality loss. For the start you only need the smallest: for the surface texts that make up most of it, Qwen3.5 9B delivered the same quality as its three‑times larger brother.

Measurement here is in characters per second, not tokens per second. That is intentional: each model brings its own tokenizer, and German text splits differently in it. Tokens are therefore only conditionally comparable across models — the amount of text that ultimately comes out is what matters. For context, Qwen3.8 27B produced 1 714 output tokens for a complete blog post in 120 seconds, i.e. about 14 tokens per second; gpt-oss-20b is roughly twice that.

The numbers say less than expected. A model three times smaller was only about 20 % faster — the bottleneck on this hardware is not compute power, but memory bandwidth. For every single token all weights must travel through the memory bus. Even more simultaneous requests helped little: between two and twenty parallel requests, the spread of measurements was higher than the effect.

More interesting are the behavioral differences.

TranslateGemma is not a conversational partner

Google’s translation model does not understand instructions — it translates what comes in. If you give it a JSON object asking to translate only the values, you get split structures back. Structure therefore must be preserved programmatically.

It also has its own call format: its chat template expects language codes as fields, not as a prompt. LM Studio passes these fields over the chat endpoint but does not forward them, so every call ends with HTTP 400. The way is via /v1/completions with a self‑built prompt — the template lives in the model folder.

It does what it should, and leaves technical terms alone. Product names, however, are not: “Exeoflow” reliably became “Exeflow” or “Exoflow”. Proper names must be masked before a pure translation model call.

Qwen3.5 9B translates one field too many

The small Qwen almost obeyed all rules — dates, authors, technology lists, even person names in quotes stayed untouched. Only at one spot it reproducibly failed in every run:

# Original                              # Translation
tags: ['ki', 'glossar', 'agents']       tags: ['ai', 'glossary', 'agents']

Tags are technical identifiers, not prose — translating them splits the filtering. Qwen3.8 27B and gpt-oss did not make this mistake. It was finally fixed by a single additional sentence in the instruction: that tags are identifiers and should never be translated.

gpt-oss-20b needs an English instruction

Here was the most instructive finding, and it almost slipped away. With a German system prompt, gpt-oss dutifully translated the frontmatter and left the entire body text unchanged in German — in all five runs. Even then, when the instruction explicitly said that the whole text should be translated.

The same prompt in English: immediately correct. The problem was not the clarity of the instruction, but its language. Instruct models are predominantly trained on English instructions. Since then all our prompts have been in English, even if neither the source nor target language is — that makes the pipeline independent of which model it runs on.

The error we almost missed

The issue with untranslated text has an unpleasant twist: our test reported “passed”.

It compared headings, table rows, bold text, link targets and code blocks between the original and the translation. Untranslated text passes this test effortlessly — it has perfectly preserved structure. Thirteen files with fully German content ended up as en.md on disk, and in a language you don’t read it is only noticed months later.

Since then the pipeline also answers the most obvious question: Was anything translated at all? It measures what fraction of the body text remained word‑for‑word. A real translation is about seven % — individual lines stay legitimately the same, e.g. proper names or technical terms. The passed German text was 100 %.

If a block fails the test, the file is not written. That sounds strict, but it is the gentler variant: a missing language file is simply skipped by the site generator, it also drops out of hreflang references. A broken file, however, goes live.

The lesson extends beyond translations: a test that only looks for damage misses cases where nothing happened at all.

What else surprised us

Qwen3.8 thinks out loud by default. In a translation that is pure waiting time — 187 reasoning tokens for one sentence, 12.4 instead of 1.4 seconds, with identical result. The switch intended for Qwen is the template arguments enable_thinking and preserve_thinking, both active by default. In LM Studio they do nothing: the answer contains exactly the same number of reasoning tokens with enable_thinking: false as without the field, and a made‑up key at the same spot causes neither error nor effect — the whole block is ignored instead of going to the chat template. The switch isn’t wrong, but the expectation that every runtime environment would carry it through is. What disables it here is reasoning_effort: 'none' — for LM Studio it even logs a warning that the setting is invalid. It still works, though.

Repeated attempts need no randomness. Our first draft raised the temperature on a failure to get a different answer. For a rule‑bound task that is counterproductive: with temperature: 0 both tested models passed the test, with 0.2 they failed. Variance costs exactly the rule fidelity it is about. An instruct model can instead use a concrete feedback — the second attempt now receives the complaint in the prompt.

Arrays are a silent trap. In JavaScript an array is also typeof 'object'. If you recursively split into it, ["a","b"] becomes unnoticed {"0":"a","1":"b"} — and writes that into the target file.

Re‑building it

The example script below is self‑contained, runs without dependencies, and shows all three building blocks: slicing blocks, checking answers, caching results.

1. Download LM Studio and install it, load an instruct model (openai/gpt-oss-20b is a good start) and start the server under Developer.
2. Download and run the script:

node lmstudio-translate-demo.mjs
node lmstudio-translate-demo.mjs --model=qwen/qwen3.8-27b --lang=French
node lmstudio-translate-demo.mjs --file=meine-texte.json

3. On the second run the cache kicks in — changed texts are retranslated, everything else comes from the file.

If you use it productively, don’t skip three things: the check for actual translation, the cache (otherwise every correction costs a full run), and a test that verifies the split against the original. Everything else can be pulled along on the way.

And the fundamental question? For a website with four languages and regular text changes, the local path already pays off because it is repeatable. You can change a sentence and pull all three translations in twenty seconds — without thinking about per‑call costs.