ARCTA — Automated Radio Communication Transcription & Analysis

About

ARCTA is an application for the automated transcription, structuring and analysis of fire service and emergency radio traffic. It turns dialect-heavy audio recordings into a normalised, standard-German incident log. Processing happens in two stages: first a speech recogniser (speech-to-text, STT) converts the audio into raw text; a large language model (LLM) then corrects and structures that text — dialect correction, sender and receiver identification, classification.

For radio traffic it is not only general textual fidelity that matters, but above all the correct recognition of numbers, coordinates and call signs. The evaluation therefore had to answer which correction LLM improves the raw STT output most, how reliably numbers are recognised, and how the accuracy achieved relates to cost and processing speed.

Evaluation tool — EvalARCTA

The evaluation was carried out with EvalARCTA, a tool developed specifically for this purpose. It compares each transcription (hypothesis) against a manually produced, correct reference (truth) at word level and derives standardised error metrics from it. Only the word content is compared — capitalisation, punctuation and timestamps have no effect on the result. The calculation builds on the established jiwer library and runs entirely locally on the CPU.

Results

The biggest jump in quality comes from the LLM correction: in radio mode the raw STT output sits at 18.5 % WER, while the best model brings the error rate down to 4.8 % — a reduction of roughly three quarters. Even the weakest correction LLM (deepseek-v4-flash, 11.9 %) still improves the raw output noticeably.

The comparison of modes also shows that radio normalisation consistently lowers the measured WER, because different ways of speaking the same number are counted as equivalent in content. The highest accuracy is not automatically the most expensive option: gemini-3.5-flash:nitro combines the best WER with a mid-range price of 0.575 USD, giving it the best overall balance of accuracy, speed and cost.

Metrics

• WER (word error rate) = (S + D + I) / N — the standard speech recognition metric, lower is better. It can in theory exceed 100 % if a lot of additional text is generated.
• CER (character error rate) = character edit distance / reference characters — finer-grained than WER and more robust with German inflections and compounds.
• Number accuracy = correctly recognised digit tokens / total digit tokens — mission-critical for coordinates, meaningful only in radio mode.
• RTF (real-time factor) = total runtime / audio duration — values below 1 mean faster than real time.
• Cost = STT cost plus LLM cost in US dollars.

Conclusion

For the automatic transcription of the emergency radio traffic examined, this first evaluation recommends the combination of the STT model microsoft/mai-transcribe-1.5 with the correction LLM gemini-3.5-flash:nitro. This combination achieves the lowest word error rate (4.8 %), recognises every digit token correctly (100 % number accuracy) and works faster than real time at a moderate cost.

Two general findings stand out: downstream LLM correction is indispensable for usable transcription quality, and radio normalisation is necessary when assessing radio texts in order to measure the professionally decisive fidelity of numbers correctly. The next sensible step is to extend the evaluation to a larger corpus in order to put the resulting ranking on a firmer statistical footing.

• TypeMaster's programme — project 2
• Period03.2026 — 06.2026
• FieldSpeech-to-text, large language models, emergency service radio
• ToolsEvalARCTA (built on jiwer)
• Recommendationmicrosoft/mai-transcribe-1.5 + gemini-3.5-flash:nitro
• StatusCompleted
• Last updated24 Aug 2026