Only you can order the cheeseburgers: Speaker verification comes to OVOS

Only you can order the cheeseburgers: speaker verification comes to OVOS

A wake word engine only checks what was said. It doesn't check who said it. A guest who shouts "Hey Mycroft, order a thousand cheeseburgers" gets through. So does a TV advert that happens to say your hotword. The OVOS Wake Word Verifier Plugin Speaker closes that gap: it checks the voice against your household's enrolled profiles before anything else happens.

Try it (the package has prerelease versions only, so ask for them):

pip install --pre ovos-ww-verifier-plugin-speaker

The enrolment command, from version 0.0.2a4:

$ ovos-speaker-enroll --help
usage: ovos-speaker-enroll [-h] [--model MODEL] [--profiles-path PROFILES_PATH] name wavs [wavs ...]

Enroll a household member so the OVOS speaker verifier will accept their voice commands.

positional arguments:
  name                  Speaker name (e.g. Alice)
  wavs                  One or more WAV files

options:
  --model MODEL         speakeronnx model alias (default: wespeaker-resnet34)
  --profiles-path PROFILES_PATH

So for example, if you are Alice and have recorded your voice as wavs called clip1 - clip3, the next command would be:

ovos-speaker-enroll Alice clip1.wav clip2.wav clip3.wav

You can add multiple accounts. More clips (5 to 30 seconds total per person) give a more robust profile.

Enable it in ~/.config/mycroft/mycroft.conf:

{
  "listener": {
    "ww_verifiers": {
      "ovos-ww-verifier-speaker": {
        "model": "wespeaker-resnet34",
        "threshold": 0.45,
        "fail_open": true
      }
    }
  }
}

Restart your OVOS service. Enroll first, then tighten threshold. With no profiles enrolled the plugin accepts everyone by design, so installing it never locks you out. That is what the plugin does in a fresh environment before anyone is enrolled:

import numpy as np
from ovos_ww_verifier_plugin_speaker import SpeakerVerifier

v = SpeakerVerifier(config={"fail_open": True})
print(v.verify(np.zeros(16000, dtype=np.int16).tobytes()))

Would give:

True

A gate after the gate

A wake word engine's job stops at detection: it can't tell you from a guest, or from the dog's squeaky toy. ovos-dinkum-listener PR #191 adds a wake word verifier framework to close that gap. After a wake word fires, the captured audio passes through any installed verifier plugins before recording or intent handling starts. ovos-plugin-manager PR #341 defines the HotWordVerifier template and a new opm.wake_word.verifier entry point, so verifiers are discovered the same way wake word engines, STT, and TTS plugins already are.

The voting rule is strict:

If any verifier returns False, the detection is silently discarded.
If all verifiers accept (or none are configured), the wake proceeds normally.
The chain is fail-open on error: if a verifier raises an exception, the listener logs it and lets the detection through instead of locking you out.

The speaker verifier is the first plugin built on this hook, and the third gate in a pattern OVOS has been building for a while: Pre-Wake-VAD asks "is this even speech?" before the wake word engine runs; the Transcription Validator Plugin asks "does this make sense?" after transcription; the speaker verifier asks "are you allowed to say it?"

How it works

A wake word engine detects an activation ("Hey Mycroft").
The verifier extracts a speaker embedding, a fixed-length numeric fingerprint of the voice, using the speakeronnx library.
It compares that embedding against enrolled household profiles with cosine similarity.
A match lets the wake through. No match, and it's dropped silently: recognizer_loop:record_begin never fires on the bus, so the listener never starts recording.

Two effects fall out of this directly. Random audio from the TV or radio won't match an enrolled voice, so it stops tripping the assistant. And a visitor who knows the hotword still can't fire routines, play music, or touch your smart home. Only enrolled voices can.

Privacy by design

Profiles are the L2-normalised mean of your enrollment clips' embeddings, stored locally in ~/.local/share/ovos_speaker_verifier/profiles.json.
No audio is retained after the embedding is extracted.
Embeddings cannot be reversed back into audio.
Everything runs locally, with no cloud and no account.

Your voiceprint stays on your hardware, as a list of numbers.

Tuning

threshold (default 0.45) is calibrated for the default wespeaker-resnet34 model, and does not transfer between models: cosine-similarity scales differ by architecture. In our tests, the same enrolled-vs-guest pair scored ~0.95 / 0.89 on titanet-small but ~0.17 / 0.14 on campplus. Swap model for one of the other supported options (wespeaker-ecapa512, campplus, eres2net, titanet-large, redimnet-b2 and more, all downloaded and cached on first use) and re-tune threshold to match. per_profile_thresholds lets you set a stricter bar for some people than others.

Two "fail-open" settings look similar but do different jobs. The framework-level one is about errors: a crashing verifier never suppresses a wake. The plugin's fail_open: true is about enrolment: before you've enrolled anyone, the verifier accepts everything, so installing it doesn't brick the assistant.

What's still rough on the edges

The verifier only sees the wake-word window itself, roughly a second of audio, not the full command that follows. Speaker identity from a clip that short is weaker than from a whole sentence, and audio under about half a second can produce unreliable embeddings. A clearly spoken hotword works best; a mumbled one is where a genuine household member is most likely to get turned away wrongly. This is a real gate, not a biometric lock.

Enrolment is also still command-line only: record a few WAV clips, run ovos-speaker-enroll. That's fine at a terminal, but it's not yet the guided "repeat after me" voice experience this deserves.

The verifier framework and the matching engine are tested end-to-end through a real listener in the test suite. The enrolment experience is what still needs work: spoken enrollment, a GUI and easier profile management are next. Bugs and questions go to the issue tracker.

Built in the Open, Funded for the Commons

The speaker verifier was developed by TigreGótico for OpenVoiceOS. As we shared when OpenVoiceOS received its NGI Zero Commons Fund grant, that funding goes toward the plumbing a community project rarely has the resources to polish: privacy, reliability, listener internals. Speaker verification fits the pattern: privacy-first, built in the open, no cloud account required.

This work is part of the OpenVoiceOS From Beta to Breakthrough milestone, funded through the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101135429. Additional funding is made available by the Swiss State Secretariat for Education, Research and Innovation (SERI).

Help Us Build Voice for Everyone

OpenVoiceOS is more than software, it's a mission. If you believe voice assistants should be open, inclusive, and user-controlled, here's how you can help:

💸 Donate: Help us fund development, infrastructure, and legal protection.
📣 Contribute Open Data: Share voice samples and transcriptions under open licenses.
🌍 Translate: Help make OVOS accessible in every language.

We're not building this for profit. We're building it for people. With your support, we can keep voice tech transparent, private, and community-owned.

👉 Support the project here