A fully local voice assistant setup

Build a fully local voice assistant in 2026 https://blog.platypush.tech/article/Local-voice-assistant

A practical setup for a Raspberry Pi-friendly voice assistant based on Platypush.

Those who have followed me for a while know of my personal obsession with self-built voice assistants.

My experiments over the years can be summarized as it follows:

2007 : Voxifera , my very first attempt at building a primitive voice assistant using Hidden Markov models . Definitely not good for general-purpose usage, but good enough in 2007 to distinguish between a dozen of simple voice commands.

2019 : First voice assistant built on top of Platypush . It used the now deprecated Google Assistant Library on top of a Raspberry Pi with a microphone and a speaker, and it could hook any automation routines and custom commands to it through event hooks.

2020 : Second iteration on #platypush , this time supporting other assistant plugins too - Alexa (integration now removed), Snowboy (also removed, since the project is dead), Mozilla DeepSpeech (also removed now, since Mozilla discontinued it), PicoVoice , and mimic3 (the text-to-speech engine built on top of Mycroft, now bankrupt).

2024 : Third iteration on Platypush, this time with an enhanced PicoVoice integration and new speech-to-text and text-to-speech plugins based on the OpenAI APIs.

But it's now 2026, and perhaps both the hardware and the software are now mature enough for fully on-device voice assistants based on fully open solutions likely to stick around for a while.

In this article we'll wire that gap closed with Platypush:

The result is not another cloud assistant with a different coat of paint. The hotword engine, speech recognition, command dispatch and speech synthesis can all run on-device. If the openai step points to a local OpenAI-compatible server, then the whole pipeline can stay on your LAN too.

The pipeline

The architecture can be summarized as follows:

listens

emits

hotword detected

emits

speech recognized

phrase matches local command

generic response

text to speech

text to speech

process intent

play speech response

follow up

conversation end

conversation end

Microphone

assistant.openwakeword

HotwordDetectedEvent

assistant.vosk.start_conversation

ConversationStartEvent

SpeechRecognizedEvent

Local command hooks

openai.get_response

tts.piper

Speaker

ConversationEndEvent

Hotword detection ("OK Google", "Alexa" etc.) is a continuous, low-latency workload, and it should not need the network.

Speech-to-text is also a good fit for local inference: Vosk models are small enough to run on modest hardware, including Raspberry Pis, and they are perfectly adequate for short home automation commands.

Text-to-speech is another place where local models are good enough nowadays: Piper voices are fast, small and much nicer than the old robotic espeak -style fallback.

The only optional network-shaped piece is the language model .

But that is a policy choice, not a requirement of the voice stack.

Setup

Clone the assistant sample repository:

Models

The next step is to download the voice models used by the voice stack.

When the service starts the first time, it will automatically download all the available models.

You can then use the following command to list the available models once the service is running:

Where $PLATYPUSH_TOKEN is the token of the user that is running the service.

You can retrieve it by connecting to http://localhost:8008 when the service starts for the first time. Create your credentials, then select Settings -> Tokens -> Generate API Token .

A full list of the Vosk voice models is available here .

Some feedback about the quality of the English models:

Download the selected model to the Docker volume working directory:

Download a speech synthesis model from here .

Audio samples are also available to get an idea of the type of voice before downloading.

The model usually consists of a *.onnx and a *.onnx.json file. Download both of them to the Docker volume working directory:

Configuration

Copy and edit the example configuration file .

The assistant becomes useful once recognized speech can reach the rest of the house.

For example, Hue lights:

And MPD/Mopidy for music:

Those are just regular Platypush plugins .

The assistant does not need special knowledge about Hue, MPD, Chromecast, Zigbee, MQTT or anything else.

It only needs to emit events; your hooks decide what to do with them.

Build

Build the container image for the assistant service:

Run

The assistant needs access to the host microphone and speakers. The container routes ALSA through PulseAudio, so the examples below connect it to a PulseAudio server running on the host.

Linux

With PulseAudio or pipewire-pulseaudio installed:

macOS

Install and start PulseAudio on the host:

Then start the container:

If pactl load-module reports that the module is already loaded, you can keep using the existing PulseAudio daemon.

Windows

Install PulseAudio for Windows, then create a default.pa file in the same directory as pulseaudio.exe :

Start PulseAudio from PowerShell:

Then start the container from the repository directory:

Make sure microphone access is enabled for desktop applications under Windows privacy settings, and allow PulseAudio through the firewall if prompted.

Usage

Once the service is running, you can start interact with it with voice commands (the default activation word is "Alexa").

Any questions about the weather will be resolved by the weather plugin if it's been enabled.

If the music or lights plugins are enabled, they can be controlled with voice commands ("stop the music", "turn on the lights", etc.)

Otherwise, the assistant will use the openai plugin to respond to your questions, with follow-up turns when the response from OpenAI is also a question.

Extending the Assistant

The assistant logic is modeled through simple Platypush hooks under config/scripts .

You can extend it as you like by defining your own hooks or modifying the existing ones.

Starting a conversation

Conversations are started by hooking to the HotwordDetectedEvent .

Deterministic commands

For common home automation commands, regular event hooks are still the best tool. They are fast, inspectable, and they do not hallucinate.

AI Commands

If the openai plugin is enabled, you can use it to help you answer questions.

There are two generic use-cases for voice assistants where an AI plugin is beneficial:

You may want this for general questions, for commands that do not fit a neat regular expression, or for transforming a raw sentence such as:

make it a bit darker and reduce the music volume

into a structured action plan like.

An example provided in the assistant sample is that of weather forecasting .

Note in particular the usage of openai.get_response with a well crafted system prompt that turns a natural language request like:

What's the weather tomorrow in San Francisco?

Into:

You can also use the model for intermediate transformation instead of direct answers. For example, ask it to return a tiny JSON object with action and args , then dispatch only actions you explicitly allow:

That last validation step matters. A model may be useful for interpretation, but it should not get arbitrary access to run() .

If a request doesn't match any of the commands you have defined, you can use a generic SpeechRecognizedEvent hook to forward the request to an AI plugin, and render the response as speech through the text-to-speech plugin.

When a response from the LLM ends with a question mark, the assistant will automatically listen for a follow-up command and fire a new SpeechRecognizedEvent .

Pausing music while listening

One nice touch is to pause the music when a conversation starts and resume it after the assistant is done.

That makes the interaction feel much less clumsy: wake word, music ducks or pauses, command is recognized, answer is spoken, music resumes a few seconds later.

Going fully local

With the configuration above, hotword detection, speech-to-text, automation and text-to-speech are already local. The only non-local component is the openai plugin, if it points to OpenAI's servers.

To make the last step local too, run a model server that exposes an OpenAI-compatible API. Ollama, llama.cpp server, vLLM and LocalAI can all expose some version of /v1/chat/completions .

For example, with Ollama:

The OpenAI-compatible endpoint is then usually available at:

If your Platypush openai plugin version supports a custom API base URL, the configuration is the whole change:

If it does not, keep the rest of the assistant exactly the same and replace only the fallback action with a tiny local request:

That is enough to turn the assistant into a fully local stack:

OpenWakeWord

Vosk

Platypush Hooks

Local OpenAI compatible model

Piper

OpenWakeWord

Vosk

Platypush Hooks

Local OpenAI compatible model

Piper

On a Raspberry Pi, I would still keep expectations realistic. Hotword detection, Vosk and Piper are fine on small machines. Local LLMs are the heavy piece. A Pi 5 with enough RAM can run small quantized models, but latency will not feel like a cloud model or a GPU-backed workstation. For many home automation workflows, that is acceptable because the LLM is only the fallback; the frequent commands stay deterministic.

Why this architecture ages well

Voice assistants have been a graveyard of abandoned SDKs and cloud products. Snowboy is gone. Mycroft is gone. The old Google Assistant SDK is deprecated. Vendor assistants are increasingly shaped around vendor ecosystems rather than user-controlled automation.

The safer long-term bet is not one monolithic assistant. It is a pipeline of small replaceable parts:

Platypush is a good fit for this because its event system is already the boundary between perception and action. Speech recognition emits an event. Hooks decide what to do. Plugins execute the actions.

That separation is what makes the assistant inspectable. It is also what makes it possible to keep most of it on a Raspberry Pi in your house, instead of outsourcing the entire audio loop to a cloud service that may disappear, get worse, or decide one day that your use case is no longer part of the roadmap.

Final notes

The minimal version of this setup is small:

Start with the deterministic commands. Add the model fallback later. That way the assistant stays fast for the commands you use every day, while still being flexible enough to answer questions or interpret messy speech when you need it.

To interact via Webmentions , send an activity that references this URL from a platform that supports Webmentions, such as Lemmy , WordPress with Webmention plugins , or any IndieWeb-compatible site .

#weather apps are one of those things that nowadays we take for granted, and most of us consider a largely solved problem.

After all, a weather app mostly consists in a simple UI that fetches the weather conditions from some API, optionally by retrieving your current location, and then it displays the information to the user.

Optionally, it can provide little perks like weather notifications.

The average usual app is simple and boring, and it's usually nothing that a decent student at the last year of engineering college wouldn't be able to put together in a weekend.

And yet, because of how easy it is to build a simple weather app, how pervasive these apps are on everyone's phones, and the amount of data that they collect about the user (which vastly justifies the small development cost), weather apps have become a favourite vector for tracking users .

They are also favourite ads delivery instruments .

The American government itself uses weather apps to track its citizens, and ICE buys data from weather apps to track potential targets.

After all, weather apps are among the few apps that:

What if it could be different?

Of course, there are many weather apps on F-Droid that actually do a decent job respecting users' privacy, and if you want something simple that only runs on your phone many of them may match your needs. But in my opinion that only solves part of the problem.

Those apps still need to be installed on each of your mobile or tablet devices. And on desktop you'll need a different service anyway.

And any weather notifications will be limited to the mobile device that receives them, which limits what you can do with them - what if I want to use my weather app to record all the weather measurements in a certain area? What if I also want to send a message on my family's messaging group if it starts snowing in my area?

This is where a weather solution based on Platypush comes handy.

Pros:

Fully self-hosted . Run the #platypush service on any device that can run a #python interpreter, from a cheap VPS to a RaspberryPi, expose it over an HTTPS URL, and any device can use the app.

No mobile apps required . Platypush provides a Progressive Web Application ( PWA ), which means that you can open the Web interface from your browser, install the PWA directly from your browser, and have it on your home screen just like a native app.

Full control over weather notifications . Weather updates are handled as standard Platypush events , which means that you can write your own custom hooks to deliver notifications, store weather measurements, send notifications over other channels, and so on.

Getting started

The first step is to get an OpenWeatherMap API key .

Then create a simple configuration for Platypush under /your/platypush/config/config.yaml :

Then install Platypush , or run it directly through the Docker image :

Once started, you can open the Web interface at http://localhost:8008 to register your user.

Once logged in, you can click on the weather.openweathermap tab from the left menu to immediately access your weather forecast:

HTTPS configuration

A PWA requires an HTTPS connection, or the Web service to be installed on localhost .

The localhost installation of Platypush is also possible on Android via Termux , but it's out of the scope of this article.

We can use a reverse proxy and Certbot to make the Web interface available at e.g. https://weather.platypush.example.com :

This requires:

[Optional] Set up a VPN for the reverse proxy

This step is optional.

You can also run the Platypush weather service on the same box as the reverse proxy, and in such cases you don't need to set up a VPN for the reverse proxy.

If your Platypush service runs on the same machine as the reverse proxy, you can skip to Reverse proxy configuration .

Otherwise, It's recommended if you want your reverse proxy to tunnel HTTP requests to e.g. your RaspberryPi or old Android tablet at home that runs the Platypush service, without exposing those IP addresses directly to the Internet.

A quick solution involves setting up your machine with a public IP to also run a Wireguard tunnel to your local machine, so the reverse proxy can directly access your local Platypush installation without leaking your own IP to the Internet.

A common set up, if your machine runs Linux with systemd, involves using the wg-quick utility to create a Wireguard tunnel, and then setting up a systemd service to start the tunnel at boot time.

wg-quick is usually provided by the wireguard-tools package on most of the UNIX-like installations.

Wireguard peers authenticate each other through public keys. In this example:

Start by generating the server keypair on the VPS:

Then generate the client keypair on the Platypush machine:

Now go back to the VPS and create the Wireguard server configuration.

Replace <The public key of the client> with the public key printed by the Platypush machine in the previous step:

The AllowedIPs = 10.0.0.2/32 line is important: it tells Wireguard that only the peer with the configured client public key is allowed to use the 10.0.0.2 tunnel address. Unknown clients, or clients with a different private key, won't be able to complete the tunnel handshake.

If your VPS firewall blocks inbound traffic by default, allow the Wireguard UDP port. For example, with ufw :

Now start the tunnel on the VPS:

The Platypush machine now needs a configuration that points back to the VPS.

Run the following commands on your Platypush machine:

AllowedIPs = 10.0.0.1/32 keeps the client configuration narrow: only traffic for the VPN address of the reverse proxy goes through this tunnel. PersistentKeepalive = 25 is useful when the Platypush machine is behind a home router or mobile NAT, because it keeps the tunnel mapping alive so the reverse proxy can reach 10.0.0.2:8008 .

Once the client is up, verify from the VPS that the reverse proxy can reach the Platypush Web service through the tunnel:

Reverse proxy

Assuming that these conditions are met, proceed with creating a reverse proxy configuration for the Platypush Web interface :

Apply the configuration and reload your reverse proxy:

Then verify that the reverse proxy can reach the Platypush Web service through the tunnel:

Generate a certificate

Run the following commands on your reverse proxy machine:

Then verify that the reverse proxy can reach the Platypush Web service through the tunnel over HTTPS:

Installing the mobile app

Open https://<weather.platypush.example.com>/plugin/weather.openweathermap from your browser on a mobile device.

Tap on your browser's menu. You should see an entry like " Add to Home Screen " or " Install app ".

Select to install the app and add it to your home screen.

You can search for other locations directly in the search bar, or use the GPS button to find your current location.

The GPS access permissions are optional, they are requested directly in your browser and only when you use the app, and there's nothing monitoring your location in the background (unless you actually want to monitor it explicitly).

[Optional] Handling weather events

If you only need a weather app that works on desktop and mobile when you want to check it, then you can skip this section.

If instead you would also like to handle events from the weather service, then you can create Platypush event hooks to handle NewWeatherConditionEvent .

Every time the Platypush service processed a weather update, it will emit events with the following payload:

Values:

You can subscribe to these events, for example, for:

ntfy

We'll use ntfy to send notifications to your mobile device, paired with the Platypush ntfy plugin .

You can install the Android ( F-Droid link ) or iOS app to receive notifications on your mobile device.

By default, the app and the Platypush plugin will connect to the default ntfy server ( https://ntfy.sh/app ).

That's an option, but in that case make sure to always use authentication or topic names with randomized strings.

Otherwise, you can run your own ntfy instance to send and receive your notifications on a fully self-hosted setup.

The easiest way is perhaps through Docker :

Then create a reverse proxy configuration with a certificate like shown in Reverse proxy configuration .

Plugin configuration

Add the ntfy plugin to your config.yaml file for Platypush:

Notifying of precipitation events

Create a Platypush event hook to send notifications to your mobile device:

Then install the ntfy app on your mobile device or use the Web interface, and subscribe to the weather-notifications-1234 topic on the configured server.

You'll be notified whenever there's precipitation in your area.

Of course, you can modify your hook to deliver any kind of relevant notifications in your area - about wind, temperature, humidity, etc.

And actions are not limited to ntfy. If you prefer, you can deliver the notification over email , ActivityPub , Matrix , Telegram , SMS , XMPP or anything that has a Platypush plugin .

Weather summaries

Another useful feature of many mainstream weather apps is that of a daily summary (usually in the morning) of the weather in a certain location, so you can plan your day accordingly.

This can be easily achieved too through a Platypush cronjob .

Restart the Platypush service, and every day at 6 AM, you'll get a summary of the weather in your area.

[Optional] Voice Assistant

A common use-case for voice assistant is to ask information about the weather, and Platypush can cover that too by running a voice assistant directly on your hardware.

The linked article describes how to run a fully local voice assistant, with local speech-to-text and text-to-speech engines.

But things also work if you decide to use remote models through the assistant.openai or tts.openai plugins.

You can use the assistant-sample repository to quickly get started with a Docker image with a Platypush installation configured to run a voice assistant.

Some sample configuration, using assistant.openwakeword for hotword detection together with assistant.openai and tts.openai :

You can then add a script with an event hook on SpeechRecognizedEvent and reacts to weather requests:

Voice weather assistant flow

No

Yes

No

Yes

No default

Hotword detected assistant.openwakeword

Start conversation assistant.openai.start_conversation

Speech recognized SpeechRecognizedEvent

Weather request detected?

Ignore or let other handlers continue

Parse free-text request

OpenAI extracts structured request location + delta_days

Location available?

Use default location from weather.openweathermap config

Geocode location via Nominatim

Fetch coordinates

Weather API via Platypush weather.openweathermap.get_forecast

Filter forecast by requested time range

Build structured weather JSON now / today / tomorrow / ndays

OpenAI turns weather JSON into a short spoken weather report

assistant.openai.render_response

TTS via tts.openai

Spoken answer to the user

Fallback apology if no weather info is found

A full demo of how it looks and sounds like:

Content version: 61d29501a9dc