, Alexa, and Siri are the dominating voice assistants accessible for on a regular basis use. These assistants have develop into ubiquitous in virtually each residence, finishing up duties from residence automation, notice taking, recipe steerage and answering easy questions. In relation to answering questions although, within the age of LLMs, getting a concise and context-based reply from these voice assistants might be difficult, if not non-existent. For instance, when you ask Google Assistant how the market is reacting to Jerome Powell’s speech in Jackson Gap on Aug 22, it should merely reply that it doesn’t know the reply and provides a number of hyperlinks that you would be able to peruse. That’s when you have the screen-based Google Assistant.
Typically you simply need a fast reply on present occasions, otherwise you wish to know if an Apple tree would survive the winter in Ohio, and sometimes voice assistants like Google and Siri fall wanting offering a satisfying reply. This received me serious about constructing my very own voice assistant, one that will give me a easy, single sentence reply primarily based on its search of the net.
Of the varied LLM powered engines like google accessible, I’ve been an avid person of Perplexity for greater than a yr now and I exploit it completely for all my searches besides for easy ones the place I nonetheless return to Google or Bing. Perplexity, along with its dwell internet index, which permits it to offer up-to-date, correct, sourced solutions, permits customers entry to its performance by way of a strong API. Utilizing this performance and integrating it with a easy Raspberry Pi, I supposed to create a voice assistant that will:
- Reply to a wake phrase and be able to reply my query
- Reply my query in a easy, concise sentence
- Return to passive listening with out promoting my information or giving my pointless adverts
The {Hardware} for the Assistant
To construct our voice assistant, a number of key {hardware} elements are required. The core of the mission is a Raspberry Pi 5, which serves because the central processor for our software. For the assistant’s audio enter, I selected a easy USB gooseneck microphone. One of these microphone is omnidirectional, making it efficient at listening to the wake phrase from totally different elements of a room, and its plug-and-play nature simplifies the setup. For the assistant’s output, a compact USB-powered speaker gives the audio output. A key benefit of this speaker is that it makes use of a single USB cable for each its energy and audio sign, which minimizes cable muddle.
This strategy of utilizing available USB peripherals makes the {hardware} meeting simple, permitting us to focus our efforts on the software program.
Getting the atmosphere prepared
So as to question Perplexity utilizing customized queries and with a purpose to have a wake phrase for the voice assistant, we have to generate a few API keys. So as to generate a Perplexity API key one can join a Perplexity account, go to the Settings menu, choose the API tab, and click on “Generate API Key” to create and duplicate their private key to be used in functions. Entry to API key technology often requires a paid plan or cost technique, so make sure the account is eligible earlier than continuing.
Platforms that supply wake phrase customization embody PicoVoice Porcupine, Sensory TrulyHandsfree, and Snowboy, with PicoVoice Porcupine offering a straightforward on-line console for producing, testing, and deploying customized wake phrases throughout desktop, cellular, and embedded units. A brand new person can generate a customized phrase for PicoVoice Porcupine by signing up for a free Picovoice Console account, navigating to the Porcupine web page, deciding on the specified language, typing within the customized wake phrase, and clicking “Practice” to provide and obtain the platform-specific mannequin file (.ppn) to be used. Be certain that to check the wake phrase for efficiency earlier than finalizing, as this ensures dependable detection and minimal false positives. The wake phrase I’ve educated and can use is “Hey Krishna”.
Coding the Assistant
The entire Python script for this mission is accessible on my GitHub repository. On this part, let’s take a look at the important thing elements of the code to grasp how the assistant capabilities.
The script is organized into a number of core capabilities that deal with the assistant’s senses and intelligence, all managed by a central loop.
Configuration and Initialization
The primary a part of the script is devoted to setup. It handles loading the mandatory API keys, mannequin information, and initializing the shoppers for the companies we’ll use.
# --- 1. Configuration ---
load_dotenv()
PICOVOICE_ACCESS_KEY = os.environ.get("PICOVOICE_ACCESS_KEY")
PERPLEXITY_API_KEY = os.environ.get("PERPLEXITY_API_KEY")
KEYWORD_PATHS = ["Krishna_raspberry-pi.ppn"] # My wake phrase pat
MODEL_NAME = "sonar"This part makes use of the dotenv library to securely load your secret API keys from a .env file, which is a finest follow that retains them out of your supply code. It additionally defines key variables like the trail to your customized wake phrase file and the precise Perplexity mannequin we wish to question.
Wake Phrase Detection
For the assistant to be really hands-free, it must hear repeatedly for a particular wake phrase with out utilizing vital system sources. That is dealt with by the whereas True: loop within the primary perform, which makes use of the PicoVoice Porcupine engine.
# That is the primary loop that runs repeatedly
whereas True:
# Learn a small chunk of uncooked audio information from the microphone
pcm = audio_stream.learn(porcupine.frame_length)
pcm = struct.unpack_from("h" * porcupine.frame_length, pcm)
# Feed the audio chunk into the Porcupine engine for evaluation
keyword_index = porcupine.course of(pcm)
if keyword_index >= 0:
# Wake phrase was detected, proceed to deal with the command...
print("Wake phrase detected!")This loop is the center of the assistant’s “passive listening” state. It repeatedly reads small, uncooked audio frames from the microphone stream. Every body is then handed to the porcupine.course of() perform. It is a extremely environment friendly, offline course of that analyzes the audio for the precise acoustic sample of your customized wake phrase (“Krishna”). If the sample is detected, porcupine.course of() returns a non-negative quantity, and the script proceeds to the lively part of listening for a full command.
Speech-to-Textual content — Changing person inquiries to textual content
After the wake phrase is detected, the assistant must hear for and perceive the person’s query. That is dealt with by the Speech-to-Textual content (STT) element.
# --- This logic is inside the primary 'if keyword_index >= 0:' block ---
print("Listening for command...")
frames = []
# Report audio from the stream for a set length (~10 seconds)
for _ in vary(0, int(porcupine.sample_rate / porcupine.frame_length * 10)):
frames.append(audio_stream.learn(porcupine.frame_length))
# Convert the uncooked audio frames into an object the library can use
audio_data = sr.AudioData(b"".be a part of(frames), porcupine.sample_rate, 2)
attempt:
# Ship the audio information to Google's service for transcription
command = recognizer.recognize_google(audio_data)
print(f"You (command): {command}")
besides sr.UnknownValueError:
speak_text("Sorry, I did not catch that.")
As soon as the wake phrase is detected, the code actively information audio from the microphone for roughly 10 seconds, capturing the person’s spoken command. It then packages this uncooked audio information and sends it to Google’s speech recognition service utilizing the speech_recognition library. The service processes the audio and returns the transcribed textual content, which is then saved within the command variable.
Getting Solutions from Perplexity
As soon as the person’s command has been transformed to textual content, it’s despatched to the Perplexity API to get an clever, up-to-date reply.
# --- This logic runs if a command was efficiently transcribed ---
if command:
# Outline the directions and context for the AI
messages = [{"role": "system", "content": "You are an AI assistant. You are located in Twinsburg, Ohio. All answers must be relevant to Cleveland, Ohio unless asked for differently by the user. You MUST answer all questions in a single and VERY concise sentence."}]
messages.append({"function": "person", "content material": command})
# Ship the request to the Perplexity API
response = perplexity_client.chat.completions.create(
mannequin=MODEL_NAME,
messages=messages
)
assistant_response_text = response.selections[0].message.content material.strip()
speak_text(assistant_response_text)
This code block is the “mind” of the operation. It first constructs a messages listing, which features a crucial system immediate. This immediate provides the AI its character and guidelines, equivalent to answering in a single sentence and being conscious of its location in Ohio. The person’s command is then added to this listing, and all the bundle is shipped to the Perplexity API. The script then extracts the textual content from the AI’s response and passes it to the speak_text perform to be learn aloud.
Textual content-to-Speech — Changing Perplexity response to Voice
The speak_text perform is what provides the assistant its voice.
def speak_text(text_to_speak, lang='en'):
# Outline a perform that converts textual content to speech, default language is English
print(f"Assistant (talking): {text_to_speak}")
# Print the textual content for reference so the person can see what's being spoken
attempt:
pygame.mixer.init()
# Initialize the Pygame mixer module for audio playback
tts = gTTS(textual content=text_to_speak, lang=lang, sluggish=False)
# Create a Google Textual content-to-Speech (gTTS) object with the offered textual content and language
# 'sluggish=False' makes the speech sound extra pure (not slow-paced)
mp3_filename = "response_audio.mp3"
# Set the filename the place the generated speech will probably be saved
tts.save(mp3_filename)
# Save the generated speech as an MP3 file
pygame.mixer.music.load(mp3_filename)
# Load the MP3 file into Pygame's music participant for playback
pygame.mixer.music.play()
# Begin taking part in the speech audio
whereas pygame.mixer.music.get_busy():
pygame.time.Clock().tick(10)
# Hold this system operating (by checking if playback is ongoing)
# This prevents the script from ending earlier than the speech finishes
# The clock.tick(10) ensures it checks 10 instances per second
pygame.mixer.give up()
# Give up the Pygame mixer as soon as playback is full to free sources
os.take away(mp3_filename)
# Delete the momentary MP3 file after playback to scrub up
besides Exception as e:
print(f"Error in Textual content-to-Speech: {e}")
# Catch and show any errors that happen throughout the speech technology or playbackThis perform takes a textual content string, prints it for reference, then makes use of the gTTS (Google Textual content-to-Speech) library to generate a brief MP3 audio file. It performs the file by way of the system’s audio system utilizing the pygame library, waits till playback is completed, after which deletes the file. Error dealing with is included to catch points throughout the course of.
Testing the assistant
Beneath is an indication of the functioning of the customized voice assistant. To match its efficiency with Google Assistant, I’ve requested the identical query from Google in addition to from the customized assistant.
As you may see, Google gives hyperlinks to the reply slightly than offering a quick abstract of what the person desires. The customized assistant goes additional and gives a abstract and is extra useful and informational.
Conclusion
On this article, we regarded on the means of constructing a completely purposeful, hands-free voice assistant on a Raspberry Pi. By combining the facility of a customized wake phrase and the Perplexity API by utilizing Python, we created a easy voice assistant machine that helps in getting data shortly.
The important thing benefit of this LLM-based strategy is its means to ship direct, synthesized solutions to advanced and present questions — a job the place assistants like Google Assistant usually fall brief by merely offering an inventory of search hyperlinks. As an alternative of appearing as a mere voice interface for a search engine, our assistant capabilities as a real reply engine, parsing real-time internet outcomes to present a single, concise response. The way forward for voice assistants lies on this deeper, extra clever integration, and constructing your personal is one of the best ways to discover it.
