Make a Google AIY Assistant without the kit on the Raspberry Pi 2 and 3

Here is a short demo in case you know nothing of Google AIY (Aritificial Intelligence Yourself).

If you prefer you can watch the full instructional video on YouTube.

In mid-2017 Google released a kit for use with the Raspberry Pi under the AIY brand, it came with a Voice HAT, Voice HAT microphone board, a 3″ speaker,  a cardboard box,  some wires and some lovely arcade buttons. This kit was surpassed with the V2 release that used the Pi Zero WH.

 

But you don’t need the kit to take advantage of the image that they have provided. All we need to do is make a few changes to some configuration files to get the Google AIY to work without the Voice Hat stuff.

I will be using a USB sound card (This is the one I bought from Amazon UKUS) instead of the Voice Hat along with an external speaker and a 3.5mm jack microphone. Wanna follow along? Take a look below.

Contents

  1. What you will need
  2. Where to begin
  3. Setting up the software
    1. General settings
    2. Edit audio settings and test
    3. Enable Google Cloud stuff
    4. Getting it to run on boot
  4. What to do next?
  5. Problems?

What you will need

    • Raspberry Pi 2/3 (Amazon UKUS)
    • Micro-SD card (I would recommend a minimum of 8GB)
    • External powered speaker (Anker make a fairly good one – Amazon UKUS but any will do)
    • USB soundcard (This is the one I bought from Amazon UKUS)
      • This option gives you 3.5mm input for audio and 3.5mm output
      • You will need a cheap 3.5mm microphone for this approach (This is the one I bought from Amazon UKUS but any will do)
    • WiFi dongle or Ethernet connection if using the Pi 2
    • A breadboard
    • A momentary button
    • A Google Account (This is so you can make use of the Google Assistant API)

Where to begin

I will be using Windows for this process, steps are similar but Linux and Mac users will have to muddle through I am afraid.

Step 1 – Building Blocks
This is probably a little obvious to most of you, connect all your components together but hold off on power for now.

You will need to connect the button up to GPIO23, you can still use this without the button if you don’t have one. Using your breadboard, connect one side of your button to GPIO23 and the other side to GND, the pins are mark in pink and blue on the image.

 

Step 2 – Getting Software
Download and burn the image to your SD card, it will be zipped up in a Tar format that you can extract with 7-Zip and burned with something like Win32DiskImager or you can burn it directly to you SD card using Etcher.

Step 3 – Getting Connected

Easy
Here is where things get a little messy, depending upon what you have available. If you have a monitor, keyboard and mouse available then you can set up your WiFi in the GUI environment that this image will boot straight into. This is likely the easiest method to get up and running quickly. I won’t go into the specifics here, setting up the WiFi should be easy as Pi in the GUI these days. You can achieve everything in this tutoral through the GUI interface but I am going to go the rest of the way with a terminal window.

Medium
If you don’t have want to mess up your desktop setup or you are using an all-in-one, you should connect through Ethernet to your home network. This is my preferred method to set up my Raspberry Pi’s.

Hard
Grab yourself a USB to serial converter, and hook it up to the Pi’s RX and TX lines. You will still need an internet connection so this is not the best route to take but it can get you in to connect up your WiFi module.

We are going to connect to the Pi for the first time using SSH or through a serial interface. To do this I am going to use Putty but feel free to use your terminal/serial console of choice, there are even browser based ones these days.

When you open Putty you can choose between SSH or Serial (choose Serial if you are using that interface and select the correct COM port) at this point you will need to know the IP address of your Pi if running the SSH route.

Raspberry Pi Google AIY Speak Button Pin

Setting up the software

Now you will need to create a project on the Google Cloud Platform the instructions on AIY are great for this.

Time-saving tip
You can copy these commands by selecting them and copying with
ctrl-c then paste into Putty with a mouse right-click.

General settings

Login to your Pi, the username is pi and the password will be raspberry

First we’ll update. This can take ages (30 minutes) but it is important to make sure your Pi can talk to your hardware, it may need newer drivers. Runs these commands separately:


sudo apt-get update

sudo apt-get upgrade

sudo reboot

Second, let’s modify some settings in the Raspberry Pi Software Configuration Tool:

sudo raspi-config

(Optional) 3 Boot Options –> B1 Desktop / CLI

Select Console Autologin

Now jump back and select Wait for Network at Boot

Now under 2. Network Option we will need to connect to WiFi if you are using it. You will need to know your home network SSID (The name that pops up when searching for WiFi) and the password.

Edit audio settings and test

Now we need to delve into some of the audio configuration files, these are used by the Pi (Specifically ALSA) to determine what device to use as the default for audio input and output.

You will need to run two commands which will open files in the terminal, delete the contents of that file and paste in the Audio Configuration code listed below.

Command 1

sudo raspi-config
sudo nano /etc/asound.conf

Command 2

sudo nano ~/.asoundrc

Audio Configuration code
This code tells the OS which card you want to use, here it is specified as a USB PnP Sound Device. If your USB sound card reports to the OS differently you can alter the title, use the command cat /proc/asound/cards to see how your appears.

options snd-usb-audio index=1
pcm.!default {
type hw
card 1
}
ctl.!default {
type hw
card 1
}

Next let’s change some of the ALSA configuration files so it always uses our default device (we specified that it would be device 1)

sudo nano /usr/share/alsa/alsa.conf

Now find the section that contains the following, it is on page 2:

#defaults.ctl.card 0
#defaults.pcm.card 0

Replace those sections with:

defaults.ctl.card 1
defaults.pcm.card 1

Now lets go in an change our volume settings. Type alsamixer to bring up this window.

I changed mine so that they were at 50% for both speaker and Mic, you can navigate to your devive with F6. Once you exit you can store those settings with the following command:

sudo alsactl store

Now let’s reboot and see it our changes have take effect:

sudo reboot

Let’s test our audio now, to do that we are going to record from the microphone and play it back through the speakers.

This command will record audio through your microphone for 5 seconds:

arecord -d 5 -f S16_LE -r 44100 -t wav they_hear_me_recording_they_hating.wav

This command will play back what you have recorded, hopefully you will hear whatever you recorded through your external speaker:

aplay they_hear_me_recording_they_hating.wav

Enable Google Cloud stuff

You will need to grab some credentials in order to work with the Google Assiatant. Make sure you have a Google Account and you are logged in on your PC.

Go to the the Google Cloud platform website and create a project, make sure you select/activate Google Assistant

Next you need to grab some credentials for an OAuth 2.0 client once you have configured your ‘app’ you can download them from the credentials list. This will be a JSON file.

Now we need to get that onto the Pi, the easiest way is just to create a file in the terminal, open the downloaded file on your PC and copy and paste the contents to the terminal window inside the JSON file we will create:

sudo nano /home/pi/assistant.json

Now to activate!

cd AIY-voice-kit-python

The run this:

src/examples/voice/assistant_grpc_demo.py

It is going to ask you to visit a website, bit of a pain because copying the URL from the terminal will close the Python script. Copy the long URL and paste it into your desktop browser and follow the authorisation steps to get your key. Copy the response key from your browser and then run that command again in the terminal, this time you can paste the response in and it will activate your device.

Getting it to run on boot

Copy this example src/examples/voice/assistant_library_with_button_demo.py using this command:

cp src/examples/voice/assistant_library_with_button_demo.py src/main.py

Now we need to create a service, this is something that runs at boot and remains running. To do this we need to create a ‘service’ file that describes our service, the environment it will run in and the location of the Python file. We have called out file main.py but you can call it what every you like, just be sure to change it in the service too.

Lets begin by creating the file, we will move it later.

sudo nano pi-assistant.service

Now copy the code below and paste it into your terminal.

Description=Pi Google Assistant
[Service]
Environment=XDG_RUNTIME_DIR=/run/user/1000
ExecStart=/bin/bash -c 'python3 -u src/main.py'
WorkingDirectory=/home/pi/AIY-projects-python Restart=always User=pi [Install] WantedBy=multi-user.target

Now that we have created our service file we will need to move it to the right folder with this command:

sudo mv pi-assistant.service /lib/systemd/system/

Next we will need to enable the service so that it will work on boot with this command:

sudo systemctl enable pi-assistant.service

At this point I had a few problems with the user (Pi) being able to run the script,  now the next set of commands are not entirely necessary but if you get permission errors with your service this should solve them:

Run them one at a time.

sudo chown -R pi /home/pi/.config/pulse
chmod 755 /home/pi/.config/pulse

Now let’s start the service:

sudo service pi-assistant start

If all has gone well and once your Pi has fully booted, you should just be able to press the button or call out ‘Hey Google’ to speak to the assistant.

If it hasn’t and your service doesn’t start, run these commands separately and then reboot.

chown -R pi /run/user/1000
chmod 755 /run/user/1000

What to do next?

Adding a power button for a headless setup is really useful to safely shutdown the pi and to reboot it.

I have implemented rpi-shutdown from Halofx on GitHub and it is working great. One press of the button will shutdown the Pi and once shutdown a second press will reboot it. To get this set up we’ll need a button on GPIO 3, I have marker the pins for you:

Raspberry Pi Shutdown Button Pin

Download the code from GitHub and SFTP it over to your Pi or just create a new file with sudo nano /home/pi/shutdown_pi.py and paste the code in.

Now lets edit a file which will run our script at boot, type – sudo nano /etc/rc.local

You will see a file that looks like this:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Print the IP address
_IP=$(hostname -I) || true
if [ “$_IP” ]; then
printf “My IP address is %s\n” “$_IP”
fi

exit 0


No you will need to add our Python file to rc.local file like this:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Print the IP address
_IP=$(hostname -I) || true
if [ “$_IP” ]; then
printf “My IP address is %s\n” “$_IP”
fi
sudo python /home/pi/shutdown_pi.py &
exit 0

Hopefully and there are no guarantees, you should be up and running. Audio configurations can be fiddly, Python and Linux are quite sensitive to syntax and spacing for commands, you may have a very different USB sound card to me, there are lots of things that can make you efforts fail. I am happy to help in the comments or on YouTube but I am by no means a Linux expert.

Problems?

Yeah, unfortuately that is pretty likely, things can go wrong. Often times it is a case of going back and repeating some of the steps which change settings, permissions or just plain starting again.

  • Run the command speaker-test to see if you get output on your speakers
  • You service isn’t running! Run the command sudo service pi-assistant status to see what error message it is flagging up. If it is to do with failing to create a directory repeat the chmod and chown steps
  • Python is sensitive to spacing and specifically tabs, make sure your code looks like it does above
  • My audio output is really low! Try ‘Hey Google, what’s the volume?’ then set is higher with – ‘Hey Google, set volume to *1-10 or 0-100%*’

24 thoughts on “Make a Google AIY Assistant without the kit on the Raspberry Pi 2 and 3

  1. I got this error when I ran

    src/examples/voice/assistant_grpc_demo.py

    everything had gone great until then what is wrong thanks

    Traceback (most recent call last):
    File “src/examples/voice/assistant_grpc_demo.py”, line 54, in
    main()
    File “src/examples/voice/assistant_grpc_demo.py”, line 33, in main
    assistant = aiy.assistant.grpc.get_assistant()
    File “/opt/aiy/projects-python/src/aiy/assistant/grpc.py”, line 80, in get_ass istant
    credentials = aiy.assistant.auth_helpers.get_assistant_credentials()
    File “/opt/aiy/projects-python/src/aiy/assistant/auth_helpers.py”, line 132, i n get_assistant_credentials
    return _try_to_get_credentials(credentials_file)
    File “/opt/aiy/projects-python/src/aiy/assistant/auth_helpers.py”, line 123, i n _try_to_get_credentials
    credentials = _credentials_flow_interactive(client_secrets)
    File “/opt/aiy/projects-python/src/aiy/assistant/auth_helpers.py”, line 78, in _credentials_flow_interactive
    credentials = flow.run_console()
    File “/usr/local/lib/python3.5/dist-packages/google_auth_oauthlib/flow.py”, li ne 364, in run_console
    self.fetch_token(code=code)
    File “/usr/local/lib/python3.5/dist-packages/google_auth_oauthlib/flow.py”, li ne 239, in fetch_token
    kwargs.setdefault(‘client_secret’, self.client_config[‘client_secret’])
    KeyError: ‘client_secret’

    1. Hi James, It looks like you need to have a go at the credentials section again, no harm in running it again. KeyError: ‘client_secret’ tells me that it cannot read you assistant.json file properly.

      1. hola a mi tambien me ha salido lo mismo. he vuelto hacer las credenciales y no me hace nada. todo me funciona menos eso… como hago la configuracion de credenciales??

  2. Is there anyway to add a light indicator and an audial feedback when you say “ok google”

    Your instruction were very clear. Thank you sooo much!

    1. Sure, you can use the RPi.GPIO to turn on an off LEDs. You should be able to play WAV files via aplay or VLC if you wish. I imagine aplay is probably easier. You could put it all in the conditional statements within the main python program. For example, I use the EventType.ON_ALERT_STARTED flag from AIY to tun set a GPIO pin high.

      1. You can do the wav file playing within Python pretty easily, and better than calling an external program for most purposes.

        I have a program that runs on my PC, and alerts me with a sound clip when a remote embedded system hiccups. The code is pretty simple. At the top:

        import pygame
        import wave

        To set up the player:
        try:
        waver = wave.open(Restart_sound_file_name, mode=’rb’)
        sample_rate = waver.getframerate()
        num_channels = waver.getnchannels()
        waver.close()
        pygame.mixer.pre_init(frequency=sample_rate, channels=num_channels)
        pygame.init()
        self.restart_sound_exists = 1
        # print (‘Sound initialized’) #debug
        # sys.stdout.flush()
        except:
        print sys.exc_info() #DEBUG
        tkMessageBox.showinfo(“Uh-oh!”, Restart_sound_file_name + ” doesn’t exist, or is a bad .WAV!”)

        You’d want to handle the exception a little differently if you’re not running a GUI. Restart_sound_file_name is passed as a command line parameter, so I have to make sure that I didn’t mistype it before using it. Less paranoia _may_ be okay if you’re building a production package and you’re “certain” it’s okay 😉

        To play the sound:
        pygame.mixer.music.load(self.alert_sound_file_name)
        pygame.mixer.music.play(0)

        Note that my alert sounds are ones that I generated with the same sample rate and in mono. If you’re using multiple sounds with different sample rates and/or number of channels, you probably need to reinitialize the mixer for each.

  3. When asking the assistant to call me by my name, it replies that it does not have permissions set in the assistant settings to allow it to do so… It canot say my name or get my location etc…On my Google account all permissions is granted! I guess it must be a setting in Raspbian?
    Please help!

    1. Hi, when you ask Google to tell you your name it will tell you about the permissions. This should trigger something on you phone (assuming you have a Google account on your phone) and it will prompt you to change some settings in the app. I didnt run into any problems so it isn’t something I have had to troubleshoot.

  4. Great work, I have got it running and now I wonder how to change the Assistant language. Can you help me?

  5. Hi, i install aiyprojects-2018-08-03 image to raspberry pi 3B+.
    I doing step by step following your tutorial.
    after run command: sudo service pi-assistant start
    everything ok.
    but when i reboot raspberry pi.
    nothing response when i say “Ok google” or push button

  6. Hi,

    i want to use this together with my smart mirror.
    My question is how you can burn the pi´s operating system and the aiy image on the same sd card???

  7. The audio input is way too low….Google assistant can hardly hear when I say hey Google…..but when I about hey Google it definitely works…lol

  8. how do i configure it without internet. simply just respond me a one question i ask..?

  9. Hi David, please I need a step by step explanation how to add a led to the assistant feddbak response, I read what you response Alim, But..I don’t understand, Im new in Rpi. thanks a lot

  10. Hi David im a Uni student and i really loved the video, i’ve been trying too do this for months and yours does seem the best way. But i have found a problem with the mic, it doesn’t pick anything up i have it turned up to max on both volume control and alsamixer. I would really love to hear back from you and your thoughts. Thanks

  11. So, I need to use my headphone jack for my speaker, but it seems this install has removed that card..? How can I get it back in there?

    Here’s all I’m seeing:

    pi@raspberrypi:~ $ cat /proc/asound/cards
    1 [Headset ]: USB-Audio – Logitech USB Headset
    Logitech USB Headset Logitech USB Headset at usb-3f980000.usb-1.3, full speed

    I know it works with other installs.

    Thanks

  12. Good day, I seem to have a problem:
    when I run the command sudo nano /usr/share/alsa/alsa.conf the file opens but it stops at “/cards/aliases.conf” therefore I can not see the default things. The previous files I opened to paste this code
    options snd-usb-audio index=1
    pcm.!default {
    type hw
    card 1
    }
    ctl.!default {
    type hw
    card 1
    }
    was empty

  13. Hell o David…..My partner and I are doing our final project for both my College Electronic Engineering program and his Computer Engineering program. We have built a working amplifier and are attempting to incorporate a touchscreen GUI interface similar to your YouTube “Rune, Audio Player with a Raspberry Pi – Sponsored by Reichelt”. We purchased the Raspberry PI 4 with 4 Gb of RAM, but weeks later realized that it did not have many compatible options. We just purchased a Raspberry PI 3 Model B and we still are having no luck. We have about 3 weeks to try to get this part going, so any help sooner than later would be life saving.

    Any advice, support, instruction, coaching or referrals would be greatly appreciated. We are from Canada so if there is a possibility to arrange for some communication or consulting please email us with some contact information.

    Thank you and we hope to hear from you.

  14. Hello, I got this error, could you help?

    INFO:root:Press button to start conversation…
    INFO:root:Conversation started!
    INFO:aiy.assistant.grpc:Recording started.
    aplay: playback:2787: read error
    INFO:aiy.assistant.grpc:Recording stopped.
    Traceback (most recent call last):
    File “src/examples/voice/assistant_grpc_demo.py”, line 57, in
    main()
    File “src/examples/voice/assistant_grpc_demo.py”, line 54, in main
    assistant.conversation()
    File “/opt/aiy/projects-python/src/aiy/assistant/grpc.py”, line 190, in conver sation
    keep_talking = self._assist(recorder, wrapped_play, deadline)
    File “/opt/aiy/projects-python/src/aiy/assistant/grpc.py”, line 135, in _assis t
    for response in self._assistant.Assist(self._requests(recorder), deadline):
    File “/usr/local/lib/python3.5/dist-packages/grpc/_channel.py”, line 363, in _ _next__
    return self._next()
    File “/usr/local/lib/python3.5/dist-packages/grpc/_channel.py”, line 357, in _ next
    raise self
    grpc._channel._Rendezvous:

  15. Thanks a lot. I saw AIY in Google, searched AIY components, it cost around 37500 rupees for voice hat, and 1200 rupees for raspberry Pi zero. Total more than 5000(37500+1200+others),so I dropped the plan today. After seeing your post I got hope. I can create DIY Google Assistant.

    Thank you

  16. **************************************
    raceback (most recent call last):
    File “src/examples/voice/assistant_grpc_demo.py”, line 57, in
    main()
    File “src/examples/voice/assistant_grpc_demo.py”, line 49, in main
    language_code=args.language)
    File “/home/pi/AIY-projects-python/src/aiy/assistant/grpc.py”, line 258, in __init__
    super().__init__(language_code, volume_percentage)
    File “/home/pi/AIY-projects-python/src/aiy/assistant/grpc.py”, line 94, in __init__
    credentials = auth_helpers.get_assistant_credentials()
    File “/home/pi/AIY-projects-python/src/aiy/assistant/auth_helpers.py”, line 151, in get_assistant_credentials
    return _try_to_get_credentials(credentials_file)
    File “/home/pi/AIY-projects-python/src/aiy/assistant/auth_helpers.py”, line 123, in _try_to_get_credentials
    credentials = _credentials_flow_interactive(client_secrets)
    File “/home/pi/AIY-projects-python/src/aiy/assistant/auth_helpers.py”, line 75, in _credentials_flow_interactive
    webbrowser.register(‘chromium-browser’, None, webbrowser.Chrome(‘chromium-browser’), -1)
    TypeError: register() takes from 2 to 3 positional arguments but 4 were given
    ***********************************************
    I got this error when I ran `src/assistant_library_demo.py`
    google voice kit error
    help me pls

Leave a Reply to David Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.