RaspberryPi + Pocketsphinx + ps3eye Error: Failed to open audio device

Question

just installed pocketsphinx on my raspberry pi. Think i'm going crazy but not sure if I'm providing the correct device.

Whenever I run:

src/programs/pocketsphinx_continuous -adcdev plughw:1,0 -nfft 2048 -samprate 48000

I get the following:

root@scarlettpi:/usr/install/pocketsphinx-0.8# src/programs/pocketsphinx_continuous -adcdev plughw:1,0 -nfft 2048 -samprate 48000 INFO: cmd_ln.c(691): Parsing command line: /usr/install/pocketsphinx-0.8/src/programs/.libs/lt-pocketsphinx_continuous \ -adcdev plughw:1,0 \ -nfft 2048 \ -samprate 48000

Current configuration: [NAME] [DEFLT] [VALUE] -adcdev plughw:1,0 -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -argfile
-ascale 20.0 2.000000e+01 -aw 1 1 -backtrace no no -beam 1e-48 1.000000e-48 -bestpath yes yes -bestpathlw 9.5 9.500000e+00 -bghist no no -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -compallsen no no -debug 0 -dict
-dictcase no no -dither no no -doublebw no no -ds 1 1 -fdict
-feat 1s_c_d_dd 1s_c_d_dd -featparams
-fillprob 1e-8 1.000000e-08 -frate 100 100 -fsg
-fsgusealtpron yes yes -fsgusefiller yes yes -fwdflat yes yes -fwdflatbeam 1e-64 1.000000e-64 -fwdflatefwid 4 4 -fwdflatlw 8.5 8.500000e+00 -fwdflatsfwin 25 25 -fwdflatwbeam 7e-29 7.000000e-29 -fwdtree yes yes -hmm
-infile
-input_endian little little -jsgf
-kdmaxbbi -1 -1 -kdmaxdepth 0 0 -kdtree
-latsize 5000 5000 -lda
-ldadim 0 0 -lextreedump 0 0 -lifter 0 0 -lm
-lmctl
-lmname default default -logbase 1.0001 1.000100e+00 -logfn
-logspec no no -lowerf 133.33334 1.333333e+02 -lpbeam 1e-40 1.000000e-40 -lponlybeam 7e-29 7.000000e-29 -lw 6.5 6.500000e+00 -maxhmmpf -1 -1 -maxnewoov 20 20 -maxwpf -1 -1 -mdef
-mean
-mfclogdir
-min_endfr 0 0 -mixw
-mixwfloor 0.0000001 1.000000e-07 -mllr
-mmap yes yes -ncep 13 13 -nfft 512 2048 -nfilt 40 40 -nwpen 1.0 1.000000e+00 -pbeam 1e-48 1.000000e-48 -pip 1.0 1.000000e+00 -pl_beam 1e-10 1.000000e-10 -pl_pbeam 1e-5 1.000000e-05 -pl_window 0 0 -rawlogdir
-remove_dc no no -round_filters yes yes -samprate 16000 4.800000e+04 -seed -1 -1 -sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03 -smoothspec no no -svspec
-time no no -tmat
-tmatfloor 0.0001 1.000000e-04 -topn 4 4 -topn_beam 0 0 -toprule
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -usewdphones no no -uw 1.0 1.000000e+00 -var
-varfloor 0.0001 1.000000e-04 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wbeam 7e-29 7.000000e-29 -wip 0.65 6.500000e-01 -wlen 0.025625 2.562500e-02

INFO: cmd_ln.c(691): Parsing command line: \ -nfilt 20 \ -lowerf 1 \ -upperf 4000 \ -wlen 0.025 \ -transform dct \ -round_filters no \ -remove_dc yes \ -svspec 0-12/13-25/26-38 \ -feat 1s_c_d_dd \ -agc none \ -cmn current \ -cmninit 56,-3,1 \ -varnorm no

Current configuration: [NAME] [DEFLT] [VALUE] -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -ceplen 13 13 -cmn current current -cmninit 8.0 56,-3,1 -dither no no -doublebw no no -feat 1s_c_d_dd 1s_c_d_dd -frate 100 100 -input_endian little little -lda
-ldadim 0 0 -lifter 0 0 -logspec no no -lowerf 133.33334 1.000000e+00 -ncep 13 13 -nfft 512 2048 -nfilt 40 20 -remove_dc no yes -round_filters yes no -samprate 16000 4.800000e+04 -seed -1 -1 -smoothspec no no -svspec 0-12/13-25/26-38 -transform legacy dct -unit_area yes yes -upperf 6855.4976 4.000000e+03 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wlen 0.025625 2.500000e-02

INFO: acmod.c(246): Parsed model-specific feature parameters from /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/feat.params INFO: feat.c(713): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none' INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0 INFO: acmod.c(167): Using subvector specification 0-12/13-25/26-38 INFO: mdef.c(517): Reading model definition: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/mdef INFO: mdef.c(528): Found byte-order mark BMDF, assuming this is a binary mdef file INFO: bin_mdef.c(336): Reading binary model definition: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/mdef INFO: bin_mdef.c(513): 50 CI-phone, 143047 CD-phone, 3 emitstate/phone, 150 CI-sen, 5150 Sen, 27135 Sen-Seq INFO: tmat.c(205): Reading HMM transition probability matrices: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/transition_matrices INFO: acmod.c(121): Attempting to use SCHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/means INFO: ms_gauden.c(292): 1 codebook, 3 feature, size: INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/variances INFO: ms_gauden.c(292): 1 codebook, 3 feature, size: INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(354): 0 variance values floored INFO: s2_semi_mgau.c(903): Loading senones from dump file /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/sendump INFO: s2_semi_mgau.c(927): BEGIN FILE FORMAT DESCRIPTION INFO: s2_semi_mgau.c(1022): Using memory-mapped I/O for senones INFO: s2_semi_mgau.c(1296): Maximum top-N: 4 Top-N beams: 0 0 0 INFO: dict.c(317): Allocating 137543 * 20 bytes (2686 KiB) for word entries INFO: dict.c(332): Reading main dictionary: /usr/local/share/pocketsphinx/model/lm/en_US/cmu07a.dic INFO: dict.c(211): Allocated 1010 KiB for strings, 1664 KiB for phones INFO: dict.c(335): 133436 words read INFO: dict.c(341): Reading filler dictionary: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/noisedict INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(344): 11 words read INFO: dict2pid.c(396): Building PID tables for dictionary INFO: dict2pid.c(404): Allocating 50^3 * 2 bytes (244 KiB) for word-initial triphones INFO: dict2pid.c(131): Allocated 30200 bytes (29 KiB) for word-final triphones INFO: dict2pid.c(195): Allocated 30200 bytes (29 KiB) for single-phone word triphones INFO: ngram_model_arpa.c(77): No \data\ mark in LM file INFO: ngram_model_dmp.c(142): Will use memory-mapped I/O for LM file INFO: ngram_model_dmp.c(196): ngrams 1=5001, 2=436879, 3=418286 INFO: ngram_model_dmp.c(242): 5001 = LM.unigrams(+trailer) read INFO: ngram_model_dmp.c(288): 436879 = LM.bigrams(+trailer) read INFO: ngram_model_dmp.c(314): 418286 = LM.trigrams read INFO: ngram_model_dmp.c(339): 37293 = LM.prob2 entries read INFO: ngram_model_dmp.c(359): 14370 = LM.bo_wt2 entries read INFO: ngram_model_dmp.c(379): 36094 = LM.prob3 entries read INFO: ngram_model_dmp.c(407): 854 = LM.tseg_base entries read INFO: ngram_model_dmp.c(463): 5001 = ascii word strings read INFO: ngram_search_fwdtree.c(99): 788 unique initial diphones INFO: ngram_search_fwdtree.c(147): 0 root, 0 non-root channels, 60 single-phone words INFO: ngram_search_fwdtree.c(186): Creating search tree INFO: ngram_search_fwdtree.c(191): before: 0 root, 0 non-root channels, 60 single-phone words INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 13428 INFO: ngram_search_fwdtree.c(338): after: 457 root, 13300 non-root channels, 26 single-phone words INFO: ngram_search_fwdflat.c(156): fwdflat: min_ef_width = 4, max_sf_win = 25 INFO: continuous.c(371): /usr/install/pocketsphinx-0.8/src/programs/.libs/lt-pocketsphinx_continuous COMPILED ON: Jul 21 2013, AT: 14:34:06

Mixer load failed: Invalid argument FATAL_ERROR: "continuous.c", line 246: Failed to open audio device

I'm using a ps3eye currently. If i do a simple:

arecord -D plughw:1,0 -d 5 -q -f cd -t wav ~/test.wav

Everything works fine ( Verified this by hooking up Raspberrypi to TV via HDMI and running aplay ~/test.wav )

What am I doing wrong guys?

Information you might need ( based on other posts i've seen ):

root@scarlettpi:/usr/install/pocketsphinx-0.8# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
root@scarlettpi:/usr/install/pocketsphinx-0.8# 
root@scarlettpi:/usr/install/pocketsphinx-0.8# aplay -L
null
    Discard all samples (playback) or generate zero samples (capture)
pulse
    PulseAudio Sound Server
sysdefault:CARD=ALSA
    bcm2835 ALSA, bcm2835 ALSA
    Default Audio Device
root@scarlettpi:/usr/install/pocketsphinx-0.8#



root@scarlettpi:/usr/install/pocketsphinx-0.8# dpkg -l | grep "alsa"
ii  alsa-base                             1.0.25+3~deb7u1                      all          ALSA driver configuration files
ii  alsa-firmware-loaders                 1.0.25-2                             armhf        ALSA software loaders for specific hardware
ii  alsa-oss                              1.0.25-1                             armhf        ALSA wrapper for OSS applications
ii  alsa-tools                            1.0.25-2                             armhf        Console based ALSA utilities for specific hardware
ii  alsa-utils                            1.0.25-4                             armhf        Utilities for configuring and using ALSA
ii  alsaplayer-alsa                       0.99.80-5.1                          armhf        PCM player designed for ALSA (ALSA output module)
ii  alsaplayer-common                     0.99.80-5.1                          armhf        PCM player designed for ALSA (common files)
ii  alsaplayer-gtk                        0.99.80-5.1                          armhf        PCM player designed for ALSA (GTK+ version)
ii  gstreamer0.10-alsa:armhf              0.10.36-1.1                          armhf        GStreamer plugin for ALSA
ii  libsox-fmt-alsa                       14.4.0-3                             armhf        SoX alsa format I/O library
root@scarlettpi:/usr/install/pocketsphinx-0.8#



root@scarlettpi:/usr/install/pocketsphinx-0.8# dpkg -l | grep pulseaudio
ii  gstreamer0.10-pulseaudio:armhf        0.10.31-3+nmu1                       armhf        GStreamer plugin for PulseAudio
root@scarlettpi:/usr/install/pocketsphinx-0.8# 

Also in terms of installing pocket sphinx I did the following:

# uninstall pulse audio if its already installed 
apt-get remove pulseaudio -y 
aptitude purge pulseaudio -y

# sphinxbase install
apt-get install bison -y
cd /usr/install
wget  http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz
tar -xvf sphinxbase-0.8.tar.gz
cd sphinxbase-0.8
./configure --enable-fixed
make
make install
cd -

# pocketsphinx installwget http://sourceforge.net/projects/cmusphinx/files/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
tar -xvf pocketsphinx-0.8.tar.gz
cd pocketsphinx-0.8
./configure
make
make install

Any ideas or advice in the right direction would be extremely helpful.

Answer 1

Took me a while, with with some help from a couple sources ( they will be listed in my answer ) and some helpful hints from nikolay-shmyrev ( https://stackoverflow.com/users/432021/nikolay-shmyrev ), I finally came up with an answer that worked for me.

Key assumptions:

1. running these commands as the pi user ( previously I was running them as root, which was incorrect )

2. I'm using continuous recognition and I was ONLY looking for the ability to "wake-up" my raspberry pi. Upon waking it up, I have other plans on how it should interact.

My setup: 1. Canakit RaspberryPi

2. HDMI cable to my Toshiba TV

3. usb wifi dongle

4. Playstation 3 Eye for speech recognition

Moving forward. I ran the following commands on my RaspberryPi to get PulseAudio + pocketsphinx working together w/ my Playstation 3 Eye. ( If you see any places for improvement please let me know )

# Install pulse audio / development packages
sudo apt-get install gstreamer0.10-pulseaudio libao4 libasound2-plugins libgconfmm-2.6-1c2 libglademm-2.4-1c2a libpulse-dev libpulse-mainloop-glib0 libpulse-mainloop-glib0-dbg libpulse0 libpulse0-dbg libsox-fmt-pulse paman paprefs pavucontrol pavumeter pulseaudio pulseaudio-dbg pulseaudio-esound-compat pulseaudio-esound-compat-dbg pulseaudio-module-bluetooth pulseaudio-module-gconf pulseaudio-module-jack pulseaudio-module-lirc pulseaudio-module-lirc-dbg pulseaudio-module-x11 pulseaudio-module-zeroconf pulseaudio-module-zeroconf-dbg pulseaudio-utils -y

# per instructions from http://forums.debian.net/viewtopic.php?f=16&t=12497
sudo \cp -pf /etc/asound.conf /etc/asound.conf.ORIG 
sudo echo 'pcm.pulse {
    type pulse
}

ctl.pulse {
    type pulse
}

pcm.!default {
    type pulse
}

ctl.!default {
    type pulse
}' > /etc/asound.conf

_DEVICE_LOAD_ON_START=$(grep "snd.bcm2835" /etc/modules | wc -l)
if [[ "${_DEVICE_LOAD_ON_START}" = "0" ]]; then

  sudo \cp -pf /etc/modules /etc/modules.ORIG
  sudo echo "snd-bcm2835" >> /etc/modules

fi

_DISALLOW_MODULE_LOADING=$(grep "DISALLOW_MODULE_LOADING=1" /etc/default/pulseaudio | wc -l)
if [[ "${_DISALLOW_MODULE_LOADING}" = "0" ]]; then

  sudo \cp -pf /etc/default/pulseaudio /etc/default/pulseaudio.ORIG
  sudo sed -i "s,DISALLOW_MODULE_LOADING=1,DISALLOW_MODULE_LOADING=0,g" /etc/default/pulseaudio

fi

sudo \cp -fvp /etc/pulse/system.pa /etc/pulse/system.pa.ORIG
sudo echo "
# ScarlettPi ADDED THIS
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24 auth-anonymous=1
load-module module-zeroconf-publish" >> /etc/pulse/system.pa

sudo echo "
# ScarlettPi added this
#load-module module-native-protocol-tcp
#load-module module-zeroconf-publish
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24 auth-anonymous=1
load-module module-zeroconf-publish" >> /etc/pulse/default.pa

# check to make sure it looks okay
cat /etc/pulse/default.pa

# change default driver frm alsa to pulse
sudo \cp -fvp /etc/libao.conf /etc/libao.conf.ORIG
sudo sed -i "s,default_driver=alsa,default_driver=pulse,g" /etc/libao.conf 

# settings according to Pi-Musicbox ( https://github.com/woutervanwijk/Pi-MusicBox )
sudo \cp -fvp /etc/pulse/daemon.conf /etc/pulse/daemon.conf.ORIG

sudo echo "
# ScarlettPi added this
high-priority = yes
nice-level = 5
exit-idle-time = -1
resample-method = src-sinc-medium-quality
default-sample-format = s16le
default-sample-rate = 48000
default-sample-channels = 2" >> /etc/pulse/daemon.conf.ORIG

# add pi to the pulse access group
sudo adduser pi pulse-access

sudo shutdown -r now


# install python dev packages
sudo apt-get install python2.7-dev -y

# sphinxbase install
sudo apt-get install bison -y
cd ~pi/
wget  http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz
tar -xvf sphinxbase-0.8.tar.gz
cd sphinxbase-0.8
./configure
make
sudo make install

export LD_LIBRARY_PATH=/usr/local/lib
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

echo "
# scarlettPi added this
export LD_LIBRARY_PATH=/usr/local/lib
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig" >> ~/.bashrc

cd ~pi/
# pocketsphinx install
# set this: LD_LIBRARY_PATH=/path/to/pocketsphinxlibs /usr/local/bin/pocketsphinx_continuous
# http://www.voxforge.org/home/forums/message-boards/speech-recognition-engines/howto-use-pocketsphinx
wget http://sourceforge.net/projects/cmusphinx/files/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
tar -xvf pocketsphinx-0.8.tar.gz
cd pocketsphinx-0.8
./configure
make
sudo make install

cd ~pi/
wget http://sourceforge.net/projects/cmusphinx/files/sphinxtrain/1.0.8/sphinxtrain-1.0.8.tar.gz
tar -xvf sphinxtrain-1.0.8
cd sphinxtrain-1.0.8
./configure
make
sudo make install

# try using it:

sudo apt-get install oss-compat -y

# check if pulse daemon is running
ps aux | grep pulse

# If it isn't, start it up yourself ( need to figure out the best way to make this run on boot...init.d script maybe? )
# forgot where I grabbed this line from, but this was put into my rc.local ( didn't start up my pulseaudio daemon automatically though )
su - pi -c '/usr/bin/pulseaudio --start --log-target=syslog --system=false'

# check again to see if pule daemon is now running
ps aux | grep pulse

cd cd ~pi/pocketsphinx-0.8
# IMPORTANT NOTE, YOU HAVE TO BE USER PI AND THE PULSEAUDIO SERVER NEEDS TO BE RUNNING
# advice taken from: http://sourceforge.net/p/cmusphinx/discussion/help/thread/49e34dff/ ( on how to calibrate pocketsphinx correctly )
# from: on how to get pocketsphinx to recognize new words via a corpus: http://www.ghatage.com/2012/12/make-pocketsphinx-recognize-new-words/
# and finally from: http://puneetk.com/basics-of-java-speech-grammar-format-jsgf BEST/Simplest explanation of how Java Speech Grammar Format ( JSGF ) works
# NOTE: assume I already created my corpus file, .jsgf file, .dic, and .lm files ( via http://www.speech.cs.cmu.edu/tools/lmtool.html ) 
pocketsphinx_continuous -lm /home/pi/scarlettPi/config/speech/lm/scarlett.lm -dict /home/pi/scarlettPi/config/speech/dict/scarlett.dic -hmm /home/pi/scarlettPi/config/speech/model/hmm/en_US/hub4wsj_sc_8k -silprob  0.1 -wip 1e-4 -bestpath 0

I plan on adding more details behind why I used certain setting, configurations in a blog post i'm writing on my home automation project, but figured, i'd share what i've done thus far incase someone else was stuck like me and would like to move forward with what they're working on. Hope this helps someone. Thanks for the advice guys.