Siri proxy & Home Automation

Well crap.
That was only a matter of time…

If you been following my blog for some time, will no doubt noticed that this is not my first attempt to create some voice control for my home automation system. However at the time of my previous attempts, I was not aware of the Siri proxy, which is pretty awesome once you get working.

I first became aware of Siri proxy being used to drive home automation system while reading one of my favorite websites, life hacker. The article talked about Elvis impersonator who it created a plug-in to open and close his garage door and set the thermostat. Needless to say I found this to be pretty amazing, and far superior to anything I've attempted so far. So I went about looking into how to get this set up.

I've been wrestling with this for some time. First off, I didn't know anything about Ubuntu nor what would be the best way to install it. Particularly because of my dexterity limitations. Anyway, check out the results!

I have to say, I'm really quite pleased with the results. It was well worth the effort.

So what was required to get this up and running?

As mentioned, I had to first figure out how to work with Ubuntu and what was the best way to install it. For the time being, I'm running a bootable partition on my desktop PC. I am going to try to get a old PC/laptop running it 24/7. I also tried running it through virtual box. While the installation went smoothly, I had some difficulties getting Siri proxy up and running.

The and to create the functions for turning the light on/off or locking/unlocking the door, I had to figure out a little bit of Ruby script. Fortunately, modifying the sample plug-in is pretty straightforward and it's very easy to insert console commands to it, and fortunately I had already figured these commands out from my previous attempts…

If you would like to give it a try, here's the links to all of the resources you will need:

• Ubuntu http://www.ubuntu.com/download
  Ubuntu is free to download and install. It is also very easy to install as bootable partition on your Windows PC
  Installation instructions http://www.ubuntu.com/download/help/install-ubuntu-with-windows
• alternatively, you can run it through virtual box inside your Windows operating system (if your PC has enough power) https://www.virtualbox.org/
• this tutorial will walk you through the process http://www.psychocats.net/ubuntu/virtualbox
• Installing Siri proxy
  once you have Ubuntu up and running in either one of these two options you will need to install Siri proxy through the terminal inside Ubuntu.
  follow these instructions, pasting each one after the other into the terminal window. https://github.com/plamoni/SiriProxy#set-up-instructions
• if you need a visual walk-through first, this webpage has a now outdated demonstration of how to install it. The process is very similar, but the commands have changed. It does give you a good feel for how to get virtual box installed and the process required to get Siri proxy up and running.
  http://www.idownloadblog.com/2011/12/09/how-to-install-siri-proxy-tutorial-video/
  be sure to use the set of instructions on GITHUB, not the ones in the video!
• Getting Siri proxy to do what you want
  the easiest way to get started, and what I've done so far is to modify the example plug-in. You can find those instructions here https://github.com/plamoni/SiriProxy/wiki/Plugin%20Developer%20Guide

Even though Siri proxy is working super well for me, they do say that it is in an alpha state development now. Meaning it does have occasional issues and going through the proxy server, some of the voice commands or functions may not be quite the same/functional. For example the open app command still does work for any of the non-default applications. However it seems to not recognize programs like contacts or music. This also means that when asking to play a particular song or album e.g. "Siri, play album X" the proxy does not recognize this. Saying I do not have that album or artist. All in all, swapping one level of convenience for another. Not too big of a deal. Easily reverted by resetting your Wi-Fi IP. So what's next? Well I've ordered a few more light nodes and those of the most immediately useful to me. I would also like to add some thermostat control, however my house is kind of badly designed for that, making it cost prohibitive. (I have a number of baseboards on independent thermostats in the same room… Stupid)

Troubleshooting: If encounter a few issues about trying to get Siri proxy up and running, try the following: I found that the DNSMASQ may be interfering. You can try killing it with terminal command: killall dnsmasq If your server IP is showing 0.0.0.0 open .siriproxy/config.yml and set the IP which should be in the first or second line of the file to whatever the IP is for your machine Apparently there has been some issues with Siri proxy and Ruby DNSnot playing nice together… Edit line 56 in /home/<username>/.rvm/gems/ruby-2.0.0-p0/gems/siriproxy-0.5.3/lib/siriproxy/dns.rb to look like the following: match(/guzzoni.apple.com/, Resolv::DNS::Resource::IN::A) do |transaction| Then perform siriproxy bundle iPhones are known to "leak" data over the cellular network even when connected to Wi-Fi. Try turning off cellular data. I performed these three steps, and Siri proxy consistently runs perfectly. Thanks to Elvis Impersonator for most of these tips!

Woo hoo!

It's apparently also possible to connect an IR blaster to my home automation system. This would then give me voice control over my TV, air conditioner and other IR devices (awesome!). This is up from more investigation…

Also in the process of trying to find a machine I can use for dedicated Siri proxy server. Raspberry_pi is an option, and you can find more about those on Elvis impersonator's page. I have to look into it a bit ourselves first to see if it's the best option for me.

One of my favorite things about the Siri proxy server is, as you see in the video I can unlock my door just by using my voice. As I walk up to my house, I reconnect to my Wi-Fi and the Siri proxy server, all I have to do is tell the door to unlock. Which is really great for me as I can no longer manage keys.

So all in all I found getting this up and running was challenging, but well worth the efforts.

---

I would just like to take a second and give a shout out to Elvis Impersonator who contacted me out of the blue with a generous offer for help and advice. Be sure to check out his YouTube page and some of the other magic that can be done with Siri proxy and a Raspberry_pi!

---

Additional Note (added 29th of July 2013)

be sure to check out the related blogs comparing the desktop installation to using a raspberry pi
http://pikespals.blogspot.ca/2013/06/raspberry-pie-versus-desktop.html

Voice Banking with Model Talker

Although that I'm not showing any definitive changes to my voice at the moment (that I cannot attribute to paranoia anyway) and knowing the progression of ALS, I decided that it would be wise to create a voice bank so that when that unfortunate day does come.  I'll still be able to communicate using speech to text technology and it will sound somewhat like me.

Creating a voice bank is best done when your voice is still clear and strong. Definitely something that should be done before symptoms start affecting your speech patterns. Reason being is any speech impediment you may develop from ALS will consequently be recorded in the voice bank. Not to mention that the recording process is actually quite long. It can take quite a significant amount of time and effort to get done properly. And even not showing symptoms, it was at times quite fatiguing.

I used a software called model talker which required me to record about 1600 phrases. Most of the phrases were quite simple, with many references to the Wizard of Oz. There were however some reasonably challenging tongue twisters. This software then takes all the phonemes and uses them to create a text to speech file. This file can then be used within the Windows text-to-speech systems or (so I'm told) used with other hardware such as Dyna Vox.

say hello to my little voice bank

Model talker is presently in beta phase. So you can expect the possibility of some instabilities at times. Personally I did not counter many of these. The issues I did have had to do with setting up the recording in my own environment. For some reason any microphone I tried on my home computer was also recording a white noise or hum. After trying multiple microphones I could not isolate the cause. Best I can figure is this EMF field somewhere in my house, or noise from the power bar which is disrupting my audio recording (for example, check this video. The noise in my recording was not quite so pronounced enough to disrupt the quality of the recording). This is not something I could blame on model talker. I ended up having to record my voice and my rehabilitation facility. It was a bit inconvenient to go there so frequently, but I felt it was a necessity to create the voice bank. Recording their went quite smoothly.

The people at model talker were extremely helpful and responsive. Whenever there was a problem on the initial calibrations they would get back to me very quickly. Also once I submitted the final files, I was expecting it to take up two weeks to get back. They had my full voice bank ready for download by the next afternoon.

On a side note, if you need to make a recording of your synthesized voice as I did for this blog and you're using speech recognition for dictation… Be sure to turn off the microphone for speech recognition. I forgot to do so the first time and the speech recognition was actually dictating from my synthesized voice… LOL!

This download included two versions of playback software which could be used in addition to the Windows text-to-speech functionalities. So far, my best results seem to have been from using model talker 2. I was a little dissatisfied with the initial playbacks and found my voice didn't sound enough like me to make the effort worthwhile. In particular there seemed to be too much of a gurgle. However after tweaking some of the settings (unfortunately model talker 2 didn't include any documentation for this) I found that setting the phonetic target cost down to about 20 made for a significant improvement in the gurgling issue. I'll probably be able to improve on it further once I figure out more about the other settings.

Here, listen for yourself ;-)

• Pike synthesized voice sample
• Pike synthesized voice chaos theory

can you hear me now?

However, being beta I did have some difficulties in recording the samples. The record option fails and changes to the synthesis settings do not seem to be retained between executions. That being said, it is presently free for you to create your own voice bank.

From my Internet Travels…

I know, I'm behind on my posts.For the last two weeks or so, I have started to realize that I do in fact have need for more home automation. Especially because winter is coming. So for the last while I've been directing my efforts to researching some of my needs. I will post more about this a little later. Suffice to say I found some interesting stuff ;-)

That said, I've come across two very interesting pieces of software that I feel are worth mention.

---

Alternative Voice Recognition.

I've been using Dragon dictate software quite a bit over the last few months and I discussed my usage of it here: Dragon gamer post . for the most part I find it works extremely well. It's dictation functionality is far superior to the speech recognition that comes with Windows7.

One of the reasons that helps me decide to buy it is that there was an option to purchase a gamer pack which would give you voice access to various games. My particular interest was World of Warcraft. This gamer pack was provided by a company called Vox enable and up until recently it work really quite well. However with the latest war craft expansion pack, I seem to have lost my voice functionality from Vox enable. I've tried numerous times to a hold of some support from the website, reinstalling and updating Vox enable. But to no avail.

From what I can tell, the version of Vox enable that I purchased his version 6 and with the WoW expansion they patched it to version 7. This seems to indicate to me that I will have to repurchase Vox enable to maintain the plug-in to war craft. In other words this will cost me another €20.

Needless to say I'm not terribly happy about this, nor am I pleased with the customer support. I have had zero responses to date.

This is led me on a bit of the quest to try and find some alternatives for voice integration to war craft.

I've tried playing around with the windows speech macros on my own to get some idea as to what I could do. I've had some success with it, but only really just gotten started…

But a piece of free software I found it seems to work very well is by a group called killers software (don't let the icon fool you). It seems to work very well but does require you to create your own templates. Once are provided are kind of tailored to their own characters. That said creating your own template is pretty easy.

I've only used it a little bit so far, but it does work really quite well and seems to be more responsive than a Vox enable. It also has a whole bunch of additional functionality for other games and software such as iTunes.

All in all, for the price of a free this software seems to be very good and well worth a try.

---

Camera Mouse.

Now, this piece of software I've not actually tried and then kind of hoping to not need to try it in the near future world for that matter… Nonetheless, dealing with ALS means that I might have to consider it at some point.

This free software offers you an alternative to using your mouse through the use of the WebCam.

By moving your head around, tipping it left or right or up and down, the camera mouse translates these movements into mouse action. To send a mouseclick this is done with a supporting program (also available through this website for free) called dwell click.

All that's required to get this working is your PC, a web cam and the free camera mouse software.

Now, I do realize that this software will not work for everybody with ALS as all our symptoms progression's are somewhat different. However if your situation is similar to mine and you seem to be losing the use of your hands first but still retain control of your head and neck (no bulbar symptoms really) software like this may just help you stay active for that much longer.

And like I said, it's free. So what you have to lose?

More Autohotkey Goodness

While paying one of my many visits to life hacker, which is one of my favorite sites they posted some new stuff for autohotkey scripts. I was really impressed with both of these scripts, they're too good not to share.

The first, is really a great way to get started with autohotkey. What life hacker has done, is created a webpage which will automatically generate your first autohotkey scripts allowing you to bind all sorts of commands to various buttons on the keyboard that don't really see a lot the use on a day-to-day basis. Not only is this a great way to create your first scripts, is really a great way to get started learning what autohotkey can do for you.

Check out how to Put Your Keyboard’s Unused Keys to Use (by Turning Them into Awesome Shortcuts)!

And as an added bonus, if you are using voice to text software such as DragonDictate these commands are now available to you via voice by simply saying "press F1" for example…

---

The next tool they recently posted, is a story about how a fella out on the Internet has used autohotkey to modify the function of the number pad to interface with the Windows Aero snap feature.

For me the Windows Aero snap feature is something I used really quite a lot and with a 16:9 monitors, it's a great way to manage your screen space. For me I would always have what I'm writing on the left and my research/references on the right. This feature was really great for that.

; my first Windows hotkey override F2:: #Left F3:: #Right LAlt:: LWin  return

However once my hands started to not work so well, I found myself really unable to manage the screen the way I was accustomed. Not having easy access to this snap feature (via pressing the Windows key and the left or right cursor arrow) is in fact what drove me to find autohotkey in the first place… I was looking for a way that I could trigger the snap features via DragonDictate, and autohotkey was letting me do this. In fact, this was some of my first attempts at scripting. For fun I've posted them here:

This was a very, very basic. Nowhere near the level of function you'll see and window pad… But I was very quickly able to use F2 and F3 to stop the window to the left or right of my screen... And it was available to DragonDictate ;-)

Now window pad is considerably more complex to modify, and its default state won't really work with DragonDictate… Because Dragon dictate does not recognize the Windows key... also, I've not really had a chance to play with window pad much yet, but I sure am planning to look into it and figuring out how to change the hotkeys so don't rely on a dual key press is going to be my personal project ;-)

WindowPad Uses Your Number Pad to Snap Windows to the Screen Edge

Both of these scripts are free and well worth the time and effort to check amount… And I feel both of which will likely make your interactions with the computer far smoother. Especially if you have Dragon ;-)

Dealing with Zippers

One of the first things that really started to get in my way after my fine dexterity started to go was dealing with zippers. Grabbing the end of the zipper just became increasingly difficult and now it's pretty much impossible.

Around about this time I also started attending a rehabilitation facility here in Montréal or had access to ergo therapist. I explained this problem to them in the solution they came back to me was... keyrings.

Now the idea of having a shiny silver keyring hanging from my zipper did not appeal to me. Sure it would work and I did find it rather humorous to have something that resembled the pin on a grenade hanging from my jeans, I felt it would look odd and attract attention to me and my disability. Now of course there's all sorts of the zipper pull options out there, but to be honest I actually had a hard time finding them in my local sporting goods stores. And yes I could have also had some made at the local shoe cobblers as well.

What's interesting here is not so much the zipper tab itself, as where I got them… And the best thing is I got about 30 of them for free! Enough for me to address every pair of pants and shorts that I own… And then some…

The first thing I did was to walk into the local Mountain equipment co-op store  showed them the ones I had on my bag, explain my situation and asked if that were possible to acquire some. The customer service person was more than happy to do so and gave me what was on hand which is only three or four of them. Once again for free.

Obviously, this wasn't quite enough to address my needs. So I got on the phone and called Mountain equipment co-op's online customer service directly. I once again explain the situation and what I was looking for to the agent and I would be more than happy to purchase number of them. He paused for a second and said "don't worry about it, I will send out a bunch classified as a repair".

I've always liked shopping at Mountain equipment co-op anyway, then this level of generosity and customer service I thought was just great. Sometimes all you have to do is ask ;-)

What I particularly like about the zipper pulls is there discrete, durable and little plastic knob at the end makes it reasonably easy to grab. There are a couple different types, the ones they sent me have a little small plastic knob. But the better ones I find is you can actually get a form longer plastic knob to grab onto. Those are the best…

Also if you prefer to purchase some or are having a hard time finding some, I find you can order  zipper pulls online at CPGear for about $2C a pair. I find this to be a little bit expensive, so maybe it would be worth shopping around at various online military surplus/tactical stores.

Cost: free!
Pro: much more discreet than a key ring, easy to grab… did I mention I got them for free?
Con: there are various types, and the wrong ones might not address your dexterity needs.