The Pilot's Modular Knee Board

I'm finally back for another, much overdue blog posts! My apologies for being away so long, it took some time to reorient my technology needs after the broken arm. I had to acquire a Windows surface and a tobii eyetracker , consequently I have been occupied With setup and configuration.

Now that has been said, on to the meat and potatoes of this article :-)

A while back I picked up a pilot modular knee board on from CP gear. The idea was to find a way to things like my iPhone and ID for adapted transport visible and easily accessible. One of my criteria has always been to minimize the amount of stuff in front of particularly things like tables attached to my wheelchair. That said, I will admit right at the
beginning that it didn't work quite as I had envisioned. But on occasion it has been extremely helpful.

I do pilot my wheelchair... right?
So I obviously need wingmen... right!

Most recently, I've been using it for things like grocery lists and managing my paper work at the hospital. These days, I usually get off at the hospital and left on my own as I can still communicate and get around quite well wheeled 2.0 I don't need an assistant following me all the time. So we start my iPhone to the need knee board along with my hospital cards. This allowed people to easily access what they needed, with my supervision and not rummaging through my bag. Kept everything nice, organized and easily accessible.

By sticking my iPhone to get on a band of Velcro tape I was also able to (relatively) easily maintain text message communication via Siri while I was there.

The board is comfortable to wear (with pants. With the shorts…meh!) on either leg and stays put quite well.  It is attached by a pair of Velcro loops that you close around your leg.

Personally, I quite like and I would say it is well worth the cost of C$50. It comes in one of three colors. Black, e Pat camouflage and optionally has a bunch of accessories that can be velcrod to it. Other pouches for example.

---

And a huge thanks

I would also like to give a shout out to William Cassan for his incredibly generous donation to this blog. It is greatly appreciated and will go to financing the exploration of other useful tools :-)

Another tweak for the Griffin Beacon

I've talked about Griffin Beacon before. It's a great way to control your to television and PVR from your iPhone or android device. But as we discussed it does have a few issues. We talked about a result the battery consumption issue but the one remaining pains in the butt is that the device will go to sleep if it is been without Bluetooth communication to your device for over an hour.

So if you come back from having left the house safer groceries and doctors appointment or something. The device is off and there's no way to wake it up without touching it. And sometimes, my helper leaves before I realized the place has been turned off. Needless to say, without the function of your arms or a stable ability to walk this is a problem.

So it occurred to me that after the move I had an extra controller for my Vera home automation which was no longer being used. So I thought, why not plug my hacked Griffin Beacon into the control switch? This way, using my iPhone I simply have to cycle the power to the beginning. When the power comes back on, the BK wakes up in searches for my phone. Automatically connecting.

BOOM! Problem solved.

The downside to this is that the typical Vera visual switch runs about $40-$50. So it might not be the best solution to run out and buy one. But in my case, it was no longer being used so this worked out perfectly :-)

Sadly however, it would seem the grifin beacon has been discontinued.

The Blue Ant

No… not this guy!

A few months back I came a cross the blue ant commute hands free device, which seem to address an issue I've been having with my iPhone. Which is, hands-free answering, and while I've jailbroken my iPhone to allow hands-free control, the one big thing that's missing is the ability to verbally answer a call. Sure, I can activate Siri and place a call. But if someone is calling me, I still have to pick up my phone and physically answer it.

Honestly, I think this is probably the single biggest accessibility feature missing from Apple products. Voice triggering. I really wish they would add it, if they had I very likely would not have jail broken my iPhone.

What caught my attention was that once this devices. With your phone, it acts like any other hands-free device but allows voice activation. I could toggle Siri simply by saying "launch phone control", which is essentially the same function provided by the jailbreak with hands-free control. But if someone were to call me, the blue ant would ring, in the same way as it would a Bluetooth headset but, having integrated with my phonebook it will verbally tell me who's calling and all I have to do is send say "answer".

Needless to say, with failing dexterity this is rather interesting.

I've been using it on and off for couple months and surprisingly I'm not finding it quite as useful as I had hoped. First of all, I'm told that the audio quality for the person I'm talking to is quite good. Better than with other hands-free devices. However there are three things which I find really get in the way of regular usage.

1. Voice to text speech recognition… Not so hot.
   I use this far more frequently than I do for making or answering phone calls.. Updating my status on Facebook, sending text messages or quickly replying to an e-mail. I use voice to text quite a bit. I have found that going through the blue and commute significantly diminishes the recognition accuracy. Which is really quite unfortunate.
2. Another device to carry around…
   For me, the whole point of going to the iPhone and keeping with my was the reduction of things that they needed to have attached to my waist. Providing the maximum amount of functionality for the least amount of stuff. The blue ant commute, would require me to carry around another device. And at this point, verbally being able to answer calls does not add sufficient utility to justify.
3. Cannot verbally hang up

   

   we've all been there

   It's great to be able to answer hands-free, but if you can't have some other means to end the call. You're still required to manually terminated either by manipulating your iPhone or the blue ant itself. This might not sound like that big of a deal, but when you want to get off the phone with the telemarketer or someone who just… Won't… Shut… Up…

The blue ant commute usually retails for around $100 Canadian. I however found it on sale for about $70.

Don't get me wrong, the device does have its usefulness and is I've said, the audio quality for regular phone calls is really quite good. It is small (almost the same dimensions as the iPhone 5), easily portable and is something that can easily be attached to the side wheelchair, your belt and so forth as it's intended to be mounted on the sun visor of your car. I should however point out, I've not used it in the car yet.

So, if you do not want to jailbreak your iPhone (and with IOS 7, you presently can't anyway) this could be a pretty good way to get some hands-free control back. Sure, the Bluetooth range on the blue ant isquite good and I can probably use it throughout my house, but because of the limitations mentioned above, I still prefer to carry around my iPhone. It is presently Velcro to my waist. 

The utility of the blue ant is good, but just not sufficient to completely address my needs for day-to-day use at this point in time. So in conclusion:

Remote PC Start up Tools & Siri proxy

This is a little project that is kept me occupied for some time now, partly to do with a few difficulties in implementation. But the idea is to allow me to remotely start my computer without having to find my way to the basement to do so.

So why would someone want to remotely start the computer? Well I download quite a bit of media which is stored on the hard drive of my desktop and I use media streaming to send this content to one of my gaming consoles so I can watch it in the comfort of my family room which is on the ground floor. My desktop PC is in the basement separated by a flight of stairs with big nasty scary teeth and with mobility issues, I don't always feel like taking the chance. Especially when the lighting conditions are a little dark. For some reason this really messes with my balance, making stairs downright hazardous at times.


I tried a couple different apps, and hit a few speed bumps on the way that were not expected. Here's what I found.

First off, the speed bumps

Every motherboard on your computer nowadays will have a wired network connection. If you go through the BIOS is just about guaranteed that it will support some sort of wake on LAN functionality. Simply enable it, and you should be good to go right?

Well, not so much.

I have an older Asus motherboard that those sales for wake on LAN. I figured this should be enough. Enabled it, set the magic package and would only work immediately after shutdown. The motherboard power savings would shut down the network connection is well after a period of time. This caused quite a bit of annoyance.

So for about $10, about it PCI express network card. This solves the problems. Once the BIOS was set, the PCI express port maintained power, even after the PC shutdown.


Secondly, not all routers will support away, and set up. In particular Linksys routers (which is what I have) don't allow you to set the subnet mask 255.255.255.255 which is required to forward the magic packet the subnet mask needs to be set to 255.255.255.128. Contact Linksys cares technical support via twitter. They were very helpful with me for sorting this out.

How does it work?

Very simply, we will be using a computer or a smart phone to send what's called a magic packet over your local area network to the desired (powered down) computer. Once you BIOSes appropriately set up, your network card essentially never sleeps and monitors the network for the appropriate magic package.

There are a couple ways to send this.

iShutdown Is probably about the easiest way to go. Simply install the app on your smart phone and then from the link here, install the service on your desired desktop. Once everything is installed, with your PC running simply search for the server via the smart phone app. It should detect it right away. This will allow you to remotely start and stop your computer and has a very nice interface. It's also very easy to monitor several computers if you so choose.

This simplicity makes it well worth the $2

Your next the best option would be to try send a magic packets with depicus.com. This is also a very good app. Works flawlessly, but is a little bit more technical as it exposes you to more of the raw details. It however also has the scanning function to find computers on your network. But what is very interesting, is that also offer a free wake on LAN packet sniffer which will allow you to view magic packets that are being sent and received by your PC. Very useful when you're setting things up to make sure things are going to the right places

This is also $2 on the App Store.

Then there is the Siri proxy… My personal favorite!

If you've already figured out how to get Siri proxy running on raspberry pie, this edition is actually really quite easy, and it's free! If you are running Debian like me. Go here for information on the install

• open terminal and type
  aptitude install wakeonlan 
• test the install
• run that but that this is packet sniffer on your PC.
  By default it should be on port 7
• type this into terminal
  wakeonlan -p 7 YOUR_MAC_ADDRESS
• if everything ends working correctly you should see something like this
  http://www.depicus.com/wake-on-lan/wake-on-lan-monitor.aspx

Once you are sending and receiving magic packets from your raspberry pie to your PC via terminal, is just a simple matter of adding the recognition block into your Siri proxy script, editing your Mac address, rebundling and restarting your Siri proxy.

#wake on LAN-------------------------------------------------    listen_for /wake up my (computer|PC|desktop)/i do           say "waking up your computer"     system 'wakeonlan -p 7 YOUR_MAC_ADDRESS'     request_completed #always complete your request! Otherwise the phone will "spin" at the user!   end

And this should be enough to get your serial proxy sending magic packet.

But having set this up, if I decide to watch a movie I simply voice activate my iPhone, tell her to wake up my computer and within a minute or so the media servers visible on the network.

Very convenient, very fun. :-)

If you're looking to set up a wake on LAN, I hope this helps you set one up.

Old IPhone, New Life As Dedicated Dragon Mic

Back in July, I did a post about various microphones for use with Dragon naturally speaking. Some working better than others. The one microphone, that has always impressed me for its ability to comprehend what I'm saying but has been terribly convenient was the dragon microphone app for iPhone. The reason I found it not terribly convenient was because my screen lock would time out and locked my iPhone. Shutting down the microphone, often in midsentence.

Yes, I could easily go into the settings and change the timeout time. But the pain in the butt is that I

would have to do this every single time. Not to mention, I would have to remember to set it when I'm done. Not terribly convenient.

Then, for some reason the other day it dawned on me that I had by old iPhone 3GS sitting around doing nothing. So I decided to install the Dragon microphone app, and plug it into my PC to give constant power (not that it really needs this as the battery is still good) then I changed the lock screen timeout to "never".

This is working out really quite well. As a matter of fact, I'm using it at the moment and the recognition is near 100% simply sitting on my desk nowhere near my mouth and I'm speaking at a regular volume.

The only downside I have encountered so far is that I can no longer use the on-screen microphone button nor the keyboard on my PC to toggle the microphone on or off. I have to touch the iPhone. It does however respond very well to the "go sleep" and "wake up" commands. Unfortunately however, the Dragon Does not allow you to use your iPhone as a regular mic to your PC. Only interfaces with Dragon NaturallySpeaking. Which is fine for me!

All in all, this is a great way to recover some use of the old technology. It was destined to be a toy for my three-year-old (cracked screening and all) or be recycled as it now has no resale value. This way I can use it just about every day when I'm in front of the computer, greatly extending its life and utility for me.

Don't have an old iPhone? All that you can find one which is off plan for cheap on craigslist, kijiji or the like ;-)

Microphones & Voice Recognition

It's been mentioned about for some time I rely quite heavily on voice recognition functionality to access my computer via Dragon naturally speaking (sometimes the Windows of voice recognition). However I have not discussed the pros and cons of the microphone/headset that I've tried. Some are obviously better than others and this of course changes as my condition deteriorates.

Something I should point out however is the exact nature of my limitations. I've lost almost all the dexterity in my hands and have very little use of my arms. I can scratch my nose, but I can no longer lift my hands/shoulders to place anything on my head.

There are some pros and cons to some of the headsets I've tried:

Wired headsets

Initially, I used wired headsets almost exclusively. However a few months ago there were two major factors that really started to be a problem for me. The first, as my dexterity degenerated it started become more and more difficult to get the headset on in a convenient and comfortable manner. The second, again to do with dexterity is at once that the headset is on, I have to contend with the wire which is often getting wrapped around my leg, falling down (then usually getting squashed under the wheel of my computer chair), getting hung up on the armrest, or otherwise being a pain in the butt.…

Since then, I've moved over to using almost exclusively a wireless headset. Nonetheless, there were some quality headsets and some work much better than others which Dragon NaturallySpeaking.

Rocketfish PS3 Headset 

This is one of the first headsets on tried. And the cost of about $20 Canadian it worked exceptionally well. It has an extremely long USB wire which is intended for the PlayStation 3. E.g. from the couch some distance away. Not really a PC, but functionally it's plug and play. What I find worked exceptionally well but this was the boom for the microphone. It was long enough that it could be placed almost directly in front of your mouth and the foamy part on it was good enough to muscle out the sounds of your breathing. Therefore Dragon's comprehension of your speech was generally very good.

The noise cancellation is so good in fact, that I could actually have music playing softly to my PC speakers (or a game) and the voice recognition would still work very well. This headset was particularly designed for voice communications in gameplay environment, so if you're a gamer. You can pipe the communication audio through your headset and still listen to the game/music through your main speakers.

If my hands didn't make it impossible for me to put this on, this would still be my preferred headset. It's light, secure on your head and reasonably comfortable for longer period of time. However without dexterity, is extremely difficult place and adjust.

Corsair HS1 Gaming Headset

These are an exceptionally good quality gaming headset. It is also USB connection with a rather long wire. There initially given to me because I was having substantial difficulty recording my voice into modeltalker. For some reason, no matter what I tried there seem to be some form of electric hum recorded with my voice. This didn't seem to matter which headset I was using.

That said, these are a little heavy but very comfortable once they're on. Because they have very large over your head ear piece I was able to place them on my desk in such a way I could sort of wedge my head in between the earpieces. Push them into position and him lift my head off of the desk with my headphones roughly in place.

Unfortunately, the microphone is a little far away from my mouth for to work well with Dragon NaturallySpeaking. I find I have to raise my voice quite a bit in order for it comprehend and even then. It doesn't work all that well for voice recognition. The other reason I think why, is even though there is a noise canceling function on the microphone it does not have a piece of foam over the microphone. Consequently the records more of your breathing and other undesired noises.

If you are using it just for gaming however, the audio quality through the headset is exceptionally good and your teammates should hear you talking to problem. The driver software also comes with a bunch of other tweaks that you can play with to modulate your voice and have all sorts of fun effects. My three-year-old particularly enjoys it rerecord his voice and make him sound like a cartoon character or a Dragon. ;-)

Again, I had to discontinue usage of these because of my dexterity limitations and fighting with the wire. There are however the heaviest of all the headphones I used and because of this, the but the progressive atrophy I doubt that I would be able to use them now for any length of time

This microphone retails for about C$100.

Wireless headsets

Plantronics Backbeat 903+

To be fair, I did not initially purchased these for use with my PC. I found that when I was out for my IVIG treatments which would last several hours, I would like to music listen to. Dealing with wired headsets with tiny little earbuds was nearly impossible. So I went looking for a set of Bluetooth headphones and found these. I found a retail for around $80-$100 in store. However with a little bit of online shopping, I found a pair on sale from Amazon for about $50 Canadian. And I have to say, I'm very pleased with the purchase and use them very regularly.

Although doesn't look it, once they are on they are extremely comfortable. The way almost nothing and is very easy to forget where. The battery life is extremely good and last seven or more hours of continual usage. As a matter of fact I would routinely leave the house for my IVIG appointment at 7 o'clock in the morning, start the music. Hardly ever stop it and be back home somewhere between one and 2 PM with plenty of battery life to spare.

The audio quality is very good, they do have mild noise cancellation which helps promote the clarity of which are listening to. That however are not designed to block out the sound of the outside world. You can still hear what's going on quite well, but the music or whatever you listen to is extremely clear. This also applies to receiving phone calls.

The built-in microphone is okay. I've been told I sound a little bit like I'm talking into a beer can when using. But other than that the audio quality is acceptable for the listener. I have tried the microphone with Dragon NaturallySpeaking on my PC. Well good enough for conversation, Dragon has substantial difficulty in understanding what I'm saying. Microphone is too far from my mouth and there seems to be too much noise detected.

So if you're in the market should be able to wear a pair of headphones almost all day, making phone calls with it and listen to music. These are very good purchase. Just don't expect to use Dragon NaturallySpeaking…

Oh, sure. My disability makes it impossible for me to put on by myself. However once they are on, I can forget that they are there and use them almost normally as anyone else would. And although it has happened, I rarely have to ask for help to put them back.

One neat little feature I have set up at home, is my raspberry pi Siri proxy home automation system. What's great, is that wearing these headphones as I approach my house once my iPhone (jail broken) detects my wireless network, it turns on hands-free mode (opens the microphone on the headset) and listen for a keyword to activate. All I have to do is say the keyword, verbally command my door to open and it unlocked. Wearing the headphones improves the voice recognition of my Siri proxy as I don't have to go fishing for my iPhone.

All in all, for me this is worth the price of admission.

Logitech BH870 - Bluetooth

This headset is now hard most commonly use voice recognition with my PC. I found it on TigerDirect at a reduced price for about C$30. At the time down from about C$110. Quite the deal, so I thought it was worth a try.

They have since reduced the price to about C$35.

All in all, from a comprehension standpoint it works quite well. Almost as well as the PlayStation three I heard headset. I do however find using this headset, Dragon comprehension seems to be a little bit slower than it otherwise would be. But I find if I slow down my rate of speech, it manages reasonably well.

The headset does have a noise canceling function, but it does seem to be consistently transmitting some form of noise to my PC. The Dragon icon frequently shows that it's processing. Even when I'm not detecting any noise myself in the environment. I most certainly cannot use this headset while gaming or playing music.

Even with my dexterity limitations, I'm still able to get this ear piece on by myself. It is a little bit tricky, but I am able to do so. One of my pet peeves about your pieces of this nature is they tend to wiggle the mirror hanging from your ear. This drives me nuts. This particular headset is reasonably secure, but still have that annoying wiggle when I move. Because of this, it also tends to pick up a scratching noise from my beard. That said, because of my dexterity issues still I'm not very comfortable wearing this headset out of the house. I prefer the backbeat headphones. They feel much more secure to me.

One nice feature I like about them, is that I compare the headset to both the USB dongle it comes with for my PC and my iPhone at the same time. So for making phone calls, this is great…I recently found an unlimited usage phone plan for my iPhone. This is great, as my phone is much easier for me to manipulate a regular phone.

And because this your pieces intended to be used with the PC, it will naturally integrate itself with a soft phone such as Skype. I've used it for Skype a couple times now. The call quality has been very good.

It comes with a bunch of headset options Such as different earpieces and in over the headband. This is great if you could be using a prolonged periods of time or if you have a larger/smaller earlobe. He viewed a few options. The one silly thing I find about it is discharging unit. It is very tight to slide the earpiece in and out of the charger. I can still do it, but it's getting increasingly difficult.

The battery will last several hours.

So far this is among the best options I have found using voice recognition wirelessly on my PC.

Dragon microphone

This is a free app available from the iTunes App Store and I know that android offers a similar app.

This app is a very pleasant surprise. It works extremely well and comprehending your speech indicating the commands to your computer. I've tried it even just sitting it on my desktop and talking added. Not even having it that close to my mouth. Again the comprehension was exceptional. I would say, by far the best of anything I've tried. That said however, it is unlikely it would work nearly as well if there is music playing or if your gaming. I've not tried either of these situations as I don't expect it to work.

Dragon microphone connects to your PC through your Wi-Fi interface. Not Bluetooth. You don't need any additional dongle or Bluetooth functionality on your PC. Just your Wi-Fi enabled router. Set up was extremely simple. Simply create a profile… Start the app… And that's about it.Upgrading from my 3GS to my iPhone 5 was even easier. Simply had to install the app on my iPhone. No profile update or anything.

The only annoying feature I have found is that if you have the power save mode on your iPhone to turn the screen off. Then it also turns off the voice recording of the app. Making you have to press the home button and unlock the screen once again. This can be rather annoying… Especially with dexterity limitations.

That said, it's something that you can turn off when the iPhone's general settings. But once again, annoying.

Especially for the price, this is really good way to go and it could well fit better with some of your dexterity limitations that does mine. So if you are running Dragon dictation and are looking for a wireless microphone, iPhone or android this is definitely worth trying before investing any money.

Conclusions?

My preferences really depend on what I'm doing. Anything to do with my PC I use the Logitech hands-free headset. It works very well and is reasonably priced. But as I mentioned, I'm not very comfortable wearing it out of house.

For general usage with my iPhone, especially when I'm out the back beat 903+ headphones are by far my preferred choice. Excellent sound quality, extremely comfortable and works quite well with Siri, but not so hot with Dragon NaturallySpeaking…

Raspberry pie versus Desktop

I know, I'm a bit overdue for a post. One of the reasons being is that I decided to try running a raspberry pie for my Siri proxy and have been occupied playing with that. Initially, I was just going to use the desktop that was arranged for me my friend. A Siri proxy is a great use for an old machine, and the one that was given to me was a beast in its day, grossly overpowered for my needs.

As you saw in the previous post about setting up a home automation system, the setup was reasonably easy. It did however require that at first the Ubuntu machine has a keyboard mouse and monitor hooked up to it. Once this is all done, I could fairly easily use VNC to connect to it. From there I can use all my assistive technology to do the installation I need. But the initial setup was a bit cumbersome for this reason only. Without hands that works very well, the required manipulation of the keyboard and mouse was challenging. The other thing is that the old desktop, being is overpowered as it was generated a fair bit of heat and consequently had a number of fans and to keep cool. The machine was a bit noisy.

So, for a friends birthday we thought a raspberry pie would be a perfect gift for him and on a bit of an impulse (also some encouragement from my girlfriend) I decided to order one for myself as well knowing it could be used as a Siri proxy. After all, this is how siri proxy first came to my attention with Elvis impersonators raspberry pie garage door opener.

I did need to help to set up initial wiring and plugging the SD card. But once that was done, set up was actually far easier than with the desktop, and not just because I'd gone through the process previously. Out-of-the-box, once the pie is plugged in it boots up by itself (no power button) it is set up to network into it immediately. No need for a keyboard, mouse or even monitor. Simply use putty.exe to get started. The raspberry pie is dead quiet. There are no fans, nothing to make noise. As a matter of fact it requires only 5 V to run this the 700 MHz, credit card sized computer. Pretty impressive, especially for just US$40.

Want to give it a try? Here's what you'll need…

I am presently running my pie headless (without monitor or keyboard) and am going to assume you're wanting to use the same here.We ordered everything from www.adafruit.com Here's the minimum parts that you will need to get started:

• A raspberry pi model B. US$40
• 5 V power adapter a micro USB cable. US$10
• SD card for Raspberry Pi preinstalled with Raspbian Wheezy US$15
• network cable.

And that's it! You can get a raspberry pi, Siri proxy server up and running for about US$65. I would however recommend also purchasing a case for it, it's always a good idea to protect and exposed circuit board. You can buy one for as little as US$10

I based this list on the starter pack listed on their website. I figured it was a good safe place to get a list of everything I would need. The starter pack includes a whole bunch of stuff for wiring and doing custom electronics, nothing I would need. Just for fun I also added a very small 4.3 inch TV monitor for an additional US$50, which I have yet to hook up. But my friend got one as well and he loves little thing.

Setting up your pi…

I chose to purchase the preinstalled SD card for pure simplicity reasons, it was only a little bit more expensive and I figured doing so would save many headaches of trying to manipulate the card between my desktop and the pie. My dexterity can't handle it anymore. This way I was sure of the install that would work right from the get go.

1. Plug everything in (Duh). Make sure the SD card is properly seated, you have power and network. When the pi is booted (should take only a few seconds) you should be rewarded with several LED lights. Red yellow and green. If you are greeted with a solid read only, your SD card is probably not well placed. This happened to us. Unplugged the pie, reseat the card and try again.
2. Find your pie on your network. In order to be able to connect to it remotely, you need the pi's IP address. You can find this fairly easily through most newer routers or you can using IP scanner.
   These are excellent instructions for your first time login using SSH
   Raspberry Pi SSH Login without Monitor
3. Through SSH login, enter this command:
   sudo raspi-config 
   then follow this first time set up guide
   As mentioned, out of the box you should be able to network into your pi using SSH. However there are a couple things you will want to make sure you do:
1. use your whole SD card
2. check for updates
3. check the boot behavior is set to desktop
4. optionally change the default password for user pi (I believe the default password was…: pi) keep in mind, Linux-based systems such as the raspberry pie are case sensitive particularly important for remembering your passwords ;-)
4. Now to install and set up VNC
1. these are the instructions I used, but a very good step-by-step guide here: http://learn.adafruit.com/adafruit-raspberry-pi-lesson-7-remote-control-with-vnc/overview
2. Personally I followed these instructions:
   http://myraspberrypiexperience.blogspot.ca/p/setting-up-vnc.html 
   once VNC is installed, you will no doubt wanted to start on boot. These instructions will walk you through the process
   http://myraspberrypiexperience.blogspot.ca/2012/08/start-vnc-automatically.html



be sure to specify which desktop
you want to connect to

5. reboot your pi (type reboot ) and reconnect using VNC
   start VNC viewer on your desktop, enter the IP address of your raspberry pie followed by :1
   the reason you want to do this is that you are specifying which desktop to connect to. You can run multiple desktops.

And there you have it. Your pie should now be up and running with a VNC connection.

One thing I should point out about VNC is while it's a great and convenient way to access your pi it does have a few limitations. You can use the accessibility tools from your Windows environment, for example I can use DragonDictate to send text and controls directly to the pi. However I cannot copy paste from my desktop through VNC onto pi, which can be a bit inconvenient (possibly because I'm using the free version).

Another reason why VNC is Uber cool
I can connect to my pi from my iPad or iPhone!

Installing Siri proxy on your pi

The best, and probably easiest way would be to follow Elvis impersonator's instructions. I installed mine through SSH, one command at a time. This step is a little bit more complicated as you will have to take out the commands from the text file one by one. Nonetheless, it's pretty straightforward and you will find the complete list of instructions here:

• https://gist.github.com/elvisimprsntr/4409751

Optionally you can set up your raspberry pi to start Siri proxy on boot. Once again, just follow Elvis impersonator's instructions:

• http://www.youtube.com/watch?v=0z6uKw8Zaas

Personally I've not set my Siri proxy of to start on boot. I kind of like having the terminal window showing me the output so I can occasionally check up with what's going on. If you if you choose to start your Siri proxies this way,  you will have to do it at the desktop level. Connected using VNC and run root terminal. In my case, for some reason it does not remember the path to the install, I simply have to rerun the following three commands in the root terminal to start the server.

• [[ -s "/etc/profile.d/rvm.sh" ]] && . "/etc/profile.d/rvm.sh"
• echo '[[ -s "/etc/profile.d/rvm.sh" ]] && . "/etc/profile.d/rvm.sh" # Load RVM function' >> ~/.bash_profile
• siriproxy server

Modifying the example plug-in

about to go through the details of how to go about modifying or creating your own plug-ins. I believe that is a project for you to figure ;-). However, I did create a little auto hotkey script which will make the process somewhat easier.

My goal was to allow myself to use my Windows environment to use the actual modifications to the example plug-in script. Doing so would give me full access to my voice control,and other accessibility features. Not to mention a nice editor. You can download my autohotkey script here:

• copy to pi

This script is just a convenient way to send a final from my Windows desktop to a raspberry. In order for to work you will need to have pscp.exe installed which of to download from the same place you  downloaded PUTTY 

http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html.

1. In the first box, this is where you want to send a file. Specify as root user. type in the full path.
2. The second box is the file that you want to copy to the raspberry pie. Press the select button, go find your file.
3. in the final box use the select button to specify where you downloaded pscp.exe
4. modify your source file, save it then press send it. This will copy the file onto your pie.

I created this only has one way send to pi because I don't expect have to copy files back from my pie , very often, if at all.

Conclusion

This should be enough of the pieces to get to well on your way. Goal here wasn't easy tutorial but more to show you where to find hold the parts. I'm finding it surprisingly easy to work with the pie in this manner. So if your voice is still strong in your house I need/use for Siri proxy, a raspberry pie is a relatively inexpensive, not overly complicated and very effective way to get it set up.

Siri proxy & Home Automation

Well crap.
That was only a matter of time…

If you been following my blog for some time, will no doubt noticed that this is not my first attempt to create some voice control for my home automation system. However at the time of my previous attempts, I was not aware of the Siri proxy, which is pretty awesome once you get working.

I first became aware of Siri proxy being used to drive home automation system while reading one of my favorite websites, life hacker. The article talked about Elvis impersonator who it created a plug-in to open and close his garage door and set the thermostat. Needless to say I found this to be pretty amazing, and far superior to anything I've attempted so far. So I went about looking into how to get this set up.

I've been wrestling with this for some time. First off, I didn't know anything about Ubuntu nor what would be the best way to install it. Particularly because of my dexterity limitations. Anyway, check out the results!

I have to say, I'm really quite pleased with the results. It was well worth the effort.

So what was required to get this up and running?

As mentioned, I had to first figure out how to work with Ubuntu and what was the best way to install it. For the time being, I'm running a bootable partition on my desktop PC. I am going to try to get a old PC/laptop running it 24/7. I also tried running it through virtual box. While the installation went smoothly, I had some difficulties getting Siri proxy up and running.

The and to create the functions for turning the light on/off or locking/unlocking the door, I had to figure out a little bit of Ruby script. Fortunately, modifying the sample plug-in is pretty straightforward and it's very easy to insert console commands to it, and fortunately I had already figured these commands out from my previous attempts…

If you would like to give it a try, here's the links to all of the resources you will need:

• Ubuntu http://www.ubuntu.com/download
  Ubuntu is free to download and install. It is also very easy to install as bootable partition on your Windows PC
  Installation instructions http://www.ubuntu.com/download/help/install-ubuntu-with-windows
• alternatively, you can run it through virtual box inside your Windows operating system (if your PC has enough power) https://www.virtualbox.org/
• this tutorial will walk you through the process http://www.psychocats.net/ubuntu/virtualbox
• Installing Siri proxy
  once you have Ubuntu up and running in either one of these two options you will need to install Siri proxy through the terminal inside Ubuntu.
  follow these instructions, pasting each one after the other into the terminal window. https://github.com/plamoni/SiriProxy#set-up-instructions
• if you need a visual walk-through first, this webpage has a now outdated demonstration of how to install it. The process is very similar, but the commands have changed. It does give you a good feel for how to get virtual box installed and the process required to get Siri proxy up and running.
  http://www.idownloadblog.com/2011/12/09/how-to-install-siri-proxy-tutorial-video/
  be sure to use the set of instructions on GITHUB, not the ones in the video!
• Getting Siri proxy to do what you want
  the easiest way to get started, and what I've done so far is to modify the example plug-in. You can find those instructions here https://github.com/plamoni/SiriProxy/wiki/Plugin%20Developer%20Guide

Even though Siri proxy is working super well for me, they do say that it is in an alpha state development now. Meaning it does have occasional issues and going through the proxy server, some of the voice commands or functions may not be quite the same/functional. For example the open app command still does work for any of the non-default applications. However it seems to not recognize programs like contacts or music. This also means that when asking to play a particular song or album e.g. "Siri, play album X" the proxy does not recognize this. Saying I do not have that album or artist. All in all, swapping one level of convenience for another. Not too big of a deal. Easily reverted by resetting your Wi-Fi IP. So what's next? Well I've ordered a few more light nodes and those of the most immediately useful to me. I would also like to add some thermostat control, however my house is kind of badly designed for that, making it cost prohibitive. (I have a number of baseboards on independent thermostats in the same room… Stupid)

Troubleshooting: If encounter a few issues about trying to get Siri proxy up and running, try the following: I found that the DNSMASQ may be interfering. You can try killing it with terminal command: killall dnsmasq If your server IP is showing 0.0.0.0 open .siriproxy/config.yml and set the IP which should be in the first or second line of the file to whatever the IP is for your machine Apparently there has been some issues with Siri proxy and Ruby DNSnot playing nice together… Edit line 56 in /home/<username>/.rvm/gems/ruby-2.0.0-p0/gems/siriproxy-0.5.3/lib/siriproxy/dns.rb to look like the following: match(/guzzoni.apple.com/, Resolv::DNS::Resource::IN::A) do |transaction| Then perform siriproxy bundle iPhones are known to "leak" data over the cellular network even when connected to Wi-Fi. Try turning off cellular data. I performed these three steps, and Siri proxy consistently runs perfectly. Thanks to Elvis Impersonator for most of these tips!

Woo hoo!

It's apparently also possible to connect an IR blaster to my home automation system. This would then give me voice control over my TV, air conditioner and other IR devices (awesome!). This is up from more investigation…

Also in the process of trying to find a machine I can use for dedicated Siri proxy server. Raspberry_pi is an option, and you can find more about those on Elvis impersonator's page. I have to look into it a bit ourselves first to see if it's the best option for me.

One of my favorite things about the Siri proxy server is, as you see in the video I can unlock my door just by using my voice. As I walk up to my house, I reconnect to my Wi-Fi and the Siri proxy server, all I have to do is tell the door to unlock. Which is really great for me as I can no longer manage keys.

So all in all I found getting this up and running was challenging, but well worth the efforts.

---

I would just like to take a second and give a shout out to Elvis Impersonator who contacted me out of the blue with a generous offer for help and advice. Be sure to check out his YouTube page and some of the other magic that can be done with Siri proxy and a Raspberry_pi!

---

Additional Note (added 29th of July 2013)

be sure to check out the related blogs comparing the desktop installation to using a raspberry pi
http://pikespals.blogspot.ca/2013/06/raspberry-pie-versus-desktop.html

IPhone Jailbreak Fun

follow my attempts on twitter
#setmyiPhonefree

If you've been following my Twitter feed, you are no doubt aware that I've been playing with jail breaking my iPhone to see if I can find some additional functionality to make my life a little bit easier. I have found a few very good apps… And a lot of other things to sift through.

Here I will give a quick tour of some of the apps I've tried out. Specifically, I'm looking to find ways to improve my access to the phone and any improvements that can be made to Siri was an eye to trying to make it work with my home automation system. But first, for those of you who may not know… What does it mean to jailbreak your iPhone/iPad?

iOS jailbreaking is the process of removing the limitations on Apple devices running the iOS operating system through the use of software and hardware exploits – such devices include the iPhone, iPod touch, iPad, and second generation Apple TV. Jailbreaking allowing the download of additional applications, extensions, and themes that are unavailable through the official Apple App Store. (Source: Wikipedia)

If you would like to give it a try, the process of jail breaking is actually quite simple. Here's the information you'll need:

• life hacker: the always up to date jailbreak guide (these are the steps I followed)
• life hacker: why jail breaking is awesome!
• life hacker: before you do, things to consider… know the risks

Great! Now that that's out of the way, let's talk about some of the cool stuff we found.

Now if this is the beginning, there's an awful lot of stuff to sift through on the Cydia store. Most of it I don't find particularly useful, and a lot of it is out of date (particularly for the new iPhone 5). There is tons of fun stuff like applying seems to your iPhone. Not useful, but entertaining. For example, using the Winterboard app I now have an animated space invaders lock screen. Here's a list of apps I've tried and how useful I found them in coping with the disabilities provided by ALS.

Octopus Keyboard This is a keyboard replacement for the standard iPhone keyboard. Essentially it's a straight up ripoff of the keyboard you'll find on blackberries Z-10. Essentially it is an auto complete function built into it, popping up the words over particular keys. To complete the word, simply swipe upwards towards you typing area. I like this keyboard very much. A substantial improvement over Apple's auto complete function and perhaps the primary reason why I chose to jailbreak my iPhone. That said, with my dexterity limitations the swiping action can be a little bit tricky for me at times. Also I found it seems to work better on the iPhone if I turn off apples AutoCorrect. Also, I'm not sure that I can attribute this to the octopus keyboard. But I have noticed when I'm creating a little bit of a larger e-mail on my iPhone (usually through a combination of typing and Siri dictation) there is does seem to be a slowdown in responsiveness. I have no problems running it with IOS6. There are also a whole bunch of dictionaries that you can add to the keyboard for free. Cost: $5 USD octopus keyboard
Hands Free Control Does exactly what the title implies. Give you hands-free control to activate Siri. When enabled, you simply have to say the keyword (default is Siri) and if hands-free control is enabled Siri wakes up and waits for your commands. You can change the activation keyword features a comes preinstalled with five options and a number of customized sounds which are played when it activates or receives a command. You can also add your own custom keyword. So far I found this works very well and quieter environments. If there is a bunch of background noise, Siri is of course going to have issues recognizing your command (this is also a problem if you're using hands free to play music from your iPhone). It operates in low power mode to minimize strain on your battery. Nonetheless it is no additional cost and you can expect to see a shortened battery life. He does however have an option to only enable when it's on AC power. This is how I use mine, when pluged-in beside the bed at night hands-free is activated. This is quite useful, but not as much as I'd hoped. I'm looking to be able to tie it into my home automation system. Unfortunately Siri cannot open bookmarks or web links so I cannot fire the command to my vera home automation system at this point in time. I am however looking at running a Siri proxy in my home. If I can figure out how to get this rolling, that will make hands-free jailbreak several levels of pure awesome. I would literally be able to say "Siri, turn on the basement light"… Cost: $3 USD hands-free control
Assistant Extensions Unfortunately this is not yet compatible with IOS6. I have installed it and it doesn't run and a few things work. For example it will search YouTube just fine through Siri. However using assistant extensions to activate any of the system settings (such as toggle Bluetooth) it crashes the iPhone and reboots in safe mode. It promises additional chat bots, translator, YouTube integration and a few other features. It is also the building block for a number of other Siri plug-ins. Unfortunately none of those will work either until this is patched. But like I said other than YouTube nothing seems to work at this point in time. Here's hoping that when is compatible with IOS6 they will have a whole bunch more features to it. It looks promising… Cost: free… But unstable. Don't bother until it's upgraded. assistant extensions
Assistant Love This is another Siri extension which looked extremely promising and unfortunately didn't turn out to be quite as useful as I had hoped. Assistant love adds music integration to spotify and grooveshark allowing you to play from any of these sources as though you would a regular music. Unfortunately, living in Canada these services are not available :-( However, it also allows you to override the default navigation service. You could replace the Apple maps searched and directions with a number of other services providing you have them installed on your iPhone. For example I've replaced by default navigation voice search with that of Google maps. This is actually pretty cool. can't use an alias for a bookmark But the real promise behind assistant love, and why I bought it was that it also offered an "alias" functionality to start an app. In other words you could specify your own custom keyword to open a specific App. For example you could say "open my drive" as a command to open your Google drive or dropbox account. I thought this was great criminal exactly what I needed, unfortunately it will not accept home screen bookmarks nor a URL. Only an app. Had it done so, this would've been exactly what I need to interface with my vera home automation system. Other than that it does provide a few additional commands to toggle some settings. For example you can use assistant love to toggle Bluetooth/flashlight/Wi-Fi and your audio mode. I have found that being able to toggle the flashlight using hands-free control kind useful in the middle of the night. No messing with the light the bedside table ;-) All in all, there are some useful functionality within assistant love even without groove shark and spotify and worth the price. Cost: $2 USD assistant love

So for me, the next step is to look at setting up a Siri proxy. This is far more complicated and if I can get it working it will answer my needs for tying it into my home automation system. I would very much have preferred to use the aliases in assistant love. But will give this a try instead. If I can get this working… Well, check out what can be done with it!

So all in all, there is some very good apps available if you jailbreak your iPhone to help make it more accessible to you. Communicate by phone Joe broken for the time being and continue to look for new apps/functionality. The is also a lot of fun apps to be had, for example check out barrel. A real fun way to turns pages on your home screen.

These also check out some other reviews and writeups on these links. If you're considering jailbreak in, this is a great place to start helping you decide if it's worthwhile or not.

• best apps on Cydia gizmag review
• best jailbreak apps and tweaks life hacker
• make your iPhone read your mind by life hacker.
• jailbreak tweaks for IOS6 Idownloadblog.com

A First Prototype…

One of the unfortunate side effects of dealing with ALS is having to deal with the progressive loss of balance. Usually resulting in the unexpected face plant. In my particular case, my balance is still pretty good but I am prone to falling from time to time and as most of my losses in my hands and arms. I am unable to properly protect myself in a fall. And once down, getting up is starting to become rather difficult. I have had a few occasions were fallen and required help to get back on my feet.

Nevermind the subject of risking more serious injury…

So this got me thinking. What happens when I'm by myself and I fall somewhere in the house and I'm unable to get up? How do I call for help?

Well, there are for sure services like the life alert bracelets and the "help by fallen and I can't get up" panic button.

This got me thinking, sure I could sign up for one of the services. But I already have a smart phone which I use quite regularly for my home automation systems and my television remote control. Not to mention it has a Siri voice recognition, which is one of the reasons why I bought it. So do I really need the life link service? Other than the built-in fall detection, my iPhone will cover my needs. Thus the question becomes, how do I keep up with my all a time? With nonfunctioning hands, I need to keep it attached to my person while remaining accessible. Not to mention, anything attached to my belt, one involving a clip to dock the iPhone are pretty much out of the question. I don't have the range of motion to access my belt, nor the dexterity to unsnap the phone from a clip.

Now, I bought my grab it pack because it had the thigh pocket for the cell phone for just these reasons when I'm out of the house. But it's not quite as practical to wear all day while I'm home. Consequently I started thinking about something similar to a fighter pilots G suit. Below is an image on the right that this pilot is flight information strapped to her thighs. I realized that this is actually a pretty safe place to carry my iPhone. Should I fall, it'll remain accessible and it is very unlikely that on the top of my thigh my iPhone will receive a direct impact. And I know from experience with my grab it pack  that Siri will work just fine at this distance (assuming there's not too much ambient noise, but that's the same anywhere)

How could I do this with my smart phone? What could I adapt?

It turns out that Belkin has an armband product to hold iPhones for when you go to the gym. It is quite secure and will allow you to use the touchscreen through a plastic cover. It is however sized for an arm and not a thigh… Just a bit of a size difference, even with atrophy. Nonetheless, this is pretty close to what I'm looking for and reasonably easy to adapt. It was also reasonably inexpensive, around C$25. It will definitely work as a first prototype.


So with the help of my re-adaptation facility, we went about extending the length of the armband strap so it would fit my thigh. We replace the Velcro with a one-inch snapdragon buckles make it easier for me to remove, and less fidgety than messing with Velcro, i.e. Once I set the size, I don't have to adjust it every time. We also added a vertical loop so I can attach it to my belt or might grab it back if I so choose. The vertical strap is essential to prevent the armband from sliding down my leg. Don't want it to be to type all the time…

I know, looks a bit dorky

This is what we came up with...


I realize that this looks a bit dorky. But the fact of the matter is these days I have to focus on function over fashion and if it gives me a degree of ease of access and safety within my own home… So be it. Dorky.

Now, this works reasonably well. But not perfectly. First of all, in order for the iPhone to fit within the Belkin case it has to be removed from any other case. E.g. the iPhone has to be "naked" to fit properly. Also is an extremely tight fit making it nearly impossible for me to insert or remove the iPhone to the Belkin case myself.

The plastic cover over the screen does work and allows me to use the screens touch functions. However it is not completely flush to the screen, removing some of its sensitivity. It is also a little bit tricky to properly access the home button. But not impossible.

Finally, the strap from the hold and position from my belt is essential. That said, options of where to attach it to the Belkin case were very limited and the only place it could be attached without compromising the accessibility of the iPhone in the case causes a slight rotation of the case when it's attached to my leg making it just a bit uncomfortable. What do you expect, first prototype ;-)

I really should mention that I am actively considering getting one of those life alert devices and it does have some significant benefits that should not be overlooked. One of the reasons why I'm looking into this is that my needs are not quite there yet and I really do want to minimize the amount of devices and things I have to drag around with me. I'm looking at the phone as a viable option because I carry it all over the place. I use it to control my TV and my home automation system. And of course, Siri for communications.

Home Automation, the 3rd Attempt

Introducing the "Jarvis Mod"

Wow. The Christmas holiday season can really get in the way of any plans you have for personal projects. I actually had the autohotkey script done at in early December, but all the obligations for this season and having the family around really slowed me down. Not that I am about that quickly anyway ;-)

Now I'm back to it. Introducing, the Jarvis mod for your Vera home automation system. But first a little back story.

We've talked before about how I'm starting to rely more and more on voice control for my computer. But that doesn't really help me out for the rest of my environment, so I started wondering if there was a way I could use voice to have some control over my environment. Ideally with minimal cost and using tools that are ready have.

The particular problem I was hoping to address was to be able to run my home automation system through my iPhone using Siri. I mean really, who would not want to be able to walk into their house speak a few instructions and have your environment respond to you in a similar way to what you see on Star Trek the next generation, or Jarvis (hence the name) from the movie Ironman.

Well, it turns out that there is!

Obviously, it's not going to be anywhere near as powerful as what you see in Star Trek or Ironman. However I can use Siri's existing functionality to send a message to my Vera Zwave home automation system. This is by definition a hack as it does use some digital duct tape and bailing wire to get this to work. But for my needs, it works very well. Check it out…

So how did I get this to work? As you can see from the video, I use the Siri to send a message to my computer via (a secret) e-mail. Microsoft outlook receives the mail, recognizes it and fires a rule to create a text file in a specific directory containing the contents of the e-mail. There is an auto hotkey script running on my PC which waits for this text file, parses its contents and then sends the appropriate notifications to my Z wave system.

It's probably easiest to visualize in the following chart.

The Jarvis script is at its core very simple. It is a simple text parser which reads the text for key phrases such as: "turn on the family room light". It doesn't use any fancy heuristic to figure out the meaning of the sentence. It simply looks for keywords in a sentence. In this case those words would would be "light", "on" and "family room". It doesn't care about the order, just their presence. I can however send multiple instructions via one e-mail using Siri. Each sentence is interpreted as one instruction. So I can have several sentences to fire several different devices. The script looks for the "." As a separator between instructions.

It is very easy to add more commands. Commands presently are:

• turn on/off the basement/living room light
• lock/unlock the door
• run scene <scene name>

Now obviously this is going to reason some security questions. As I for sure do not want some stranger firing an e-mail to my house and operating my devices. So I do have some security protocols built in. Outlook requires the message to be from a certain e-mail which will remain secret ;-) it is also easy enough to integrate a password requirement into the autohotkey  Script. This will be a simple day password (not yet implemented) where if it is absent from the script, Jarvis will not operate. It is also very easy to detect multiple, repeated attempts.

Also, I only activate my Jarvis script when I expect to need it.

Some might also ask why I chose to use autohotkey  n this way as opposed to simply firing the instructions via Outlook. While this is possible to do via the integrated VBScript in Outlook I would have to duplicate the same code into the Microsoft speech macros to get the same functionality. I.e. doubling my maintenance requirements. Any change to one would have to be made to both. In This Way, Outlook and the speech macros simply messengers. Any changes, modifications or updates are simply done in the Jarvis script only. Also by using autohotkey  it is very easy to add additional functionality to interface with any other system running on my computer. For example it would be easy enough to add commands to start iTunes and play music over my airplay system.

Unfortunately however, this Jarvis mod has been written to operate with my z wave system and PC set up. This means that unfortunately it will not be immediately portable into your environment. My intention here was to share the methods and show that it can be done. That said I will be including the text file with some additional instructions on how to do so. You will however require a basic autohotkey knowledge to know where to make the appropriate changes to your script.

Okay, I'll admit it… I just
wanted to use this picture ;-)

You can grab the Jarvis files here: the Z wave Jarvis Mod however the Windows speech macros are not yet included. Functionally, they will be nearly identical to the VBScript for Outlook.

(Update: as of 14/1/2013 speech macros are included and functional)

I will be working on a tutorial on how to properly set this up in improving the provided instructions.

So I think it's safe to say that the myth you can't extend Siri functionality or improve on your voice control functionality is pretty much busted ;-)That being said, I really should point out will once again that this script is more of a first functional prototype or a proof of concept rather than a final product. And by definition, is not really modifying the function of Siri or any of the other systems, but extending the functionality by tying them together through another script… By definition to me, this is a hack ;-) I will be periodically updating or reworking the script.