Eye Tracking, Windows Surface & a Broken Arm

If you've been following this blog for some time, you note that have been anticipating the lack of use of my hands and have been looking for solutions to allow me to continue to access my computer. For some time now I've been keeping an eye on the PC eye go by Tobii (see what I did there?) And I've been pushing my rehab facility to look into it.

Now, my rehab facility had access to one of the earlier Tobii eye tracking computer solutions but it was prohibitively expensive and not to the usual computer standards I am used to. Ringing in at about C$10,000. So once I knew about the Windows surface and the PC eye go, for total of about $3000 I was pushing my rehab facility pretty hard to get one for trial, which they recently did. And fortunately, I was the test case for it. As a matter of fact, I'm using it right now in conjunction with Dragon NaturallySpeaking. I did try Windows speech recognition which is functional, but frustrating as this comprehension was not nearly as good.

yeah, that hurt

The goal of the evaluation was to find out how easy it is to use for my day to day computer needs and how easy it is to keep calibrated. Also evaluating any physical demands it places on me. And unfortunately, my evaluation turned into a far more realistic experience than expected as about two weeks ago I fell and broke my right arm in four places. Three and the rest, one in the shoulder. Eliminating the use of my hands and any ability I had remaining to access my desktop computer. I immediately became dependent on the eye tracking.

That said, broken arm and all I have to say I'm extremely pleased with the eye tracking experience. In particular I'm glad I had it on hand after the fall (see where they did there?) As without it I would've had zero computer access. Once I figured it out, it is very easy to keep calibrated. However I do have to point out it is very sensitive to changing lighting conditions. For example in the evening when the sun is setting and the lights in the house are going on. This can really wreck calibration. So as lighting conditions change you may have to frequently recalibrate. But, if you do this often enough as you start noticing changes in the precision. It's not that big of a deal. However if you wait too long, functionality can be completely lost, in my case I have to ask for help to push the on-screen buttons.

After calibration, there are two primary ways you use the eye tracking to interact. Via mouse emulation which I don't particularly like and gaze selection, which is awesome.

Mouse emulation actively tracks ways you working on the screen and moves the mouse cursor accordingly. This can be quite frustrating as they can be a little "drift" between where you are looking at where the cursor is. This results in chasing the mouse cursor across the screen. Needless to say, annoying. Secondly, to send a click you have to dwell where you want the click that happened. A.k.a. stare at it. Now as your eyes are always making slight movements, even when you think you are fixed on a particular location this can be just enough to prevent selection from happening. And then added to it the tendency to look at the cursor from problem number one… Again annoying.

That said, it does work and it does have its place.

Now the gaze the selection is really great. There is a bar overlaid on one side of the screen which you look at to select the desired mouse function, and then you look where you want that action to happen. It also has its own on-screen keyboard which takes up about half of the screen. This may sound like a lot of screen real estate, put on the tablet it makes the buttons not that huge. But the are extremely easy to look at and type with.

I had a few issues with keyboard layout, just for my habits of typing and the tendency to want to do some programming/scripting but all in all it's very functional and very responsive. It is of course slower but if you're using it in conjunction with Dragon naturally speaking. Very pleasant experience. The eye tracking keyboard by itself is perfectly functional for shorter replies to e-mails, Facebook and twitter updates and so forth.

Using VNC & eyetrack to remote
connect to desktop to use Dragon

One of the things that have appreciated the most, is using VNC to remote connect from the tablet to my desktop. This is been extremely handy (see, did it again). Using the eye tracking on the tablet as essentially the mouse to access some specific programs and functionality I had on my desktop.

I would have to say there's only two things about the gaze selection I had issue with. The first off, is there is no auto completion reword expansion built into the on-screen keyboard. Even the default Microsoft on-screen keyboard has this as an option. As you're essentially hunting and pecking one letter at a time to type. This would've been an extremely useful addition. It is however easy enough to install a third party piece of software such as wordQ to add this functionality. And then add a little auto hotkey script on top of that for easy access (just so you don't have to turn the keyboard page to access the number pad)

The second thing has to do with the built-in scroll tool. While it is perfectly functional, it is a little frustrating to use when scrolling through say and longer webpage. Use for example have to look at the bottom of the screen while, at the corner of your vision keeping an eye on the top of the screen to know when to stop scrolling. He really needs a page up/page down button. This is something that's also easy to solve with auto hotkey scripts, as a matter of fact, already written and tested.

I also tried plugging the tablet into my windows of seven desktop with a 22 inch monitor. The functionality was nearly identical with some changes due to Windows 8 Metro interface not being present. It was again extremely easy to set up and calibrate, and of course had been the much larger screen made selections extremely easy as I had much larger look targets.

All in all, I'm extremely pleased with the Windows surface to Pro tablet and a Toby eye tracking functionality. And, seeing as the fall has pretty much put an end to the use of my arms and hands in any functional sense for computer access. I am in the process of ordering one. I am however a bit undecided as to whether or not I want to order the Windows surface Pro three.

One thing to consider however is that on battery power the Windows surface 2 Pro with the eye tracking plugged in is only given me about 2 1/2 hours of power. Not that big of a deal, but you do have to keep it in mind.

Also, if you're anticipating the loss of the function of your hands. I would suggest it's worth acquiring the eye tracker (perhaps not the tablet right away) and becoming familiar with it earlier rather than being pushed in at the deep end. Before I broke my arm, I had a few occasions where it needed to interact with it manually. Particularly when I was learning about the calibration.