And the extra point!


Part 1.5 in my quest to code for the out-dated Kinect model 1473 on an outdated Macbook pro from Mid 2010: I was never, EVER able to get Processing to control the tilt of my Kinect sensor, but now it seems as if I have that working as well! Again, things do not match the documentation/instructions 100% but with enough experience you learn where to look to resolve your issues. I Googled for “kinect tilt libfreenect” which turned up a complaint from someone that they were getting an error when running the libfreenect-glview command.

upload_firmware: failed to find firmware file.

I noticed I was getting the same weird error when running the command to test my install. It turns out the error is explained in the bottom of the FAQ on the openkinect.org site:

I get the following error accessing microphones: upload_firmware: failed to find firmware file.

The Kinect audio system requires a firmware to be sent at runtime. It should be found in your installation (in share/libfreenect/audios.bin) but it might be absent. Run share/libfreenect/fwfetcher.py to download the latest audio firmware file and move it to your installation.

Apparently you need the audio firmware in order to interact with anything more than the camera and depth sensor. There is a “fwfetcher.py” (FirmWareFetcher) Python script bundled in the libfreenect install that will fetch and install the driver for you. Running this command is not possible out of the box because it requires root privileges and is not executable. You have to run “sudo python /usr/local/Cellar/libfreenect/0.5.1/share/fwfetcher.py” (The location of the fwfetcher.py script may vary, this is its location when you install via Homebrew.) After the Python script downloads the Firmware it will crash while extracting because of case sensitivity in the path. It looks for all files under a root folder “$systemupdate” in the zip file but the file name as of the current firmware is “$SystemUpdate”. Opening the script and doing a find/replace “$systemupdate” with “$SystemUpdate” will fix the script so that it downloads and installs the driver correctly. After the install of the driver the audio and tilt devices should be operational both from native code and from the Processing wrapper.

Happy coding!

Random success story – black triangle!


Let’s say you’re plugging away at your Mac following a tutorial on getting started with Kinect 3D programming and you’re getting no where. No matter what you try you cannot get your screen to look like any of the examples as your Mac throws error after error. Hi, I’m Cliff. You’re here because you’re having a fight with your computer. You want it to draw fancy 3D imagery of your living room but it wants to remind you that it does not SEE the $99 piece of equipment that you overpaid for in 2013 just prior to the release of the newer fancier pricier identically functioning piece of equipment. I’m here because I just had a “black triangle” moment.

What does a triangle have to do with Kinect?
Anyone who has done anything with 3D programming has had at least one, what we call, “black triangle” moment. These are milestone moments where you accomplish the next big thing after spending at least an hour perusing random documentation. The term “black triangle” (coined by someone I can’t recall, my Google-fu is weak today…) comes from the typical “hello world” example you will find when learning 3D programming. Usually it involves a mild math and/or geometry lesson, along with a gentle introduction to a slew of new APIs and function calls that make little sense and it is finally climaxed by a black triangle painted on the screen of the device you are running your program on. In other words you do a whole lot of learning and work to get something really simple to show on the screen.

While the program seems trivial the door that opens as a result is huge. This is why software engineers tend to get excited over things that the average person finds ridiculous. I once tried to show my wife a needle bouncing from left to right on an analog audio app I was playing with to measure sound impulses. I digress.

My black triangle does not look triangular!
My black triangle does not look triangular!

I summoned you here today because I finally tackled an issue that consumed several weeks of my life back in 2013. I had trouble getting the Kinect model 1473 to work on OS X. This is because the OpenNI driver didn’t seem to agree with this slightly newer model of Kinect sensor. If you are playing with the Kinect you may have run across the issue as well. There are several models of Kinect sensors, the original one bundled with the Xbox 360 is the easiest to use. However the model released a few months later as a separately purchasable item (model 1473) changes somethings internally in hardware where the popular OpenNI driver can no longer detect. Then there is the Kinect sensor for the Xbox ONE, which I have yet to see/use.

My black triangle does not actually look triangular, but it’s just as exciting. I was able to get things working in just over an hour. I started by looking over all of the original tools I used in 2013 realizing none of them had changed much or received additional updates over the years. Then I finally took a leap of faith and arrived at the solution.

Getting Kinect model 1473 to work on OS X
The process is remarkably simple considering all the research I did years ago! I installed Homebrew then installed libfreenect. This entire process took about 3-5 minutes and uses only two commands.Installing Homebrew is one curl command, and installing the driver is one Homebrew command. The final step is to install the Shiffman wrapper for Processing. (You also need to install Processing itself if you haven’t done that.) This is as simple as going to “Sketch -> Import Library -> Add Library” from the menu and looking for “Open Kinect for Processing” After that I was ALMOST ready to start writing Kinect code again. There is one major issue, however.

The driver can only access the Kinect when run in a root process. I’ve logged an issue with the Open Kinect Github repo to ask about it, though the issue probably belongs on the openkinect.org site since that’s where the install instructions for OS X are lacking. (They explain how to use the Kinect as a normal user on Linux but not on OS X.) To work around the problem I had to launch Processing as a root user. I type “su” on the command line to switch to the root user account. Using Finder I right-click on show package contents and drill into the “Contents/MacOS” folder. I drag/drop the “Processing” executable from there into my terminal window and press enter in my terminal to run it. (Or you can just type the path in manually, “/Applications/Processing.app/Contents/MacOS/Processing”.) The first time you do this you’ll get a pop error with Processing complaining it can’t find the sketchbook folder. I had to make a “Documents” folder in the root user’s $HOME directory then I added a sym-link there to the “Processing” folder in my $HOME/Documents directory. only after all of that was I able to run Processing as root and get my Hello Kinect app to run. I also had to enable the root account in OS X to make it easy to add the missing sketchbook folder.

Finally, after all of the hassle I have success. In reality it should have been a simple 2 downloads, 2 installs, copy/paste a sketch and go. However things are never as straight-forward as they should be when you live in the world of computers.