Vocoding fun


I posted a question on StackOverflow a while ago trying to learn the easiest way to get MapQuest to give you turn by turn voice output from Optimus Prime. Yesterday I saw a recent comment inquiring whether or not I had a solution. Today I have a possible solution. I found an open source vocoder written in C (Zerius Vocoder) that I’ve managed to build/run on the Mac. It works pretty good, I must say. Hi, I’m Cliff. You’re here because you obsess over little robot toys that fold into/out of vehicle form. I’m here to report status on what could be a nifty little iPhone app.

Imagine for a moment that you could plug a vocoder into the pipeline of CoreAudio’s queue services API. (Imagine that there were such a pipeline!) That would allow all sorts of interesting voice output! So far I only have a compiled binary with sample files and a rough understanding of what vocoding actually means. Obvious next steps are to finish my Speex tutorial series and refine the hideousness of iPhone audio APIs. I’m really thinking of using pipelines similar to how GStreamer works. If I get distracted reading that link to GStreamer I may end up on a tangent, trying to port GStreamer to CocoaTouch. That’s all for now, party people. I’ll hit you back when I actually have something tangible to share.

Speex On iPhone Explained


*Update*
I neglected to point out that you should un comment #define _USE_SSE in the config.h as mentioned below. This preproc directive will allow you to run on device. It was also mentioned that you could get more speed out of Speex if you #define FIXED_POINT instead of FLOATING_POINT. I have not verified this and Speex runs acceptable in my implementation without it but its worth mentioning.
*Update*

I get a lot of questions on how I implemented Speex on iPhone. So I’m going to take this time to do a quick tutorial series. This is part 1, which will show you how to import the speex source into XCode and get a clean compile.

First download and explode the Speex source to any location. Run “./configure” in the exploded folder and create an iPhone subfolder within the speex in this same location. Now we’ll begin creating an XCode project inside the iPhone subfolder. I use a static lib project so that I can reused my work in several other projects, but if you’re new to XCode and iPhone you may forego static libs and just create a typical view based project. I start out being anal about my project structure. I prefer to have only the project folder and a single src folder at my project root. I realize this is non standard iPhone but it makes perfect sense from a build perspective. (I see: build description + src = build output) So a little refactoring and adjusting my target settings to find the new location of the plist and pch files.

  • Drag/drop the libspeex and include/speex folders into your XCode project.
  • Drag/drop the config.h file from the unpackaged speex tar into your project.
  • Add the include folder to your header search path
  • Double click the project in the left hand tree and add the following user defined setting: GCC_PREPROCESSOR_DEFINITIONS = HAVE_CONFIG_H
  • (User defined settings can be added by clicking the little wrench in the lower left corner.)
  • Finally, you need to take “libspeex/echo_diagnostic.m” out of the compile loop. Either uncheck the target membership box next to the left hand “Groups & Files” tree pane on the left or open and drill into your Application (or static lib) target and select the “Compile Sources” branch. Find the file under there and delete it.

By now your compiles should be squeaky clean. Stay tuned to the next parts of the series where I’ll discuss how to import ogg and wave container formats into XCode and perform Speex decompression in an app. I’ve upoaded my source (SpeexLib) to my box.net account so you can click here to download it or find it in the Box widget to the right of this post.

Missing an entire section of code!


Don’t you hate it when you put your blood, sweat, and snot (would say tears but my nose tends to run more than my eyes when I work hard) into some section of your code… you release it and it runs flawlessly for a few months then you come back to find that the one section you poured your heart into is not being executed at all? Don’t you just hate that? I’m Cliff. You’re here because you write code for the machine to ignore. (If you’re here for another reason then go away because around here we only care about stuff that compilers can optimize away!)

Maybe it came as the result of my earlier iterative optimization. Maybe it was just an oversight. Whatever the case, I’m now looking at a section of code that appears very important but is guaranteed to never execute. Because I’m not nearly as dumb as I smell on a Saturday after cutting the grass I’m going to pretend that I meant to skip this section of code. In the interim I’ll explain how I got here. If you didn’t catch it, we released version 2.0 of our popular turn by turn navigation app over the weekend but there was a small problem integrating external music apps like Pandora. We posted a break fix 1st thing on Sunday because we understood the impact of people using their phone as internet music portal. However, in the mean time I’m looking at code that I cried over… code meant to handle the audio sessions in a sensible way… code that the run time never sees as the compiler likely plays hop-scotch over it.

I don’t get it. In my mind I had localized the activation of the audio session to a single place. I had completed method interfaces (or protocols in ObjC tongue) to mirror the way I would express my intent in English. I crossed my eyes and dotted my tees but somehow I did such a good job in my optimization that it somehow became optional to start an audio session. In other words, I don’t see how/where audio sessions are started at all but yet we get music ducking with the iPod. (We also kinda get it with Pandora now too but its needing a little more work.) By the end of the day I should have a better understanding of what went wrong here.

Leopard/Snow Leopard and iPhone Simulator Audio drama


You’re demoing an iPhone game that you’ve run successfully for months now. You’re standing in front of your superiors but you aren’t concerned because you have unit test coverage that spans the seven seas. You managed to find and stomp out even the littlest, most insignificant bugs that only a developer will notice. You’re on your way to becoming a superstar as soon as your demo completes, wowing everyone in and near proximity to the room. You have remote viewers watching as well over a conference call to Los Angeles via WebEx. You launch your app in the Simulator and for no reason what so ever the room is silent! The updated sound track that was supposed to accompany your splash screen is not playing! Even worse, the title screen explosion sound is completely absent. Double check your audio cable, your volume controls, the system preferences. Everything is set to full blast. Playing a test clip in iTunes confirms. You get audio out of every orifice except when the simulator runs! What gives? Hi I’m Cliff. I’m going to explain why you have egg on your face.

Fold your Macbook Pro up and call the demo. You can explain later why everyone in the room wasted their lunch hour to watch you fumble nervously with network cords and make awkward hand gestures in an attempt to draw lines in the air tracing the sound that wasn’t there. Back at your desk yet? Don’t waste time linking different audio frameworks for your sim build. Don’t waste time debating AVAudio vs. Audio Queues. Instead, pay attention to the color of your shirt and the amount of creamer you add to your cup of joe. Your hair should be brushed from back to front and not to the side or you won’t pull this off at all. Also, any notes or paperwork should be stacked to the right side of your laptop when you launch the simulator. What I’m getting at is the impact of your environment on the iPhone simulator. There may be nothing wrong in your code or your make file. The problem exists with you’re environment.

Twice it has happened to me where I thought I had accidentally allowed a bug to slip through causing audio glitches. In both occasions I found a different external factor causing my iPhone simulator audio to fail. The first time I had been using a Bluetooth headset paired to my Mac to watch Ali on DVD. That caused Simulator audio to not play at all. Un-pairing the device remedied the situation. Now, just recently, I learned that leaving the iShowU screen recorder idle in the background can cause an echo effect on the simulator’s audio. Bottom line, if you have iPhone simulator audio issues that just weren’t there yesterday… double check the movie you rented from Blockbuster the night before because it could be the culprit.