Speex On iPhone Explained


*Update*
I neglected to point out that you should un comment #define _USE_SSE in the config.h as mentioned below. This preproc directive will allow you to run on device. It was also mentioned that you could get more speed out of Speex if you #define FIXED_POINT instead of FLOATING_POINT. I have not verified this and Speex runs acceptable in my implementation without it but its worth mentioning.
*Update*

I get a lot of questions on how I implemented Speex on iPhone. So I’m going to take this time to do a quick tutorial series. This is part 1, which will show you how to import the speex source into XCode and get a clean compile.

First download and explode the Speex source to any location. Run “./configure” in the exploded folder and create an iPhone subfolder within the speex in this same location. Now we’ll begin creating an XCode project inside the iPhone subfolder. I use a static lib project so that I can reused my work in several other projects, but if you’re new to XCode and iPhone you may forego static libs and just create a typical view based project. I start out being anal about my project structure. I prefer to have only the project folder and a single src folder at my project root. I realize this is non standard iPhone but it makes perfect sense from a build perspective. (I see: build description + src = build output) So a little refactoring and adjusting my target settings to find the new location of the plist and pch files.

  • Drag/drop the libspeex and include/speex folders into your XCode project.
  • Drag/drop the config.h file from the unpackaged speex tar into your project.
  • Add the include folder to your header search path
  • Double click the project in the left hand tree and add the following user defined setting: GCC_PREPROCESSOR_DEFINITIONS = HAVE_CONFIG_H
  • (User defined settings can be added by clicking the little wrench in the lower left corner.)
  • Finally, you need to take “libspeex/echo_diagnostic.m” out of the compile loop. Either uncheck the target membership box next to the left hand “Groups & Files” tree pane on the left or open and drill into your Application (or static lib) target and select the “Compile Sources” branch. Find the file under there and delete it.

By now your compiles should be squeaky clean. Stay tuned to the next parts of the series where I’ll discuss how to import ogg and wave container formats into XCode and perform Speex decompression in an app. I’ve upoaded my source (SpeexLib) to my box.net account so you can click here to download it or find it in the Box widget to the right of this post.

Progressive Multimedia Downloads


So I’m switching careers… I’m not getting a new job. I’m just finally getting into Android development, or so I thought. There’s one catch. I have this media server I developed that’s occupying all of my free/professional/personal/sleep hours. It’s not the media server occupying my time. Actually it’s my thirst for knowledge/expertise in the multi-media field that has me tied up. You see, I found out that the iPhone does something different when you request multimedia with the web browser. Not so different but to me it’s different. In my career I have not yet had to deal with progressive downloads. Mobile Safari likes to request multimedia in chunks rather than in its entirety. That feature has driven me to the dark edges of the HTTP manuals and beyond. Tonight, I finally have a rough implementation of a progressive download multimedia server. Apache seems to support this out of the box. But since my media server is implemented as a servlet under Tomcat I had to implement this myself. I found details on this page.

There ya’ go. Don’t say I didn’t get you anything for Christmas. If you missed it, your gift is here. Check it out and learn something new for Christmas. If I wasn’t so busy these days I’d elaborate more. You know, be more verbose and ramble about how HTTP is so tricky and why we need more high level abstractions. Last week was rough so I’m just gonna sign off with a “Hit me up party people!”

Missing an entire section of code!


Don’t you hate it when you put your blood, sweat, and snot (would say tears but my nose tends to run more than my eyes when I work hard) into some section of your code… you release it and it runs flawlessly for a few months then you come back to find that the one section you poured your heart into is not being executed at all? Don’t you just hate that? I’m Cliff. You’re here because you write code for the machine to ignore. (If you’re here for another reason then go away because around here we only care about stuff that compilers can optimize away!)

Maybe it came as the result of my earlier iterative optimization. Maybe it was just an oversight. Whatever the case, I’m now looking at a section of code that appears very important but is guaranteed to never execute. Because I’m not nearly as dumb as I smell on a Saturday after cutting the grass I’m going to pretend that I meant to skip this section of code. In the interim I’ll explain how I got here. If you didn’t catch it, we released version 2.0 of our popular turn by turn navigation app over the weekend but there was a small problem integrating external music apps like Pandora. We posted a break fix 1st thing on Sunday because we understood the impact of people using their phone as internet music portal. However, in the mean time I’m looking at code that I cried over… code meant to handle the audio sessions in a sensible way… code that the run time never sees as the compiler likely plays hop-scotch over it.

I don’t get it. In my mind I had localized the activation of the audio session to a single place. I had completed method interfaces (or protocols in ObjC tongue) to mirror the way I would express my intent in English. I crossed my eyes and dotted my tees but somehow I did such a good job in my optimization that it somehow became optional to start an audio session. In other words, I don’t see how/where audio sessions are started at all but yet we get music ducking with the iPod. (We also kinda get it with Pandora now too but its needing a little more work.) By the end of the day I should have a better understanding of what went wrong here.

Compile Speex For iPhone


Speex is an audio codec specially designed for voice audio. I’m sorry, a codec is a software component used for encoding and decoding something. Oops, I’m terribly sorry! An iPhone is kinda like a phone but it has no wires, y’see… and the letter i is prefixed to it because… Hi, I’m Cliff. You’re probably here because you already know what an iPhone is. Maybe you’ve seen one of the commercials. Maybe your cousin’s best friend’s uncle’s nephew’s best friend’s cousin has one. (In case you missed it, that was a really really verbose way of saying “your cousin”!) Maybe you’ve been around the block once or twice and happen to know about both codecs and Speex. (Maybe it was dem fools throwin’ bones by the liquor store that taught you about Speex. Don’t laugh, cause you’d be surprised by what you learn on the streets these days.) Whatever the case, I’m going to explain how I got a clean compile on the Speex library using XCode. Hold tight because the remainder of this post is designed to be informative… that is all jokes and nonsense to the side.

  1. Download the Speex C source bundle
  2. Unpack it
  3. you might wanna try building it from the cmd line. Run “./configure;make” from the folder where you unpacked it.
  4. Create a new project in XCode. This can be an iPhone or an OSX project but for consistency’s sake (and because it’s what I put in the title) let’s use an iPhone project.
  5. Create an actual folder, not an XCode group, under the root of your new project and call it “CSource”. This is where we’ll put the Speex source code.
  6. Drag/drop the folder into your XCode project or create a group that points to that folder.
  7. Double click your project icon in the left hand tree in XCode to edit the project settings.
  8. Goto the build tab and type “header” in the search field to filter your choices to the things that deal with headers. (Yes I just blatantly included a “goto” in a modern day technical writeup.)
  9. Look for the “Header Search paths” build setting. It should fall under the search paths category somewhere toward the middle of the screen. If you don’t see it finish typing the term “header search paths” in the search bar above. It does an incremental search as you type.
  10. Double click the Header Search Paths build setting to bring up the edit dialog then double click in the value field and set the value to “$(SRCROOT)/CSource”
  11. Open the folder that you unpacked Speex to in Finder and drag the “libspeex” folder directly into the CSource grouping you created in XCode. Choose yes to copy the files.
  12. Back in Finder, navigate to the include folder under the Speex unpack directory. Drag the “speex” folder out of here and next to the libspeex folder you added to your project in XCode. Choose yes to copy the files.
  13. Back in Finder, drag the “config.h” file out of the root of the speex unpacked folder and into your XCode project.
  14. Back in XCode, hit Cmd+Shift+D and type “arch.h” to find and open the arch.h file we’ve imported into our project.
  15. Add a “#include” to include “config.h” at the top.
  16. Remove the “echo_diagnostic.m” file under libspeex from the project as it will just cause complications.
  17. Hit compile and wait for the errors to roll in!

If you followed the above steps correctly then you should only see a few errors relating to duplicate symbols. If you get thousands of errors then it’s likely related to missing header files. You probably have to make sure you imported the speex folder with all the headers and double check your header search path to make sure that it points to the directory containing the imported “speex” folder. You might find a bunch of errors if you don’t remove the “echo_diagnostic.m” file from compilation. The last order of busniess would be to import the ogg container source.I downloaded libogg-1.1.4 which appears to work with speex-1.2rc1. Including the “ogg.h”, “os_types.h”, “bitwise.c”, and “framing.c” files allows me to compile code included from the “speexen.c” and “speexdec.c” examples.

Free Voice Guidance Navigation on the iPhone


Yesterday we became the first to offer free voice guided turn by turn navigation on the iPhone. Not Google, not Apple, not even AT&T but MapQuest! I know a lot of people will be all like, “Google’s free on Android!”, or “Verizon does this!” and “Garmin does that!” but let me tell you. There has been a monumental change within the walls of AOL. Things are moving in MapQuest. Free turn by turn is only the first step. And because you got a nut like me working on the project you can believe there’s a lot more coming. I don’t typically post a lot of specifics about AOL or MapQuest here but yesterday was highly important. We’re going to establish ground in the market of mobile. And we’re going to do so while you’re sleeping. …While you’re obsessing over whatever Motorola puts out. …While you’re tinkering with your Google Maps mashup. We’re going to release something that will have you saying “wha?? Why didn’t Tom-Tom think of that? Where was Google when they thunk that up?” If you don’t know, then you’ll know!

Leopard/Snow Leopard and iPhone Simulator Audio drama


You’re demoing an iPhone game that you’ve run successfully for months now. You’re standing in front of your superiors but you aren’t concerned because you have unit test coverage that spans the seven seas. You managed to find and stomp out even the littlest, most insignificant bugs that only a developer will notice. You’re on your way to becoming a superstar as soon as your demo completes, wowing everyone in and near proximity to the room. You have remote viewers watching as well over a conference call to Los Angeles via WebEx. You launch your app in the Simulator and for no reason what so ever the room is silent! The updated sound track that was supposed to accompany your splash screen is not playing! Even worse, the title screen explosion sound is completely absent. Double check your audio cable, your volume controls, the system preferences. Everything is set to full blast. Playing a test clip in iTunes confirms. You get audio out of every orifice except when the simulator runs! What gives? Hi I’m Cliff. I’m going to explain why you have egg on your face.

Fold your Macbook Pro up and call the demo. You can explain later why everyone in the room wasted their lunch hour to watch you fumble nervously with network cords and make awkward hand gestures in an attempt to draw lines in the air tracing the sound that wasn’t there. Back at your desk yet? Don’t waste time linking different audio frameworks for your sim build. Don’t waste time debating AVAudio vs. Audio Queues. Instead, pay attention to the color of your shirt and the amount of creamer you add to your cup of joe. Your hair should be brushed from back to front and not to the side or you won’t pull this off at all. Also, any notes or paperwork should be stacked to the right side of your laptop when you launch the simulator. What I’m getting at is the impact of your environment on the iPhone simulator. There may be nothing wrong in your code or your make file. The problem exists with you’re environment.

Twice it has happened to me where I thought I had accidentally allowed a bug to slip through causing audio glitches. In both occasions I found a different external factor causing my iPhone simulator audio to fail. The first time I had been using a Bluetooth headset paired to my Mac to watch Ali on DVD. That caused Simulator audio to not play at all. Un-pairing the device remedied the situation. Now, just recently, I learned that leaving the iShowU screen recorder idle in the background can cause an echo effect on the simulator’s audio. Bottom line, if you have iPhone simulator audio issues that just weren’t there yesterday… double check the movie you rented from Blockbuster the night before because it could be the culprit.

Downgrade your iPhone DFU mode


My iPhone crapped out on me this weekend. I went to use it and the screen wouldn’t show at all. I loaded iTunes which was able to recognize it and offer to restore it to factory. Word of advice, iTunes is very sneaky! I didn’t realize that the factory restore it offered came with a complimentary upgrade to OS 3.1.2! It became apparent after I went to use the device and swiped right revealing the new spotlight search. Crap! I needed to keep it on 2.2 because I need to verify 2.2 compatibility. No worries, I’l just down grade when I get to the office. I then realized that I couldn’t build and run on device using XCode. I was getting “No Provisioned device connected” errors. I got concerned thinking I had to go through the whole re-provisioning my device hassle. Then I realized the “use for development” button in XCode. One thing lead to another then I finally realized all of the ugly truths about iPhone development.

1. OS 3.1.2 only accepts builds from XCode with iPhone SDK 3.1.2.
2. The iPhone SDK 3.1.2 only installs on OSX 10.6 (Snow Leopard)
3. Apple no longer offers any iPhone OS earlier than 3.1.2 on its website.
4. If you are lucky enough to find a lingering copy of iPhone OS 3.0 or earlier, you cannot roll back from 3.1.2 (…easily) due to baseband errors.
5. DFU mode is not easy to do or understand.
6. You will likely spend the good part of a couple of days trying to recover from an unintentional upgrade as your first 5-7 rollbacks will leave you with misc errors.

The simple answer is that you cannot downgrade the OS without using DFU mode. DFU stands for Device Firmware Upgrade and is a secret mode that Apple doesn’t want you to know about. You have to use something similar to the old Nintendo Contra cheat (you remember up, up, down, down, left, right, left, right, b, a…) to enable it. I’ve tried it a couple of times so far and realized that you probably should have a completely installed OS before trying it. My second attempt (after a fresh iTunes restore) left me with an error and I’m hoping the third time is a charm. Here goes nothing…