How to Fix an App that Crashes in Release but not Debug

I recently submitted version 1.16 of Charmed to the App Store, which included a new OpenFeint version, and a new Tap Me SDK version.  In addition, this was the first version of Charmed to be built under any version of Xcode 4.x.

Everything seemed to go smoothly with the submission and it was soon posted to the App Store.  With over 71,000 updates in 4 days and no complaints, I thought everything was good.

Then a couple of days ago I got an email from a player saying that Charmed was crashing when starting a new game.  Then I got another email and another one.

I investigated first on my iPhone 4S, iPod touch 4th gen and iPad.  Charmed worked perfectly on all of them.  I was confused.

Then I tried my old iPhone 3G…BOOM!  It crashed instantly.

Debug This!

I connected the iPhone 3G to my iMac and ran Charmed in Debug mode.  No crash.  Damn!  I hate these kinds of problems!

I tried multiple times with the network connected and disconnected, with OpenFeint enabled and disabled, after upgrading from version 1.15 and after a fresh install.  It worked fine every time.

I wasn’t sure where to look next, so I started to think about what the differences could be, and decided to check the Build Settings.  The new Build Settings dialog in Xcode 4 was a big help.  Being able to see the different settings for Debug and Release mode at the same time is a great feature.

One of the differences that jumped out at me was the Optmization Level setting.  The Debug version was set to None [-O0], while the Release version was set to Fastest, Smallest [-Os].

I changed the Debug setting to Fastest, Smallest [-Os] and BOOM, it crashed on the iPhone 3G every time I ran it.  Switching it back to None [-O0] brought back the correct behavior.

I did a Google search and found a question on stackoverflow.com saying exactly the same thing.

After some testing to ensure that performance was not majorly affected, and that it ran on all my devices, I submitted version 1.17 of Charmed to iTunesConnect.

The world is safe again for people to play Charmed on older devices (the update is now live on iTunes).

I’m All Thumbs

I wanted to get a quick fix out there ASAP, so the above change was good enough for now, however, during my investigations I found out about the Thumb instruction set.

Basically, Apple CPUs understand two instructions sets, the normal ARM instructions and a reduced size instruction set called Thumb.  The one limitation of the Thumb instruction set is that it has poor floating point performance.  So you trade off slower floating point for smaller executable size.

To complicate matters though, Apples armv7 processors support Thumb-2, which has good floating point performance.

The solution appears to be to add a build setting that builds the ARM instructions for armv6 devices for older devices, but uses Thumb-2 for the new armv7 devices.

I don’t yet know if building for the ARM instruction set on armv6 devices will fix my code generation problem, but I’ll be giving it a try sometime soon.

Lesson Learned

From now on, when I am in the final testing phase, I will test with the Optimization Level set to Fastest, Smallest [-Os] in Debug mode to more closely simulate the code that will be generated & running on the user’s device.

Hopefully this will allow me to catch this type of problem in the future before it gets into customers hands.

Of course if would also be nice if Apple were to fix this code generation bug in Xcode, but given that they are not even updating iOS for these older devices, I don’t expect that they will put much effort behind this.