Month: March 2015

  • Unicorn vs. Passenger on Heroku

    I’ve been hosting my flagship SaaS app on Heroku since 2008. Overall it’s been a stable, if a bit overpriced, platform. Over the past year, however, I’ve been experiencing mysterious performance problems. The app runs fine for several weeks. Then suddenly I begin receiving exception reports about certain methods not being found on certain objects. Restarting my dynos would fix the problem for a few days or a few weeks, but eventually I would start getting errors again. It definitely felt like some sort of memory issue.

    After profiling the app and discovering nothing, I installed the Librato dashboard which offers a basic line graph of memory usage across dynos. I began noticing a correspondence between this line getting above 200 MB and my app throwing errors.

    Each dyno on Heroku theoretically has 512 MB of memory. I was running my app on Unicorn with 2 processes per dyno. I wouldn’t expect problems unless each process exceeded 256 MB. I was confused why I was seeing problems at just 200 MB of usage. True, the line would continue creeping up if I didn’t restart my dynos, and would eventually exceed 256 MB which would trigger an auto-restart of the dynos. But this took a long time to happen, and in the meantime my visitors were experiencing a slower app and/or outright errors.

    I spent several days attempting to identify where the app was leaking memory. Why did the memory usage line continue climbing? I tried various techniques to identify the problem but was unable to reproduce the leak on my local system. Eventually I decided a different tactic was necessary. Heroku has been recommending Puma as an alternative to Unicorn for a while now, so my first thought was to switch to Puma which uses threads for concurrency instead of processes. However, my app runs under MRI, not JRuby, so I wouldn’t necessarily be able to take advantage of those performance gains. Instead I opted for Passenger which now runs on Heroku.

    The results have been beyond what I expected. My memory usage line is now perfectly straight. No increase over time. No eventual errors and dyno restarts due to overconsumption. What Passenger is doing under the covers is spinning up new processes during high traffic periods and killing them during low traffic periods. My app has been running for 3 months now and I haven’t had to restart any of my dynos, nor have I encountered any performance issues with the app. Success!

    I can think of two explanations as to why Passenger fixed these problems: first, perhaps Unicorn itself was causing my app to leak memory in a strange way. Second, and more likely, Passenger’s built-in ability to spin up processes on demand is keeping memory leakage to a minimum due to processes regularly being refreshed. Regardless of which explanation is correct, I’m happy the app is no longer throwing errors at inconvenient times. Most importantly, my users are having a far more consistent experience. If they’re happy, I’m happy.

  • Moving from Gmail to FastMail

    It was January, 2005. Google had launched its invitation-only beta release of Gmail just a few months ago. The initial storage capacity of 1 GB was dramatic, with its closest competitors offering an anemic 15 or 20 MB. My beta invite had finally arrived and I was in the process of signing up for an account. The excitement was palpable, “It’s email… but by Google! 1 GB of space… who could possibly use that much? And the web interface is so fast!”

    Jump ahead ten years to January, 2015 and you’ll find me ditching my Gmail account in favor of FastMail, a move that has been long overdue. “But why ditch Gmail?” you may ask. I have my reasons.

    First and most importantly, I’ve come to the conclusion that I want my email to be reasonably private. I just don’t like the idea of Google scanning my email and pulling out little bits of information about my personal life and buying habits. The speed, storage space, and features that Gmail offered used to offset the privacy disadvantage in my mind, but they don’t any longer.

    Gmail used to be fast. Really fast. It’s not anymore. Don’t get me wrong, it’s still fairly zippy, but it’s definitely slowed down over the years as the interface has grown increasingly more complex and bloated. I’ve found FastMail to be true to its name: it’s just as fast as Gmail. And the web interface is simple and non-bloated. I like simple.

    Another change in Gmail that really bothered me was the introduction of the social tabs. I know these tabs can be disabled now, but I don’t like the way it was initially forced on us. And I definitely don’t like the impact it had on legitimate email marketing. The average user isn’t going to notice their email is suddenly getting filtered into separate tabs, much less figure out how to turn it off.

    When it comes down to it, I’d rather pay FastMail for an equal amount of storage space, reliability, and speed as Gmail. I no longer have to deal with the ads, the privacy violations, or the sudden feature changes. Don’t fool yourself: you’re already paying for a free service like Gmail, just not with cash. You yourself are the payment: a consumer to be analyzed and sold to. And Google is very good at doing just that.

    And hey, Marco Arment recommends FastMail so it’s got to be good, right? Here’s what he says about the benefit of having an email address ending in a domain name that you control:

    For something as important as email, I’ve never trusted everything to a proprietary provider. My email address has never ended in someone else’s domain name, and has never been hosted in any way that would preclude me from easily switching to another provider.

    The transition to FastMail was very smooth. It was just a matter of modifying a couple of DNS records and using FastMail’s excellent IMAP import tool to transfer a decade’s worth of email from Gmail (this did take a few hours). I’m still able to check my FastMail account from my iOS devices, and I use their web interface on my desktop through a Fluid app.

    If you’re looking for a new email provider comparable to Gmail, I can recommend FastMail without hesitation.

  • Fix Bluetooth in OS X Yosemite

    I love OS X. It’s an incredibly reliable operating system and it’s usually a joy to operate. Unfortunately, since upgrading from OS X Mavericks to Yosemite I had been plagued with Bluetooth connectivity problems:

    • My Apple keyboard would randomly disconnect from the computer. Once this happened, it became impossible to reconnect it again without restarting. Turning the keyboard off and on again wouldn’t fix it.
    • My Magic Mouse’s tracking motion would randomly become jerky and stuttering. This would last for 2 or 3 minutes and then return to normal. Turning the mouse off and on again wouldn’t fix it.
    • Devices that I hadn’t added would show up in Bluetooth Preferences as being permanently “remembered.” Whenever I would try to “forget” these devices and closed the Preferences window, they would immediately show up again after opening Bluetooth Preferences.
    • My mouse and keyboard also showed up in Preferences and could not be “forgotten.” Same as above, as soon as I removed them and closed Preferences, they would appear when I immediately opened Preferences again.

    These problems were incredibly frustrating. I did a lot of research trying to determine how best to resolve them. None of the solutions I found worked. These included:

    • Replacing the batteries in the Bluetooth device
    • Disabling and re-enabling Bluetooth
    • Clearing the PRAM
    • Resetting the SMC
    • Restarting the computer (this temporarily fixed the problems but they always came back)

    However, I believe I’ve finally fixed these strange connectivity problems for good. A couple of days ago I moved the following files to my Desktop and restarted:

    • /Library/Preferences/com.apple.Bluetooth.plist*
    • ~/Library/Preferences/com.apple.Bluetooth.plist*
    • ~/Library/Preferences/ByHost/com.apple.Bluetooth.*

    It’s important to move (not copy) the files. This forces Yosemite to re-create the files on reboot. (I could have just deleted the files but I wanted to keep them around as backups in case something went wrong.) Since doing this, my Bluetooth devices have been happily connecting and disconnecting appropriately and I have no more stuck devices in my Preferences.

  • Why I’ll never buy from Virgin Mobile again

    Why I’ll never buy from Virgin Mobile again

    Today’s post is a bit self-serving and for that I apologize, but I’m hoping that telling my story publicly will accomplish 2 things: first, it will warn my readers that they do business with Virgin Mobile at their own risk. Second, and it’s a long shot, but it might provoke a response from VM and they might return the money they’ve owed me for years. A very long shot, I realize.

    “Trust, but verify.” I learned this lesson in early 2011. I have nothing against Virgin as a brand or a company. I admire Richard Branson and all he’s accomplished. He’s a remarkable example of a self-made entrepreneur. However, I can say without hyperbole that he has some real dolts working for him at Virgin Mobile.

    In December of 2010 I was looking for a mi-fi provider. I didn’t have an iPhone to tether with yet and needed an option to connect to the Internet while on the road. Virgin Mobile seemed to have the best deal at $130 plus S&H for a MiFi 2200. They also touted a 30 day money back guarantee which gave me confidence in making the purchase.

    When the device showed up, I quickly discovered that the coverage was not satisfactory for my area. I would frequently get dropped connections from home, and when out and about coverage was even spottier. So I called VM on January 5 to request service cancellation and get instructions on how to return the device for a refund. The rep I spoke with put me on hold for 20 minutes then said she would call me back later that night. She never did. Thus began 6 months of pure and utter frustration.

    • January 7: called a second time to find out what happened. Rep said they were sending me a mailing label to return the device. I waited over a week for the label to arrive but it never did.
    • January 16: called a third time to ask where the label was. The rep wanted to transfer me to the “mi-fi group” (first I had heard of this) but actually just dumped me back out to the automated call menu.
    • January 17: called a fourth time and the rep finally gave me an address and RMA for the device. I shipped the device back the same day via UPS and included a note in the package explaining in detail about what had happened.
    • January 19: UPS reports the package was delivered.

    Between January and June 2011 I called Virgin Mobile a total of 6 times to ask why my refund had not been processed. Each time I was told that it would be processed within a week. Each time the refund failed to appear.

    I switched to a different tactic and opened a service ticket through their web site. Here’s their response:

    We do understand how frustrating could be not having the answers when you need them. Unfortunately, there are procedures we have to follow and your issue is under review at this time. All we are asking you is for a little time in order for us to resolve the issue at your satisfaction. Again, we are deeply sorry for the delays, but we need to wait for the investigation that we have opened regarding the refund of your device.

    We have already confirmed your device has been returned and it might take up to 5 business days for us to have a resolution.

    So they admit they received the package. But for some reason, issuing a refund is a challenge for these people. Subsequent service tickets were equally useless. My refund was always “in process” or “under review.”

    Eventually, they claimed they had mailed me a refund check. The check never arrived. Subsequent customer service requests yielded no help whatsoever. Refusing to explain why I hadn’t received a check yet, they instead began telling me they couldn’t help me and that I had to contact the “Broadband department” for a refund. Whatever. I give up.

    By the time August 2011 rolled around I decided it was not worth the time and effort to continue pursuing this. So Virgin Mobile kept my money and I’ve heard nothing from them since. Overall, it’s a frustrating and disappointing experience when a company steals your money. Had I anticipated what was going to happen I would have kept the device and sold it myself. I wouldn’t have gotten all my money back, but I would have gotten something. As it happened, Virgin Mobile ended up with both the device and my money.

    Be warned. When it comes to Virgin Mobile, advice from The Princess Bride is appropriate: “Get used to disappointment.”