1. A pillar. This person is nearly impossible to replace because you trust them and they add incredible value to the team. It’s even more so if you’ve built a friendship and/or working relationship with them over time. Imagine starting from square one and going to a cofounder’s mixer.. I can’t.

2. Full of faith for the vision. They are not only motivated by the outcome but also by the work and the vision. They want to be a part of that vision because it’s something they fundamentally and philosophically believe in.

3. Someone who is not competitive with you. Many people start companies with their best friends, but sometimes this is exactly the wrong approach. Human relationships are complicated and someone you’ve known for years might seem like a good person to start a company with, that is until you start a company with them. Just remember to keep an open mind and that if you’re competitive with your cofounder, it probably won’t work out.

4. Someone you would trust with your life.

5. Complementary. If you have an idea they expand on that idea. If you’re a bit too liberal they’re a bit too conservative. Complementary should not be confused with diametrically opposed. There is such a thing as too different to get a long. A good example of a complimentary founding team would be the two Steves of Apple fame.

6. Unexpected. The cofounder should impress you. Not because they want to or are trying to but because you just happen to be impressed with the quality of their work/ideas/direction or whatever.

7. Challenging. They don’t blindly follow every word and they don’t challenge you on every point. They challenge you when you need to be challenged. After all, you can’t always be right even if you’d like to be.

8. Easy for you to get along with on some other level than just work. Either you have similar political views/principals or you both love Family Guy.. Whatever it is you should easily be able to find stuff to talk about. If you can’t even have a conversation with this person, they probably shouldn’t be your cofounder.

9. Someone who sticks around even when the going gets tough. Real tough. As I’ve experienced first hand startups are extremely trying for spouses, partners, friends, family and everything else. It takes its toll and no one can cross the finish line without paying that price. No one. You’re going to need someone who doesn’t just shine in the initial stages, they also need to shine when shit hits the fan and you’re losing hair, sleep, sanity and whatever else..

10. Someone who puts their money where their mouth is. You’ll know what this means when the time comes.. But the true test of character is when people start putting their own money into the project and they do it with both feet.

11. Well grounded with their ego. You obviously want someone brilliant to cofound your startup with.. But at what price? Don’t go for the guy with 2 PhDs and the nobel prize in Physics just because his IQ is 270. You have to first make sure they fit into your universe and in a good way. As Einstein said: “God does not play dice with the universe” and neither should you with your startup. Take that Niels!

Section 0: Startup Failure Anonymous?
Hi my name is Fred and I failed at a software startup. Is there a Startup Failure Anonymous group I should post this to..?

Section 1: In the Beginning

I currently work for a small company as a software engineer and have been with them for 5 years now. Let’s call them Company X. Company X sells a software product to doctors nationwide and pays me well so I have no complaints about that. My primary role there is development on their main product, a web service, and back-end web apps that help Company X manage everything from accounting to customer information and more.

About a year ago I endeavored to create a piece of software which would help manage Company X’s information (I can’t get into specifics about what the actual product was because of contractual reasons but I can say that it was a business tool aimed at small to medium sized businesses and it was a django/jquery web app with a twist: a feature that used speech recognition so that users could interact with the app using voice commands from a mobile device like an iPhone).

I had looked countless hours on the net looking for an existing solution for Company X’s need and just couldn’t find one that fit the bill. That’s when I got this idea. The idea was I would develop this piece of software, get it working for Company X, make sure it worked well, and then pull a 37signals move and release it under my newly created LLC.

So it began… My new nighttime startup was just me, an independent contractor coder, and of course: several credit cards…!

Section 2: Moving along

It took us roughly 8-10 months of sleepless nights from concept to design to implementation. The plan was to get some buzz going for my web app by first releasing an iPhone app. The goal of the iPhone app was really threefold: getting some of that aforementioned buzz, testing out the server-side speech recognition, and making some cash to alleviate debt.

I guess two out of three ain’t bad. The app met some initial success but we realized quickly it wasn’t going to pay the bills. We were better off giving it away to encourage downloads. That certainly did the trick. Our system was tested to its limits when we set the price to free. We were successfully processing 1000s of daily requests which is more than I had hoped for. So on that note we were good.

Section 3: Rewind just a little – I forgot to mention…

We continued our march onwards. Or did we..? Well that’s when things get a bit foggy. I almost left something out. It didn’t kill my startup directly.. but it definitely hurt it tremendously. You might call it an accomplice in my startup’s murder.

This thing I’m speaking of was a bug in the 2 monster servers I purchased to handle the speech rec for the iPhone app. Speech recognition for the uninitiated is fairly memory/processor intensive even with today’s hardware hence the reason we needed some powerful servers. We thought about renting some Quads with decent memory but it would have been too expensive.. so I purchased 2 servers for a bargain at newegg.com and colocated them for dirt cheap.

At any rate after we had gotten everything set up we started noticing that either server would randomly throw a machine check exception every now and again which brought that entire machine to screeching halt. You can imagine my horror at this. They were brand spankin new..! I was pissed.

For nearly 2 months we pulled our hair out trying to figure out what was causing this behaviour. You can imagine what we tried: operating system version changes, disabling RAID, swapping memory. You name it and we tried it. Checking logs was useless. The kernel was killed before it could even write out something useful. The machine check exception was useless because it was too general to pinpoint any one piece of faulty hardware.

The one thing we had going for us was that it happened on both machines. So finally.. yes finally we figured out that the bug in question was an issue in our blazing fast Harpertowns’ microcode. One quick update to the microcode and 5 minutes later the ghost in the machine was dead and we shot’em good..!

Section 4: Painful realization…

The unfortunate result of this wild goose chase was that my funds had dried up. I didn’t have enough credit or cash left to keep paying my contractor to finish the web app. The iPhone app was subsequently released (now that our servers were fully working) but as I stated above it didn’t generate a significant amount of cash from app sales.

So there I was – an iPhone app released and used by tens of thousands… but not a damned dime to finish the job..!! After all, I initially set out to create a solution, not a feature to that solution. The iPhone app was really just the beginning in my mind or so I thought. In the overall scheme it was going to be a buzz generator and beta test of my “killer” feature that would later be incorporated into the general solution. I couldn’t have been more wrong in my approach. I made a fundamental mistake which I’ll discuss later.

So anyway, when the funds ran out it obviously hit me that I had to shut it all down and I did. I pulled the app from the store, sold off the servers and continued on with my day job at Company X.

Section 5: Reflections…

I’ve had almost a year to think about where I went wrong.. Here are the main observations I made about my experience:

1. iPhone app development is not trivial. This took up a huge portion of my time.
2. In a self-funded startup you’re always one emergency away from going bust i.e. your servers randomly generate machine check exceptions =)
3. the iTunes app store and hype are misleading. Most apps need enormous volume priced at 99 cents to make significant money from app sales and those tend to be video games or iFart type apps.
4. Decent designers cost a shitload, are usually too busy, and really hard to locate.
5. It was the most chaotic up and down roller coaster year of my entire life.
6. Startups are really really hard to do alone. I would have loved to have had a cofounder but it just didn’t work out that way.

Section 6: Reflections part II…

After much thought and reflection I finally realized that integrating speech recognition into my solution and scaling it out web 2.0 style was ridiculous. This was a really tough thing to do even for large and well funded companies. How could I have hoped in my wildest dreams to implement something like that on such a large scale..?

I was so enchanted by this magical killer feature and the promise of speech recognition that I fell into a delusion. I made the entire solution about this one feature. Even though I thought it was a killer feature, one that would separate me from the competition. I failed to realize that this feature did not solve the original problem. Sure it was shiny and really freakin cool but people aren’t usually willing to pay for just ‘cool’. People are willing to pay for a solution to their problem. If you’re not solving someone’s problem then what are you doing..?? It wasn’t my original intent to implement this feature. It’s just something I stumbled upon during my research.

Section 7: The number one reason my startup failed…

So the number one reason my startup failed was: I was distracted by a cool and shiny feature that didn’t solve anyone’s problem. The shinier and more tempting features of any software program should be regarded with a high level of suspicion. There may be a reason some things are so shiny and alluring. Traps often have this quality. My advice to anyone creating a solution is to march straight towards your initial goal, as long as the goal really does address a true need then that’s what you should focus on.

Questions for the wise: Is the feature absolutely necessary for the solution to function..? If not will this feature take less than a day or two to implement…? In other words if the shiny feature you want to implement is not absolutely necessary for your solution and it will take more than a day or two to finish then move on.

That’s a wrap.. Love to hear your thoughts.

For all of those thinking of integrating speech recognition into their apps I have a word of advice for you: Don’t.

But you’re implementing the Star Trek computer you say..?
Well in that case it’s absolutely necessary to the core functionality of your solution! Starship software definitely needs damn good speech rec for its interface. How the hell else is it going to get your Earl Grey the right temperature..!? Well at least read this article before you embark on your journey so you have a better idea of where to start.

Just so we’re straight:
Speech rec discussed in this article is the kind that understands short phrases and/or commands with no training required. It’s not free flowing dictation like that found in Dragon software. That requires training and still has a long ways to go before being viable in a web 2.0 kind of way.

Engage:

Three main ways to integrate speech rec into a web app:

1. Telephony
2. Web Services
3. Embedded

1. Telephony & VoIP:

Why do we consider telephony..? Why not. There are more phones in the world than computers. They all have built in microphones and speakers and they all speak the same crappy language designed by the evil telcos years ago. This means that anyone anywhere who has access to a phone can interface with your app, international restrictions aside. All you need to do sign up for a DID. A DID stands for Direct Inward Dialing and can be any kind of phone number like 555-555-5555. When anyone picks up a phone and dials your DID they’ll be routed to your server where the call can then be managed by telephony software.

Asterisk & FreeSwitch are the two big players here. Technically they’re soft switches, software created to handle and direct calls either between two VoIP points like Skype or they can be used to accept and/or direct calls from/to the POTS network, otherwise known as the Plain Old Telephone Service.

Both are open source but FS is definitely more so. You could say Asterisk is like MySql and FS is like Postgres. The former has a massive ecosystem and lots of commercial support, books, and a giant user base. FreeSwitch, on the other hand, is supported and developed by a smaller more cohesive group of talented and dedicated devs. By extension FS has a smaller but more dedicated user base much in the same way Postgres does.

Performance and stability wise FS easily wins the battle. However, support and availability for 3rd party extensions, tools, and dev resources definitely favor Asterisk.

For my solution I stuck with Asterisk but not because I liked it better as you’ll see why later on in the article..

So in summary telephony is great because you can operate free of operating system quirks, platform differences, and having to support several different devices.. but the real question is: does it scale..? Well yes it does but not easily. There are per minute charges for the DIDs. It’s cheap, i.e. around 2 cents a minute, but imagine supporting 1000 simulateneous calls to your server.. Quite possible performance wise as long as you offload the speech rec to another server, it’s just that you’re paying per minute every time someone calls in to interact with your app. That could get expensive pretty quickly. There is one way to get around all of this headache and that leads us to the next section of course.

2. Web Services:
The web services setup is simple architecturally speaking: a voice sample is recorded either at a device, like an iPhone for example or from a computer mic and sent to the server as a file. The voice file can be compressed before it’s sent, most likely using Speex or GSM, and POSTed to the server RESTfully, uncompressed, and then decoded using your speech rec software. The server can then send back an interpretation to the client.

The nice thing about this solution is that it scales.! As long as you have some decent power server side you can easily decode thousands of requests a minute. A request being a spoken command or phrase no longer than 10-20 seconds in length. We’re talking voice commands here, not dictation, which is much much harder to do. It’s certainly possible but you’d need a lot more power.. This kind of tech just isn’t ready for prime time IMHO. If it were we might indeed have a star trek computer available. We’d still need to work out the AI though. Even if the dictation speech rec were 100% you’d need one behemoth of a program to analyze the meaning of it all.. Not yet available folks.

The one issue of course with going the web services route is that you have to implement a client side program for each device you want to work with. This particularly sucks in the mobile world since you’re dealing with so many device quirks and platforms.. If only working with devices was like working with browsers.. Never mind IE not supporting events the right way.. the iPod touch doesn’t even include a friggin mic..! Welcome to the world of mobile devices. And you thought browser quirks were bad…!

3. Embedded
Honestly, I just put this option up there to warn you. Embedded speech rec just makes all speech rec look bad. Don’t bother. Sure it will rec a few hundred names and/or song titles.. In general though serious speech rec should be done in the cloud b/c that’s where the power is and good speech rec requires a good deal of power.

Available Speech Rec solutions:

You didn’t seriously think you were going to implement this part did you..? =) Companies who deal with this tech often have several PhDs on board and there’s a good reason for that: Recognizing human language in a meaningful way is really hard. At least by today’s standards. With all the nuances, intonations, accents, and signal distortion i.e. wind, weak signal, that annoying guy on the cell phone etc.. it’s amazing it works at all.. A real credit to the science and researchers behind it all.

That said it’s so far away from what the stupidest human you’ve ever met can do it’s almost humiliating we’re not further along.

At any rate it’s not like you’re trying to build the next starship voice interface. That’s currently impossible anyway so don’t try it. What you can do is build some basic speech rec into your app much in the same way major banks, airlines, and mobile operators have: automated voice attendants that understand a constrained set of responses, lists of names or places, and/or strings of numbers.. Here are the viable 3rd party options for doing just that:

Commercially available:

1. Lumenvox (San Diego, CA)
2. Nuance (Burlington, MA)

Open Source:

1. Sphinx (Carnegie Mellon University)
2. Julius/Julian (Kyoto University, Japan)

The real deal:
I know there are others available like IBM, AT&T, Microsoft, Acapela, Loguendo etc… but they all kind of suck. Trust me. The suits at AT&T only care about BOA and AA contracts. IBM doesn’t even know it still has a speech rec department. Microsoft made some valiant efforts in the field but I’m not a windows dev and Loquendo/Acapela are in Europe so perhaps that’s why they were hard to get a hold of. Speaking French fluently I couldn’t get these guys to return my calls or emails. They did put me on their mailing list though. =)

Also it’s not like any of them are superior to what the other guys have or else we’d all know. Everyone is basically using the same kind of tech at their core. There is no one company with superior proprietary speech recognition. At least not yet. Maybe Google will be that company at some point but for now it doesn’t exist.

So your best bet is Lumenvox or Nuance. In particular Lumenvox is your best bet because they are the most developer friendly and the nicest to work with. Nuance is alright but it’s a windows shop and they are a lot more pricey.. They also have a ton of products, some of which may even compete with what you’re trying to do..! It’s easy to get lost on their site.

Open source dreams:
Accurate open source speech recognition is still but a dream unfortunately. I have the utmost respect and admiration for both the Sphinx and Julius teams. I mean holy shit.. They’re literally giving away speech recognition tech for free so all of us can tinker with it. The only real differences between the open source and commercially available solutions lie in what’s called their Acoustic Models. AMs for speech rec are like gold. A good AM is produced from several thousand hours of good audio samples. It’s what gives speech recognition its ability to recognize. Think stats: weights, averages, means, standard deviations.. etc.. that kind of thing…

The commercial products do indeed spend most of their time building up good AMs by enlisting large groups of people to help ‘train’ the system. The open source guys just can’t afford to do this for obvious reasons. In fact Lumenvox is based in no small part on the Sphinx project. So you might say that Lumenvox is Sphinx with a really good AM and some polish. Some well intentioned efforts are being made over at Voxforge.org to provide better AMs for Sphix and Julius but they still have a ways to go.

Wrapping Up:

Stick with Asterisk for now if you go the telephony route:
The primary reason I stuck with Asterisk was because FS dropped Lumenvox support. Understandably so.. Lumenvox attempted to charge them for dev licenses. I’m not sure that was a wise move on Lumen’s part but that’s politics I guess. FS is really becoming quite a contender in the world of Telephony & VoIP so I hope they work it out. For me it was a deal breaker.

Eventually though I went the web services route. It made sense because using this approach we were able to scale out the solution web 2.0 style. Telephony becomes too expensive cost and performance wise. You can still get good performance from the telephony route, just not nearly what you can attain using web services.

Anyway, I should have just listened to the advice at the beginning of this article in the first place: forget about speech rec unless it’s absolutely central to your solution. I learned the hard way and I’m writing this so hopefully you won’t have to.

I may write some more on the subject later on. It really was an enlightening experience to submerge myself in this whole other universe.

At any rate this article should have at least moved you a little closer to building your very own Star Trek computer. If you want to hear more on the subject or have questions just submit a comment and I’ll do my best to answer.

That’s a wrap