My Photo
Blog powered by Typepad

CEO and senior exec blogs

Blogads


« The debate continues: WOMM vs. stealth marketing | Main | What REALLY happened behind Kryptonite's blogging fiasco and what it means for the rise of corporate blogs »

Comments

Barak Berkowitz

Thank you for your patience Debbie, I surely understand those who have lost it.

We are investing brains, money and time against this problem and it has generally paid off. Performance in November was not perfect but was better than October, December was even better than November until this down time. Unfortunately each time you try to make a system more reliable you also create risk. In this case the risk was assumed to be small, we assumed wrong.

Over the last few weeks we have successfully moved the entire service from San Jose to San Francisco (there are still a few network changes to make to be totally moved). We are on new machines including an expensive, reliable and fast file server. We moved terabytes of data, put it on this new system and moved tons of servers without a service failure. Thursday we were completing the instillation of a redundant front end to the file server and this installation required a reboot. For some yet to be determined reason the reboot corrupted the directory of the file system and led to the longest application down time in TypePad’s two year service history.

Apologizing at this point sounds so hollow that I am almost afraid that people will take it as an insult. But we are sorry, in fact I’m mortified. We don’t really have the right to ask for our customers continued trust, but that is what I must do and honestly believe we can earn it.

We will complete restoring data this weekend. We should finish the remaining details of the move by the first half of January. And I truly believe that risk, while already low, will be greatly reduced by then. No big new changes in hardware to make, no data to move, no networks to reconfigure, that is all behind us and we can focus on continuing to smoothly growing capacity to keep up with demand. We will keep making the system better, but what we have today is many times more bullet proof than what we had three months ago.

All this is to say that I am optimistic that Steve is right, this is the darkness before the dawn and we will be providing you a rock solid stable service for all of next year and well on into the future.

The comments to this entry are closed.

July 2017

Sun Mon Tue Wed Thu Fri Sat
            1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31