Rackspace Handled the Outage Well (IMO)
I should probably state for the record that I might be a little bias on this one because I currently serve as a member of the Client Advisory Board at Rackspace and provide feedback on various aspects of their hosted email applications. But, as I am sure a lot of you realized today, there was a disturbance in the force. Rackspace experienced an outage today and as a result, a lot of websites and email accounts experienced downtime. While Pleth doesn’t host any websites at Rackspace, but we do however house some of our clients email solutions there.
Usually when there is a major NOC failure like this I like to find a nice tight hole and crawl into it, fortunately these situations rarely happen. When they do it’s a helpless feeling for those of us whose connectivity is affected by it. Normally we immediately call into our datacenter to see what news we can get, only to find that the call center is experiencing a severe meltdown thanks to the high call volume of resellers calling in to report the obvious. At the end of the day all we want to know is the same thing, what is the ETA for everything being back online? Well, thanks to Twitter, that’s all changed…
Today Rackspace did a great job getting updates out to their customers via Twitter, and I wanted to commend Cameron Nouri, the Rackspace Apps Evangelist who manages the @RackApps for doing an excellent job keeping us updated. I know it sounds weird to give a company Kudos on a day that they experienced an outage, but let’s face it, these things are going to happen. When these things do happen, keeping clients informed should be one of the top priorities for a provider in my opinion…
I know that there are going to be thousands of people out there that were affected that aren’t as calm about the outage as I am, and I promise I understand where you are coming from. I just wanted to point out how well they did on keeping us all in the loop.
Also, here’s a list of system status updates from the outage…
Kudos to the Planet
Yesterday The Planet Datacenter experienced a widespread outage that affected literally thousands of clients around the world. Since our network is housed at The Planet our clients experienced a brief outage. The details of the outage still aren’t for certain, we know that it had to do something with a router that was not functioning correctly.
The timing for this outage could not have been any worse, our server administrator was boarding an airplane headed for France and my other partner Stephen just so happened to be in Florida. For the past 2 weeks we have been doing some gradual client migrations and phasing out some old equipment. I scrambled and called my partner Greg who was at his sons baseball game and he scrambled into the office so we could put our heads together to see what was going on.
Initially we thought that the issue may have been on our end given the fact that we have been doing a lot of migrations but about the time Greg and I got connected I glanced at my Twitter window and noticed several other hosting providers talking about their systems were down. I put two and two together and figured that a lot of these providers were probably all housing their infrastructure at the planet as well. A few minutes later Kevin from Planet updated his status to let us know what was going on with the outage.
Since the phone lines coming into the support center were busy, just having this twitter account out there giving us periodic updates about the situation really did give my partners and I peace of mind knowing that the Planet had the situation under control.
Kudos to the Planet for using Twitter to communicate effectively to your clients and providing real time updates.
The Planet – Kevin (theplanetdotcom) on Twitter
Google Container Data Center Tour
I thought that it was pretty interesting that Google posted these tours of their new container data center storage facilities on YouTube. If you are a nerd like me and you look at a Data Center Tour like others would a trip to Disneyland, you will love this…
I got a kick out of how advanced this data center is laid out, and all of the research and planning that went into it, pretty much under the radar screen of a lot of us in the industry. Way to go Google:
Extremist Websites Using US Hosts
This is a practice that has been going on for a while now but is just starting to get some attention. Apparently some of these Muslim extremist websites that are touting anti-American sentiments are actually using web hosting infrastructures located here in the United States.
I personally know of at least two other incidents similar to this where websites were being hosted that were in support of Al Qaida. This is something that is very hard to police, but I think that as an industry we should do our best to make sure our clients are all on the up and up. Before any clients gain hosting access onto any of our web servers, my partners and I always take a close look at the project and if there is anything we aren’t sure about we always try to error on the side of caution.
Honestly I hate it that this happened to the Planet, but at the end of the day there really isn’t a whole lot that they can do to prevent it from happening again, especially given the high volume of hosting accounts that they house in their data centers.
(WEB HOST INDUSTRY REVIEW) — After taking credit for a deadly March 25 attack on coalition forces, a Taliban website claiming to be the voice of the "Islamic Emirate of Afghanistan" was found to be hosted by Texas web host The Planet (www.theplanet.com).
Unbeknownst to the US web host, according to a Washington Post report Wednesday, the militant group used The Planet’s hosting services for more than a year "to rally its followers and keep a running tally of suicide bombings, rocket attacks and raids against US and allied troops." As one of the world’s largest hosting providers, The Planet was unaware of its Taliban connections until a blogger noticed the connection last week and The Planet pulled the plug on it.
The Post article connected The Planet’s incident with a greater trend of extremist groups using US hosting because of its reliability, ease-of-use, and terms of use that allow for anonymity. At around $70 a month, payable by credit card, Site Intelligence Group co-founder and extremist Muslim communication researcher Rita Katz told the Post that low-cost, high-quality US hosting has resulted in many extremist groups willing to pay US companies to host their anti-US messages.
Using Free Web Town (www.freewebtown.com), run by Atlanta-based Tulix Systems (www.tulix.com), Tora Bora Front (www.toorabora.com), another Taliban Web site, continues to post regularly about attacks on US-led coalition forces and other inflammatory material.
Such incidents fuel the debate on whether US webhosts should shut down these websites or let them continue to operate, giving US intelligence access to clues about the leadership and structure of terrorist groups. This also reminds web hosts of the importance of being aware of the companies with whom they are dealing.
GoDaddy Denial of Service
I was just reading on WHIR this morning about GoDaddy’s most recent denial of service that brought down several thousand of their clients websites yesterday. In their response they noted that even though it was several thousands of websites, it only represents a small percentage of their clients.
This is one of the downsides to bargain website hosting in my opinion. Granted, any webhost can fall victim to a denial of service attack, but the old saying, you get what you pay for resonates loudly here in my opinion. There is no estimate on the amount of lost revenue and site traffic from this outage yet, and there likely won’t be, but trust me on this, it cost some people some money.
(WEB HOST INDUSTRY REVIEW) — Due to a mysterious distributed denial-of-service attack, several thousand websites hosted by GoDaddy.com (www.godaddy.com) intermittently flickered on and offline Wednesday morning for a period of hours.
CNET blogger Bob Walsh reported the outage Wednesday, writing that GoDaddy communications manager Nick Fuller stated that while many sites were interrupted, neither e-mail nor DNS services were interrupted. Also, while tech support reported that the DDoS attack made several thousand sites unreachable, this only represents a very small percentage of GoDaddy’s customer base.
Regardless, complaints began circulating shortly after the outage. Los Angeles lifestyle blog LA Snark was one of the sites affected, which wrote about being confused and irritated.
"It has been brought to our attention that you have been experiencing brief periods of site outtages. In order to best assist you, we will need to re-produce the errors that you are experiencing," read an email sent to LA Snark’s Jack Stands from a GoDaddy hosting support representative.
The email on to ask the site operator for a "traceroute to the domain while the issue occurs." Which, typical and understandable of shared hosting customers, Stands had no idea what that meant.
A similar denial-of-service attack affected 600,000 GoDaddy hosting customers in November 2005, which lasted roughly an hour.
Data Loss Dooms Journalspace
I was just reading a post on Data Center Knowledge this morning about a blog hosting service that I had actually heard of, Journalspace, who is apparently shutting their doors after they apparently lost all of their clients data due to one of the poorest backup strategies I have every heard of. This is yet another glowing, bright red & blinking example as to why you get what you pay for with your hosting provider, in my opinion.
At Pleth, we have one of the most comprehensive offsite backup solutions on the market. Of course by making this offering a part of our base managed hosting solution we are forced to relay those costs onto our clients, but as you will see below, this peace of mind is hard to put a price tag on. If you don’t believe me, just ask some of the Journalspace clients that are staring at the daunting task of starting all over again in 2009. I wouldn’t be surprised to hear about some lawsuits over this in the next year…
Here’s an excerpt of the article on Data Center Knowledge:
Also, if any Journalspace clients out there that would like to discuss webhosting for their blogs on a safe, redundant, secure, and reliable hosting platform, please drop us a note.
The blog hosting service Journalspace is shutting down after all its user data was lost. Rather than backing up its data at another location, Journalspace mirrored the data on a separate drive on the same server. When the data on both drives went missing, the service apparently had no backups it could use to restore the data.
“There was no hardware failure,” reads the notice to users. “Both drives are operating fine. The data was simply gone. Overwritten. … Clearly, we failed to take the steps to prevent this from happening. And for that we are very sorry. So, after nearly six years, journalspace is no more.”
The site had a rank of 106,000 on Alexa, suggesting some meaningful traffic. Obviously, a cautionary tale about back and recovery practices. See Slashdot for more discussion.
Data Loss Dooms Blog Hosting Service « Data Center Knowledge
Mailtrust Advisory Board
I am going to be on the Mailtrust Customer Advisory Board for the next 6 months. I was talking w/ one of their representatives today about how Mailtrust’s Noteworthy product is probably my most favorite SAAS solution on the market! I didn’t really realize how handy the webmail interface was because I usually download everything into Outlook when I am in the office or onto my phone when I am on the road, but over the Christmas break I used the webmail interface exclusively and it was awesome! I am excited about providing feedback that might impact the future path of this service.
As a member of the advisory board I am going to participate in surveys and analysis provided by the mailtrust team. I will also provide some real world insight as to how I and my clients interact with their product.
Business Email Hosting & Webmail Services | Mailtrust
Wal-Mart Website Issues
I was out of state visiting family over the holidays but still managed to catch wind of this one on CNBC black friday. It appears that Wal-Mart’s website went down due to heavy traffic and slow pageload times. There are no estimates as to how much revenue was lost during the outage but with wal-mart.com being the 21st most popular website on the web, you can assume that it’s considerable. Here’s an excerpt from BizReport.com regarding the outage:
While many of the big stores experienced high traffic and a small amount of disruption, even Amazon.com shut down for 15 minutes, Wal-Mart came out the worst hit.
Linda Blakley, a Wal-Mart spokeswoman, said, “Due to a higher-than-anticipated traffic surge, we made the decision to shut down the site temporarily. The site is open for business and the problem resolved.”
Wal-Mart’s problems began early in the day. At 4.30am EST visitors were already experiencing blank pages and delays. By midday a virtual sign was tacked to their virtual door advising people to “come back later”. Not a great start to the holiday season for the 21st most popular website in the U.S.. By 2.00pm the site was up and running again, but with a noticeable delay.
Amazon.com’s problems began at around 11.00am EST when they offered the first 1,000 customers to buy an Xbox a bargain price of $100, compared to the usual retail price of around $300. The Xbox sold out in 29 seconds.
“We saw dramatically more traffic than what we anticipated,’’ Amazon spokesman Craig Berman said on Friday.
Ben Rushlo, senior manager of competitive research at Keynote Systems Inc., said retailers get better each year bracing for the volumes, but they also make their sites increasingly complex, adding 360-degree views of products and other features. Nonetheless, with the exception of Wal-Mart, online retailers were generally performing well.
“There were a few glitches here and there, minor problems,” Rushlo said.
Heavy Traffic Weighs Down Wal-Mart Website – E-commerce – BizReport
Valuation of Annual Accounts
Several times my partners and I have played around with the idea of acquiring a smaller hosting company that was for sale on the open market. We have the infrastructure to accommodate several times over the number of clients we currently have and purchasing a smaller company seems like a logical choice for us to quickly grow our client base.
One of the concerns we have about purchasing another hosting company is that our operation is completely different than that of most hosting operations, for instance GoDaddy, they do bargain basement hosting and offer annual renewals on their hosting customers. Several other large bargain basement, barebones hosting operations out there are setup the same way or similar as well.
The vast majority of our clients are billed on a monthly basis. The reasoning for this is because we actually bring a few other products and services to the table and also have a retainer agreement that certain clients take advantage of. We also offer other services outside the realm of hosting such as development.
I ran across this article today, and I actually have an email into this guy to discuss some of this but haven’t heard back from him yet. This is great information if you are in the hosting industry and I recommend you check out Furlow Consulting’s website.
If you are acquiring web hosting companies and a specific target primarily has annual customer accounts, don’t run for the hills, quantify it. Some buyers are immediately concerned about the increased risk of customer renewals for annual accounts which are 6-12 months out. At the same time they do not want to provide service and support each month for free for many of the customers for that period of time. There is a solution.
The two main value drivers in this equation are customer renewal rates and the “Total Customer Support Months” figure, (or “TCSM”). The later is as follows. If a customer signed up or renewed their annual contract 2 months ago, then this customer counts as 10 Customer Support Months. If a customer signed up or renewed 5 months ago then the contributing Customer Support Months is 7. Add this for every customer and you get a Total Customer Support Months. Multiply the total, times the average monthly cost to provide service and support to the average customer and you get the TCSMs figure in dollars. This needs to be subtracted from the valuation of the company if all of the customer accounts were monthly.
For example, if you are analyzing a customer base of 1,200 accounts and they are all billed annually and spread evenly throughout the year so each month 100 are up for renewal, then the TCSM is 7,800 for all of the accounts and the average monthly cost to provide service and support to the average customer is $8, then the value of this issue is $62,400. This figure is of special relevance if the renewals are not spread evenly across the 12 months of the year. It is common for companies to do “marketing blitz’s” from time to time and sign up a lot of annual accounts in a 1-3 month period.
Click here to see the example chart provided in his article…
Customer renewal rates is the other value driver. I will use two examples.
Higher Renewal Rates: The target company has 1,200 customer accounts which pay $240/year ($288,000/year), and they are evenly spread throughout the year so 100 are up for renewal in each of the next 12 months. In looking at the trailing 12 months, on average each month 95% of the customers renewed for another year, so the initial forecast for the next 12 months is that 1,140 will renew and pay $273,600, not counting new customers.
Lower Renewal Rates: The target company has 1,200 customer accounts which pay $240/year ($288,000/year), and they are evenly spread throughout the year so 100 are up for renewal in each of the next 12 months. In looking at the trailing 12 months, on average each month 50% of the customers renewed for another year, so the initial forecast for the next 12 months is that 600 will renew and pay $144,000, not counting new customers.
Forecasting Renewal Rates: I wish forecasting annual customer renewals would be as easy as taking the seller’s historical rates and forecasting it out 12 months, but it’s not.
- It is important to realize at first the target company’s renewal rates will be par for the course, but over time the buyer’s renewal rate will be the correct rate to forecast. If the buyer’s rate of renewal is 60% and the seller’s rate is 90%, it goes without saying the buyer needs to think hard and fast about why and DO NOT forecast the acquired base at 90% renewal for a long period of time.
- Some of the factors which will affect a change in the renewal rates of the acquired base are the acquirer’s billing methods, pricing, service, support etc. Changes in each of these will affect renewal rates and should be factored in to the forecast.
- Look at the quarterly changes in customer renewal rates prior to closing. Look at the trailing 4 quarters renewal rates. Are they getting better or worse? … take into consideration the trend as opposed to using just last years total renewal rate.
- Figure out why did the renewal rate change? Did the seller keep lowering prices faster than others in the industry, hence the renewal rates increased a bit quarter over quarter? Or, did the renewal rates fall slightly because a year ago there were 300 accounts per support employee, six months ago there were 500 and for the last quarter there were 700 accounts per support employee.
In conclusion, yes, there is a reduction in the value of a web hosting company which has primarily annual customer accounts, but it has a lot more to do with Total Customer Support Months “TCSM’s”than it does renewal rates, because even monthly customer accounts have renewal rates.
Furlow Consulting – Valuation of Annually Billed Customer Accounts
Hosting Locations of Busiest Websites
Being a partner and co-founder of a managed hosting company, Pleth Networks, I try to stay up with what’s going on in the industry. I stumbled onto Netcraft this morning and found this hosting research data pretty interesting. Notice how big a piece of the pie our datacenter, The Planet, has in comparison to the competition.
I wonder sometimes if our clients even realize that their websites reside in the exact same datacenter, with the same backbone and hardware components, as some of the busiest websites in the world? This is one area that my partners and I have never skimped on, I would put our network infrastructure and hosting products against any other company in the industry.
I was a little surprised that Google showed up on this list but I am assuming that it got most of it’s clout from some their internal online properties. Apparently this research covered one million of the most popular websites.
Netcraft has developed a dataset which shows the hosting locations of the million busiest websites, as determined by visits from users of the Netcraft Toolbar. The dataset gives a guide to the market share of companies hosting the sites responsible for the great majority of web traffic, and is uninfluenced by parked domains, personal sites, shared hosting accounts or the majority of blogs.
Although the top 1000 sites are concentrated amongst the web superpowers, Google, Microsoft, Yahoo and EBay, the hosting locations of the top million sites are widely fragmented, with a little over 3.25% sufficient for top spot.
The dataset is presented in an Excel spreadsheet and provides a variety of different filters and selections. Using the dataset, a hosting company can identify its relative position and closest competitors in each of the top 10,000, 100,000 and million tiers of site traffic, and also by region, country, and operating system. If this data is of value to you, you can check out: Netcraft to learn how you can purchase this data.
