I've said it in a previous DotCloud post and I'll say it again: If they can pull it off then they'll be golden.
But I'm extremely skeptical they can.
Many of the components they list require intimate domain knowledge. E.g. there are entire companies dedicated to designing and babysitting Hadoop clusters, yet DotCloud just lists it casually alongside other bleeding edge mammoths such as Riak or Cassandra.
DotCloud will need to not only stay on top of things for so many components and provide smooth upgrades for an inherently growing diversity of deployments. They'll also have to support old versions effectively forever and deal with all those little customizations that people need.
Of course all that is doable, given enough man-power. However, after a certain point it's not very automatable anymore.
So, that said, I remain curious if this will really come out as the holy grail that they seem to be shooting for. Or if it will gradually degrade into "just another managed hosting provider" (which isn't a bad thing either, of course).
We get this all the time. Yes, Riak and Cassandra require intimate domain knowledge. The question is: what's the most efficient way for you to consume that knowledge? Hire a full-time expert for each? Or use the battle-tested Riak setup that our Riak expert maintains for thousands of developers?
As you point out, beyond a certain point your needs will be specific enough to justify serious customization by the Basho and Clouderas of the world. We don't replace these guys. In fact we plan on facilitating integration with them, so that you don't have to chose between a well-oiled dotcloud stack and that 1% of customization.
As for how scalable and automatable our business is - don't worry about us. Having been in your shoes before, I can tell you with great certainty that dotcloud is much more than a bunch of custom AMIs, chef scripts and clever role-based provisioning scripts.
Our goal is to make you, an expert at these power tools, happy to use us instead, because it's so much freakin simpler. And don't worry, we will add all the customization hooks you need. We're taking one step at a time.
That was exactly my concern when I first heard about it. I got a chance to sit down and chat with one of the co-founders a few weeks ago and came away very impressed - he demonstrated deep level knowledge of a bunch of software I care about (I particularly quizzed him on Solr and Redis) and managed to convince me that his team is very sharp when it comes to keeping on top of the infrastructure elements they support. I think they might be able to pull it off.
Yes, I didn't mean to criticize the team. I'm just in the same business (making my living with what they are trying to automate) and know the hurdles they are going to meet.
For example, my provisioning system looks nearly identical to theirs. I also have the central config file to map out roles/services/dependencies, network relations, and the CLI tools to 'manifest' a layout in the form of server instances, EBS volumes etc. And finally puppet to beat everything into shape. I can compose and deploy a production-ready runtime environment with many permutations of commonly used components in a single day.
The thing is, every seasoned admin builds that stuff over the course of his career. It cuts down manual labor significantly.
But you eventually reach a point of diminishing returns. Each individual component has corner cases, special optimization potential, and more than one valid operation mode (e.g. cluster vs master/slave).
Automating and maintaining(!) all permutations that customers will request is nothing short of a herculean task. That's why every managed hosting provider limits itself to a tiny subset of components ("We are experts in X"), charges significant fees for their services, and refers to specialized consultants for requirements such as "let's have a hadoop cluster".
But I'm extremely skeptical they can.
Many of the components they list require intimate domain knowledge. E.g. there are entire companies dedicated to designing and babysitting Hadoop clusters, yet DotCloud just lists it casually alongside other bleeding edge mammoths such as Riak or Cassandra.
DotCloud will need to not only stay on top of things for so many components and provide smooth upgrades for an inherently growing diversity of deployments. They'll also have to support old versions effectively forever and deal with all those little customizations that people need.
Of course all that is doable, given enough man-power. However, after a certain point it's not very automatable anymore.
So, that said, I remain curious if this will really come out as the holy grail that they seem to be shooting for. Or if it will gradually degrade into "just another managed hosting provider" (which isn't a bad thing either, of course).