Fedora's Cloud Future (and, self (re-)introduction)

[mattdm]

(This is a reprise of a message I posted to the Fedora devel mailing list yesterday. I know not everyone reads that list and I want to get wide input and hopefully excitement.)

I've been active with Fedora for a while, and I know a lot of you and I hope a lot of you know me. I helped with the first few FUDCons back when I was at Boston University, and have been moderately active on the mailing lists, in Bugzilla, and as a package maintainer, and with other things Fedora as my employment has allowed.

Well, now, my employment will allow it a lot, as I've been hired by Red Hat full time to work on Fedora. Specifically, I'm going to work on bringing some sense to the whole "Cloud" thing – what that buzzword practically means for us as a project both now and in the future, and (critically) what we should do about it.

There is some seriously awesome cloud-related work going on in Fedora — many of the features for F18, for example — but there's no real overall vision for how this will all work together for us. So, I'm going to indulge in some strategic planning, starting with some basic questions about our stakeholders for cloud in Fedora, and working on a mission and vision within Fedora as a whole. I hope you'll join me in working on this.

If any of this sounds interesting, please come on over to the Cloud SIG, centered around the mailing list

https://lists.fedoraproject.org/mailman/listinfo/cloud

and pitch in. (Or post here, of course, but joining the SIG will have more impact.)

And if the words "strategic" and "planning" make your eyes glaze over, don't worry. This is going to be a doing thing, not just a talking thing. Computing as a whole is really at one of those big inflection points, and it's going to take a lot of action to bring us on over to the other side — where we've got a lot to contribute that the world shouldn't miss out on.

(Working on bringing the Cloud SIG Fedora wiki page up to reflect current activity is one of my low-hanging-fruit tasks as well, but I wanted to get this intro out there first thing.)

And if you want to talk about any of this in any medium _beyond_ the Cloud SIG, please feel free. My inbox is open, I'll be seeing a lot of you at various conferences, and if you're in the Boston/Cambridge/Somerville area, we have a lot of nice places serving local microbrews. (I will buy you a drink if you can refrain from using the word "nebulous" in a conversation about cloud.)

[Dan]

Ok. Welcome back, Matt. It's been a while since we've seen you in these parts.

But as you might have guessed if you've been reading the forum for a while, there are a lot of misunderstandings, misconceptions and down right concern for what the "cloud" means to fedora, and more importantly, fedora users, and what, if any, benefits it may have for both or either. In short ...

First ... you've got to sell us on the idea. And that means, a brief overview.

So ... Whatiz, howiz, wheniz and whuzzinit for us?


<....>

[mattdm]

Originally Posted by Dan

But as you might have guessed if you've been reading the forum for a while, there are a lot of misunderstandings, misconceptions and down right concern for what the "cloud" means to fedora, and more importantly, fedora users, and what, if any, benefits it may have for both or either. In short ...

First ... you've got to sell us on the idea. And that means, a brief overview.

So ... Whatiz, howiz, wheniz and whuzzinit for us?

Well, actually I think you've captured an important part of it here. The first step is defining exactly that. Specifically:

• What does cloud mean to Fedora and to Fedora users?
• How can we benefit from cloud computing technology, as a distribution and as users of that distribution?
• What are the risks of cloud computing, and how can we avoid, mitigate, or even benefit from them?
• How can we bring our mission, vision, and values into the upcoming computing landscape?

I have a lot of ideas (and a lot of input) already, but this is a big topic with wide-reaching implications, so I very much want to make it an inclusive, community-wide effort.

"Cloud" is a buzzword, but it isn't just a buzzword. It describes a shift that is really happening, right now. We can lead, we can struggle to keep up, or we can become irrelevant.

[ocratato]

One of the main concerns that spawned the open source movement was that our data would be locked behind some proprietary software; that even though the data was on our equipment we were still potentially held hostage by the software vendors.

I suspect that the idea of moving our data to a cloud raises similar (and even stronger) concerns. Our data is now to be held on someone elses equipment, and potentially processed with software that have no control over whatsoever.

I can see where a cloud service would be useful. Running up a set of virtual machines to solve a large genetic algorithm problem for example, or a business that needs to ramp up their capability for a temporary scenario are obvious and valid uses. Using cloud services as your primary computing platform, not so much

In the near term we also need to be mindful of the costs associated with a network oriented solution. For many of us network costs outweigh data storage and even processing costs. And that is where there is a reliable and fast network - which is not the case for most of the world.

However, from an ordinary user's perspective, I'd like to see a platform where processing and storage requirements that exceeded my local capability were automatically offloaded to "the cloud" in a totally transparent manner.

We also need a variety of definitions for what the cloud might be. It might be a local server, or server farm, on my network; it might be a set of servers shared by a community, scattered across the neighbourhood; or it might be some commercial offering.

This is going to be an interest trip.

[billybob linux]

In my view “ The Cloud” is simply the re-branding or re-making of an old computing concept, with its origins going back to the days of the mainframe. As the mainframe was expensive then subsequently computer time was expensive. This resulted in “Timeshare” using terminals to access the mainframe across a network as a shared hardware and software resource. One of the challenges being to derive methods to optimize and organize the most efficient use of Timeshare.

With “Cloud” computing I think the only thing that has changed is that data centres have replaced the mainframe. So is “Cloud” just “Timeshare”, if we agree that it is then “Timeshare” might be a better term to understand the concept of the “Cloud”. Perhaps you could argue that the difference between the system of “Timeshare” of a mainframe and the “Cloud” is one of scale and scale-ability, the same basic idea (relatively simple hardware, such as dumb terminals to access more computing power) but on a potentially huge scale.

However as others have pointed out the advantages (economy of scale, flexibility of capacity, etc) I would be asking if there are standards in place, will vendor lock-in be an issue, the potential for access to data to be restricted if new terms by providers can’t be agreed upon, quality of service guarantees, security of data, how the data might be used. As more data is stored in the “Cloud” how long will it be before governments will start demanding access to it, and what about the different standards of data use and storage around the world ?

[DBelton]

About the only time I would even think about using cloud services is if I were in control of the servers.

As already stated above, I see it as a major security risk to hand my data over to unknown servers to be processed by unknown software, and possibly ending up in unknown locations.

I really don't see the point in it, either. At least not for individuals. Large companies might benefit some by consolidating their hardware and software, making upgrades and maintenance simpler, but for the average user, not a single benefit, and numerous drawbacks.

[Bazu135]

I'll add my voice to the sceptics' camp too. I can see how the Cloud would be beneficial for large organisations, universities etc, but I see absolutely no benefit for me as a home user (quite the opposite; I expect it would inconvenience me), nor for the company I work for - our offices include 20 workstations, and at any given time a maximum fifteen of them are in use; some days there are only 7-8 in use. This therefore means we don't have particularly large amounts of network traffic nor particularly large volumes of data to store - not even 1TB at present. A large quantity of that data is sensitive, though, and I've not seen anything that convinces me our data would be more secure stored on someone else's servers.

With all that said, I do like the idea of the Cloud, and am open to being shown how it could work for SOHO-level users - my current mindset on the topic falls into "eyeing it suspiciously and poking it with a stick" though

[mattdm]

Originally Posted by billybob linux

In my view “ The Cloud” is simply the re-branding or re-making of an old computing concept, with its origins going back to the days of the mainframe.

I agree that this is a swing of the large pendulum that goes between centralization and decentralization in computing. It's not just a rehash of time sharing, though. Scale, as you say, is an important aspect, but there's more. There's a recent NIST definition, which puts forth these essential characteristics:

• On-demand self-service
• Broad network access
• Resource pooling
• Rapid elasticity
• Measured service

There's something different here than just scale and beyond just mainframe→datacenter. The whole document above worth reading (it's really less than 2 pages long), but I also recommend the "OSSM" definition (outlined in this Rackspace blog article:

• On-demand: the server is already setup and ready to be deployed
• Self-service: you chooses what you want, when you want it
• Scalable: You can choose how much you want and ramp up if necessary
• Measurable: there’s metering and reporting so you know you are getting what you pay for

I guess we could quibble about how this is or isn't different from timesharing, but either way, that shift from mainframe to minicomputers to PCs was a shake-up, and there's plenty of reason to believe that this will be too.

However as others have pointed out the advantages (economy of scale, flexibility of capacity, etc) I would be asking if there are standards in place, will vendor lock-in be an issue, the potential for access to data to be restricted if new terms by providers can’t be agreed upon, quality of service guarantees, security of data, how the data might be used. As more data is stored in the “Cloud” how long will it be before governments will start demanding access to it, and what about the different standards of data use and storage around the world ?

Yes, these are all huge issues and we need to make sure the open source and free software world has a meaningful voice in that future. Some people are already arguing that free software is now irrelevant, and that open data and open apis take its place. I'm not so sure that's the case.

---------- Post added at 02:35 PM ---------- Previous post was at 02:28 PM ----------

Originally Posted by DBelton

About the only time I would even think about using cloud services is if I were in control of the servers.

As already stated above, I see it as a major security risk to hand my data over to unknown servers to be processed by unknown software, and possibly ending up in unknown locations.

I really don't see the point in it, either. At least not for individuals. Large companies might benefit some by consolidating their hardware and software, making upgrades and maintenance simpler, but for the average user, not a single benefit, and numerous drawbacks.

Consumer-level benefits are probably more in the software-as-a-service aspect of the cloud. What to do about that from a Linux distribution's perspective may actually be the hardest aspect — doing things like making ideal Fedora images to run in EC2 and packaging up OpenStack are a lot of work, but they're the kind of work we're used to and have well-oiled processes for handling.

As for data security: definitely a risk, although at least from a small business point of view (and probably also for non-technical users) it's easy to overestimate the security of a local but networked computing system.

---------- Post added at 02:46 PM ---------- Previous post was at 02:35 PM ----------

Originally Posted by Bazu135

I'll add my voice to the sceptics' camp too. I can see how the Cloud would be beneficial for large organisations, universities etc, but I see absolutely no benefit for me as a home user (quite the opposite; I expect it would inconvenience me), nor for the company I work for - our offices include 20 workstations, and at any given time a maximum fifteen of them are in use; some days there are only 7-8 in use. This therefore means we don't have particularly large amounts of network traffic nor particularly large volumes of data to store - not even 1TB at present. A large quantity of that data is sensitive, though, and I've not seen anything that convinces me our data would be more secure stored on someone else's servers.

Actually, I think small companies have a lot to benefit. Cloud computing is the hot thing for tech startups, where low initial investment and high scalability are killer. But I think your example also provides something to think about — you've got 20 workstations, but are using less than half of the capacity. In fact, since you're probably not 24/7, you may be using as little as an eighth of what you've got sitting around. And since your network and data needs are small, the cost of getting things in and out is correspondingly small.

From a business perspective, data security is a problem of risk management, for legal reasons, business continuity reasons, and of course reputation. These things can all be managed. From a compliance point of view, having a legal agreement with a big cloud provider might even be preferable.

Now, from a Fedora perspective, maybe there's some particular things we can do to help address the data security concern.

[DBelton]

With linux being a free and open source OS, and numerous applications available as free open source, I don't see any benefit to the software-as-a-service aspect of the cloud.

Also, from a data storage standpoint, I don't see any benefit either, and would actually cost more than keeping the data locally.

For example, my hard drives average life span has been 8 years. The last 2 TB hard drive I purchased was a WD Caviar black, and costed me US$129.00.

So, calculating the cost over the average lifetime of my drives, that 2TB of storage costs me US$1.34 per month.

Can I get 2TB of cloud storage for $1.34 per month?

That is not even factoring the cost of the bandwidth to transfer data back and forth over the course of those 8 years. Local network is much cheaper than internet data transfers.

Plus, do I have a 24/7/365 data access guarantee? If my data is local, I can be assured that I can access my data anytime I wish. If it's in the cloud, other factors could limit data access, like internet being down, cloud server down. a storm comes through and takes out the communications.

With many ISP's going to bandwidth caps on internet data transfers, cloud storage is looking less and less usable. I have no bandwidth cap on local network transfers.

[Dan]

My whole understanding of this "cloud" thing is a little cloudy itself. That being said, I would hazard a guess that the following list of things I have used at one time or another might qualify as a "cloud" experience.

• Search engine (any)
• Wikipedia
• Google maps
• Google docs
• Blogger
• Pidgin/empathy/etc chat
• US Census data
• Assorted LEN stats
• NFIRS data
• CAMEO/MARPLOT/ALOHA data
• NOAA data/mapping
• US-CERT data
• Various software Repositories
• My own web site/pages
• Fedora Forums

But it seems to me that with the exception of Google Docs for active collaboration, most of the above would just as easily fall into the category of a normal web surfing/research experience, and not specifically a "cloud" defined function.

So ... my definition of the "cloud" is obviously erroneous. No?

[ocratato]

This discussion is becoming polarised. On one side Matt is concentrating on how to use Fedora as the basis for a cloud service. On the other Dan, and perhaps most of the rest of us, are looking at it from the user's perspective.

I suspect the problem many of have is that the cloud appears to be a solution looking for a problem. The open source nature of Fedora and the applications that run on it negate the benefits of most "software-as-a-service" aspects. The cost of network access puts a big dampener on the hardware side. There doesn't seem to be much left for us ordinary home or small office users - the core users of Fedora.

As I said before, there are good cases to be made for the cloud if you want to quickly ramp up the capability of a business, or farm out some large calculation. However these seem to be problems that are already solved. I was using a Fedora image on a virtual machine provisioned by a government department as an experiment in cloud computing just a few days ago - it was no different than using any other remotely accessed server.

Perhaps we need to think about how we can improve the experience of the ordinary Fedora user. What benefits could a cloud like service provide? The big one that comes to my mind is collaboration - allowing a team to work together seamlessly would be an impressive achievement.

[Dan]

... Perhaps we need to think about how we can improve the experience of the ordinary Fedora user. What benefits could a cloud like service provide? The big one that comes to my mind is collaboration - allowing a team to work together seamlessly would be an impressive achievement.

Indeed, it would. But we're talking the cloud here ... not mandatory ego control. <....>

[stevea]

Hmmm when Iook at that list of benefits ...
On-demand: the server is already setup and ready to be deployed
Self-service: you chooses what you want, when you want it
Scalable: You can choose how much you want and ramp up if necessary
Measurable: there’s metering and reporting so you know you are getting what you pay for
or
On-demand self-service
Broad network access
Resource pooling
Rapid elasticity
Measured service

Then consider a home/soho system ...
- on demand is a one time setup cost, modest negative.
- Self-service or resource pooling does make some sense wrt potential cost.
- Broadnetwork access is a don't care for most not running a high volume website
- Measurable is a don't care too.
- Scalable could be a real advantage, but who needs that much scalability ? Most ppl can't saturate a dual core processor for more than a few minutes a day.

Probably more important that any of the above to end-users, the cloud potentially give 'anywhere' access to your data&apps.

On the downside
- huge concerns about data security, both in transit and in the cloud storage.
- all transactions have a network cost.
- can't support gaming vid performance (maybe in the future), nor custom hardware.
- Who want's to play & experiment on a system that you are paying a rate for access and use ? I don't mind running a benchmark or building an odd kernel just for the experience. It costs me a fraction of a cent for power, as I already own excess capacity.

firefox bookmark sync, and dropbox are great, useful, simple cloudish ideas, but who would expect real privacy ?

[DJRavine]

As with a lot of the views above, I am very sceptical on off-loading processing/storage to unknown WAN servers due to security factors. The 'cloud' idea is very appealing though. If functionality was available to offload processing/storage to a LAN server, this could be very appealing. This would not add to the current WAN network load for countries like Australia which already have a choked network distribution, not to mention the standard 'GB per month download' limits.

As many people these days have their own LAN servers (DLNA media servers or w/e), being able to upgrade a primary server in the LAN with more RAM and CPU capabilities could be a real attraction. Even if its all about configuration, if you want to use a WAN network server to offload processing to help render CGI or the like.

A very interesting topic nonetheless.

[stevea]

Originally Posted by DJRavine

As with a lot of the views above, I am very sceptical on off-loading processing/storage to unknown WAN servers due to security factors

I'm told the post-modern viewpoint is that there is no privacy or security and you'd better get used to the notion. OTOH I hear this from ppl who find it necessary to twitter a report on every bodily function.