A Look At Hidden Costs In Cloud Solutions
Cloud computing has real business benefits that can help the bottom line of most organizations. However, you may have heard about (or directly experienced) cases of sticker shock where actual costs were higher than expectations.
“These costs aren’t really hidden, of course: it’s more that they’re overlooked, misunderstood, or underestimated.” He says, as he examines and identifies these commonly overlooked costs in cloud based solutions.
Hidden Cost #1: Dimensions of Pricing
According to David the #1 source of surprises is not taking into account the various dimensions the provider services are metered. Every service utilized adds more facets the offering can be metered in terms of bandwidth, storage, transaction costs, service fees etc.
In effect, everything in the cloud is cheap but every kind of service represents an additional level of charge. To make it worse, as new features and services are added to the platform the number of billing considerations continues to increase.
Hidden Cost #2: Bandwidth
”Bandwidth is often overlooked or underappreciated in estimating cloud computing charges.” he says, He suggests that we model the bandwidth usage using tools such as Fiddler to give an give a ballpark estimate based on key usage scenarios. One could also throttle bandwidth overages or model the architecture to provide the path of least traffic given any usage scenario.
Hidden Cost #3: Leaving the Faucet Running
He suggests we review the usage charges and billing often to avoid surprises at the end
Leaving an application deployed that you forgot about is a surefire way to get a surprising bill. Once you put applications or data into the cloud, they continue to cost you money, month after month, until such time as you remove them. It’s very easy to put something in the cloud and forget about it.
Hidden Cost #4: Compute Charges Are Not Based on Usage
Hidden Cost #6: A Suspended Application is a Billable Application
He emphasizes that if the application is not used or suspended it does not mean the billing charges do not apply. He urges users to check the billing policies.
Since the general message of cloud computing is consumption-based pricing, some people assume their hourly compute charges are based on how much their application is used. It’s not the case: hourly charges for compute time do not work that way in Windows Azure.
Hidden Cost #5: Staging Costs the Same as Production
Many have mistakenly concluded that only Production is billed for when in fact Production and Staging are both charged for, and at the same rates.
Use Staging as a temporary area and set policies that anything deployed there must be promoted to Production or shut down within a certain amount of time. Give someone the job of checking for forgotten Staging deployments and deleting them—or even better, automate this process.
Hidden Cost #7: Seeing Double
[Y]ou need a minimum of 2 servers per farm if you want the Windows Azure SLA to be upheld, which boils down to 3 9’s of availability. If you’re not aware of this, your estimates of hosting costs could be off by 100%!
Hidden Cost #8: Polling
Polling data in the cloud is a costly activity and incurs transaction fees. Very soon the costs could add up based on the quantity of polling. He suggests
Either find an alternative to polling, or do your polling in a way that is cost-efficient. There is an efficient way to implement polling using an algorithm that varies the sleep time between polls based on whether any data has been seen recently.
Hidden Cost #9: Unwanted Traffic and Denial of Service Attacks
He warns that unintended traffic in the form of DOS attacks or spiders etc. could increase traffic in unexpected ways. he suggests the best way to deal with such unintended charges is to audit the security of the application and provide measures of controls such as CAPTCHA’s.
If your application is hosted in the cloud, you may find it is being accessed by more than your intended user base. That can include curious or accidental web users, search engine spiders, and openly hostile denial of service attacks by hackers or competitors. What happens to your bandwidth charges if your web site or storage assets are being constantly accessed by a bot?
Hidden Cost #10: Management
Finally he states that as a result of all of these factors clouds come with an inherent cost to manage these services for efficiency in usage and consequently billing.
Regularly monitor the health of your applications. Regularly monitor your billing. Regularly review whether what’s in the cloud still needs to be in the cloud. Regularly monitor the amount of load on your applications. Adjust the size of your deployments to match load.
The cloud’s marvelous IT dollar efficiency is based on adjusting deployment larger or smaller to fit demand. This only works if you regularly perform monitoring and adjustment.
He concludes his article saying that its very possible that one might not get things right the first time around and to expect some experimentation; possibly get a assessment of the infrastructure and seek the guidance from experts.
Cloud computing is too valuable to pass by and too important to remain a diamond in the rough.
Be sure to check out the original post and do enrich the comments section with your experiences.
An metering approach based on ABC has advantages across many mgmt domains including performance, capacity, security, reliability/availability,.....
We should also push for standardization of how metering information is made available in particular we need it at the point of interaction which in the real world is generally the norm
This technology allows GridGain customers to safely over-provision yet pay for the actual use only. This removes the cost penalty for such frequent over-provisioning scenarios such as planned over-capacity for anticipated load spikes, staging and QA environments, hot standbys, scheduled builds, disaster recovery sites, etc.
GridGain = Compute + Data + Grid
Re: GridGain 3.0
Remember you don't manage cost you manage its cause. Cause is specific to the customers goal and generally inaccessible to generic runtime frameworks.
Metering points for remote service integration: opencore.jinspired.com/?p=1583
Meter plugins for local runtime integration: williamlouth.wordpress.com/2010/04/03/if-it-can...
Re: GridGain 3.0
1. What is the incentive for us (vendor) to even look at it? We have something that is 100% integrated into our system/billing and no standard will ever provide anything close to what we specifically need.
2. *ANY* standards in cloud industry are pie in the sky right now. I'm saying it with pretty dead-on authority...
Idea is good, but I think we are 3-5 years away from even considering it at this point...
Re: GridGain 3.0
We are going to move to a (composite) service supply chain. Whilst its nice to make local cost & performance optimization at one service point the bigger savings will be made in managing the complete supply chain cost & value models.
Re: GridGain 3.0
This is the problem with a real lack of competition and when the market is not sufficiently mature to protect itself and effectively manage & audit a suppliers charges. That said it is not an all or nothing choice. You can still have you billing but still make available execution context (thread generally) specific meters.
Metering != Billing, Metering > Billing
Re: GridGain 3.0
Quincy: Fair Scheduling for Distributed Computing Clusters
"We introduce a powerful and flexible new framework for scheduling concurrent distributed obs with fine-grain resource sharing. The scheduling problem is mapped to a graph datastructure, where edge weights and capacities encode the competing demands of data locality, fairness, and starvation-freedom, and a standard solver computes the optimal online schedule according to a global cost model."
OpenCore can fully support such a solution today that takes into costs associated with metered services and resources consumed directly and indirectly as a result of them. Maybe it is still too far off but hey we need to live up to the company name.
Re: GridGain 3.0
Again, these ideas are great and some will come to fruition at some day. But we need to have something today and most grid and cloud standards have been an object failure. I'm on JSR-107 ("Data Grid" JSR). It's been... almost 10 years (if I'm not mistaken) and nothing is done. Metering and billing is a way more generalized than that.
I wish we could have an industry wide standard for that - we'd jump on it. Until then - we'll provide our own technology.
GridGain = Compute + Data + Cloud
Shane Hastie Aug 30, 2014
Roy Rapoport Aug 28, 2014