Introduction .uk



Enterprise Cloud Service Requirements and SolutionsVersion 1.2IntroductionLast year we started work on a pathfinder project to provide a possible cloud service for the non-research needs of the university. During that project we purchased some server hardware for a prototype service (10 servers) and learnt what might be involved to build an Openstack service and what might be involved with adopting a managed Openstack service from a provider and a vCloud service from VMware. We now want to build on that earlier work to deliver a cloud service. As part of this project we have undertaken a requirements gathering exercise by consulting with colleagues across Information Services and with representatives from the colleges. This document summarises the requirements and the possible solutions available with the pros and cons of each.This service offering needs to take into account the existing virtual hosting service. Any new service needs to be sufficiently differentiated from that to make it worthwhile but it should also complement and not compete with rmation Services requirementsThe requirements for a cloud service in IS mainly come from the Applications Division and Edina but the Library have also expressed an interest.Application Division requirementsThe Applications Division mainly require a development platform to allow them to speed up the development/test cycle. They would like the ability to rapidly spin up new virtual machines for development, test them and then repeat the process. The current virtual hosting service would remain as the platform for long term hosting of services but it doesn't provide the rapid turn around of virtual machines that is necessary for an agile development cycle.Applications don't currently require an elastic growth service and have not requested one. However, we think that if such a service were provided that they may use this service in preference to the existing virtual hosting service.Edina requirementsEdina's services often have long periods where they are not being heavily used but at other periods they can come under much more intense use. An example of this kind of service would be Digimap for Schools, which is used intensively for a few weeks or months in the year but otherwise is lightly loaded. Edina would like a facility to elastically grow server resource during these intensive periods.Edina also require an API-based programmatic stand-up/teardown for agile development and a very strong desire for an ObjectStore. Library requirementsThe library have no immediate requirement for a production burst capacity service but they are interested in an environment to support agile development similar to the Applications Division.Schools and CollegesSchools and Colleges have a requirement for a cloud service for both research and non-research computing needs. The Openstack service being developed to support the Eddie platform will not completely satisfy these requirements. What was clear from the discussion with them was that they would like a consistent, supported, single offering from IS even if underlying that there were multiple solutions providing that service.Medicine and Veterinary MedicineMVM researchers need quick setup and short term hosting platforms that are responsive and flexible. They don't know what compute, storage and power requirements might be but these are probably more aligned to the Research Services Openstack service.They do currently use the Virtual Hosting service for long term hosting but have trouble accessing storage from Datastore (this will be followed up through the college support channels, as we don't believe there should be any barrier to this). They would like to have a rapid-deployment development platform to develop applications and test them at low cost and that would allow them to then transfer them simply to the virtual hosting service for long-term deployment.They have a current problem in the college with managing projects where people are “rolling their own” cloud-based services using public cloud services such as Amazon Web Services (AWS). They have major concerns regarding the data and security governance issues that this causes and the college support teams feel there is no control over what is being done. They would prefer to have a coherent strategy and have things hosted in-house. IS will also be investigating the use of public cloud services through another public cloud pathfinder project.Science and EngineeringScience and Engineering have similar requirements to MVM and have similar concerns about data and security governance.CSE still wants an enterprise service where they get better guarantees of service as opposed to a research platform where SLAs might not be so guaranteed.SCE may also have a small requirement to support teaching projects e.g. Informatics may like to use a cloud service to support third-year undergraduate teaching projects. This would only likely be taken up if the service had a free-at-the-point-of-use default service level.CSE might want a free base allocation for short-lived VMs not being charged but if they run for more than a few days then they would be charged.Humanities and Social ScienceHSS have no preference for a private versus a public cloud hosted service but whatever was provided should be a “badged” University of Edinburgh service supported locally and provided in a coherent manner.They want a service that has the ease and speed of use of AWS to fire up a virtual machine. HSS demand on this is likely to be very low but they would want to support researchers and rapid development. They might also want to prototype stuff and then migrate things on to the virtual hosting environment.HSS suggest things might be setup initially as pilot and not need strong SLA but this might develop something that needs it. This brings up the question of how easy it is to migrate it from one service to an another.HSS would prefer not to top-slice but do cost-recovery. HSS might cover the cost as a college if lots of schools wanted to use but otherwise not.Requirements summaryThere appear to be three use cases:an enterprise service guaranteed service available 24x7 that would be charged fora quick turn-around development platform with a small baseline quota of free service and otherwise charged-for servicea research platform for quick throughput but without the same guaranteed levels of service guarantee for the above options – this is the service likely to be met by the Research Services Openstack platform providing support for Hadoop and Spark (Sahara).The service should nevertheless appear as a single IS-managed service with “service catalogue” management envelope to make it appear as a coherent service offering even though there may be two different levels of service guarantee depending on the particular use case.Possible solutionsTaking into account that Research Services are rolling their own Openstack service from source to provide services on top of the Eddie platform some but not all of the above requirements will be met by that service. There is still a remaining use case from both inside IS and from colleges tha would require a platform with more stable guaranteed levels of service (not running bleeding-edge code) to support development work and for elastic loads. Possible solutions to augment the Research Services offering would be:Roll our own Openstack service in a similar way to Research Servicescon: would require very high FTE staff to setup and maintain the service (1 FTE); would not provide a differentiated service from that provided by Research Services; would not guarantee high levels of service;pro: would provide a consistent platform; would meet the elastic demand requirement; would provide a self-service rapid deployment development platformcost: ?30K for additional servers to dual site the service; ?15K for storage; ?10K for HBAs; 2 FTEBuy a managed Openstack service from e.g. Mirantispro: would provide a fully supported solution with low FTE setup and ongoing FTE support cost ; would meet the elastic demand requirement; would provide a self-service rapid deployment development platformcon: would be very expensive to purchase and would require large on-going recurrent budget to supportcost: ?360K for initial setup and support; ?40K for recurrent support; ?30K for additional servers to dual site the service; ?15K for storage; ?10K for HBAs; 0.5 FTEBuy the vCloud suite of software from VMware to augment our virtual hosting servicepro: would integrate well with our existing virtual hosting platform and be managed through our existing vCentre; would require moderate FTE requirement to setup and support; would meet the elastic demand requirement; would provide a self-service rapid deployment development platformcon: would be extremely costly; would be costly in FTE during initial setup; an Enterprise Licence Agreement would probably be required to get a reasonable discount level to even approach it making it affordable and this requires us to make an additional $150K of business to VMware; our existing budget is not sufficient to meet the requirement even with consolidating multiple budgets; an ELA only lasts three years with a possible single year extension and we would be put in a position in year four or five where we would need to negotiate a new ELA and provide another $150K of new business, which is not something we could guarantee to do; would be running on a proprietary platform with vendor lock-incost: ?410K for three year ELA; ?30K for additional servers to dual site the service; ?10K for HBAs; ?15K for additional storage; 1 FTEUse the VMware-branded Openstack software stackpro: would be very quick to setup needing minimum FTE; would require moderate FTE for on-going support; could use existing hardware bought for previous pathfinder project and build on it to provide a service with sufficient capacity to meet initial demand; is free to deploy the software but does require the hosts to have Enterprise+ VMware licences, but these costs are well known and constrained and can be met by the existing capital budget and ongoing recurrent support uplift; would provide a similar, consistent platform as Research Services but with higher levels of service guarantee; may provide a simpler platform to migrate services between cloud and virtual hosting service; allows us to meet existing needs without boxing us into a proprietary system; would meet the elastic demand requirement; would provide a self-service rapid deployment development platformcon: requires more staff effort than some of the other options and requires us to buy more Enterprise+ licences for VMwarecost: ?46K for Enterprise+ licences for 20 CPUs for existing servers (single site) or ?90K for 40 CPUs to dual site the service; ?30K for additional servers to dual site the service; ?15K for storage; ?10K for HBAs; ?15K recurrent support budget; 0.75 FTE.Buy an allocation of time from a public cloud providerpro: would be minimal FTE requirement to provide; would be provided from an existing supplier providing quick turnaround and ease of use e.g. AWScon: the funding for this project is capital money from the IS 5 year roadmap funding plan and this money could not be used to buy a service from a public cloud service provider; it is therefore not clear how a service could be funded but it might be done through up-front purchasing and then cost-recovery from schools; increased institutional cost might be an issue; integration could be an issue/lack of knowledge could be an issue; we would still need to provide local support for this service and it is unclear who would provide that support and how the cost would be met; the purchasing framework through ArcusGlobal for AWS has limits on the use for the discount of data egress fees that limit it to only academic and research use not including web hosting and MOOCs and the total discount is capped at 15% – most of the services we would use this type of service for would likely be web services; Amazon pricing is opaque (the Addendum contains pricing documents recently used as part of Edina’s purchase of service from AWS – our only real experience so far of buying services – we have been unable to find any real life existing user data from people within the university but as a rough guide a 100 instances of 4 vCPU 16GB VMs would cost approx $17.5K (?15K) per month assuming 100% utilisation and no data egress fees)cost: unclear what budget would be available – example costs are given in the Appendix and above; date egrees fees would probably apply for most use cases; 0.25 FTE; Use Research Services Openstack cloud service to meet all requirementspro: would be a consistent single, service platform; would provide a self-service rapid deployment development platformcon: existing service is not aimed to meet 24x7 SLA; would be running bleeding-edge code and may not be reliable; may be harder to migrate services to the virtual hosting service or to the public cloud; would not meet the elastic demand requirementcost: ?0; 0 FTEBuy a vCloud service that was scaled to the budget we have available for this projectpro: would be affordablecon: would not be of sufficient capacity to meet likely demand (estimate of CPU and memory capacity); would be costly in FTE during initial setup; would require a higher on-going support cost and budget than other options; would be running on a proprietary platform with vendor lock-in; funding for support beyond the second year is not currently budgeted for.cost: ?100K this year + ?100K next year and nothing thereafter; 1 FTEDo nothingpro: no costcon: does not meet any of the requirementscost: ?0; 0 FTEConclusionThe option that meets most of the requirements and within the existing capital and recurrent budgets that also requires the least initial and ongoing FTE support costs is the VMware branded Openstack service. We recommend building such a service on the existing hardware. This would need to be augmented with VMware Enterprise+ licences, some HBAs to connect them to the SAN and some additional storage. This can all be met within the ?100K budget allocated to the project with some additional funding from existing budgets.AddendumAmazon pricingAmazon S3 Pricing Pay only for what you use. There is no minimum fee. Estimate your monthly bill using the AWS Simple Monthly Calculator. We charge less where our costs are less, and prices are based on the location of your Amazon S3 bucket. Storage Pricing Standard Storage Reduced Redundancy Storage Glacier Storage First 1 TB / month ?0.054 per GB ?0.043 per GB ?0.007 per GB Next 49 TB / month ?0.047 per GB ?0.038 per GB ?0.007 per GB Next 450 TB / month ?0.038 per GB ?0.030 per GB ?0.007 per GB Next 500 TB / month ?0.035 per GB ?0.028 per GB ?0.007 per GB Next 4000 TB / month ?0.032 per GB ?0.026 per GB ?0.007 per GB Over 5000 TB / month ?0.027 per GB ?0.022 per GB ?0.007 per GB Request Pricing Pricing PUT, COPY, POST, or LIST Requests ?0.003 per 1,000 requests Glacier Archive and Restore Requests ?0.035 per 1,000 requests Delete Requests ?Free ? per 1,000 requests GET and all other Requests ?0.003 per 10,000 requests Glacier Data Restores ?Free ?? ? No charge for delete requests of Standard or RRS objects. For objects that are archived to Glacier, there is a pro-rated charge of $0.033 per gigabyte for objects deleted prior to 90 days. Learn more.?? Glacier is designed with the expectation that restores are infrequent and unusual, and data will be stored for extended periods of time. You can restore up to 5% of your average monthly Glacier storage (pro-rated daily) for free each month. If you choose to restore more than this amount of data in a month, you are charged a restore fee starting at $0.011 per gigabyte. Learn more. Data Transfer Pricing The pricing below is based on data transferred "in" to and "out" of Amazon S3. Pricing Data Transfer IN To Amazon S3 All data transfer in ?0.000 per GB Data Transfer OUT From Amazon S3 To Amazon EC2 in the same region ?0.000 per GB Another AWS Region or Amazon CloudFront ?0.013 per GB Data Transfer OUT From Amazon S3 To Internet First 1 GB / month ?0.000 per GB Up to 10 TB / month ?0.076 per GB Next 40 TB / month ?0.057 per GB Next 100 TB / month ?0.044 per GB Next 350 TB / month ?0.032 per GB Next 524 TB / month Contact us Next 4 PB / month Contact us Greater than 5 PB / month Contact us Storage and bandwidth size includes all file overhead.Rate tiers take into account your aggregate usage for Data Transfer Out to the Internet across Amazon EC2, Amazon S3, Amazon Glacier, Amazon RDS, Amazon SimpleDB, Amazon SQS, Amazon SNS, Amazon DynamoDB, and AWS Storage Gateway. Amazon Glacier Pay only for what you use. There is no minimum fee. Storage Pricing Request Pricing ? Glacier is designed with the expectation that retrievals are infrequent and unusual, and data will be stored for extended periods of time. You can retrieve up to 5% of your average monthly storage (pro-rated daily) for free each month. If you choose to retrieve more than this amount of data in a month, you are charged a retrieval fee starting at $0.011 per gigabyte. Learn more. In addition, there is a pro-rated charge of $0.033 per gigabyte for items deleted prior to 90 days. Learn more. Data Transfer Pricing The pricing below is based on data transferred "in" to and "out" of Amazon Glacier. ?0.007 per GB / month Pricing UPLOAD and RETRIEVAL Requests ?0.035 per 1,000 requests LISTVAULTS, GETJOBOUTPUT, DELETE and all other Requests free Data Retrievals free? Pricing Data Transfer IN To Amazon Glacier All data transfer in ?0.000 per GB Data Transfer OUT From Amazon Glacier To Amazon EC2 in the same region ?0.000 per GB Another AWS Region ?0.013 per GB Data Transfer OUT From Amazon Glacier To Internet First 1 GB / month ?0.000 per GB Up to 10 TB / month ?0.076 per GB Next 40 TB / month ?0.057 per GB Next 100 TB / month ?0.044 per GB Next 350 TB / month ?0.032 per GB Next 524 TB / month Contact us Next 4 PB / month Contact us Greater than 5 PB / month Contact us Redshift On-Demand Pricing On-Demand pricing lets you pay for capacity by the hour. Your data warehouse cluster requires no long-term commitments or upfront costs. This frees you from the cost and complexity of planning and purchasing data warehouse capacity ahead of your needs. On-Demand pricing lets you pay as you go. vCPU ECU Memory (GiB) Storage I/O Price DW1 - Dense Storage dw1.xlarge 2 4.4 15 2TB HDD 0.30GB/s ?0.602 per Hour dw1.8xlarge 16 35 120 16TB HDD 2.40GB/s ?4.813 per Hour DW2 - Dense Compute dw2.large 2 7 15 0.16TB SSD 0.20GB/s ?0.190 per Hour dw2.8xlarge 32 104 244 2.56TB SSD 3.70GB/s ?3.547 per Hour Reserved Instance Pricing Reserved Instance pricing offers significant savings over On-Demand pricing. Reserved Instances are appropriate for steady-state production workloads. You commit to the entire duration of the Reserved Instance term. Using Reserved Instance pricing, you can save 41% with a 1-year term and 73% with a 3-year term over On-Demand pricing. 1 Year Reserved Instance Pricing 1yr Reserved Instance DW Node Class (On-Demand) Upfront Hourly dw1.xlarge ?1,583.333 ?0.215 per Hour dw1.8xlarge ?12,666.667 ?1.723 per Hour dw2.large ?475.000 ?0.070 per Hour dw2.8xlarge ?7,600.000 ?1.583 per Hour , 3 year Reserved Instance Pricing 3yr Reserved Instance DW Node Class (On-Demand) Upfront Hourly dw1.xlarge ?1,900.000 ?0.108 per Hour dw1.8xlarge ?15,200.000 ?0.861 per Hour dw2.large ?839.167 ?0.044 per Hour dw2.8xlarge ?13,426.667 ?0.709 per Hour Calculating Your Effective priced per TB per Year For On-Demand, the effective priced per TB per year is the hourly priced for the instance times the number of hours in a year divided by the number of TB per instance. Let's consider this in the context of US East (N. Virginia). In this case, using On Demand pricing, this works out to ?2357.900 per TB per year for both XL and 8XL nodes. For Reserved Instances, the effective priced per TB per year is the upfront payment plus the hourly payment times the number of hours in the term divided by the number of years in the term and the number of TB per node. Again using US East (N. Virginia), for 1 year Reserved Instances, this works out to ?1387.000 per TB per Year. For 3 year Reserved Instances, the effective priced is ?632.700 per TB per year. Backup Storage Backup storage is the storage associated with your automated and manual snapshots for your data warehouse. Increasing your backup retention period or taking additional snapshots increases the backup storage consumed by your data warehouse. There is no additional charge for backup storage up to 100% of your provisioned storage for an active data warehouse cluster. For example, if you have an active single XL node cluster with 2TB of storage, we will provide up to 2TB-Month of backup storage at no additional charge. Backup storage beyond the provisioned storage size and backups stored after your cluster is terminated are billed at standard Amazon S3 rates. Data Transfer Amazon Redshift does not charge for data transfer. In particular, you will not incur charges for data transferred between Amazon Redshift and Amazon S3 for backup, restore, load, or unload operations. However, if you run your Amazon Redshift clusters in Amazon VPC, you will see a charge from AWS at standard AWS Data Transfer rates for the data transferred over JDBC/ODBC to your Amazon Redshift cluster endpoint. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download