This is for
The Reference Model Chapter 3 focuses on cloud infrastructure abstractions. While these are generic abstractions they and the associated capabilities of the cloud infrastructure are specified for data centres, central office and colocation centres. The environmental conditions, facility and other constraints, and the variability of deployments on the edge are significantly different and, thus, require separate consideration.
It is unrealistic to expect that a private cloud can cost effectively meet the need of all loads, including peak loads and disaster recovery. It is for that reason that enterprises will implement a hybrid cloud. In a hybrid cloud deployment, at least two or more distinct cloud infrastructures are inter-connected together. In a multi-cloud the distinct cloud infrastructures of the hybrid cloud may be implemented using one or more technologies. The hybrid multi-cloud infrastructure has differences requiring different abstractions. These hybrid multi-clouds can be considered to be federated.
In the Reference Model Chapter 3, the cloud infrastructure is defined. The tenants are required to provide certain needed services (such as Load Balancer (LB), messaging). Thus, the VNF/CNFs incorporate different versions of the same services with the resultant issues related to an explosion of services, their integration and management complexities. To mitigate these issues, the Reference Model must specify the common services that every Telco cloud must support and thereby require workload developers to utilise these pre-specified services.
A generic Telco cloud is a hybrid multi-cloud or a Federated cloud that has deployments in large data centres, central offices or colocation facilities, and the edge. This chapter discusses the characteristics of Telco Edge and hybrid multi-cloud.
The GSMA whitepaper on "Operator Platform Concept Phase 1: Edge Cloud Computing" (January 2020) states, "Given the wide diversity of use cases that the operators will tasked to address, from healthcare to industrial IoT, it seems logical for operators to create a generic platform that can package the existing assets and capabilities (e.g., voice messaging, IP data services, billing, security, identity management, etc. ...) as well as the new ones that 5G makes available (e.g., Edge cloud, network slicing, etc.) in such a way as to create the necessary flexibility required by this new breed of enterprise customers."
Cloud computing has evolved and matured since 2010 when NIST published its definition of cloud computing, with its 5 essential characteristics, 3 service models and 4 deployment models.
The generic model for an enterprise cloud has to be "hybrid" with the special cases of purely private or public clouds as subsets of the generic hybrid cloud deployment model. In a hybrid cloud deployment, at least two or more distinct cloud infrastructures are inter-connected together.
Cloud deployments can be created using a variety of technologies (e.g., OpenStack, Kubernetes) and commercial technologies (e.g., VMware, AWS, Azure, etc.). A multi-cloud deployment can consist of the use of more than one technology.
A generic Telco cloud is a hybrid multi-cloud. A better designation would be a federation of clouds - a federated cloud:
In this section we will further explore the characteristics of the federated cloud architecture, and architecture building blocks that constitute the federated cloud. For example, Figure 8-1 shows a Telco Cloud that consists of 4 sub-clouds: Private on premise, Cloud Vendor provided on premise, Private outsourced (Commercial Cloud Provider such as a Hyperscaler Cloud Provider (HCP), and Public outsourced (see diagram below). Such an implementation of a Telco Cloud allows for mix'n'match of price points, flexibility in market positioning and time to market, capacity with the objective of attaining near "unlimited" capacity, scaling within a sub-cloud or through bursting across sub-clouds, access to "local" capacity near user base, and access to specialised services.
Figure 8-1: Example Hybrid Multi-Cloud Component Cloud
The Figure 8-2 presents a visualisation of a Telco operator cloud (or simply, Telco cloud) with clouds and cloud components distributed across Regional Data Centres, Metro locations (such as Central Office or a Colocation site) and at the Edge, that are interconnected using a partial mesh network. Please note that at the Regional centre level the interconnections are likely to be a "fuller" mesh while being a sparser mesh at the Edges.
Figure 8-2: Telco Cloud: Data Centre to Edge
The Telco Operator may own and/or have partnerships and network connections to utilize multiple Clouds for network services, IT workloads, and external subscribers. The types of the component clouds include:
In general, a Telco Cloud consists of multiple interconnected very large data centres that serve trans-continental areas (Regions). A Telco Cloud Region may connect to multiple regions of another Telco Cloud via large capacity networks. A Telco Cloud also consists of interconnected local/metro sites (multiple possible scenarios). A local site cloud may connect to multiple Regions within that Telco Cloud or another Telco Cloud. A Telco Cloud also consists of a large number of interconnected edge nodes where these edge nodes maybe impermanent. A Telco Cloud's Edge node may connect to multiple local sites within that Telco Cloud or another Telco Cloud; an Edge node may rarely connect to a Telco Cloud Region.
Table 8-1 captures the essential information about the types of deployments, and responsible parties for cloud artefacts.
Type | System Developer | System Maintenance | System Operated & Managed by | Location where Deployed | Primary Resource Consumption Models |
---|---|---|---|---|---|
Private (Internal Users) | Open Source | Self/Vendor | Self/Vendor | On Premise | Reserved, Dedicated |
Private | Vendor | HCP | Self/Vendor | Self/Vendor | On Premise | Reserved, Dedicated |
Public | Vendor | HCP | Self/Vendor | Self/Vendor | On Premise | Reserved, On Demand |
Private | HCP | Vendor | Vendor | Vendor Locations | Reserved, Dedicated |
Public (All Users) | HCP | Vendor | Vendor | Vendor Locations | On Demand, Reserved |
Table 8-1. Cloud Types and the Parties Responsible for Artefacts
Figure 8-3 shows a conceptual Telco Operator Platform Architecture. The Cloud Infrastructure Resources Layer exposes virtualised (including containerised) resources on the physical infrastructure resources and also consists of various virtualisation and management software (see details later in this chapter). The Cloud Platform Components Layer makes available both elementary and composite objects for use by application and service developers, and for use by Services during runtime. The Cloud Services Layer exposes the Services and Applications that are available to the Users; some of the Services and Applications may be sourced from or execute on other cloud platforms. Please note that while the architecture is shown as a set of layers, this is not an isolation mechanism and, thus, for example, Users may access the Cloud Infrastructure Resources directly without interacting with a Broker.
Figure 8-3: Conceptual Architecture of a Telco Operator Platform
The Cloud Services and the Cloud Resources Brokers provide value-added services in addition to the fundamental capabilities like service and resource discovery. These Brokers are critical for a multi-cloud environment to function and utilise cloud specific plugins to perform the necessary activities. These Brokers can, for example, provision and manage environments with resources and services for Machine Learning (ML) services, Augmented/Virtual Reality, or specific industries.
NOTE: Published Lakelse (removed from working materials)
8.2.4.2 Stereo-Typical Scenarios
NOTE: Published Lakelse (removed from working materials)
8.2.4.2 Stereo-Typical Scenarios
NOTE: Published Lakelse (removed from working materials)
The Communcations Service Provider (CSP) is both a provider and consumer of Cloud based services. When the CSP is actings as:
These two stances will drive differing approaches to how a CSP would look to manage how it interacts within a Multi-Cloud environment.
As a consumer of cloud to support internal Business operations and BSS/OSS focus is on meeting the applications needs of organisation which historically came with need to operate support the infrastructure needs of these. This resulted in split of CIO organisation into Delivery and Operations groups. At the same time that CIO application workloads have been moving to SaaS Cloud Providers, CTO Network Systems have been moving from running on dedicated infrastructure to being virtualised and IMS, 3GPP (4G & 5G) functions, IP Routers and Firewalls are being provided as VNFs and CNFs.
As outline in section "8.2.2 Telco Cloud" the result is that the future CSP "network" is a set of distributed NFVi's (Cloud Infrastructure) which will be connected to Cloud Providers and hence the "Hybrid Multi-Cloud" and the need for CSP to be able to support this is both inevitable and essential.
So as a consumer and provider of Cloud Services the CSP will continue to need to build and manage it own Cloud Infrastructure as well as provide network orchestration solution to management interconnectivity across its own and other Cloud Providers. The interactions for this are outlined in the "Multi-Cloud Interactions Model", however to realise this the CSP will need to adopt and sponsor a set of standards that are nessasary to support the interactions.
The follow summary table aims to identify what standards and/or technologies that are seen are applicable.
The criteria for inclusion are
NOTE: Where are the gaps and where is there need to focus and raise priority ?
Manage (Cloud) | Comment | ||||||||
---|---|---|---|---|---|---|---|---|---|
Scope | Standard / Technology | Type | Account | Connectivity | Resource | App / VNF / CNF | Transactions / Conversations | Stereotypes | |
All | OASIS TOSCA | Industry Consortium | N | Y | N | Y | N | OAOSIS TOSCA or ETSI Adoption ? | |
Apache JClouds | Open Source | N | N | Y | Y | N | API Brokerage | Provide coverage, but how broadly adopted this this ? | |
Canonical MAAS/JuJu | Vendor ? | N | N | Y | Y | N | Multi-Cloud Provisioning | Is this an Open Source project or a Vendor solution ? There are other alternates for Infrastructure Automation / Provisioning | |
Airship | N | N | Y | Y | N |
This section presents the characteristics and capabilities of different Edge cloud deployment locations, infrastructure, footprint, etc. Please note that in the literature many terms are used and, thus, this section includes a table that tries to map these different terms.
Telco Edge Cloud (TEC) deployment locations can be environmentally friendly such as indoors (offices, buildings, etc.) or environmentally challenged such as outdoors (near network radios, curb side, etc.) or environmentally harsh environments (factories, noise, chemical, heat and electromagnetic exposure, etc). Some of the more salient characteristics are captured in Table 8-2.
Facility Type | Environmental Characteristics | Capabilities | Physical Security | Implications | Deployment Locations | |
---|---|---|---|---|---|---|
Environmentally friendly | Indoors: typical commercial or residential structures | Protected Safe for common infrastructure | Easy access to continuous electric power High/Medium bandwidth Fixed and/or wireless network access | Controlled Access | Commoditised infrastructure with no or minimal need for hardening/ruggedisation Operational benefits for installation and maintenance | Indoor venues: homes, shops, offices, stationary and secure cabinets Data centers, central offices, co-location facilities, Vendor premises, Customer premises |
Environmentally challenged | Outdoors and/or exposed to environmentally harsh conditions | maybe unprotected Exposure to abnormal levels of noise, vibration, heat, chemical, electromagnetic pollution | May only have battery power Low/Medium bandwidth Fixed and/or mobile network access | No or minimal access control | Expensive ruggedisation Operationally complex | Example locations: curb side, near cellular radios, |
Table 8-2. TEC Deployment Location Characteristics & Capabilities
Commodity hardware is only suited for environmentally friendly environments. Commodity hardware have standardised designs and form factors. Cloud deployments in data centres typically use such commodity hardware with standardised configurations resulting in operational benefits for procurement, installation and ongoing operations.
In addition to the type of infrastructure hosted in data centre clouds, facilities with smaller sized infrastructure deployments, such as central offices or co-location facilities, may also host non-standard hardware designs including specialised components. The introduction of specialised hardware and custom configurations increases the cloud operations and management complexity.
At the edge, the infrastructure may further include ruggedised hardware for harsh environments and hardware with different form factors.
The Cloud Infrastructure Profiles section specifies two infrastructure profiles:
The Basic cloud infrastructure profile is intended for use by both IT and Network Function workloads that have low to medium network throughput requirements.
The High Performance cloud infrastructure profile is intended for use by applications that have high network throughput requirements (up to 50Gbps).
The High Performance profile can specify extensions for hardware offloading; please see Hardware Acceleration Abstraction. The Reference Model High Performance profile includes an initial set of High Performance profile extensions.
Based on the infrastructure deployed at the edge, Table 8-3 specifies the Infrastructure Profile features and requirements that would need to be relaxed.
Table 8-3. TEC Exceptions to Infrastructure Profile features and requirements
Reference | Feature | Description | As Specified in RM Chapter 05 | Exception for Edge | ||
---|---|---|---|---|---|---|
Basic Type | High Performance | Basic Type | High Performance | |||
infra.stg.cfg.003 | Storage with replication | N | Y | N | Optional | |
infra.stg.cfg.004 | Storage with encryption | Y | Y | N | Optional | |
infra.hw.cpu.cfg.001 | Minimum Number of CPU sockets | This determines the minimum number of CPU sockets within each host | 2 | 2 | 1 | 1 |
infra.hw.cpu.cfg.002 | Minimum Number of cores per CPU | This determines the number of cores needed per CPU. | 20 | 20 | 1 | 1 |
infra.hw.cpu.cfg.003 | NUMA alignment | NUMA alignment support and BIOS configured to enable NUMA | N | Y | N | Y* |
* immaterial if the number of CPU sockets (infra.hw.cpu.cfg.001) is 1
Please note that none of the listed parameters form part of a typical OpenStack flavour except that the vCPU and memory requirements of a flavour cannot exceed the available hardware capacity.
This section characterises the hardware capabilities for different edge deployments and the Platform services that run on the infrastructure. Please note, that the Platform services are containerised to save resources, and benefit from intrinsic availability and auto-scaling capabilities.
Table 8-4. Characteristics of Infrastructure nodes
Platform Services | Storage | Network Services | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Identity | Image | Placement | Compute | Networking | Message Queue | DB Server | Ephemeral | Persistent Block | Persistent Object | Management | Underlay (Provider) | Overlay | |||
Control Nodes | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ||||
Workload Nodes (Compute) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||
Storage Nodes | ✅ | ✅ | ✅ | ✅ | ✅ |
Depending on the facility capabilities, deployments at the edge may be similar to one of the following:
Table 8-5. Comparison of Deployment Topologies
This Specification | Compute | Storage | Networking | RTT | Security | Scalability | Elasticity | Resiliency | Preferred Workload Architecture | Upgrades | OpenStack | OPNFV Edge | Edge Glossary | GSMA | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Regional Data Centre (DC) Fixed | 1000's Standardised >1 CPU >20 cores/CPU | 10's EB Standardised HDD and NVMe Permanence | >100 Gbps Standardised | ~100 ms | Highly Secure | Horizontal and unlimited scaling | Rapid spin up and down | Infrastructure architected for resiliency Redundancy for FT and HA | Microservices based Stateless Hosted on Containers | HW Refresh: ? Firmware: When required Platform SW: CD | Central Data Centre | ||||
Metro Data Centres Fixed | 10's to 100's Standardised >1 CPU >20 cores/CPU | 100's PB Standardised NVMe on PCIe Permanence | > 100 Gbps Standardised | ~10 ms | Highly Secure | Horizontal but limited scaling | Rapid spin up and down | Infrastructure architected for some level of resiliency Redundancy for limited FT and HA | Microservices based Stateless Hosted on Containers | HW Refresh: ? Firmware: When required Platform SW: CD | Edge Site | Large Edge | Aggregation Edge | ||
Edge Fixed / Mobile | 10's Some Variability >=1 CPU >10 cores/CPU | 100 TB Standardised NVMe on PCIe Permanence / Ephemeral | 50 Gbps Standardised | ~5 ms | Low Level of Trust | Horizontal but highly constrained scaling, if any | Rapid spin up (when possible) and down | Applications designed for resiliency against infra failures No or highly limited redundancy | Microservices based Stateless Hosted on Containers | HW Refresh: ? Firmware: When required Platform SW: CD | Far Edge Site | Medium Edge | Access Edge / Aggregation Edge | ||
Mini-/Micro-Edge Mobile / Fixed | 1's High Variability Harsh Environments 1 CPU >2 cores/CPU | 10's GB NVMe Ephemeral Caching | 10 Gbps Connectivity not Guaranteed | <2 ms Located in network proximity of EUD/IoT | Untrusted | Limited Vertical Scaling (resizing) | Constrained | Applications designed for resiliency against infra failures No or highly limited redundancy | Microservices based or monolithic Stateless or Stateful Hosted on Containers or VMs Subject to QoS, adaptive to resource availability, viz. reduce resource consumption as they saturate | HW Refresh: ? Firmware: ? Platform SW: ? | Fog Computing (Mostly deprecated terminology) Extreme Edge Far Edge | Small Edge | Access Edge |