Comware V7 system architecture

Planes

Comware V7 comprises four planes: management plane, control plane, data plane, and infrastructure plane.

Figure 1 Comware V7 planes

Infrastructure plane

The infrastructure plane provides basic Linux services and Comware support functions. Basic Linux services comprise basic Linux functions, C language library functions, data structure operations, and standard algorithms. Comware support functions provide software and service infrastructures for Comware processes, including all basic functions.

Data plane

The data plane provides data forwarding for local packets and received IPv4 and IPv6 packets at different layers.

Control plane

The control plane comprises all routing, signaling, and control protocols, such as MPLS, OSPF, and security control protocols. It generates forwarding tables for the data plane.

Management plane

The management plane provides management interfaces for operators to configure, monitor, and manage Comware V7. Common management interfaces include Telnet, SSH, SNMP, HTTP, and Web.

Modular design

Comware V7 implements full modularization based on Linux. All software features run independent processes in different spaces. A failed process does not affect other processes. Preemptive scheduling used by Linux threads enables Comware V7 to provide high-speed services. In addition, Linux supports multi-core, multi-CPU, and Symmetrical Multi-Processing (SMP) technologies, which can maximize multi-CPU performance.

The modular design makes Comware V7 totally different from earlier versions.

Figure 2 Modular design

Process isolation

The modular design isolates processes to improve system availability. Each process can be managed separately. This refined management method enhances system stability and performance.

Multi-core support

Comware V7 supports multi-core CPU and SMP technologies besides multi-core support at the data plane. The modular design enables concurrent running of threads through direct Linux scheduling, which can maximize multi-CPU performance. More CPUs improve performance for the whole system to implement faster route convergence and less recovery time.

Comware V7 allows a group of processes to run on a dedicated CPU set to ensure enough resources for key services. The thread preemptive scheduling and priority marking mechanisms enable real-time services to fast respond to requests even when the CPU system is busy.

The multi-core CPU replaces auxiliary CPUs to complete related tasks, simplifying software operations.

On-demand operation

The modular design enables a process to load only needed functions to complete a task. Unused functions do not occupy any system resources and thus will not be attacked. This mechanism improves both system performance and security.

Modular upgrading

The modular design allows you to upgrade a single feature or add new features without affecting other services.

Simplified software tailoring

Comware V7 software processes are independent of one another. This makes software tailoring much easier, without the need of re-compiling.

Multi-package releasing

The modular design allows a Comware V7 software version to be released in multiple software packages, one basic package and multiple function packages, satisfying the requirements of different products.

Function-based licenses

The modular design allows for function-based licenses, which can improve software flexibility and avoid interference from unnecessary functions.

Open interfaces

Comware V7 provides APIs through dynamic Link Library for users to develop their own applications. This method is more flexible than OAA.

Although modularization has many advantages, it has limitations in resource occupation and inter-process communication overheads. Therefore, a balance between the degree of modularization and current resources must be considered. To achieve maximum modularization, H3C bases Comware V7 on deep understanding of networking technologies, and proper feature implementations and module classification, enabling Comware V7 to have better scalability and extensibility than other networking operating systems.

Distribution

Distributed structure

Comware V7 has only one distributed structure that supports various types of hardware structures and virtualization.

In the distributed structure, all nodes are fully meshed. A main processing unit (MPU) has complete functions, and an I/O node (line card) has local processing functions. This software structure is irrelevant with topology. Therefore, different products have consistent software processing, improving scalability and stability.

Figure 3 Distributed structure of Comware V7

The distributed structure implements load sharing among MPUs, which is different from earlier structures that use one main mode as active and others as standby. Modular control processes can run on different nodes.

Distributed computing

Distributed computing based the distributed software structure can maximize resource utilization and improve system performance.

Hardware-based distributed computing

Each card has its own control functions, which can run only on the card to implement distributed computing. For example, link layer protocols run only on ports, so an interface card can independently complete protocol computing for the link layer protocols.

Figure 4 Hardware-based distributed computing

Feature-based distributed computing

Comware V7 enables global features such as routing protocols and MPLS to run on different MPUs rather than on a single MPU. An MPU is the active node for some processes and is a standby node for some other processes. Users can determine the positions of processes or let the system place processes. This mechanism implements load sharing among CPUs, improving the performance of the whole system.

Figure 5 Feature-based distributed computing

Single-feature distributed computing

Feature-based distributed computing cannot fully utilize system resources in some scenarios. For example, if only one or two features are running, placing the features on any CPU will leave other CPUs unused. To solve this problem, Comware V7 supports distributed computing for a single feature.

To achieve this purpose, a common practice is to assign the sub functions of a feature to different MPUs. However, this method is not suitable for concurrent processing because the sub functions must exchange large amounts of data. Another practice is to assign the targets of a feature to different MPUs. Because different targets are independent of each other, this practice has good concurrent processing capability and can fully utilize system resources. For example, BGP route computing can be distributed to different MPUs based on BGP peers.

Figure 6 Single-feature distributed computing

High availability

HA structure

A major object of HA is to minimize failures and impact of failures. Comware V7 can fast detect and recover process failures to ensure HA.

Comware V7 has an HA structure to implement coordinated cooperation among processes. This structure provides a uniform standard for HA functions of processes.

This HA structure provides reliable inter-process communication (IPC), HA control, and database in memory (DBM) real-time database for processes.

Figure 7 HA structure

IPC

IPC provides the following functions:

Reliable cross-node unicast and multicast communication among processes.

Data synchronization between an active process and standby processes, and between different processes.

Continuous communication between processes during a switchover. For example, the switchover of a process during software upgrading cannot be sensed by other processes that are communicating with that process, so the processes can communicate continuously without data loss.

HA control

HA control provides the following functions:

Elects active and standby processes.

Monitors processes, performs active/standby switchover upon active failure, and recovers failures.

Provides communication channels between active and standby processes.

DBM

DBM is an embedded real-time database. It provides reliable process data storage. Processes save real-time data into DBM. The saved data of a process does not get lost when the process terminates or fails. DBM can provide data recovery for a restarted process.

Figure 8 DBM

DBM adopts non-SQL database design to solve database performance problems. It features high reliability while delivering HA services for processes. DBM performs automatic data backup to ensure data consistency during active/standby switchover. In addition, it can save data to other storage media to avoid data loss in extreme conditions. All these data storage methods are transparent to applications, which makes the HA structure more generic, more scalable, and more applicable to different hardware forms.

Process-level GR

Comware V7 supports process-level GR. GR performs backup for each running process. If a process restarts, relevant software and hardware do not delete the data for the process. After restart, the process recovers data, and resumes running. During the restart process, no service interruption occurs, no other services are affected, and neighbors cannot sense the restart.

Process-level GR has two backup methods. One method performs data backup, and the other method performs process backup. Process-level GR comprises single-process GR and active-standby-process GR.

Single-process GR performs data backup for a single process. The data is saved in the local memory database. When the process fails, the system automatically restarts it. After restart, the process resumes running by using the backed up data and exchanges data with other relevant processes to get events that occurred during its restart to complete the recovery. This backup method requires fewer resources and is more suitable for centralized devices or line cards of distributed devices.

Figure 9 Single-process GR

Active-standby-process GR performs process backup. Each standby process receives state information from the active process. When the active process fails, a standby process immediately becomes the new active process to takes over services. The previous active process becomes a standby process after restart. This method has shorter recovery time and higher availability but needs more resources. It is more suitable for MPUs on distributed devices.

Figure 10 Active-standby-process GR

Process-level GR improves system availability without affecting other nodes. For example, if a routing protocol process fails, process-level GR performs active/standby process switchover to implement NSR without affecting the data forwarding plane, and other devices cannot sense the switchover.

System-level HA

System-level HA is implemented through active and standby MPUs. When the active MPU fails, the standby MPU becomes the new active MPU to take over all services. The previous active MPU becomes the standby MPU after restart.

System-level switchovers should rarely occur and should have least impact on operation.

Process-level GR can solve most software problems that can only be solved through a system-level switchover before, improving system-level HA.

The distributed control plane of Comware V7 assigns active control processes to different MPUs. When an MPU fails, only the active control processes running on the MPU are affected, so the impact of the system-level switchover is reduced.

Figure 11 System-level HA

In addition, the Comware V7 MDC feature can virtualize a device into multiple logical devices. With MDC, each MPU is virtualized to multiple virtual systems. A system switchover does not affect other logical devices, so the impact of the system-level switchover is further reduced.

Virtualization

Virtualization includes N:1 virtualization that creates a logical device from multiple physical devices, and 1:N virtualization that changes a physical device to multiple logical devices. Comware V7 supports both virtualization technologies and allows mixing of them (hybrid virtualization).

IRF (N:1 virtualization)

IRF can virtualize multiple physical devices into a logical device, called an IRF fabric. The logical device is considered as a single node from the perspective of NMS, control protocols, and data forwarding. A user can configure and manage the whole IRF fabric by logging in to an IRF member device through the console port or Telnet.

IRF is a generic N:1 virtualization technology. It is applicable to multiple types of networking devices and can implement the following types of virtualization:

Virtualizing multiple centralized devices into a distributed device. Each centralized device is an MPU of the distributed device.

Virtualizing multiple distributed devices into a single distributed device that has multiple MPUs and line cards.

Virtualizing a centralized device and another device into a logical device. The centralized device is a line card of the logical device.

Figure 12 IRF virtualization

An IRF fabric has all the functions of a physical distributed device but has more MPUs and line cards. Users can manage IRF fabrics as common devices.

Figure 13 IRF fabrics

IRF enables multiple low-end devices to form a high-end device that has the advantages of both low-end and high-end devices. On one hand, it has high-density ports, high bandwidth, and active/standby MPU redundancy. On the other hand, it features low costs.

An IRF fabric brings no limitations to virtualized physical devices. For example, an IRF fabric that comprises multiple distributed devices has improved system availability and each distributed device still has its own active/standby MPU redundancy.

An IRF link failure might split an IRF fabric into two IRF fabrics that have identical Layer 3 configurations, resulting in address conflicts. Comware V7 can fast detect the failure by using LACP, ND, ARP or BFD to avoid address conflicts.

In summary, IRF brings the following benefits:

Simplifies network topology.

Facilitates network management.

Good scalability.

Protects user investments.

Improves system availability and performance.

Supports various types of networking devices and comprehensive functions.

MDC (1:N virtualization)

Multitenant device context (MDC) is a 1:N virtualization technology. It virtualizes all the data plane, control plane, and management plane of a physical device to create multiple logical devices called MDCs. MDCs use the same kernel, but their data is separated. Each MDC has its own interfaces and CPU resources. Rebooting an MDC does not affect the configuration or service of any other MDC.

Figure 14 MDC virtualization

The modular design of Comware V7 enables each MDC to run its own control protocol processes on a separate control plane. A process failure of an MDC does not affect other MDCs. Each MDC has a reserved memory space, which enables each MDC to run normally and prevents an MDC from occupying excessive memory resources.

Each MDC has its own forwarding tables, protocol stacks, and physical ports. Data forwarding on MDCs does not affect each other.

Each MDC has its own configuration files, serves its own users, and is managed by its own administrators. Managing an MDC is the same as managing a common device and has no impact to other MDCs.

Each MDC has its own infrastructures that enable processes to run on an MDC in the same way they run on a physical device.

Each MDC uses its own software. The default MDC used for MDC management cannot directly control other MDCs, and it can manage other MDCs only after logging in to their console. The default MDC manages and assigns hardware resources for other MDCs. The only management interface is virtualized into multiple management interfaces that each correspond to an MDC.

To add an MDC, create it on the default MDC, and assign a slot, ports, and CPU resources to it. Then you can log in to the MDC to perform configurations.

An MDC runs only on the specified slot and its resources can be seen during operation. Data synchronization on the MDC only occurs on the cards within the MDC. Other cards cannot receive the information of the MDC.

Figure 15 MDC data synchronization

Each MDC has all the functions of a physical device. An MDC supports GR, system-level switchover, VRFs (or called VPN instances), and VLAN. Different MDCs do not affect each other. For example, the same VLAN can be created on different MDCs.

MDC is suitable for device renting, service hosting, and student labs to reduce device investments and improve efficiency.

Hybrid (N:M virtualization)

Comware V7 allows mixing of I:N virtualization and N:1 virtualization to implement N:M virtualization. You can use IRF to create an IRF fabric that comprises multiple physical devices, and then uses MDC to virtualize the IRF fabric into multiple logical devices.

Figure 16 Hybrid virtualization

Hybrid virtualization can best integrate device resources and improve efficiency. It combines the advantages of both I:N virtualization and N:1 virtualization. For example, it provides IRF active/standby MPU redundancy for each MDC, improving availability.

Hybrid virtualization also improves scalability and flexibility. On a device configured with MDC, you can use IRF to add ports, bandwidth, and processing capability for the device without changing network deployment. On an IRF fabric, you can create MDCs to deploy new networks without affecting existing networks.

ISSU

In-Service Software Upgrade (ISSU) features minimum impact to services, faster upgrades, and higher availability.

Upgrading methods

Patching

Patching aims at solving specific software bugs. Comware V7 supports patching all software components. A patching operation does not require a system reboot, and has no impact to services. Patching is not used for feature upgrading.

Feature upgrading

The Comware V7 software comprises a basic image and multiple feature images. To upgrade features, you only need to load the corresponding feature image, reducing impact to the system.

Basic-image upgrading

Sometimes, you need to upgrade the basic image to complete software upgrading. Some low-end devices only use a single image. Therefore, you only need to perform basic-image upgrading for those devices.

ISSU process

On a centralized or distributed device that has only one MPU, an ISSU upgrades all cards. On a distributed device that has redundant MPUs, an ISSU upgrades the MPUs in turn so that at least one MPU is available during ISSU to ensure stable and reliable operation.

Figure 17 ISSU process

During an ISSU on a device with redundant MPUs, ISSU first upgrades the standby MPU, and then performs a switchover to upgrade the previous active MPU and all line cards. To complete the ISSU, a commit operation is needed. During the ISSU, software rollback is allowed to restore the previous version.

Incremental upgrade

The modular design of Comware V7 allows for upgrading only specific software modules without needing a system reboot. This incremental upgrading method can implement perfect ISSU with minimum impact. During an upgrade, the system automatically compares the new and old versions. If they have no basic differences, the system upgrades only the different parts. Otherwise, the system upgrades the entire software. New features added by incremental upgrading have no operational impact. If a feature to be upgraded is running, process-level GR can ensure service continuity during the upgrade.

Figure 18 Incremental upgrading

Figure 18 shows an incremental upgrade for process 1 on a distributed device and on a centralized device. The upgrade does not affect other processes, and process 1 uses GR to ensure service continuity during switchover.

ISSU soft reboot

During an ISSU, at least one MPU is always available to control the system, and system-level HA ensures global service continuity. However, it is difficult to implement nonstop forwarding on line cards when they are upgraded. Earlier solutions can reduce downtime but cannot avoid service interruption. Comware V7 uses a soft reboot technology to solve this problem.

Soft reboot implements the following functions:

Keeps hardware state unchanged during an ISSU, so the hardware performs data forwarding by using original forwarding entries without interruption, even on an upgraded card.

Has no impact to data forwarding and control functions performed by protocols that are not upgraded.

Saves data for protocols to be upgraded into non-volatile storage media before an ISSU, so that the new software can use the saved data to directly take over hardware control functions after the ISSU.

Uses a protocol agent on an MPU to replace a protocol on an upgraded card when the protocol is suspended, to avoid network oscillation. This mechanism is suitable for delay-sensitive protocols.

Figure 19 Soft reboot process

In summary, soft reboot ensures nonstop forwarding for all services, nonstop control plane processing for protocols that are not upgraded, and fast control plane recovery for upgraded protocols during an ISSU.

Soft reboot cannot implement protocol agent function on a centralized device or a distributed device that has no redundant MPU. Therefore, before an ISSU, increase the keepalive interval for relevant protocols or disable keepalive function for those protocols, and enable GR for Layer 3 protocols such as routing protocols so that neighbors can help re-create forwarding entries after the soft reboot.

Figure 20 Soft reboot process on a centralized device

Maintainability

In addition to traditional maintenance methods for viewing operational states, logs, and debug and core dump information, Comware V7 provides a more flexible feature called EAA.

EAA comprises a group of functional components for users to subscribe software and hardware events such as link down event, and to apply policies. According to user-defined policies, EAA can send event information via e-mail to technical support or user email accounts.

EAA includes the following components:

Real-time event manager (RTM)—Performs event subscription and policy enforcement.

Real-time notify (RTN)—Performs message assembly and email notification.

Real-time detector (RTD)—Detects software and hardware faults and triggers RTM to perform policies such automatic recovery and information collection.

Scheduler—Executes one-time and regular jobs, similar to Linux cron/atd.

Figure 21 RTM structure

Event sources are software and hardware components that generate events. For example, executing the comsh command generates a command execution event, and a syslog generates a syslog generation event.

RTM supports the following event sources:

More event sources will be added to RTM over time.

RTM filters events generated by event sources according to user subscriptions, and executes policies for qualified events by running scripts.

Policy enforcement executes policy scripts according to the requirements of RTM. Policy scripts include comsh command line scripts and TCL scripts.

RTM supports multiple actions, such as reboot, cli, syslog, trap, and switchover. RTM provides an open structure for users to easily add actions.

EAA greatly improves system maintainability by providing more flexible and intelligent methods.

Data center features

Comware V7 provides the following features to satisfy the new requirements of data centers.

TRILL

TRansparent Interconnect of Lots of Links (TRILL) uses IS-IS to provide transparent Layer 2 forwarding.

TRILL combines the simplicity and flexibility of Layer 2 switching with the stability, scalability, and rapid convergence capability of Layer 3 routing. All these advantages make TRILL very suitable for large flat Layer 2 networks in data centers.

EVI

Ethernet Virtual Interconnect (EVI) is a MAC-in-IP technology that provides Layer 2 connectivity between distant Layer 2 network sites across an IP routed network. It is used for connecting geographically dispersed sites of a virtualized large-scale data center that requires Layer 2 adjacency. For example, virtual machines must move between data center sites without changing their IP addresses, so their movements are transparent to users.

EVB

Edge Virtual Bridging (EVB) allows virtual machines (VMs) on a physical server to obtain bridge relay services through a common bridge port. It enables coordinated configuration and management of bridge services for VMs.

Data center virtualization includes network virtualization, storage virtualization, and server virtualization. Server virtualization uses specific virtualization software such as VMware to create VMs on a single physical server. Each VM operates independently and has its own operating system, applications, and virtual hardware environments.

VMs on a physical server communicate with each other or with the outside network through a Virtual Ethernet Bridge (VEB). VEBs are implemented through software or hardware such as NICs. Both implementation methods have the following limitations:

Lack of traffic monitoring capabilities such as packets statistics, traffic mirroring, and NetStream.

Lack of network policy enforcement capabilities, such as QoS.

Lack of management scalability, especially in unified deployment of the internal server network and the external network.

EVB solves these limitations. It uses a physical switch (called EVB bridge) to switch traffic for VMs on a directly connected physical server (called EVB station). EVB implements traffic monitoring, network policy enforcement, and unified network deployment and management for VMs.

FCoE

A data center using the FC SAN technology usually comprises separate local area networks (LANs) and storage area networks (SANs). LANs carry traditional Ethernet/IP services, and SANs carry network storage services.

To provide services for LANs and use SANs for storage simultaneously, the servers must use independent Ethernet adapters and FC adapters. In addition, the IP switches and the FC switches are also independent and have independent network connections. Therefore, the data center needs more switches, network adapters, and cables. This solution increases investment and maintenance workload and has low scalability.

FCoE can solve this problem. In an FCoE solution, the server uses an FCoE-capable Ethernet adapter, and the FCoE switch (FCoE forwarder) integrates the functions of both the traditional IP switch and FC switch. FCoE reduces the number of network adapters, switches, and cables, and maintenance workload.

Comware V7 advantages

Unified system

Comprehensive networking functions, including Layer 2, Layer 3, storage, and MPLS, virtualization functions, enable Comware V7 to support all types of networking devices such as data center devices and cloud computing devices. Comware V7 supports integration of IP and storage networks, data center Ethernet, virtualization, and high availability. Comware V7 provides highly reliable, high-performance, and large-capacity routing functions, MPLS, and device monitoring and management functions for service provider devices, and provides rich features for access devices.

Figure 22 Unified networking operating system

Comware V7 is suitable for centralized, distributed, multi-chassis distributed hardware structures, and WAN, LAN, and data center devices. All these devices only need to run the single Comware V7 operating system to provide all functions.

The unified system enables unified deployment and management, reduces networking and management complexity, and implements full compatibility. Users only need to learn one system to use all software functions. In addition, the unified system allows for flexible integration of functions, improving scalability.

Full virtualization

Comware V7 supports N:1 virtualization technology IRF, 1:N virtualization technology MDC, and hybrid virtualization by mixing IRF and MDC.

High performance

Improved resource utilization: Modular design ensures unused functions to not occupy any resources.

Multi-core, multi-CPU: Data and control planes support multi-core functions, which enable improving system performance by adding CPUs. CPU resources can be reserved for processes to satisfy their performance requirements without needing dedicated hardware, improving usability.

Distributed computing: Implements both load sharing and active/standby redundancy, improving performance and resource utilization.

High reliability

Fault isolation: Process isolation implements fault isolation. The failure of a process does not affect other processes and the kernel. A failed process can be recovered without hardware impact.

Process-level GR: Ensures graceful restart of processes without operational impact.

System-level high availability: Reduces impact of active/standby switchover through control plane distributed functions.

ISSU: Supports both MPU ISSU and interface card ISSU for multi-MPU, distributed single-MPU, centralized, and IRF devices. Incremental upgrade and soft reboot ensure non-stop forwarding during ISSU.

Extensibility

Feature tailoring: Modular design allows for free tailoring of software features, without the need of re-compiling.

Feature addition: Modular design enables new features to be easily added to Comware V7 with affecting other features.

ISSU: Upgrades features without operational impact, ensuring service continuity.

IRF: Increases ports and bandwidth by adding devices into a virtual system, improving system performance and capacity while protecting user investment.

Openness

TCL scripts: Comware V7 is embedded with TCL. Users can compile TCL scripts by using commands and SNMP get and set operations to implement needed functions.

Open APIs: Based on Linux, Comware V7 provides open APIs for users to develop their own functions.

EAA: Enforces user-defined policies upon system changes, improving manageability and satisfying individual needs.

Open architecture: Modular design enables running of all kinds of applications. Open APIs enable users to develop and run applications on Comware V7.

Continuity

The user interfaces of Comware V7 inherit the style of earlier versions. In addition, unchanged functions inherit previous commands. Users familiar with previous Comware can easily learn how to operate Comware V7. New maintenance methods such as EAA provided by Comware V7 enable users to get more clear and detailed information on operational states.

Comware V7 adds new features while keeping the features of earlier versions, ensuring version continuity.