Managing IT infrastructure in a data center is becoming increasingly difficult due to growing complexity in advancing technologies and user requirements.

The evolving requirement to change data center infrastructure can affect stability, performance, costs, and upgradability. Dell 3-Tier Platform solutions are proven, tested, and optimized to help organizations meet their long-term data center needs for mixed workloads. The 3-Tier Platform solution that this guide describes provides the simplicity of a flexible, validated converged infrastructure (CI) that is based on:

• Dell PowerMax-V4, 2nd Generation Dell PowerStore, Dell PowerScale, and Dell Unity XT storage
• Cisco Nexus switches
• Cisco UCS or UCS-X compute
• VMware vSphere virtualization

The 3-Tier Platform for CI environments offers the following benefits:

• Resiliency: The 3-Tier Platform architecture eliminates any single point of failure by incorporating platform redundancy in server high availability (HA) features, networking, and multipath storage.
• Scalability: The 3-Tier Platform is configurable to meet your virtualized infrastructure needs. It enables you to choose from various configuration options, such as the number, model, and memory capacity of compute servers and the storage type and capacity.
• Validated interoperability: The 3-Tier Platform hardware and software components have been tested and validated for HA and compatibility to ensure known quality in platform run-time.

This guide helps you design a 3-Tier Platform solution by providing design principles, best practices, and architectural guidance for compute, management, networking, and storage.

The guide includes component specifications for validated configuration and scaling options and considerations.

This guide helps you design a 3-Tier Platform solution by providing design principles, best practices, and architectural guidance for compute, management, networking, and storage.

The guide includes component specifications for validated configuration and scaling options and considerations.

Dell Technologies and the authors of this document welcome your feedback on the solution and the solution documentation.

Contact the Dell Technologies Solutions team by email.

This chapter provides an overview of the components and design of the Dell 3-Tier Platform Architecture for Converged Infrastructure Environments.

The Dell 3-Tier Platform includes compute, network, storage, management, and virtualization resources.

The Cisco compute portfolio, which is designed to optimize production and ensure a stable, worry-free environment, encompasses fabric interconnects (FIs), rackmount servers, and blade servers.

The portfolio consists of:

• Cisco UCS Manager and Cisco Intersight
• Cisco B-series, C-series, and X-series servers
• Cisco 4th-generation and 5th-generation FIs

Cisco Intersight provides software as a service (SaaS) delivery and full life cycle management (LCM) of distributed infrastructure and workloads across data centers, remote sites, branch offices, and edge environments. Cisco Intersight enables efficient deployment, configuration, security, maintenance, automation, and scaling of your environment through a cloud-operating model. It enhances your operational consistency and compliance, and strengthens your security posture to accelerate the delivery of IT infrastructure, resources, and applications to support business growth.

For more information, see the Cisco Intersight website (access requires Cisco login credentials).

For LAN connectivity, the 3-Tier Platform provides a data plane and a management plane.

• Data plane: The Top of Rack (TOR) switches carry all the production network traffic that flows through the three-tiered architecture. Cisco Nexus 9000 series switches provide an Ethernet switching fabric for the production traffic flow between the Cisco UCS and the core customer network. The Cisco 9000 series switches provide high-performance ports (10 GbE, 25 GbE, 40 GbE, 50 GbE, and 100 GbE), density, and low latency. They feature superior power efficiency across various compact form factors. Dell Technologies recommends configuring data plane network switches in pairs for redundancy and scalability.
• Management plane: Cisco Nexus 9000 series switches provide management plane network connectivity for out of band (OOB) management of the various 3-Tier Platform components. Dell Technologies recommends configuring management network switches in pairs for redundancy and scalability.

This solution incorporates the following Dell storage systems: PowerMax, PowerStore, PowerScale, and Unity XT.

• The PowerStore advantage begins with a comprehensive line-up of enterprise capabilities that provide maximum headroom and flexibility for change. The PowerStore platform adaptable architecture and integrated intelligence ensure that it is continuously modern. Consistent and predictable performance ensures lasting value across multiple solution life cycles to help build long-term business success.
• PowerMax is a powerful, simple, and trusted storage array. Designed uncompromisingly for the mission-critical applications of today and tomorrow, PowerMax incorporates end-to-end NVMe, next-generation storage media (SCM), real-time machine learning (ML), and inline deduplication and compression. It also delivers the essential features and data services that businesses require.
• PowerScale is an exceptionally flexible and secure scale-out NAS solution that is adept at managing demanding, data-intensive projects. Powered by the PowerScale OneFS operating system, PowerScale hybrid and archive nodes are the ideal platform for efficient storage, management, security, protection, and analysis of unstructured data. They support a broad range of applications and workloads.
• Unity XT is the latest family member in Dell Technologies midrange storage. Unity XT is a unified storage solution that provides all-inclusive data storage software, optimized efficiency, multicloud enablement, and all-flash performance. Unity XT is available in two formats: All-Flash Arrays (AFA) and Hybrid Flash Arrays (HFA).

The management solution uses workloads to adapt and respond to evolving management capabilities for one or more 3-Tier Platforms.

The solution runs core applications, optional Dell Technologies applications, and ecosystem applications to provide:

• Health monitoring
• Performance and capacity management
• Fault domain isolation

Both single-system and multisystem management solutions are supported on the 3-Tier Platform.

The 3-Tier Platform uses VMware vSphere for virtualization to manage and orchestrate infrastructure resources such as compute, storage, and networking. This solution provides users with a seamless, versatile, and dynamic experience.

vSphere encompasses the following components and features:

• VMware ESXi: A Type 1 hypervisor that abstracts processors, memory, storage, and other resources for use by one or more guest operating systems that are configured as virtual machines (VMs).
• VMware vCenter Server: A central control point for data center management services, providing a single view of all ESXi hosts.
• VMware vSphere HA: Services that restart failed VMs on other available servers.
• VMware vSphere DRS: Services that perform resource scheduling and load balancing.
• VMware vSphere vMotion: Feature that enables live migration for powered-up VMs in the same data center.
• VMware vSphere Storage vMotion: Feature that enables the live migration of virtual disks or configuration files to a new datastore while a VM is running.
• VMware vSphere Client: HTML5-based management interface that enables users to connect remotely to VMware vCenter Server.

When installing and configuring a production-ready CI, keep these considerations in mind:

• Distributing system and virtualization software.
• Allocating cluster services and data storage to physical nodes and arrays.
• Selecting appropriate server hardware.
• Determining size and scalability.

Challenges include:

• Understanding the types of workloads that run on the cluster.
• Managing a system that is designed for multiple general-purpose, virtualized workloads.
• Keeping up with the fast-moving pace of the industry.

Dell Technologies recommends that the data center hosting the 3-Tier Platform includes the following infrastructure elements:

• Sufficient power and cooling to support the 3-Tier Platform.
• Panduit power distribution units (PDUs) and cabinets for optimal results. For more information, see Panduit PDUs.

This chapter describes the 3-Tier Platform compute components that are supported in Cisco UCS-Managed Mode.

The Cisco UCS Manager (UCSM) system runs on the Fabric Interconnects (FIs). It enables you to access, configure, administer, and monitor the network and server resources across all the connected chassis, blades, and rackmount servers.

Cisco UCS 4th and 5th generation FI components provide compute power in the 3-Tier Platform.

The following table shows the Cisco Nexus and Cisco UCS components.

Each FI reserves multiple ports to connect Cisco SAN switches and carry FC traffic between the compute layer and the storage layer. SAN port channels carry FC traffic between the FIs and upstream Cisco MDS switches. The FEX within the blade server chassis connects to the FIs over converged networking. By default, servers in the 3-Tier Platform boot VMware vSphere ESXi from local M.2 cards. Booting from SAN is optional.

Regardless of the number of blade servers in the blade server chassis, the FEX connect northbound to the FIs with:

• Four or eight Cisco UCS fourth generation 10/25 GbE connections with either the 2408IOM or the X9108-25G IFM
• Two or four Cisco UCS fifth generation 40 GbE connections from the 2304v2 40 GbE IOM
• Four or eight Cisco UCS fifth-generation 100 GbE connections from the 9108-100G IOM

The Cisco UCS 2408 FEX has eight external 25 GbE ports and 32 internal 10 GbE ports. Each FEX connects northbound to the FIs, regardless of the number of blade servers in the blade server chassis. The Cisco UCS 2408 FEX works with the Cisco UCS 6454 or 64108 FI at 25 Gbps.

Cisco UCS C-Series rack servers connect directly to the following FIs:

• Cisco UCS 6454
• Cisco UCS 64108
• Cisco UCS 6536

Each 3-Tier Platform contains at least two Cisco UCS FIs to form a HA cluster. Each FI pair:

• Provides network connectivity and management capability for the Cisco UCS 5108 blade server chassis and blades.
• Forms a Cisco UCS compute domain.
• Runs the Cisco UCS Manager application and database.

The Cisco UCS 4th-generation components are:

• Cisco UCS 6454 FIs: Switches with 54-port, 16 Gbps or 32 Gbps FC, 10 GbE or 25 GbE and 6 QSFP 100 GbE capabilities. These FIs connect to a pair of 100 Gbps-capable Cisco Nexus 9336C-FX2 data plane switches that support up to seven Cisco UCS compute domains.

  For more information, see the Cisco UCS 6454 FI data sheet.

• Cisco UCS 64108 FIs: Switches with 108-port, 16 Gbps or 32 Gbps FC, 10 GbE or 25 GbE and 12 QSFP 100 GbE capabilities. These FIs connect to a pair of 100 Gbps capable Cisco Nexus 9336C-FX2 switches. The Cisco Nexus 9336C-FX2 switches support up to seven Cisco UCS compute domains.

  For more information, see the Cisco UCS 64108 FI data sheet.

The Cisco UCS 6536 Fabric Interconnects are 36 port, 32 Gbps FC, 25 GbE, or 100 GbE capable switches.

The Cisco UCS 6536 Fabric Interconnects connect to two 100 Gbps-capable Cisco Nexus 9336C-FX2 data plane switches that support up to seven Cisco UCS compute domains.

For more information, see the Cisco UCS 6536 Fabric Interconnect data sheet.

Cisco UCS M6 and M7 servers combine compute, network, storage access, and virtualization resources into a single converged system. Cisco UCS integrates a low-latency, lossless 10 GbE, 25 GbE, 40 GbE, or 100 GbE unified network fabric with x86-architecture servers.

Cisco UCS B-Series blade servers and Cisco UCS C-Series rack servers provide:

• Integrated redundancy for high availability
• Hot-swappable components for serviceability, upgrade, or expansion
• Reduced cabling
• Improved energy efficiency over traditional blade server chassis

For VMware deployments, the 3-Tier Platform can scale from three to hundreds of Cisco UCS servers.

The following table shows the maximum number of Cisco UCS servers that the 3-Tier Platform supports.

The following table shows the maximum number of Cisco UCS B-Series blade servers to deploy that are directly connected with Cisco UCS C-Series rack servers in the same domain.

The following table shows the CPUs and memory that the Tier-3 Platform supports.

Order CPUs from the factory or by requesting an additional server through the expansion services. All Cisco UCS servers are preconfigured with local M.2 cards for operating system booting. For issues and concerns that are related to CPUs that are ordered as part of a converged system, Dell Technologies liaises directly with Cisco support.

Cisco UCS B-Series blade servers and Cisco UCS C-Series rack servers contain at least one physical Cisco UCS virtual interface card (VIC) for server connectivity.

The VIC passes converged FC and IP network traffic through the Cisco UCS 5108 blade server chassis midplane to the FEX.

The Cisco UCS C-Series rack servers can connect directly to each FI.

Cisco UCS-X Compute Nodes, UCS B-Series Blade Servers, and Cisco UCS C-Series Rack Servers contain at least one physical Cisco UCS VIC for server connectivity.

For the Cisco UCS servers, the VIC passes converged FC and IP network traffic through the Cisco UCS 5108 Blade Server Chassis midplane to the FEX.

The Cisco UCS C-Series Rack Servers can connect directly to each FI with direct connect mode, or to a pair of FEX connected to the FIs.

The Cisco UCS X9508 supports the Cisco UCS X210c M6 and M7, and the X410c M7 Compute Node.

Cisco UCS B-Series blade servers contain half-width blades with 4th-generation Cisco UCS VICs on supported servers.

Half-width M6 servers can contain a 4th-generation Cisco UCS VIC 1440 in the motherboard (mLOM) mezzanine slot. The Cisco UCS VIC 1440 is installable with a port expander card for native 40 Gbps connectivity to each fabric.

The following table shows the UCS B-Series network connectivity.

The Cisco UCS X9508 chassis contains Cisco UCS X210c M6 and M7, and X410c M7 Compute Nodes with 14000 and 15000-series VICs.

The following table shows the uplink connectivity from the Cisco X9508 chassis to the fabric interconnect.

Cisco UCS C220/240 rackmount servers and X210c and X410c M7 compute nodes continue to only be supported in Intersight in new deployments. M7 servers are supported in UCSM with firmware release 4.3(2b). However, each UCSM environment with M7 servers must purchase Intersight licensing and deploy a SaaS, CVA, or PVA environment even if they are running UCSM.

In this case, Cisco prescribes that the UCSM domain be claimed as a target within Intersight. Then, Intersight licenses can be distributed to M7 servers that are managed in UCSM. In CVA and PVA environments, all servers that are in a UCSM domain that are claimed to Intersight automatically take up a license (including M6 servers). However, in a SaaS environment you can have dedicated Intersight licenses to only M7 servers in your UCSM domain. Dell Technologies highly recommends that you deploy an Intersight SaaS environment in this case, to avoid having to license every server in your UCSM environment.

The use of M7 servers within UCSM has no limitations on functionality. As of right now you get a warning for each M7 server that does not have an Intersight license. However, Cisco has stated that future firmware releases will likely block functionality of M7 servers in UCSM that do not have an Intersight license assigned.

If you implement M7 servers into a new or existing UCSM domain without Intersight licenses assigned, you must understand that this configuration is unsupported. You must understand that each M7 server within that UCSM domain will likely lose functionality within UCSM during a future firmware release.

The following figure shows an eight-link l/O module (IOM) at 25 Gbps port speed on a Cisco UCS blade server chassis with FEX to FI connections.

Cisco UCS Manager 4.1 (2a) or later allows the configuration of Cisco UCS 2408 FEX (IOM) with Cisco UCS VIC 1440 adapters and port expanders. This configuration supports up to 40 Gbps aggregate flow and 25 Gbps single flow per virtual Network Interface Card (vNIC).

Cisco UCS C-Series rack servers connect directly to the FIs. The Direct Connect mode allows single-cable management and data traffic for these servers by the Cisco UCS Manager. Each Cisco UCS VIC connects to the FI using a single link. This approach is preferred for configurations with fewer servers.

A single server does not support different generations of VICs. Use either the Cisco UCS VIC 15xxx or 14xx series.

In Direct Connect mode, 4th-generation Cisco UCS C-Series rack servers include the following features:

• Cisco UCS VIC 1467, a Small Form-Factor Pluggable (SFP28) mLOM card for the M6 servers.
  • The card supports 10/25-Gbps Ethernet or FCoE.Cisco UCS VIC 1477, a dual-port Quad Small Form-Factor (QSFP28) mLOM card for the M6 servers, and 40-Gbps or 100-Gbps Ethernet or FCoE.
• Cisco UCS VIC 15428, a Small Form-Factor Pluggable (SFP28) mLOM card for the M6 servers.
  • The card supports 10, 25, or 50-Gbps Ethernet or FCoE.
• Cisco UCS VIC 15238, a Small Form-Factor Pluggable (SFP28) mLOM card for the M6 servers.
  • The card supports 40, 100, 200-Gbps Ethernet or FCoE.

On Cisco UCS C220/240 M7 rackmounts only the following VICs are supported:

• 15428 mLOM (25 G)
• 15425 PCIe (25 G)

The UCS 15238 mLOM can be paired with a 15235 in the PCIe slot of a M7 rackmount server to achieve 4 x 100 Gb connections (200 Gb per fabric). Fifth-generation

• 100 Gbps per fabric on a Cisco UCS VIC 1497 or 1495 on Cisco UCS M5 servers.
• The Cisco UCS VIC 1477 is a dual-port Quad Small Form-Factor (QSFP28) mLOM card for the M6 servers. The card supports 40/100-Gbps Ethernet or FCoE.

On Cisco UCS C220/240 M7 rackmounts only the following VICs are supported:

• 15238 mLOM (100 Gb)
• 15235 PCIe (100 Gb)

The UCS 15428 mLOM can be paired with a 15425 in the PCIe slot of a M7 rackmount server to achieve 8 x 25 Gb connections (100 Gb per fabric).

The following table shows the port-to-FI connections of the Cisco UCS VICs.

The Cisco UCS architecture uniquely combines compute, data network, and storage network access into a unified set of components that is managed through a single interface.

Cisco UCS integrates access layer networking and servers. Its unified fabric, supported by hardware and software components, runs diverse data center traffic over one converged network adapter. The elimination of switching in a Cisco UCS 5108 blade server chassis reduces network access-layer fragmentation. It cuts down the number of switches, cables, adapters, and management points by up to two-thirds. All devices in a Cisco UCS domain are managed under a single domain, which remains highly available by using redundant components.

The following figure shows a mixed server configuration with Cisco UCS B-Series blade servers and Cisco UCS C-Series rack servers.

For best performance, maintain an oversubscription rate of 8:1 or lower across all compute domains. In this ratio, 8 denotes the committed host bandwidth, while 1 denotes the Ethernet and FC uplinks.

The following table shows a 5108 chassis with 4-link and 8-link 25 G connections to the FI. A 6454 FI can have two, four, or six uplink options to a 9336C ToR switch. Oversubscribed areas are marked in bold italics. As an example, 11 chassis with eight links per side provide 2,200 GB of bandwidth to the FI per side. Two uplinks to the ToR provide only 200 GB of bandwidth. This configuration results in an oversubscription of 2,200 / 200 = 11. To avoid oversubscription, provide four uplinks from the 6454 FI to the 9336C ToR switch.

When planning your CI environment, follow the guidance in the previous table to avoid oversubscribing on Ethernet or FC connections.

Starting with the Cisco UCS firmware release 4.2, the Cisco UCS Manager (FC-NVMe) supports NVMe over Fibre Channel.

Version 4.2 of Cisco UCS Manager supports FC-NVMe on the following FIs:

• UCS 6454
• UCS 64108

3-Tier Platform support for FC-NVMe requires:

• Cisco UCS VIC 14xx series adapters on M6 blade and rack mount servers
• VMware vSphere ESXi release versions 7.0 or later

In a Disjoint Layer 2 configuration, traffic is split between two or more networks at the FI to support two or more discrete Ethernet clouds. Configure Disjoint Layer 2 on the FI on ports that are not used for the standard LAN uplink traffic.

On the 4th-generation Cisco UCS 6454 or 64108 FIs, the network ports are:

• Non-FC 10 Gbps or 25 Gbps unified ports
• 40 Gb or 100 GbE Uplink ports

Upstream Disjoint Layer 2 networks give access to the cloud to servers or VMs in the same Cisco UCS domain.

The following figure shows an example of Disjoint Layer 2 networking into a Cisco UCS domain.

• vPCs 101 and 102 are production uplinks that connect to the network layer of the 3-Tier Platform.
• vPCs 105 and 106 are external uplinks that connect to other switches.

You can also configure Disjoint Layer 2 network connectivity with an individual uplink on each FI.

The following tables show the supported cabling.

For more information about the supported cabling on a specific Cisco FI, see the corresponding data sheet:

• Cisco 6454 FI data sheet
• Cisco 64108 FI data sheet

For more information about the supported cabling on a specific Cisco FI, see the corresponding data sheet:

• Cisco 6454 FI data sheet
• Cisco 64108 FI data sheet
• Cisco 6536 FI data sheet

This chapter describes the 3-Tier Platform compute components that Cisco Intersight Managed Mode (IMM) supports.

Cisco Intersight is a management platform that is delivered as a service for your data center environment.

Cisco IMM enables you to administer and monitor multiple domains through one interface. Cisco UCS X-Series and 4th-generation components provide compute power in the 3-Tier Platform.

The following table shows the Cisco Nexus and Cisco UCS components that are used in Intersight-managed domains.

Each FI reserves several ports to connect Cisco SAN switches and carry FC traffic between the compute layer and the storage layer. SAN port channels carry FC traffic between the FIs and upstream Cisco MDS switches. The fabric extenders (FEX) within the blade server chassis connect to FIs over converged networking. By default, servers in the 3-Tier Platform boot VMware vSphere ESXi from local mirrored SD cards or M.2 cards, with booting from SAN is optional. Regardless of the number of blade servers in the chassis, the FEX establishes northbound connections to the FIs using four or eight of the following IOMs:

• Cisco UCS 4th-generation 10 Gbps with 2204/2208XP IOMs
• Cisco UCS 25 GbE connections with 2408 IOM or 9108 25 Gbps IFM
• Cisco UCS 100 GbE connections with X9108 100 Gbps IFMs (on the 6536 FI only)

The Cisco UCS 2408 FEX has eight external 25 Gbps ports and 32 internal 10 Gbps ports. Each FEX connects northbound to the FIs, regardless of the number of blade servers in the blade server chassis. The Cisco UCS 2408 FEX works with the Cisco UCS 6454 or 64108 FI at 25 Gbps.

The Cisco UCS X9108 25 Gbps IFM is equipped with eight SFP28 external connectors on an IFM to interface with FI 6400 series FIs. Internally, the IFM provides eight 25 Gbps connections to each UCS X210c or X410c compute node in the Cisco UCS X9508 chassis.

Cisco UCS M6 and M7 C-Series rack servers connect directly to the following FIs:

• Cisco UCS 6454 FIs
• Cisco UCS 64108 FIs
• Cisco UCS 6536 FIs

The Cisco UCS X9108-100G IFM is equipped with eight QSFP28 external connectors on an IFM, enabling itto interface with FI 6 series FIs. Internally, the IFM provides either one 100 Gb or four 25 Gb connections to each UCS X210c or X410c compute node in the Cisco X9508 chassis.

Cisco UCS M6 and M7 C-Series rack servers connect directly to the following FIs:

• Cisco UCS 6454 FIs
• Cisco UCS 64108 FIs
• Cisco UCS 6536 FIs

Use a pair of Cisco Nexus 2000 Series FEX to increase the scale for Cisco UCS C-Series rack servers on Cisco UCS 6332-16UP FIs. The Cisco UCS 6454 FIs can use a pair of Cisco Nexus 2232PP FEX to increase the scale of Cisco UCS C-Series rack servers. These connections carry IP and FC traffic. The Cisco UCS 64108 and 6536 FIs do not have a supported FEX for rack mounting.

You can deploy Cisco Intersight as SaaS on Intersight.com, or running on your premises as a Cisco Intersight virtual appliance. The virtual appliance provides the benefits of Cisco Intersight with more flexibility for deployments that require additional data locality and security requirements. The Centera Virtual Archive virtual appliance software is deployable on-premises, allowing users to take advantage of the SaaS functionality. The Private Virtual Appliance (PVA) can be deployed on-premises with further security restrictions.

You can use the following Cisco Intersight deployment types on the 3-Tier Platform:

• SaaS (Cloud)
• Centera Virtual Archive
• PVA

The following table compares the different Cisco Intersight deployment types.

Cisco Intersight virtual appliance deployments support the following 3-Tier Management solutions:

• 3-Tier Management Solution Central with Dell Unity and Unity XT storage
• 3-Tier Management Solution Central with vSAN

A single Cisco Virtual Appliance instance of Intersight can manage multiple 3-Tier Platforms.

If resources are available on the 3-Tier Management solution, you can host multiple virtual appliance instances on the platform. However, the discovered devices and virtual infrastructure must meet specific requirements, as shown in the following table.

Cisco Intersight offers subscription-based licensing with a choice of license types.

As you move up the licensing tiers, more capabilities become available. You can purchase a subscription for one, three, or five years, and then select the Cisco UCS server volume tier that suits your needs for the selected subscription period.

For more information, see Cisco Intersight Licensing.

Cisco Smart Accounts and Smart Licensing are required for Cisco Intersight.

A Cisco Connection Online (CCO) user account is required for the Cisco Intersight UI.

For more information about Cisco Intersight licensing tiers and ordering, see:

• Cisco Intersight Infrastructure Service Data Sheet
• Introduction to Intersight Licensing
• Infrastructure Services License

Cisco Intersight discovers and monitors targets that Cisco UCS Manager manages, a process known as Cisco UCS-Managed Mode (UMM). In this mode, you can claim Cisco UCS domains as targets known as Cisco UCS-managed targets. Cisco Intersight enables you to monitor multiple Cisco UMM domains through a single interface.

The following table shows the software versions that Cisco UCS hardware and Cisco UCS Manager support.

Cisco Intersight supports VMware 7.0 and 8.0.

The device connector is an embedded management controller that enables the Cisco Intersight capabilities. It provides a secure way for the connected targets to send information to Cisco Intersight and receive control instructions from it.

To enable Intersight Management, turn on the device connector and select Admin > Device Connector > Cisco UCS Manager.

The following table shows the minimum supported firmware versions that are required on the FI to claim targets from the Intersight Virtual Appliance.

Cisco UCS and blade servers contain half-width blades with Cisco UCS VICs on supported Cisco UCS B-Series blade servers.

Half-width Cisco UCS B-Series M6 blade servers can contain a 4th-generation Cisco UCS VIC 1440 in the motherboard (mLOM) mezzanine slot.

4th-generation Cisco UCS VIC 1480 M6 servers are installed in the PCIe mezzanine slot. The slot is next to the Cisco UCS VIC 1440. Both VICs can direct network traffic to a separate physical adapter.

The Cisco UCS VIC 1440 is installed with a port expander card for native 40 Gbps connectivity to each fabric. The Cisco UCS VIC 1440 supports a port expander with a Cisco UCS 2408 FEX IOM. The VIC 1440 does not support port expanders with Cisco UCS 2204XP or the Cisco UCS 2208XP IOMs.

Each vNIC on a VIC 14000 provides:

• An aggregate bandwidth of 50 Gbps across multiple flows.
• A single-flow maximum of 25 Gbps.
• 100 Gbps across both IFMs with two vNICs.
• 200 Gbps across both IFMs with four vNICs.

For a 14825 UCSX-V4-Q25GME rear mezzanine VIC card, a 14425 mLOM VIC (UCSX-V4-Q25GML) is required. A UCSX-V4-BRIDGE VIC bridge card is included, and connects the mLOM to the mezzanine adapter. The UCSX rear mezzanine card has Ethernet connectivity to the IFM using the UCSX-V4-BRIDGE, and a PCIE Gen3 x16 connectivity towards CPU2.

The Cisco UCS 15000 VICs provide:

• An aggregate bandwidth of 50 Gbps or 100 Gbps across multiple flows
• A single-flow maximum of 25 Gbps
• 100 Gbps across both IFMs with one or two vNICS

If a UCSX 15422 mezzanine VIC card is installed, a 15420 mLOM is required. A bridge connector (UCSX-V5-BRIDGE-D) is required and included when a 15422 mezzanine is paired with a 15420 mLOM.

The following table shows the network connectivity requirements for a Cisco UCS 5108 chassis.

The following table shows the network connectivity requirements for a Cisco UCSX 9508 chassis.

The network connectivity for a Cisco UCS X9508 chassis is:

• Compute node type: Full-height
• Cisco UCS VIC combination: 14425
• Bandwidth to each FEX: 50 Gbps
• Total available bandwidth: 100 Gbps

The following figure shows an eight-link IOM at 25 Gbps port speed with FEX to FI connections on a Cisco UCS or UCS X server chassis.

The following figure shows an eight-link IFM at 100 Gbps port speed with FEX to FI connections on a Cisco UCS X server chassis with a 9108-100G IFM.

Every 3-Tier Platform that Cisco Intersight manages is equipped with two Cisco UCS 4th- or 5th-generation FIs to form an HA cluster.

The following table shows the supported 3-Tier FI configurations.

Cisco UCS 2408 IOM requires Cisco UCS Intersight Infrastructure 4.1 (2b) or later.

To run IMM on your FIs, use IMM version 4.1 (3) or later.

Cisco UCS X210c M7 compute node on chassis 9508 and C-Series 220 and 240 M7 servers are supported on FIs running 4.2(3b) or greater. Cisco Infrastructure bundle 4.3(2c) and later supports Cisco UCS X410c M7 servers.

Each FI pair performs the following functions on the 3-Tier Platform:

• Provides network connectivity and management capability for the Cisco UCS 5108 chassis or the Cisco UCS X9508 chassis
• Forms a Cisco UCS compute domain
• Runs the Cisco Intersight application and database

Cisco UCS M6 blade and rackmount servers combine compute, network, storage access, and virtualization resources into a single converged system.

Cisco UCS integrates a low-latency, lossless 10 GbE or 25 GbE unified network fabric with x86-architecture servers.

Cisco UCS B-Series blade servers, Cisco USC X210c compute nodes, and Cisco UCS C-Series rack servers provide:

• Integrated redundancy for high availability.
• Hot-swappable components for serviceability, upgrade, or expansion.
• Fewer physical components than in a comparable system, built piece by piece.
• Reduced cabling.
• Improved energy efficiency over traditional blade server chassis.

For VMware deployments, the 3-Tier Platform can scale from three to hundreds of Cisco UCS servers. The maximum number of servers requires:

• A maximum of seven Cisco UCS domains
• Ten chassis per domain with the two-link connection option for Cisco UCS B-Series blade servers
• The underlying storage array configuration
• Consistent server models within the domain or system
• Connected storage arrays that support the configured host count.

Mixed server models are supported on a Cisco UCS 5108 chassis, but they may impact the maximum permitted quantity of servers. A mixed chassis environment of a Cisco UCS 5108 chassis and a Cisco UCS X9508 chassis in the same domain is also supported.

The following table shows the maximum number of supported Cisco UCS servers by domain and by system:

The Cisco UCS 64108 FI and Cisco UCS 6536 FI are not supported with a FEX. All rackmount servers are directly connected.

The following table shows the maximum number of Cisco UCS B-Series blade servers to deploy. The limits apply when the blade servers are directly connected to Cisco UCS C-Series rack servers in the same domain.

The following table shows the maximum supported CPUs and memory per Cisco UCS server.

Order CPUs from the factory or by ordering an additional server through the expansion services. By default, all Cisco UCS servers come with local M.2 cards installed for booting the operating system.

Dell Technologies contacts Cisco support for issues and concerns with CPUs that are ordered as part of a converged system.

Cisco UCS-X compute nodes, UCS B-Series blade servers, and Cisco UCS C-Series rack servers contain at least one physical Cisco UCS VIC for server connectivity.

• For Cisco UCS servers, the VIC passes converged FC and IP network traffic through the Cisco UCS 5108 blade server chassis midplane to the FEX.
• Cisco UCS C-Series rack servers can connect directly to each FI.
• Cisco UCS X9508 supports the Cisco UCS X210c M6, X210c M7 and X410c M7compute nodes.

The Cisco UCS X9508 chassis contains Cisco UCS X210c M6, X210c M7 and X410c M7 compute nodes along with 14000 or 15000-series VICs.

The following tables show the uplink connectivity from the Cisco X9508 chassis to the FI.

Directly connecting rackmount servers to the FIs lets Cisco Intersight manage the Cisco UCS C-Series rack servers using a single cable for both management and data traffic. The Cisco UCS VIC connects to the FI using a single connection from each VIC to each FI. This method is used for configurations that require fewer servers.

Different generations of VICs are not supported in a single server. Use the Cisco UCS VIC 14xx or 15xxx series. These servers include the following features:

• 4th-generation Cisco UCS C-Series rack servers with:
  • 25 Gbps per fabric on the Cisco UCS VIC 15428 or 1467 mLOMs and a 1455 PCIe on Cisco UCSM6 servers.
  • Cisco UCS VIC 15428, 1467, and 1455 Small Form-Factor Pluggable (SFP28) cards for the M6 servers. These cards support 10/25-Gbps Ethernet or FCoE.
  • Cisco UCS C-Series rack server Direct Connect license.
• On Cisco UCS C220/240 M7 rackmounts only the following VICs are supported:
  • 15428 mLOM (25G)
  • 15425 PCIe (25G)

The UCS 15238 mLOM can be paired with a 15235 in the PCIe slot of a M7 rackmount server to achieve 4 x 100 Gb connections (200 Gb per fabric).

• 5th-generation Cisco UCS C-Series rack servers in Direct Connect mode with:
  • 100 Gbps per fabric on a Cisco UCS VIC 15238 or 1477 mLOMs and a 1495 PCIe on Cisco UCS M6 servers.
  • The Cisco UCS VIC 15238, 1477, and 1495 are dual-port Quad Small Form-Factor (QSFP28) mLOM cards for the M6 servers. These cards support 40/100-Gbps Ethernet or FCoE.
  • On Cisco UCS C220/240 M7 rackmounts only the following VICs are supported:
    • 15238 mLOM (100 Gb)
    • 15235 PCIe (100 Gb)

The UCS 15428 mLOM can be paired with a 15425 in the PCIe slot of a M7 rackmount to achieve 8 x 25 Gb connections (100 Gb per fabric).

The following table shows the port-to-FI connections on the Cisco UCS VICs:

Cisco UCS has a unique architecture that integrates compute, data network, and storage network access into a common set of components under a single management interface.

Cisco UCS fuses access layer networking and servers. The hardware and software components support the Cisco UCS unified fabric, which runs multiple types of data center traffic over a single converged network adapter. When switching is not used in a Cisco UCS X9508 chassis, network access-layer fragmentation is reduced. All devices in a Cisco UCS domain remain under a single management domain, which uses redundant components to remain highly available.

The following figure shows a mixed server configuration with Cisco UCS B-Series blade servers and Cisco UCS C-Series rack servers.

The following figure shows a mixed-server configuration with the Cisco UCS X9508 compute node chassis and Cisco UCS C-Series rack servers.

For optimal performance, keep all compute domains at an oversubscription rate of 8:1 or lower. The 8 in this ratio is the committed host bandwidth and the Ethernet uplinks, while the 1 is for the FC uplinks.

The following table shows an example of a 5108 chassis with 4-link and 8-link 25G connections to the FI. The FI shows 2-, 4-, or 6-uplink options from a 6454 FI to a 9336C ToR switch. The area that is marked in bold italics is oversubscribed. The 11 chassis with 8-links per side provides 2,200 GB or bandwidth to the FI per side, as compared with 2-uplinks to the ToR, providing only 200 GB of bandwidth. That yields 2.200/200 = 11. To avoid oversubscription, provide 4-upinks from the 6454 FI to the 9336C ToR switch, as shown in the table.

When planning your environment, use the guidance in the preceding table to ensure that you do not oversubscribe on Ethernet or FC connections.

In a Disjoint Layer 2 configuration, traffic is split between two or more networks at the FI to support two or more discrete Ethernet clouds.

Disjoint Layer 2 on the FI must be configured on ports that are not used for the standard LAN uplink traffic.

On the 4th-generation Cisco UCS 6454 or 64108 FIs, the network ports are:

• Non-FC 10 Gbps or 25 Gbps unified ports
• 40 GbE or 100 GbE Uplink ports

Cisco UCS servers connect to different Ethernet clouds that otherwise never connect. Upstream Disjoint Layer 2 networks give access to the cloud to servers or VMs that reside in the same Cisco UCS domain.

The following figure shows an example of Disjoint Layer 2 networking into a Cisco UCS domain.

The 101 and 102 vPCs uplinks are production uplinks that connect to the network layer of the 3-Tier Platform. vPCs 105 and 106 are external uplinks that connect to other switches.

Disjoint Layer 2 network connectivity is also configurable with an individual uplink on each FI.

The following tables provide information about supported cabling on the specified FIs.

For more information about supported cabling on the specified FIs, see:

• Cisco 6454 FI data sheet
• Cisco 64108 FI data sheet
• Cisco 6536 FI data sheet

This chapter describes the 3-Tier network layer requirements, components, topologies, and switches.

The following table shows the Dell recommendations for the 3-Tier LAN network components.

The following table shows the SAN component requirements for 3-Tier Platforms.

LAN connectivity includes Cisco Nexus data plane switches. The compute layer connects to Ethernet components in the network layer.

The FIs connect to the Cisco Nexus data plane switches through 10 GbE, 40 GbE, or 100 GbE port channels. The LAN supports 10 Gbps, 40 Gbps, and 100 Gbps uplinks to the external network. It also supports 10 Gbps, 25 Gbps, 40 Gbps, or 100 Gbps access for attached devices.

When incorporating a LAN switch into a 3-Tier platform, consider the airflow in your data center design.

The LAN 93108TC-FX3P, 92348GC-X, or 9336C-FX2 switches may be configured with either port-side intake or port-side exhaust airflow. Follow these best practices:

• If the switch is configured with port-side intake, the ports face the cold aisle.
• If the switch is configured with port-side exhaust, the ports face the hot aisle.

The 3-Tier platform LAN layer consists of switches that manage, route, and store data.

The following figure shows the high-level architecture of the 3-Tier Platform with Layer 3 uplinks and Open Shortest Path First (OSPF).

The 3-Tier Platform supports a Layer 2 uplink topology or a Layer 3 routing-protocol-based uplink topology.

Both routing topologies are supported on the management plane and the data plane. Configure the uplinks as 10 Gbps, 40 Gbps, or 100 Gbps to the external network. The 3-Tier Platform is configured with a pair of ToR Cisco Nexus 9300 Series switches.

The 3-Tier Platform is connected to the external network over L2 links. Routing for all 3-Tier Platform and external VLANs occurs at the external network.

This scenario applies if you need L2 adjacency between endpoints both inside and outside the3-Tier Platform. L2 uplinks allow workload mobility without requiring a network overlay such as VXLAN, because a subnet can be consumed both inside and outside the 3-Tier Platform.

L2 uplinks reduce the amount of traffic that traverses the uplinks to the external network for 3-Tier Platform inter-VLAN communication, improving overall performance.

With an L3 uplink scenario, the 3-Tier Platform implements the OSPF, Enhanced Interior Gateway Routing Protocol (EIGRP), or static routing protocol.

These protocols provide L3 routing services for east-west traffic within the ToR switches and north-south flows for traffic that is leaving the 3-Tier Platform.

Management switches provide OOB management connectivity to all the connected devices, such as:

• Cisco UCS Fis
• Cisco ToR switches
• Management solution components
• Dell storage devices
• Dell data protection devices

The management network layer of the 3-Tier Platform includes at least one pair of Cisco Nexus 93108TC-FX3P or Cisco Nexus 92348GC-X management switches.

For systems with more than one pair of Cisco Nexus 93108TC-FX3P or 92348GC-X management switches, use two 9336C-FX2 management aggregation switches to aggregate management traffic.

• 93108TC-FX3P switch: Provides 100 Mbps, 1 Gbps, or 10 Gbps connectivity to the management interfaces of the 3-Tier Platform components and the management solution components.
• 92348GC-X switch: Provides 1 Gbps, 10 Gbps, or 25 Gbps connectivity to the management interfaces of the Tier 3 Platform components and the management solution components.

The following table shows the management network layer components.

The Cisco Nexus 92348GC-X management switches are deployed in pairs.

The following components connect to the Cisco Nexus 92348GC-X switches using Category 6 connections at 100 Mbps, or 1 Gbps:

• FI management
• Cisco Nexus 9336C-FX2 switch management aggregation (if present)
• Cisco MDS SAN switch management
• Storage array management
• Data protection appliance connections
• Management solution servers Cisco IMC
• Management plane connections for either a single system or multisystem management solution
• Management solution Dell Unity XT management and iSCSI
  • Management solution configurations using the Cisco Nexus 92348GC-X switch can be single system or multisystem management solutions with a Dell Unity XT 480 array. The array is configured with 25 GbE optical connectivity through the mezzanine adapter, which connects to the Cisco Nexus 92348GC-X switch using SFP28 25 GbE ports.
  • Connects the servers to the Cisco Nexus 92348GC-X switch using QSFP 4 x10 GbE breakout ports.

The Cisco Nexus 92348GC-X switches uplink to the external network using multimode fiber at 10 Gbps, 25 Gbps, or 40 Gbps.

Uplink connectivity options are limited to QSFP 10 GbE breakout ports using multimode fiber if:

• The Cisco Nexus 92348GC-X supports either a single system or multisystem management solution with a Unity XT 480 array that is configured with optical connectivity.

The Cisco Nexus 92348GC-X management switch supports four fixed 10 Gbps, 25 Gbps SFP28, or two 40 Gbps QSFP28 uplink ports for the management layer.

Two uplinks are required per management switch.

The base Cisco Nexus 93108TC-FX3P switches are deployed in pairs. Each switch provides:

• 48 fixed ports with speeds of 100 Mbps, 1 GbE, or 10 GbE Base-T
• Six QSFP28 ports to provide four 10 GbE, 25 GbE, 40 GbE, or 100 GbE connections.

The following components connect to the Cisco Nexus 93108TC-FX3P switches using Category 6 connections at 100 Mbps, 1 Gbps, or 10 Gbps:

• FI management
• Cisco Nexus 9336C-FX2 Switch management aggregation (if present)
• Cisco MDS SAN switch management
• Storage array management
• Data protection appliance connection
• Management solution servers Cisco IMC
• Management plane connections for either a single system or multisystem management solution
• Management solution Dell Unity XT management and iSCSI

The Cisco Nexus 93108TC-FX3P switches uplink to the external network using multimode fiber at 10 Gbps, 25 Gbps, 40 Gbps, or 100 Gbps.

At least two uplinks to the external core per management switch are required.

The Cisco Nexus 9336C-FX2 switch provides thirty-six 40 Gbps or 100 Gbps QSFP28 ports. The 9336C-FX2 switch is used in the management plane to aggregate pairs of Cisco Nexus 92348GC-X or Cisco Nexus 93108TC-FX3P switches.

The management aggregation layer enables a larger scale for the management network and is needed only if the solution requires more than 96 management ports.

Each Cisco Nexus 92348GC-X switch or Cisco Nexus 93108TC-FX3P switch has one or two uplinks to each Cisco Nexus 9336C-FX2 switch. The 9336C-FX2 switch provides:

• Up to 40 Gbps QSFP ports to aggregate traffic between multiple pairs of Cisco Nexus 92348GC-X or Cisco Nexus 93108TC-FX3P switches for management.
• 100 Gbps QSFP ports for the vPC peer link
• 10 Gbps, 40 Gbps, or 100 Gbps uplinks to the external network

The 3-Tier Platform network layer supports small or large-scale environments.

For a single 3-Tier Platform with no management aggregation, the least complex management network deployment includes two Cisco Nexus management switches that are connected directly to the external network in L3 mode.

The base management network topology allows management network Switch Virtual Interfaces (SVIs) and Hot Standby Router Protocol (HSRP) addresses to reside on either:

• The 3-Tier Platform management switches (L3)
• The external network (L2)

The following figure shows the base management network topology using a pair of Cisco Nexus management switches.

For a single 3-Tier Platform, the management aggregation layer enables a larger scale for the management network.

The management network L3 SVIs and HSRP addresses can reside on either:

• The 3-Tier Platform management aggregation switches
• The external network

There is one uplink from each management switch to each management aggregation switch at 40 Gbps.

The following figure shows the network topology for the 3-Tier Platform with the management aggregation layer using a pair of Cisco Nexus management switches.

The management aggregation switches provide the management network for multiple 3-Tier Platforms:

• All management network Layer 3 SVIs and HSRP addresses reside on the 3-Tier Platform management aggregation switches.
• Each connected 3-Tier Platform has a unique VLAN ID and subnet for OOB management.

When adding a second 3-Tier Platform to an existing management plane, you can add two Cisco Nexus 9336C-FX2 management aggregation switches to the second 3-Tier Platform.

The following figure shows multiple Tier 3 systems with management aggregation using two pairs of Cisco Nexus management switches.

The following table shows the LAN Layer 2 and 3 uplink model requirements for LAN licenses.

For solutions that include management aggregation switches, see the information about Cisco Nexus 9336C-FX2 Data Plane Switch software licenses in Software licensing. Only the management aggregation switch pair is licensed, if present.

The data plane network layer carries all the production network traffic that flows through the 3-Tier Platform.

The Cisco Nexus 9336C-FX2 switch provides 36 x 40 Gbps or 36 x 100 Gbps QSFP28 ports.

Each QSFP28 port can be divided into four 10 Gb connections or four 25 Gb connections to facilitate connections to SFP+ or SFP28 based interfaces. There are no restrictions on dividing ports on a Cisco Nexus 9336C-FX2 data plane switch. This switch can be used for the ToR network and for additional switch pairs to support PowerScale connectivity.

Two 100 Gbps direct-attach connections form the vPC peer link with the Cisco Nexus 9336C-FX2 switches.

Uplinks from the Cisco UCS 6454, Cisco UCS 64108, and Cisco UCS 6536 FIs to the Cisco Nexus 9336C-FX2 switch use:

• Cisco 100 Gbps QSFP Active Optical cables for the first two Cisco UCS domains
• QSFP optical bi-directional transceivers for Cisco UCS domains 3 through 7

The Cisco UCS 6454 FIs can consume four, eight, or 12 total ports on a pair of Cisco Nexus 9336C-FX2 switches for a standard or enhanced bandwidth uplink topology:

• (Standard) FI 2-uplink (each FI has two uplink ports): Each pair of Cisco UCS 6454 FIs consumes four QSFP ports.
• (Enhanced) FI 4-uplink (each FI has four uplink ports): Each pair of Cisco UCS 6454 FIs consumes eight QSFP ports.
• (Enhanced) FI 6-uplink (each FI has six uplink ports): Each pair of Cisco UCS 6454 FIs consumes 12 QSFP ports.

The Cisco UCS 64108 FIs can consume eight, 12, or 16 ports on a pair of Cisco Nexus 9336C-FX2 switches for a standard or enhanced bandwidth uplink topology:

• (Standard) FI 4-uplink (each FI has four uplink ports): Each pair of Cisco UCS 64108 FIs consumes eight QSFP ports.
• (Enhanced) FI 6-uplink (each FI has six uplink ports): Each pair of Cisco UCS 64108 FIs consumes 12 QSFP ports.
• (Enhanced) FI 8-Uplink (each FI has eight uplink ports): Each pair of Cisco UCS 64108 FIs consumes 16 QSFP ports.

The Cisco UCS 6536 FIs can consume four, eight, 12, or 16 total ports on a pair of Cisco Nexus 9336C-FX2 switches for a standard or enhanced bandwidth uplink topology:

• (Standard) FI 2-uplink (each FI has two uplink ports): Each pair of Cisco UCS 6536 FIs consumes four QSFP ports.
• (Standard) FI 4-uplink (each FI has four uplink ports): Each pair of Cisco UCS 6536 FIs consumes eight QSFP ports.
• (Enhanced) FI 6-uplink (each FI has six uplink ports): Each pair of Cisco UCS 6536 FIs consumes 12 QSFP ports.
• (Enhanced) FI 8-uplink (each FI has eight uplink ports): Each pair of Cisco UCS 6536 FIs consumes 16 QSFP ports.

The following table shows the supported connection types to the Cisco Nexus 9336C-FX2 ToR switches for Cisco UCS 4th-generation and 5th-generation FI uplinks.

A Cisco QSFP-40G-SR4 or QSFP-100G-SR-S transceiver connects the 10 Gbps, 25 Gbps, or 40 Gbps NAS ports on a storage array to the Cisco Nexus 9336C-FX2 switch.

A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes provide fiber connectivity to the endpoint. NAS devices are cross-connected to both switches.

The following table shows the transceiver connections for each ToR switch.

A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes provide four 25 Gbps LC fiber connectivity from each QSFP module to the endpoint. These connections are used for SDNAS and intracluster communication.

The following table shows the transceiver connections for each PowerStore T model appliance.

For each node, mezzanine ports 0 and 1 are an LACP vPC pair across the ToR switches. Mezzanine ports connect to the ToR switches as follows:

• Node A Mezz Port 0 connects to ToR switch A
• Node A Mezz Port 1 connects to ToR switch B
• Node B Mezz Port 0 connects to ToR switch B
• Node B Mezz Port 1 connects to ToR switch A

Starting from PowerStoreOS 3.0, you can update the LACP pair from two ports to four ports. Other mezzanine ports are cabled as:

• Node A Mezz Port 2 connects to ToR switch A
• Node A Mezz Port 3 connects to ToR switch B
• Node B Mezz Port 2 connects to ToR switch B
• Node B Mezz Port 3 connects to ToR switch A

The following table shows the transceiver connections for each ToR switch.

The following table shows connections to the external network for Cisco Nexus uplinks.

The Cisco Nexus 9336C-FX2 switch has a tiered licensing model with two options:

• Layer 3 uplink model: The NX-OS Essentials or the NX-OS Advanced package. Both packages include all L3 protocols.
• Layer 2 uplink model: A license bundle is not required. Subordinate switches that are connected to 9336C-FX2 switches do not require a tiered license package because they uplink to the ToR switch at Layer 2.

Cisco is moving away from the traditional PAK-based licensing model for Nexus switches. Cisco Smart Licensing is a flexible software licensing method that is designed to simplify the way you activate and manage licenses across your organization.

Smart Licensing Using Policy (SLP) is an enhanced version of Smart Licensing. The aim is twofold:

• To provide a licensing solution that does not interrupt the operations of your network.
• To enable a compliance relationship to account for the hardware and software licenses that you purchase and use.

For Smart Licensing, Cisco requires you to register Cisco Nexus switches with the Cisco Smart Account portal (CSSM) for easier license management. Familiarize yourself with the following guidelines and terminology:

• Smart Account Creation: Connect to the CSSM and create a Cisco Smart account for your organization. You can use an existing smart account or virtual account for Nexus switch registration and license management.

• Communication method: The method that the Cisco Nexus switch uses to communicate with CSSM for registration, license, and usage information.
• Offline - air gap: No direct connectivity is required to Cisco or CSSM. The RUM report is generated on the switch and manually uploaded to CSSM to register the switch. You must manually download the ACK file from the CSSM portal and import it back into the switch.
• Direct Internet: The switch requires connectivity to CSSM. The switch communicates with CSSM over the Internet for license registration and management.
• Direct Internet with proxy server: The switch communicates with the Cisco Smart Account portal using the Internet through a proxy server. The switch communicates with the CSSM for license registration and management.
• On-premises Cisco Smart License Utility: The Cisco Smart License Utility (CSLU) is deployed at the customer site to consolidate the reports from all the Cisco end devices. The CSLU can communicate to the CSSM on behalf of the end devices over the Internet or offline by uploading reports manually.

For more information, see the Cisco Nexus 9000 and 3000 Series NX-OS Smart Licensing Using Policy User Guide.

This chapter describes the SAN network layer requirements and storage array components that are used in the 3-Tier Platform architecture.

Two Cisco MDS switches or directors provide connectivity between the compute layer and the storage layer:

• The Cisco MDS model depends on the speed and total number of SAN ports that are required to support the compute and storage components. Connections from the storage components are over 32 Gbps connections.
• The number of storage connections varies based on the storage platform and size.

Cisco UCS 4th and 5th generation FIs provide an FC port channel with one of the following connections:

• Four 32 Gbps connections (128 Gbps)
• Eight 32 Gbps connections (256 Gbps)
• Sixteen 32 Gbps connections (512 Gbps) for 5th-Gen FI

The 32 Gbps connections require 32 Gbps SFPs and SAN ports. The Cisco MDS 9148T multilayer fabric switch supports eight connections from the Cisco UCS domain and up to two domains.

The Cisco MDS switches provide:

• FC connectivity between the compute and storage layer components
• Connectivity for backup and business continuity requirements (if configured)

The Cisco MDS switches automatically negotiate the port speed, based on the SFPs in the FIs. Front-end IOMs in the storage array connect to the Cisco MDS switches in the network layer over 32 Gbps FC links.

When incorporating any SAN switch into a 3-Tier Platform, consider airflow in the data center design.

Some of the switches can be configured with either port-side intake or port-side exhaust airflow, while others can only be configured with port-side intake.

• If the switch is configured with port-side intake, the ports face the cold aisle.
• If the switch is configured with port-side exhaust, the ports face the hot aisle.

The following table describes Cisco MDS switches.

The Cisco MDS 9148T multilayer fabric switch provides 24 to 48 line-rate ports for nonblocking, 32 Gbps throughput. In the base configuration, 24 ports are licensed. More ports can be licensed as necessary in eight-port increments.

The following table shows core connectivity for the Cisco MDS 9148T multilayer fabric switch per fabric.

The base license includes 48 ports. More ports can be licensed in 16-port increments. The Cisco MDS 9396T 32G Multilayer Fabric Switch is a 96-port fixed switch with no IOM expansion slots.

The following table describes core connectivity for the Cisco MDS 9396T 32 G Multilayer Fabric Switch per fabric.

Cisco MDS 9706-v2 and 9710-v2 multilayer directors are modular directors that provide greater scalability than the fixed SAN switches.

Both models deploy two supervisor modules for redundancy and three fabric modules for intradirector communication.

The Cisco MDS 9700 multilayer directors are director-class SAN switches with IOM expansion slots for 48-port 32 Gbps FC line cards. The Cisco MDS 9706-v2 switches and the Cisco MDS 9710-v2 multilayer directors have four and eight slots available for line cards, respectively. Each line card must be ordered in pairs per director and added in pairs to each director. Port licenses are not required for the 9700 multilayer directors.

Only four fabric-3 modules for each Cisco MDS 9706-v2 and 9710-v2 multilayer directors are required for N+1 redundancy and for full-line rate support for all 32 Gbps FC line cards.

The following table shows core connectivity per fabric for the Cisco MDS 9706-v2 and 9710-v2 multilayer directors.

The following figure shows a 3-Tier network block storage configuration using PowerMax arrays.

The PowerMax family of products is designed to provide a fully active/active storage solution. It meets the ever-increasing levels of performance, storage density, intelligent automation, and high security required by enterprise data centers and applications.

The PowerMax family consists of the legacy PowerMax 2000 and 8000 models, and the latest next generation PowerMax 2500 and 8500 systems. The architecture, features, and functions of these systems provide the following core value propositions for Dell Technologies customers:

• Performance: PowerMax systems provide high performance and low latency for mission-critical applications.
• Efficiency: PowerMax systems offer high storage density and intelligent automation to reduce operational costs.
• Resiliency: PowerMax systems provide high availability and data protection to ensure business continuity.
• Security: PowerMax systems offer end-to-end security and compliance to protect data from cyber threats.

When introduced, the PowerMax 2000 and 8000 arrays raised standards again for enterprise storage, delivering unmatched levels of performance and consolidation for high-value, high-demand workloads. These arrays support 32 Gb/s NVMe/FC to deliver on the promise of end-to-end NVMe, along with Storage Class Memory Drives (SCMs) used as persistent storage.

The PowerMax 2000 and 8000 arrays provide all the features and proven data services that are demanded of an enterprise active/active controller array. These features include security, protection, availability, scalability, and massive consolidation. They are now delivered at latencies that are measured in microseconds, not milliseconds.

The Next Generation PowerMax storage platform is designed to offer industry-leading cyber resiliency, security, and intelligent automation. It provides a balance of high performance with remarkable efficiency.

The Next Generation PowerMax family consists of two storage systems:

• The PowerMax 2500 model provides organizations with an attractive, highly efficient entry point into mission-critical storage. It includes up to 7x1 more storage capacity (8PBe) and a 2.8 x improvement in IOPS per watt over the PowerMax 2000 in half the footprint.
• The PowerMax 8500 model delivers extreme performance at scale. Customers can start small with two nodes and grow large to 16 nodes and up to 18 PBe of total capacity. The 8500 model is ideal for the most demanding mixed workloads that require always-on operations with industry-leading, advanced cyber resiliency including the PowerMax cyber vault solution.

Both the PowerMax 2500 and 8500 models use industry-leading rich data services. They excel at workload consolidation because both systems provide storage for block, file, and mainframe workloads.

Based on the powerful Dynamic Fabric architecture and Flexible RAID, the next generation PowerMax systems offer a powerful yet flexible design. This design enables them to independently grow nodes and storage capacity in increments of a single drive. The PowerMax 2500 and 8500 arrays use Intel® Xeon® Scalable processors and the most advanced storage technologies available today. These features include end-to-end NVMe, InfiniBand 100 Gb/s, dual-ported NVMe flash drives, NVMe/TCP, and NVMe/FC connectivity. Both models employ hardware-based data reduction and come with a 5:1 data reduction guarantee for Open Systems and 3:1 for Mainframe. Each PowerMax model is designed for six nines of availability and ships with new intelligent PDUs that provide real-time power consumption monitoring and alerting.

The following table provides a high-level comparison between the PowerMax legacy and next-generation systems.

For more information about the PowerMax family of products go to:

• Dell Technologies PowerMax Storage Home Page: PowerMax overview content, product comparisons, value propositions, use cases, and white papers.
• Dell Technologies PowerMax Support Home Page: PowerMax product guides, installation guides, and other “how to” manuals.

Dell Unity XT systems provide a powerful storage system with a cost-efficient and space-efficient profile. Some of the Dell Unity highlight features include:

• Dual-Active architecture: Unity XT uses both Storage Processors (SPs) to serve host I/O and run data operations in an active/active manner to make efficient use of all available hardware resources. Doing so optimizes performance, cost, and density in customer data centers.
• Truly unified offering: Unity XT delivers a full block and file unified environment in a single 2U enclosure. Use the same pool to provision and host LUNs, Consistency Groups, NAS Servers, File Systems, and vVols alike. The Unisphere management interface offers a consistent look and feel whether you are managing block resources, file resources, or both.
• A modern, simple interface: Unisphere, the Unity XT management interface, has been built with the modern-day data center administrator in mind. Using browser-native HTML5, Unisphere can be used across various operating systems and web browsers without the need for additional plug-ins. The interface has been designed to mimic the practical flow of the daily life of an administrator. It organizes the provisioning and management functions into easy-to-find categories and sections.
• Flexible deployment options: Unity XT enables deployment offerings for a range of different use cases and budgets. These scenarios range from the virtual offering of Dell UnityVSA to the purpose-built Unity XT platform. The purpose-built system can be configured as an All-Flash system with only solid state drives. It can also be configured as a Hybrid system with solid state and spinning media to deliver the best on both performance and economics.
• Inline data reduction: Data reduction technologies play a critical role in environments in which storage administrators are attempting to do more with less. The Unity XT Data Reduction feature aids in this effort by attempting to reduce the amount of physical storage that is required to save a dataset. This feature helps reduce the total cost of ownership of the storage system. Data Reduction provides space savings by using data deduplication and compression. Data Reduction is easy to manage, and once enabled, the storage system controls it intelligently.
• Optional I/O modules: A wide variety of connectivity options are supported on the purpose-built platform. Also, I/O Modules that support iSCSI and NAS may be used for both simultaneously.
• Expanded file system: At its heart, the Unity XT File System is a 64-bit based file system architecture that provides increased maximums to keep pace with the modern data center. Provision file systems and VMware NFS Datastores in sizes as large as 256 TB and enjoy creating multiple millions of files per directory and subdirectories per directory.
• Native data protection: Security and availability of data are critical concerns for many customers, and Unity XT offers multiple solutions to address this need. Unified snapshots provide point-in-time copies of block and file data that can be used for backup and restoration purposes. Asynchronous Replication offers an IP-based replication strategy within a system or between two systems. Synchronous Block Replication benefits FC environments that are close together and require a zero-data loss schema. Data at Rest Encryption (D@RE) ensures that user data on the system is protected from physical theft. It can substitute for drive disposal processes, such as shredding.
• VMware integration – Discovery of a VMware environment has never been easier, with Unity XT VMware Aware Integration (VAI). Use VAI to retrieve the ESXi host and vCenter environment details into Unisphere for efficient management of your virtualization environment. Support for VMware vStorage APIs for Storage Awareness (VASA) and later enables the provisioning and use of VMware Virtual Volumes (vVols), a VMware ESXi virtualization storage technology. Unity XT supports vVols for both block and file configurations.
• Multiple management paths – Configure and manage your Unity XT system in the way you are most comfortable. The Unisphere UI is browser-based and provides a graphical view of your system and its resources. Use Unisphere CLI (UEMCLI) over SSH or over a Windows host to run CLI commands against the system. Unity XT also has a full REST API library available. Any function possible in Unisphere is also possible using REST API. Developing scripts or integrating management of your system into existing frameworks has never been easier.

For more information about Unity XT, see the Unity XT information page or the Unity XT Info Hub.

In a constantly changing world of increasing complexity and scale, the need for an easy-to-use, intelligent storage system continues to grow. Organizations that use new applications and solutions require dependable storage and often face the challenge of doing more with less. PowerStore addresses this challenge by packaging a powerful storage system into a cost-efficient and space-efficient profile. Key PowerStore features and benefits include:

• Active/active architecture: PowerStore uses both nodes to serve host I/O and run data operations in an active/active manner. This design efficiently uses all available hardware resources and optimizes performance, cost, and density in data centers.
• NVMe platform: PowerStore is designed to use the latest storage and interface technologies to maximize application performance and eliminate bottlenecks. PowerStore can maximize performance with NVMe flash storage. PowerStore supports front-end NVMe connectivity with NVMe over Fibre Channel and NVMe over TCP, for a complete end-to-end NVMe solution.
• VMware integration: PowerStore is designed for deep integration with VMware vSphere, including:
• VAAI
• VASA
• Event notifications
• Snapshot management
• VMware vSphere Virtual Volumes (vVols)
• Virtual machine discovery and monitoring in PowerStore Manager
• Unified offering: PowerStore has a single architecture for block, file, and vVols storage. This architecture uses the latest technologies to provide flexible functionality without sacrificing the cost-effective nature of enterprise storage. PowerStore provides storage in multiple formats to applications, ranging from physical and virtual volumes, to containers and traditional files. This ability provides the ultimate workload flexibility and enables IT to simplify and consolidate infrastructure.
• A modern, simple interface: PowerStore Manager, the PowerStore management interface, is built with the data-center administrator in mind. Using browser-native HTML5, PowerStore Manager can be used across various operating systems and web browsers without requiring an external management server or appliance.
• Inline data reduction: Data reduction technologies play a critical role in environments in which storage administrators are attempting to do more with less. PowerStore data reduction supports this effort by optimally reducing the amount of physical storage that is required to save a dataset. PowerStore data reduction provides space savings by using software data deduplication and compression through hardware assist. The storage system always enables and intelligently controls data reduction.
• Native data protection: Security and availability of data are critical concerns for many organizations, and PowerStore storage offers multiple solutions to address this need. Snapshots provide point-in-time copies of block, file, and virtual machine data that can be used for backup and restoration purposes. Asynchronous replication offers an IP-based replication strategy within a system or between two systems. Data at Rest Encryption (D@RE) ensures that user data on the system is protected from physical theft. It can substitute for drive disposal processes, such as shredding.

For more information about PowerStore, see the Dell PowerStore information page or the PowerStore Info Hub.

The PowerScale family includes PowerScale storage arrays. The scale-out NAS storage solution delivers increased performance for file-based data applications and workflows from a single file-system architecture.

The PowerScale all-flash storage arrays can exist in the same cluster with Isilon Gen 6 nodes to drive your traditional and modern applications.

PowerScale provides scale-out capacity for use as NFS and SMB CIFS shares in the VMware vSphere VMs. PowerScale is available in the following configurations:

• PowerScale All-Flash Scale-out NAS
• PowerScale Hybrid Scale-out NAS
• PowerScale Archive Scale-out NAS

The following table shows the hardware components that are used with each configuration.

The Cisco Nexus 9336C-FX2 is the only supported switch for PowerScale node front-end connectivity.

The following table shows the PowerScale back-end switches that support Isilon Gen 6 nodes and PowerScale Gen 6.5 nodes.

* The Arista 7304 ships with two line cards, each with 48 10 GbE ports. You can add:

• One additional line card with 48 x 10 GbE ports
• Two additional line cards with 32 x 40 GbE ports

** The Arista 7308 ships with two line cards, each with thirty-two 40 GbE ports. You can add:

• Three additional line cards with 32 x 40 GbE ports
• Two additional line cards with 48 x 10 GbE ports

The PowerScale back-end Ethernet switches provide:

• 10 GbE 24 port
• 10 GbE 48 port
• 10 GbE 96 port (2 x 48-port leaf modules)
• 10 GbE 48-port leaf module
• 40 GbE or 100 GbE 32 port
• 40 GbE 64 port (2 x 32-port leaf modules)
• 40 GbE 32-port leaf module
• 40 GbE or 100 GbE 32-port leaf module
• 40 GbE or 100 GbE 64 port

For leaf and spine implementations:

• The Dell S5232 and Z9264 switches are supported for leaf-spine architecture.
• The same switch is used for both the leaf and the spine.

• Front end: 10 GbE, 25 GbE, 40 GbE, or 100 GbE optical (depending on the node type and generation)
• Back end: 10 GbE, 25 GbE, 40 GbE, or 100 GbE optical (depending on the node type)
• Single onboard 1 GbE
• DB-9 serial connection

• Up to 2 x 2.5 in. SSD for cache
• Up to 2 x M.2 vault disks
• Dual 1100 W or 1450 W PDUs

For more information, see Guidelines and Limitations for Layer 2 Interfaces.

PowerScale All-Flash F210 model is a 1U node with the following components:

• Four NVMe SSDs per node
• Single-socket CPU Intel® Sapphire Rapids®, 2G, 12C
• 128 GB ECC memory per node
• The front-end is a dual-port NIC:
  • Dual-port 25 G NIC supporting 10 G or 25 G connections (SFP+/SFP28)
  • Dual-port 100 G NIC supporting 40 G or 100 G connections

• The back-end is a dual-port 100 G NIC supporting 100 G or 25 G connections.
• Dual redundant 286 W PSU that operates at 100 V to 240 V.

The following table shows the PowerScale F210 array hardware and software specifications.

PowerScale All-Flash F710 model is a 1U node with the following components:

• Ten NVMe SSDs per node
• Dual socket CPU Intel® Sapphire Rapids®, 2G, 12C
• 512 GB ECC memory per node
• The front-end is a dual-port NIC:
  • 100 G NIC supporting 100 connections (QSFP+/QSFP28)
  • 25 G NIC supporting 25 connections (SFP+/SFP28)

• The back-end is a Dual port 100 G NIC supporting 40 G or 100 G connections (QSFP+/QSFP28).
• Dual redundant 769 W PSU that operates at 100 V to 240 V.

The following table shows the PowerScale F710 array hardware and software specifications.

PowerScale All-Flash F910 model is a 2U node with the following components:

• 24 NVMe SSDs per node
• Dual socket CPU Intel® Sapphire Rapids®, 2.6G, 24C
• 512 GB ECC memory per node
• The front-end is a dual-port NIC:
  • 100 G NIC supporting 100 G connections (QSFP+/QSFP28)
  • 25 G NIC supporting 25 G connections (SFP+/SFP28)
• The back-end is a Dual port 100 G NIC supporting 40 G or 100 G connections (QSFP+/QSFP28)
• Dual redundant 877 W PSU that operates at 100 V to 240 V.

The following table shows the PowerScale F910 array hardware and software specifications.

PowerScale All-Flash F200 model is a 1U node with the following components:

• Four 25-inch SSD drives
• Single-socket CPU
• 48 GB or 96 GB ECC memory (per node)
• The front-end is a dual-port 25 G NIC supporting 10 G or 25 G connections (SFP+/SFP28)
• The back-end is a dual-port 25 G NIC supporting 10 G or 25 G connections (SFP+/SFP28)
• Dual redundant 750 W PSU that operates at 100 V to 240 V, and 50 Hz or 60 Hz.

The following table shows the PowerScale F200 array hardware and software specifications.

The PowerScale All-Flash F600 model is a 1U node with the following components:

• Eight 2.5-inch NVMe SSD drives
• Dual-socket Intel® Cascade Lake® CPU
• 8 or 12 DDR4 DIMMs
• Front-end dual-port 25 G NIC supporting 10 Gb or 25 Gb connections (SFP+ or SFP28) or dual-port 100 G NIC supporting 40 Gb or 100 Gb connections (QSFP+ or QSFP28)
• Back-end dual-port 100 G NIC supporting 40 Gb or 100 Gb connections (QSFP+ or QSFP28)
• Dual redundant 750 W PSU that operates at 100 V to 240 V, and 50 Hz or 60 Hz.

The following table shows the PowerScale F600 model hardware and software specifications.

The PowerScale All-Flash F900 is a 2U node containing the following components:

• Twenty-four 2.5-inch NVMe SSD drives
• Dual Socket Intel® Cascade Lake® 6240R (2.4 GHz, 24C)
• 23 DDR4 DIMMs
• Front-end dual-port 25 Gb NIC supporting 10 Gb or 25 Gb connections (SFP+ or SFP28) or dual-port 100 Gb NIC supporting 40 Gb or 100 Gb connections (QSFP+ or QSFP28)
• Back-end dual-port 100 Gb NIC supporting 40 Gb or 100 Gb connections (QSFP+or QSFP28)
• Dual redundant 750 W PSU that operates at 100 V to 240 V, and 50 Hz or 60 Hz.

The following table shows the hardware and software specifications for the PowerScale F900 array.

PowerScale Gen-6 hybrid and archive models are contained in a four-node chassis.

This topic describes the PowerScale A300 and A3000 nodes.

PowerScale uses uplinks and peer-links to connect the ToR Cisco Nexus 9000 series switches to the 3-Tier Platform.

The PowerScale network topology uses the following port channels:

• Port channel 3: Uplinks to connect the PowerScale ToR switch and the 3-Tier Platform ToR switch.
• Port channel 50: Peer-links to the 3-Tier Platform ToR switch. More Cisco Nexus 9000 Series Switch pair uplinks start from port channel or vPC ID 4 and increase for each switch pair.
• Port channel 51: Peer-links to the Converged Technology Extension for PowerScale ToR switches. More Cisco Nexus 9000 series switch pair peer-links start from port channel 52, and increase for each switch pair.
• Port channel 1001: PowerScale node and ToR switch. More PowerScale nodes start from port channel or vPC ID 1002 and increase for each LC node.

Adhere to the following restrictions:

• The last four ports on the PowerScale ToR switches are reserved for uplinks. Depending on the switch, more uplinks may be used.
• vPC connections between the PowerScale switches and the 3-Tier Platform switches must be cross-connected.
• The two ports immediately preceding the uplink ports on the PowerScale switches are reserved for peer-links.
• All the ports that are not uplinks or peer-links are reserved for nodes.

Create a port channel for the nodes starting at PC or vPC 1001 to directly connect the Isilon nodes to the 3-Tier Platform ToR switches.

The following table shows the vPC maximum limit for Cisco Nexus 9000 Series NX-OS release 9.3 (1) and later.

With PowerScale OneFS 9.x, the leaf-spine back-end or flat back-end topology supports a maximum of 252 nodes on PowerScale clusters.

PowerScale leaf-and-spine architecture is based on the maximum internal bandwidth and 32-port count of Dell Z9100 and Dell S5232 switches or the 64-port count of Dell Z9264 switches.

A leaf-and-spine architecture provides the following benefits:

• Supports high scalability.
• Minimizes latency and the likelihood of bottlenecks in the back-end network.

PowerScale supports only Ethernet as the back-end network.

The following table shows the minimum OneFS software version that is required on PowerScale nodes.

Back-end leaf-and-spine network deployments can have a minimum of one spine switch and two leaf switches. For small-to-medium clusters, the back-end network includes a pair of redundant ToR switches. Dell S5232 and Dell Z9264 Ethernet switches are supported in the leaf-and-spine architecture. The same switch is used for both the leaf and the spine.

The back-end architecture contains a leaf-and-spine layer. The nodes connect to leaf switches in the leaf layer. The aggregation and core network layers are condensed into a single spine layer.

The leaf-and-spine architecture requires that:

• Every leaf switch connects to every spine switch.
• Connections from the leaf switch to the spine switch must be evenly distributed.
• There should be the same number of connections to each spine switch from each leaf switch.
• Cluster nodes connect to leaf switches that use spine switches to communicate.
• Same-type switches (leaf or spine) do not connect to one another.
• Downlinks (links to PowerScale nodes) support one 40 Gbps or four 10 Gbps using a breakout cable.
• Downlinks (links to PowerScale nodes) support one 100 Gbps or four 25 Gbps using a breakout cable.
• Downlinks (links to PowerScale nodes) support one 40 Gbps or one 100 Gbps connectivity.

The following guidelines apply to the back-end switch configuration for PowerScale clusters:

• OneFS version 9.x requires switch operating system version 10.5.0.6C2 or higher.
• The switch operating system version 10.5.0.6C2 or later requires manual designation of a personality through the CLI for Dell back-end network switches. The exception is the Dell S5232 switch, which requires operating system version 10.5.2.7 or later for a flat network topology.
• You may have to:
• Upgrade the switch operating system version to 10.5.0.6C2 or later.
• Manually designate a personality through the CLI from the switches for a flat network or leaf-and-spine network configuration.
• For information about those procedures, look for the following documents on Dell Support:
  • Dell SmartFabric OS10 Installation Guide
  • Upgrade and Downgrade Guide
  • Leaf-Spine Installation Guide
• To support leaf-spine network topology, the Dell S5232 switch requires the minimum switch operating system version to be 10.5.2.9. If you add S5232 to a leaf-spine network consisting of Z9100 or Z9264 switches, the operating system of all the switches in the cluster must be upgraded to 10.5.2.9 to support leaf-spine topology.

Scale planning prevents recabling of the backend network. Scale planning makes it easier to upgrade by installing the projected number of spine switches and scaling the cluster by adding leaf switches. The leaf-spine back-end network supports a maximum of 252 nodes with 10 GbE, 25 GbE, or 40 GbE back-end connectivity. It supports a maximum of 128 nodes using 100 GbE front-end connectivity.

The following table shows the switch requirements for the interfaces connecting to the A side network as the cluster scales. The same number of additional switches are required for the B side network. The A and B networks provide redundant paths.

With a 22-downlink 40 GbE connection, you can have up to 88 10 GbE nodes using four 10 GbE breakout cables.

With a 16-downlink 100 GbE connection, you can have up to 64 25 GbE nodes using four 25 GbE break out cables.

• With a 44-downlink 40 GbE connection, you can have up to 128 10 GbE nodes using four 10 GbE breakout cables.
• With a 32-downlink 100 GbE connection, you can have up to 128 25 GbE nodes using four 25 GbE break out cables.
• For 100 GbE nodes, the 3-Tier Platform supports only 128 nodes on a 100 GbE front-end connection. Do not exceed 12 spine or leaf switches per side. (Interface A and Interface B equal 24 switches per cluster.)
• For 40 GbE nodes, the 3-Tier Platform supports up to 252 nodes on a 40 GbE front-end connection. Do not exceed 17 spine or leaf switches per side. (Interface A and interface B equal 34 switches per cluster.)

The uplink bandwidth must be equal to or more than the total bandwidth of all the nodes that are connected to the leaf switch.

For example, each switch has nine downlink connections that at 40 Gbps require 360 Gbps of bandwidth. At least four 100 Gbps uplink connections to the spine layer should be made from that leaf.

The following table shows the recommended uplink (leaf to spine) and downlink (back-end node connectivity) port reservations on the Z9264 and S5232 switches.

OneFS 8.2 and later uses SmartConnect with multiple SmartConnect Service IPs (SSIP) per subnet.

Dell Technologies recommends that:

• The number of SSIPs available per subnet depends on the SmartConnect license. SmartConnect Basic allows two SSIPs per subnet, while SmartConnect Advanced allows six SSIPs per subnet.
• SmartConnect Multi-SSIP is not an extra layer of load balancing for client connections. More SSIPs provide redundancy and reduce failure points in the client connection sequence.
• SSIPs are supported only for use by a DNS server. Although SSIPs may be used in other configurations, the design intent was for a DNS server. Other implementations with SSIPs are not supported.
• The SmartConnect Service IP addresses and SmartConnect Zone names must not have reverse DNS entries, also known as pointer (PTR) records.
• The number of SSIPs available per subnet depends on the SmartConnect license. SmartConnect Basic allows two SSIPs per subnet, while SmartConnect Advanced allows six SSIPs per subnet. SmartConnect Multi-SSIP is not an extra layer of load balancing for client connections. More SSIPs provide redundancy and reduce failure points in the client connection sequence.
• SSIPs are supported only for use by a DNS server. Other implementations are not supported. The SSIP addresses and SmartConnect Zone names must not have reverse DNS entries, also known as pointer records.

PowerScale contains the OneFS operating system, which provides encryption, file storage, and replication features.

The OneFS operating system is available as a cluster of PowerScale OneFS nodes that contain only self-encrypting drives (SEDs).

The system requirements and management of data-at-rest on self-encrypting nodes are identical to the nodes without self-encrypting drives. Clusters of mixed node types are not supported.

Self-encrypting drives store data on a PowerScale cluster that is designed for Data at Rest Encryption (D@RE). D@RE on self-encrypted drives occurs when data that is stored on a device is encrypted to prevent unauthorized data access. All data that is written to the storage device is stored and encrypted. All data that is read from the storage device is decrypted when it is read.

The OneFS operating system combines file systems, volume managers, and data protection into a single intelligent distributed file system that runs on a PowerScale storage cluster.

The 3-Tier Platform configures the two front-end interfaces of each node in an LACP port channel. The front-end interfaces use SmartConnect to load balance share traffic across the nodes in the cluster depending on the configuration.

The OneFS operating system leverages the SyncIQ licensed feature for replication.

SyncIQ is an application that enables you to manage and automate data replication between two PowerScale clusters. SyncIQ delivers a unique, highly parallel replication performance that scales with the dataset to provide a solid foundation for disaster recovery. SyncIQ can send and receive data on every node in the PowerScale cluster, which means that replication performance is increased as your data grows.

InsightIQ provides performance monitoring and reporting tools to help you maximize the performance of a PowerScale scale-out NAS platform.

With InsightIQ, you can identify performance bottlenecks in workflows and optimize the amount of high-performance storage that is required in an environment. InsightIQ provides advanced analytics to optimize applications, correlate workflow and network events, and monitor storage requirements.

PowerScale OneFS is available in a perpetual and a subscription model with various bundles.

The following table shows the PowerScale license features.

The number of PowerScale nodes that are supported depends on the ports that are available in the ToR switch.

All the node front-end ports are placed in LACP port channels. The front-end ports for each of the nodes are connected to a pair of redundant network switches.

The following table shows the number of nodes that are supported.

You can add as many storage arrays or platforms as the connectivity model allows.

You can combine multiple storage arrays for block-level storage.

Implement the following best practices when combining multiple storage arrays and storage array platforms for block-level storage:

• Each port group per array is spread across the SAN fabric.
• Hosts do not share array workloads when performance is required.
• VMware vSphere ESXi advanced settings are configured.
• Common settings are used to mix storage array platforms.

You can combine multiple storage arrays for file-level storage.

Implement the following best practices when combining multiple storage arrays and storage array platforms for file-level storage:

• Storage array Ethernet ports are spread across the LAN switches.
• Each pair of ports is configured in an LACP port channel.
• Host-level and array-level maximum boundaries are maintained per array.

This chapter describes the high-level design of Integrated Data Protection solutions for a 3-Tier platform.

Integrated Data Protection solutions provide backup systems and business continuity and disaster recovery services. The following table describes the functions that these solutions provide.

Dell Integrated Data Protection provides business continuity services through VPLEX and related technologies.

VPLEX delivers enhanced availability, characterized by zero loss of data and near-zero application downtime. VPLEX also delivers enhanced mobility, which is the migration of applications and data between systems without the overhead of work, planning, and downtime that is associated with traditional migrations.

A VPLEX cluster resides in the data path between the converged system servers and storage, where it can create data copies. Copies are created locally or over distance and read and write access is available.

VPLEX enables dynamic workload mobility and continuous availability within and between converged systems over distance. VPLEX also provides simultaneous access to storage devices at two sites through the creation of VPLEX distributed virtual volumes (vVols). vVols are supported on each side by a VPLEX cluster.

The VPLEX integrated data protection cluster for converged systems uses the following components and technologies:

• One or more converged systems
• VPLEX
• VMware vCenter Server
• VMware vSphere High Availability (HA)
• VMware vSphere vMotion

Dell supports the following VPLEX systems for integrated data protection solutions:

• VPLEX Local: A VPLEX cluster that enables you to manage and mirror data between multiple converged systems from a single interface in a single data center.
• VPLEX Metro: Two VPLEX clusters that are connected over distance by intercluster links. VPLEX Metro enables concurrent read and write access to data by multiple hosts across two locations. By mirroring data between two sites, VPLEX Metro provides nonstop data access if there is a component failure or a site failure.
• The Cluster Witness is an optional component for Metro implementations of VPLEX Metro and Dell Metro node. If cluster failures and intercluster communication loss occur, the Cluster Witness is deployable at a third site to improve data availability. The Cluster Witness is deployed as a VM for VPLEX VS6. Its VMware ESXi host must reside in a separate failure domain from both VPLEX clusters. This separation eliminates the possibility of a single fault affecting both a VPLEX cluster and the Cluster Witness. A customer-supplied VM or physical server running SLES 15 SP3 is required to run the Cluster Witness with Metro node.

The Cluster Witness observes the state of the clusters and can distinguish between an outage of the intercluster link and a cluster failure. The Cluster Witness uses this information together with the preconfigured detach-rules to guide the clusters to resume or suspend I/O.

The following table describes VPLEX use cases in converged systems.

The Metro node hardware platform is based on the PowerEdge R640 server platform. Metro node is designed with embedded management services, I/O path simplification, and 32 Gbps FC connectivity.

Metro node supports the same use cases as the VPLEX VS6 platform with the following exceptions:

• No support for RecoverPoint
• No support for MetroPoint
• No IPv6 support

Metro node is available in a two-node cluster only (single engine). By contrast, the VPLEX VS6 scales out to dual and quad engines. Like the VPLEX VS6, The Metro node supports both the Local and Metro configurations.

The VPLEX product is offered for sale in the following models:

• VPLEX VS6:
  • The VS6 is positioned for customers that have a performance-intensive environment. These customers run all-flash storage arrays or a hybrid mix of storage arrays and want to refresh their VPLEX hardware from the VS2 platform.
  • The VS6 platform provides front-end and back-end speeds at a maximum of 16 Gbps over FC. The VS6 platform uses 40 Gbps InfiniBand for intracluster communication and offers 10 GbE IP or 16 Gbps FC for WAN replication.
• Metro node:
  • Metro node has a small footprint (2U) and does not require a dedicated data protection cabinet. The nodes are racked in the 3-Tier Platform system cabinets.
  • Metro node provides front-end and back-end speeds at a maximum of 32 Gbps over FC and 10 GbE for IP WAN replication.

The following table shows the VPLEX VS6 and Metro node hardware specifications for converged systems.

A VPLEX cluster consists of the following hardware components:

• One VPLEX engine:
  • Backup power protects each engine, which contains two directors. The VS6 Battery Backup Unit (BBU) is internal to the engine.
• One management server consisting of:
  • VS6: The VS6 has two Management Module and Control Stations (MMCS) in the base engine with internal storage. The MMCSs are named MMCS-A and MMCS-B.
  • Cluster Witness, residing in a third failure domain up to one second away.
  • A lightweight VM with one vCPU and 1 GB RAM, a 2.54 GB disk, and IP connectivity.
• Metro node is available in a single engine configuration only:
  • The engine contains two nodes.
  • With Global Cache removed, a dedicated battery backup is no longer required. It is a best practice to supply backup power to the data center through battery backup or a power generator.
  • With the availability of powerful CPUs, a dedicated management server is no longer required. The management services run on both Metro node servers.
  • Intercluster communication is done over four peer-to-peer connections.

The following figure shows the VPLEX VS6 engine management and I/O modules.

A single Metro node cluster has two directors.

The following figure shows one Metro node director.

The following table shows Dell-recommended configurations for business continuity and workload mobility.

VPLEX and Metro node support only the FC protocol. In a vSphere environment, VPLEX and Metro node support only VMFS datastores.

VPLEX and Metro node do not support VMware vVols data stores.

Dell Integrated Data Protection provides disaster recovery services through RecoverPoint for Virtual Machines and related technologies.

RecoverPoint for Virtual Machines software protects VMs residing in a VMware vSphere environment. The product provides local and remote replication capabilities with virtual machine-level granularity.

No version of RecoverPoint for Virtual Machines supports 3-Tier Platform systems that run vSphere 8.0.

RecoverPoint for Virtual Machines and RecoverPoint are different products. RecoverPoint for Virtual Machines refers to the VMware-related product. RecoverPoint refers to the hardware-based product. The following table shows the differences.

RecoverPoint for Virtual Machines vRPAs are installed in the VMware vSphere environment and provide the plug-in into the VMware vSphere web client.

VMware vRPAs use the IP protocol to communicate with the VMware vSphere data store. Each vSphere ESXi host that participates in protecting virtual machines requires the RecoverPoint for Virtual Machines installed splitter.

VMware administrators use the vSphere web client to actively protect and recover virtual machines to any point in time using integrated orchestration and automation capabilities. RecoverPoint for Virtual Machines fully supports the following standard RecoverPoint operations:

• Test Copy
• Recover Production
• Failover

Because RecoverPoint for Virtual Machines provides automation and orchestration capabilities, VMware Site Recovery Manager (SRM) is not required. Use VMware SRM only with RecoverPoint Classic.

Several RecoverPoint for Virtual Machines configurations are supported on converged systems, as shown in the following table.

The RecoverPoint for Virtual Machines system consists of:

• One or more RecoverPoint for Virtual Machines clusters. For the maximum number of clusters that a RecoverPoint for Virtual Machines system can contain, see the latest RecoverPoint for Virtual Machines Release Notes on the Dell Technologies Support website.
• A minimum of one vRPA and a maximum of eight vRPAs for each cluster. The vRPAs manage all aspects of the data replication process.

• A RecoverPoint for Virtual Machines splitter installed on:
  • The VMware ESXi hypervisor.
  • Every VMware ESXi host in the VMware vSphere cluster that hosts virtual machines that RecoverPoint for Virtual Machines protects.

The splitter separates the write coming from the host and sends it to the vRPA and the virtual machine disk format (VMDK). The vRPAs handle all traffic to the journals and replicas as in a physical RecoverPoint system. Storage traffic between the vRPAs and the vSphere datastores uses the IP protocol.

• VMware vSphere Web client plug-in. This plug-in provides the UI for managing the RecoverPoint for Virtual Machines system. The vAdmin manages all operations from this UI. Starting from RecoverPoint for Virtual Machines 5.3 the plug-in comes in two versions:
  • vSphere Flex plug-in for vSphere 6.7 and earlier.

This plug-in is the UI for managing the RecoverPoint for Virtual Machines system. The plug-in communicates directly with the vRPA clusters:

• vSphere HTML5 plug-in for vSphere 6.7 U1 and later.

This plug-in communicates with the vRPA clusters through the HTML5 plug-in server.

The following figure shows the RecoverPoint for Virtual Machines system architecture with the vSphere HTML5 plug-in.

This dedicated plug-in server provides replication management for one or more RecoverPoint for Virtual Machines systems and communicates through the REST API.

The server deploys from an OVA and runs on SUSE Linux 12.5. The plug-in server hosts the HTML5 plug-in and the UI logic. It serves as a single endpoint for the new REST API, through which it manages all RecoverPoint for Virtual Machines systems running 5.3 or later on the vCenter server. A single HTML5 plug-in server can manage one or more vRPA clusters per vCenter Server. Every vCenter server and HTML5 plug-in server supports a maximum of 50 vRPA clusters.

The plug-in server communicates securely with every registered vCenter Server and with every vRPA cluster that is registered with a vCenter Server.

Environments running vCenter in Embedded Linked Mode can install one of the following servers:

• HTML5 plug-in server per vCenter server where vRPA clusters are registered.
• HTML5 plug-in server to manage all vRPA clusters that are registered across the different vCenter servers. RecoverPoint for Virtual Machines supports up to seven vCenter servers using vCenter Enhanced Linked Mode.

A Dell-certified specialist carries out a sizing analysis with the customer using the following considerations:

• Amount of data to be replicated
• Data rate of change
• Maximum rollback window required, by disk volume
• Available network bandwidth

RecoverPoint for Virtual Machines enables a single cluster to replicate to multiple clusters, or multiple clusters to replicate to a single cluster.

RecoverPoint for Virtual Machines supports the following multisite designs:

• Fan-out replication from a local site to up to four remote copies, or to one local and three remote copies. Each copy contains data from the same CG.
• Fan-in replication from up to four remote sites to one central site. Each copy contains data from a different CG.
• Starting from RecoverPoint for VMs 5.2.2 and Cloud DR 19.1 the maximum number of noncloud replica copies per CG is two.

In addition to these multisite options, RecoverPoint for Virtual Machines supports a fully connected system where all sites are connected with each other.

RecoverPoint for Virtual Machines provides the vAdmin with integrated automation and orchestration capabilities to simplify the recovery of virtual machines. The vAdmin accesses these capabilities using the vSphere web client snap-in.

The following table shows the automation capabilities and orchestration capabilities that are available with RecoverPoint for Virtual Machines.

RecoverPoint for Virtual Machines systems provide additional configuration settings to optimize the 3-Tier Platform and allow for maximum growth based on the analyzed data. Here are the key areas that require careful planning for converged systems.

During installation, replace the default admin user password with a unique password. This password is also the password for the root user across all vRPAs in the system.

The predefined admin user is granted all permissions for managing the system, including system engineer (SE) permissions. Starting from release 5.2.0.2 of RecoverPoint for Virtual Machines, the admin account is the only predefined user. For upgrades from RecoverPoint for Virtual Machines 5.1.1.4, the other predefined users remain as they were before the upgrade.

Root access is disabled by default for remote and local access. Access can be enabled or disabled from the Boxmgmt CLI.

RecoverPoint for Virtual Machines appliances use the following Federal Information Processing Standard (FIPS) 140-2 validated cryptographic modules:

• RSA BSAFE® Crypto-J, JSAFE, and JCE Software Modules (FIPS certificate number 2057)
• RSA BSAFE® Crypto-C Micro Edition (FIPS certificate number 2056)
• OpenSSL FIPS Object Module (FIPS certificate number 1747)

All the modules are installed in accordance with the applicable security policies.

All encrypted network connections to and from RecoverPoint for Virtual Machines appliances use only the previously mentioned FIPS 140-2 validated cryptographic modules. On new installations, the vRPA communication security level is set by default to Authenticated and Encrypted.

To make the RecoverPoint for VMs system FIPS-140-2 compliant, verify that the vRPA communication security level is set to Authenticated and Encrypted on all vRPAs. To verify, log in to Boxmgmt as an admin and run the following CLI command: get_security_settings.

RecoverPoint for Virtual Machines supports Message Passing Interface (MPI) security for communication between vRPAs, between vRPA clusters, with storage, and with VMware vCenter servers. MPI security is applied to vRPA communications in the cluster and communications between clusters over WAN (IP). The following table describes the vRPA communication security levels.

The following table shows important scale limitations on RecoverPoint for Virtual Machines. For the complete list of limitations, see the RecoverPoint for Virtual Machines 5.x Scale and Performance Guide on the Dell Technologies Support website.

When configuring RecoverPoint for Virtual Machines solutions, follow the guidelines in the corresponding Security Guide. To obtain the guide:

1. Log in to Dell Technologies Support.
2. Search for RecoverPoint for Virtual Machines Security Configuration Guide.
3. In the search results, click RecoverPoint for Virtual Machines x.x Security Configuration Guide, where x.x stands for the RecoverPoint for Virtual Machines software version.

A CG is a group of virtual machines and their disks that are replicated together in a way that ensures write-order consistency.

The CG is the basic building block where most replication operations occur. Recovering production or enabling Image access is all done per CG. Virtual machines are the entities that are protected, and they reside inside the CG.

CGs ensure write-order consistency on multiple disks in a single virtual machine, but also across multiple virtual machines that reside in the CG. For example, the system can ensure consistency between the application and database servers that reside in the same CG for any point in time. Newly created CGs are load-balanced across the vRPAs in the cluster by the number of CGs per vRPA. For converged systems, after the CG has been replicated for at least a week, check the I/O load on all vRPAs and manually rebalance the CGs across the vRPAs in the cluster.

Unlike RecoverPoint, RecoverPoint for Virtual Machines does not use the concept of distributed CGs. A single CG runs on a single vRPA and is limited by the resources of the vRPA.

Careful planning is required to create CGs and assign virtual machines to each. Review the following guidelines:

• Within a RecoverPoint for Virtual Machines system, which can consist of multiple vRPA clusters, you can replicate up to 256 CGs.
• Fewer CGs improve performance.
• If the load of two virtual machines is greater than what a single vRPA can replicate, protect the virtual machines using two CGs. In this way, each CG runs on a separate vRPA.

If you require write-order consistency between these two CGs, place them together in a group set on which parallel bookmarking is enabled.

Group sets with parallel bookmarking enabled have the following limitations:

• A single group set may contain up to 32 CGs.
• The minimum frequency for parallel bookmarking is every 20 seconds, which means that CGs that use parallel bookmarking do not support recovery to any point in time.

RecoverPoint for VMs supports both VMFS and vVols datastores when used in a 3-Tier Platform system.

Journals are crucial to the performance of RecoverPoint for Virtual Machines systems. Two types of journals are required for any CG:

• Production journals: Product journals are required when a failover event has occurred and the direction of replication has changed. Production journals are required for any CG.
• Copy journals: Copy journals receive successive writes that are written in production. The write-order is maintained, so it is possible to apply or undo writes so that the copy image can reflect any point-in-time. Verify that the underlying performance of the datastore where the journal resides has been configured properly to support the incoming writes for the CG.

Use the values in the following table as a starting point for journal sizing.

To start sizing:

• Begin with a number that is 15 to 25 percent of the total protected VM capacity. Add storage as required. The maximum journal size is three times the initial journal size and is subject to the following exceptions:
  • If three times the initial journal size is less than 1.09 TB, use 1.09 TB as the maximum journal size.
  • If three times the initial journal size is greater than 1.09 TB, use 10.00 TB as the maximum journal size.

Like RecoverPoint, RecoverPoint for Virtual Machines has a limit on the minimum and maximum number of vRPAs that are deployable in a single vRPA cluster. The minimum is one, and the maximum is eight.

vRPAs can have three different virtual hardware specifications or configuration profiles. The supported vRPA configuration profiles are:

• Two vCPUs and 8 GB RAM (Bronze +)
• Four vCPUs and 8 GB RAM (Silver)
• Eight vCPUs and 8 GB RAM (Gold)

In VMware vCenter, the OVA deployment process automatically reserves all the memory that is assigned to the virtual machines and reserves 3,400 MHz for the CPUs. Adhere to the following guidelines:

• The four-vCPU and 8 GB RAM (Silver) configuration is suitable for most situations.
• If low throughput is expected, use a two-CPU and 8 GB RAM (Bronze +) configuration.
• For synchronous replication or when deduplication is required, use the eight-CPU and 8 GB RAM (Gold) configuration.
• Deploy all vRPAs in a single cluster with the same configuration profile. If impossible, then ensure that the first and second vRPAs in the vRPA cluster use a more powerful configuration profile. They are candidates to run cluster control. The cluster control vRPAs run additional logic and require additional CPU and memory resources.
• When deploying vRPAs with different configuration profiles, deploy vRPAs in pairs of the same configuration profile to provide, at a minimum, enough failover capacity and performance if there is a vRPA failure.
• The more memory the vRPA has, the better it can handle stronger and longer peaks of write I/Os before entering high load.
• The performance of the vRPAs does not scale up linearly when moving to a larger configuration profile. It is sometimes more cost-effective to use two smaller-sized vRPAs instead of a single larger vRPA. See vRPA configuration details for replication use cases and vRPA configuration sizing for the performance data.
• The performance of the vRPA cluster scales up linearly as vRPAs are added.
• If possible, spread out the vRPAs over the VMware ESXi hosts. If impossible, ensure that at least the first two vRPAs never reside on the same VMware ESXi host. The first two vRPAs are the only two vRPAs that can run the cluster logic and placing both on the same host creates a single point of failure.
• Ensure that VMware vCenter HA is enabled. If a VMware ESXi host fails, the vRPA restarts on another VMware ESXi host.
• Depending on the configuration of the VMware ESXi host, you can configure the vRPA to gain up to 20 percent performance. For example, if the VMware ESXi host has two NUMA nodes with eight CPU cores each, the recommended 8-CPU and 8 GB RAM vRPA configuration is to use one physical socket and eight cores per socket. This configuration keeps all the memory as close as possible to the CPU cores, maximizing performance.

The following table shows vRPA configurations for different replication use cases.

The following table provides a starting point for deciding how many vRPAs are required. The calculations are based on the vRPA Configuration Profile 4 vCPU and 4 GB RAM. A certified specialist must perform the official sizing.

Implement the following best practices:

• Configure the RP_Data networks with an MTU value of 9000.
• For asynchronous replication, ensure that the RTT between RecoverPoint for Virtual Machines sites does not exceed 200 milliseconds and that packet loss is below one percent.
• For synchronous replication, ensure that the RTT between RecoverPoint for Virtual Machines sites does not exceed 10 milliseconds and that packet loss is below one percent.
• Ensure that compression is enabled. Compression enhances throughput because the compressed data is written to the journal.
• For optimal performance and HA, ensure that the three networks (LAN, WAN, and RP_Data) are on separate virtual switches.

The Dell 3-Tier engineering team obtained the performance data based on the following parameters:

• vRPAs are configured with all cores on a single socket.
• The WAN connection is 10 GbE with an RTT of 0 ms, and 64 or 128 or 256 VMs are protected.
• Each VM has 5 VMDKs.
• The I/O pattern is 100 percent random writes, with up to 320 or 640 or 1,280 concurrent outstanding I/Os over all replicated VMDKs.
• A compression ratio of two when compression was enabled.

The following table shows the performance of the RecoverPoint for Virtual Machines splitter on a single VMware ESXi host:

For additional information, see the RecoverPoint for Virtual Machines Product Guide and the RecoverPoint for Virtual Machines Administrator’s Guide on the Dell Technologies Support website.

RecoverPoint for Virtual Machines 5.2.2 adds the capability to protect VMs directly to Amazon Simple Storage Service (S3) using proprietary snap-based replication with an RPO of 15 minutes.

Use RecoverPoint for Virtual Machines to protect production VMs while a Cloud DR Server (CDRS) instance is deployed to Amazon Elastic Compute Cloud (EC2) to manage the cloud copies and run orchestrated recovery flows to native Amazon EC2 instances.

Optionally, deploy the Cloud DR Add-on appliance to fail back from Amazon Web Services (AWS) to an on-premises vCenter server or to VMware Cloud on AWS. If the source VMs or sockets are licensed in the same way as a noncloud solution, the CDRA is a no-cost option with RecoverPoint for Virtual Machines.

By implementing the RecoverPoint for Virtual Machines Cloud Solution, you can:

• Allow multiple local and remote copies on-premises and to the cloud and test or failover to AWS EC2.
• Fail back to on-premises VMware environments or VMware cloud on AWS (SDDC) by using the Cloud DR appliance (CDRA)
• Orchestrate in-cloud DR.

The HTML5 plug-in for RecoverPoint for Virtual Machines 5.3 cannot configure and manage the RecoverPoint for Virtual Machines Cloud Solution. Manage the Cloud Solution through the RecoverPoint for Virtual Machines 5.3 Flex plug-in for vSphere vCenter servers supporting FLEX or the RecoverPoint for Virtual Machines RESTful API.

Replicating to AWS using the RecoverPoint for Virtual Machines cloud solution offers the benefit of cost reduction. RecoverPoint for Virtual Machines replicates data on a CG basis, so the replication topology is decided on a CG basis.

A combination of the following use cases is permitted:

• Cloud as a DR site:
  • An on-premises DR site or secondary site is not required. An RPO as low as 15 minutes and an RPO of seconds or zero requires RecoverPoint for Virtual Machines replication to a noncloud site.
  • Recover to EC2 or VMware cloud on AWS.
• Protection tiering:
  • Tier-1 VMs are protected by replication to a remote VMware vCenter (DR site).
  • Tier-2 VMs can be recovered directly from the cloud to the DR site.
• Extra protection layer
  • Production VMs are protected by replication to a remote on-premises VMware vCenter DR site).
  • A secondary copy is replicated to the AWS cloud. The use case is:
    • Distance: A production VM is replicated to a location far enough away that local or regional disasters do not impact the cloud VM.
    • Long time retention: The RecoverPoint for Virtual Machines Cloud Solution enables you to retain a VM copy for up to 90 days.

The following Cloud DR server orchestration features are available for RecoverPoint for Virtual Machines copies:

• Automated recovery workflows:
  • You select the VM to be recovered.
  • You select the point-in-time to which the VM is to be recovered.
  • You select the target (AWS EC2 or vCenter).
  • You select the network to which the VM is connected.
  • The CDRS starts the rehydration and conversion process, and then starts the instance.

The rehydration and conversion process is slow. Using Rapid Recovery Copies (RRCs) mitigates this issue as follows:

• A maximum of five RRCs are re-created for a short RTO and the slow rehydration and conversion process is avoided.
• The RRC copy interval is configured 6 to 24 hours.
• DR plans:
  • Configure multiple DR plans for recovery of multiple VMs.

The DR plan can customize the VM power-up order, and configure network and private IP details, cloud instance type, and security group association.

• The DR plan VM power-up order has five levels of priority. One level is the highest priority and starts first. The default is three, which is the medium priority. The lowest priority is five, which starts last.

The following table shows the Cloud DR scalability values.

For optimal performance when replicating to AWS cloud, adhere to these best practices:

• Use the following configuration profiles:
  • Four vCPUs and 8 GB RAM (Silver)
  • Eight vCPUs and 8 GB RAM (Gold)
• Distribute VMs evenly across the maximum of CGs. When multiple VMs share a single CG, the recovery operation in the cloud remains per VM.
• To calculate the size of the snap replication datastore, use 40 percent of the total size of the protected VM.
• RecoverPoint for Virtual Machines supports up to 128 CGs per vRPA cluster when replicating to the AWS cloud.
• Use a maximum of 256 CGs in a hybrid RecoverPoint for Virtual Machines system where both cloud and noncloud replica copies exist. Do not use more than 128 CGs for AWS cloud copies.
• The recommended snapshot interval is one hour. For smaller environments, a snapshot interval of 15 minutes is available.

Sizing a solution based on RecoverPoint for Virtual Machines replication to AWS is different to sizing a non-AWS continuous replication solution.

RecoverPoint for Virtual Machines can achieve an RPO as low as 1 second for asynchronous replication in an on-premises solution. An RPO of 0 seconds is achievable with synchronous replication. The RPO parameter whose default value is 25 seconds is only used for monitoring to validate the required RPO.

The Snap Replication interval plays a large part in sizing a RecoverPoint for Virtual Machines cloud solution. It impacts the replication frequency, the number of points-in-time, the storage required on S3, and more.

For replication to AWS, the periodic Snap Replication interval is the minimum achievable RPO. The minimum selectable RPO is the snap interval plus 30 minutes. For example, a snap interval of 15 minutes gives a minimum selectable RPO of 45 minutes. The maximum selectable RPO is one day. The RPO parameter is also used for monitoring.

The best practice when replicating to AWS is to size the production journal at 3 GB per CG.

When sizing the snap replication volume, ensure that the maximum size the volume can consume per CG is 40 percent of the total size of all production VMDKs in the CG. Two snap replication volumes protect each CG during replication to the cloud. When a snap replication volume is full, a replication cycle starts automatically, regardless of the timing.

Perform a realistic sizing calculation with the help of a Certified Data Protection sizing specialist. Use the following formula:

Volume size = 2 x ((CG Size in GB x Daily Change Rate in %) / (1440 minutes /
                Interval in minutes))

Where:

• CG Size in GB = the total size of all VMDKs in the CG in GB
• Daily Change Rate = the percentage of change for all VMDKs (virtual machines) in the CG
• 1440 minutes = 1 day
• Interval in minutes = interval in minutes when the snap-based replication occurs. This value is 15 <= Interval <= 1440
• Example:
• CG size in GB = 150 GB
• Daily Change Rate = 3%
• Interval in Minutes = 60 minutes
• Volume size = 2 x ((150 x 3%) / (1440 / 60)) => 0.375 GB

A calculation of the required number of vRPAs requires the snap-based interval and the amount of data to be replicated during each interval. When calculating the required number of vRPAs to use for replicating to AWS, Dell Technologies recommends using the Gold profile (8 vCPUs/8 GB RAM). This profile can sustain a maximum throughput of 250 MB/s of 64 K random IOs.

Use the following formula for this exercise:

Required number of vRPAs = 1 + Round (((Change Rate in GB * 1024) / (Interval in min * 60)) / Max Throughput)

Where:

• Change Rate = the amount of data that changes during the interval in GB.
• 1,024 = the value used to convert from GB to MB.
• Interval in minutes = interval in minutes when the snap-based replication occurs. This value is 15 <= Interval <= 1,440.
• 60 = the value used to convert the interval from minutes to seconds.
• Max Throughput = the maximum throughput for a Gold profile vRPA in MB per second.
• Example 1:
• Change Rate in GB = 16 GB
• Interval in Minutes = 60 Minutes
• Max Throughput = 250
• MB/s calculation:
  • Required number of vRPAs = 1 + Round (((16 * 1024) / (15 * 60)) / 250)
  • Required number of vRPAs = 1 + Round (0.072817778)
  • Required number of vRPAs = 2
  • Example 2:
  • Change Rate in GB = 200 GB
  • Interval in Minutes = 60 Minutes
  • Max Throughput = 250 MB/s Calculation: Required number of vRPAs = 1 + Round (((200 * 1024) / (15 * 60)) / 250)
  • Required number of vRPAs = 1 + Round (0.910222222)
  • Required number of vRPAs = 2

For more information, look for the latest versions of the following documents on the Dell Technologies Support website.

• RecoverPoint for Virtual Machines Product Guide
• RecoverPoint for Virtual Machines Administrator's Guide
• RecoverPoint for Virtual Machines - Replication to AWS - Planning and Deploying Replication to the Cloud with RecoverPoint for Virtual Machines

The following sections of this chapter provide recommendations and considerations for designing business continuity and DR solutions for converged systems.

Two FC-WAN configurations are supported on converged systems:

• External FC-WAN
• Internal FC-WAN (the most commonly used configuration)

When you deploy VPLEX Metro with internal FC-WAN connectivity, ensure that you:

• Keep VSAN 10 and 11 local and isolated to a single Cisco MDS switch. More VSANs are required to facilitate VPLEX FC-WAN replication traffic.
• Configure VSAN 30 for Fabric A and VSAN 31 for Fabric B as follows:
  • Dedicate VSAN 30 and 31 to FC-WAN replication traffic.
  • Extend VSAN 30 between both Fabric A switches.
  • Extend VSAN 31 between both Fabric B switches.
  • Configure VSAN 30 and 31 with unique FC domain Identifiers between converged systems. Use even numbers for Fabric A and odd numbers for Fabric B.
• To avoid a configuration mismatch when attempting to extend (merge) VSANs, do not complete any zoning activities until after the required VSAN has extended successfully.

Any new switch can become the principal switch when it joins a stable fabric. During the principal switch selection phase, the switch with the highest priority becomes the principal switch.

• Ensure that you:
  • Configure VSAN 30 and 31 with a higher switch priority on the primary converged system to enforce wanted roles.

By default, the configured priority is 128 and the valid range is 1 to 254.

• Do not use a switch priority value of 1. Adherence to this rule is necessary to allow for future migration scenarios.

If two switches have the same priority, the switch with the lower WWN becomes the principal switch.

• If you extend VSANs between new and existing Cisco MDS switches or unified Cisco Nexus switches, verify that the merge conditions, such as zoning mode and zoneset distribution, are correct. These switches can have different default configurations.

The following recommendations apply to VPLEX and Metro-node-based Integrated Data Protection solutions providing business continuity and DRR, including:

• General design recommendations for VPLEX and Metro node
• Design recommendations for VPLEX
• Design recommendations for Metro node

Adhere to the following best practices when configuring the management network for VPLEX on converged systems:

• The VPLEX Management Server in each converged system must connect to the converged system management network (101 or 1801 or OOB).
• VPLEX Metro requires a routable connection between the VPLEX management servers for each cluster, and between each management server and the Cluster Witness server. If a firewall exists between any of these servers, it must allow ICMP and IPsec traffic in both directions. For more information, see the VPLEX Security Configuration Guide on the Dell Technologies Support website.
• Metro node in a Metro configuration requires a routable connection between the director management IP addresses of each cluster in the Metro system. If a firewall exists between any of these servers, see the information in the Metro Node Security Configuration Guide on the Dell Technologies Support website.

Ideally, this routable connection does not use the L2 DCI link. You do not have to extend the management VLAN.

• Link latency between the two VPLEX management servers and the VPLEX Cluster Witness server must not exceed 1 second.
• The IP management network must not be able to route packets to the reserved IPv4 VPLEX subnets 128.221.252.0/24 and 128.221.253.0/24.
• The IP management network must not be able to route packets to the following reserved IPv4 Metro node subnets: 128.221.250.0/24, 128.221.251.0/24, 128.221.252.0/24, and 128.221.253.0/24.
• The management servers (MMCSs) for the VPLEX VS6 require two management network connections and two IP addresses per VPLEX cluster.

Even though the VPLEX VS6 uses two MMCSs, the second MMCS does not provide any function (such as HA or failover). However, it must be configured with an IP address and be always operational. Failure of MMCS-B creates alerts and becomes a Field Replaceable Unit (FRU) event.

When you configure workload mobility solutions, adhere to the following best practices:

• Ensure that there is Layer-2 (L2) adjacency between the two converged system production networks. Doing so ensures connectivity following a live vSphere vMotion migration or a VMware HA-triggered VM restart event.
• At a minimum, extend the ESXi Management (205 or 1611) VLAN that is used to host the VMware vSphere management VMs between sites. Doing so necessitates trunking a subset of VLANs that are considered to be internal outside the converged system.
• The customer is responsible for the L2 extension between sites. The chosen technology (OTV or back-to-back vPC) must comply with latency requirements.
• Effective from VMware vSphere 6.0, VMware introduced Layer 3 vMotion capabilities. You do not have to extend the Layer 2 VLAN 206 that is used for VMware vSphere vMotion in the compute hosts.
• For full resilience, provide gateway redundancy between sites. Ideally, implement FHRP isolation to avoid hosts and virtual machines unnecessarily crossing the DCI link to reach their gateway.

When configuring VPLEX Metro, bear in mind the following considerations:

• A detach rule is defined for each VPLEX Metro distributed vVol.
• When a communication failure occurs between the two clusters in a VPLEX Metro configuration, the detach rule identifies which VPLEX cluster must detach its mirror leg. This rule enables services to continue.
• The purpose of a defined preferred site is to ensure that there is no possibility of a split-brain scenario with both VPLEX clusters continuing to allow I/O during a communication failure.
• After a complete communication failure between the two VPLEX clusters, the preferred site continues to provide service to the distributed vVol.
• The other VPLEX cluster suspends I/O service to the volumes. That cluster is known as the nonpreferred site.
• The detach rule is at the distributed vVol level. Either converged system in a VPLEX Metro configuration can be the preferred site for some distributed vVols and the nonpreferred site for others.
• A VPLEX Metro instance can support up to 5,000 distributed vVols, and each volume has its own detach rule.
• VPLEX Witness failure-handling semantics apply only to the distributed vVols in a CG. CGs have a bias rule set similar to a detach rule to determine the preferred site.
• All distributed vVols common to the same set of VMs must be in one CG. All VMs that are associated with that CG must be at the preferred site.

The following failure conditions invoke VPLEX detach rules:

• A total VPLEX cluster failure at one site (all directors in a cluster)
• A VPLEX WAN partition

The following conditions occur during the failure and after the failure is resolved:

• A complete VPLEX cluster failure triggers detach rule behavior because the surviving VPLEX cluster cannot determine between intersite communication loss and complete VPLEX cluster failure.
• The distributed vVols whose preferred site is the surviving VPLEX cluster continue to run without interruption.
• The distributed vVols whose preferred site is the site with the failed VPLEX cluster site enter into I/O suspension.
• After the VPLEX cluster failure is resolved, the distributed vVols are reestablished, enabling I/O on both VPLEX clusters in a Metro configuration.

The following conditions occur during the failure and after the failure is resolved:

• The VPLEX cluster WAN partition (intersite communication failure) also triggers execution of the detach rule.
• Each distributed vVol allows I/O to continue at the preferred site and suspends I/O at the nonpreferred site.
• After the VPLEX cluster WAN partition condition is resolved, the distributed vVols are reestablished, enabling I/O on both VPLEX clusters in a Metro configuration.

VMware vSphere does not recognize all types of total path failure when running in VPLEX environments. To remedy this issue, configure VPLEX Metro for persistent device loss under the advanced VMware ESXi settings, following these guidelines:

• VMware vSphere recognizes two types of total path failure to a VMware ESXi server. The VMware ESXi server can declare either condition following a failure condition.
• All paths down (APD).
• Persistent device loss (PDL): PDL is a state that a VMware ESXi server declares when a SCSI sense code is sent from the underlying storage array (in this case, VPLEX) to the VMware ESXi host. It informs the host that the paths can no longer be used. This condition can occur if VPLEX suffers a WAN partition, causing storage volumes at the nonpreferred location to suspend. VPLEX sends the PDL SCSI sense code to the VMware ESXi server from the site that is being suspended (the nonpreferred site).

VMware HA does not automatically recognize that a SCSI PDL state causes a VM to invoke an HA failover, which causes an outage. This scenario is not acceptable when VMware HA is used with VPLEX in a stretched cluster configuration.

• VMware vSphere can act on the SCSI PDL state by powering off the VM, invoking HA failover. This behavior requires additional settings in the VMware vSphere cluster. For more information, see Advanced VMware parameters to regulate PDL conditions in the Tier-3 Platform Logical Build Guide. To obtain a copy of the guide, contact your Dell Technologies sales representative.

When implementing HA for VMware vCenter Server, adhere to the following best practices:

• In a stretched cluster solution, VMware vCenter Server requires site mobility capabilities to enable automatic failover to either converged system if there is an outage.
• Without VPLEX Metro and a VMware stretched clustering solution, each converged system would use its own instance of vCenter Server in the 3-Tier Management solution to manage its ESXi hosts. To deploy an active/active stretched clustering solution with VPLEX Metro, use a single VMware vCenter Server instance to manage all the hosts in the two converged systems that participate in VMware stretched clusters.

The following options are supported on converged systems:

• If all ESXi hosts from Site-A and Site-B participate in a VMware stretched cluster, one vCenter server hosts all 3-Tier Management solution VMs from Site-A and Site-B. All VMs on VPLEX distributed volumes that are attached to VMware ESXi hosts in Site-A and Site-B.
• For any VMware ESXi hosts in Site-A or Site-B or both that do not participate in a VMware stretched cluster:
  • Use the vCenter Server in each converged system to manage the 3-Tier Management solution and the local VMware ESXi hosts.
  • Deploy a third vCenter server to manage the participating VMware ESXi hosts.
• Converged system management VMs include the VMware vCenter Server Appliance. All other management functions that reside in the 3-Tier Management solution remain local to their converged system.
• Ensure that you apply the additional VMware parameters to Cisco UCS C-Series servers when building a VMware stretched cluster. For more information, see Advanced VMware parameters to regulate PDL conditions in the Tier 3 Platform Logical Build Guide. To obtain a copy of the guide, contact your Dell Technologies sales representative.

Dell Technologies recommends the following best practices for VPLEX on converged systems.

Consider the following general best practices when configuring back-end connectivity:

• Each VPLEX director has redundant physical connections to the back-end storage array across dual fabrics.
• For single converged systems, only 50 percent of the available VPLEX back-end ports, A1-FC00, A1-FC01, B1-FC00, and B1-FC01, are connected as standard for each engine.
• Use the remaining VPLEX back-end ports, A1-FC02, A1-FC03, B1-FC02, and B1-FC03, in a high-bandwidth environment or for connecting a second back-end storage array.
• VMware vVols are unsupported on VPLEX.
• Moving the ESXi host boot LUN behind VPLEX is not supported. Boot LUNs must be provided to the host directly from the storage array, and not from VPLEX.

Consider the following best practices when configuring back-end connectivity.

symconfigure -sid xxx -cmd "set port
8e:0 SCSI_Support1=ENABLE;" preview symconfigure -sid xxx -cmd "set
port 8e:0 SCSI_Support1=ENABLE;"
commit

symaccess -sid xxx -wwn <VPLEX
initiator> set hba_flags on os2007 -enable symaccess -sid xxx
-wwn <VPLEX initiator> list logins -dirport 8e:0
-v

symaccess -sid xxx set hba_flags on
OS2007 –enable -wwn <VPLEX initiator> -dirport
ALL:ALL

Adhere to the following best practices when configuring front-end zoning for VPLEX on converged systems:

• Ensure that the front-end I/O module on each director has a physical connection to each fabric. Connect even ports to Fabric A, and odd ports to Fabric B.
• Ensure that each host has at least one path to an A director and one path to a B director on each fabric, totaling four logical paths. A minimum of four paths are required for network data usage (NDU).
• Ensure that a VMware ESXi cluster connects to exactly two VPLEX directors.
• For a dual or quad engine, spread the host I/O paths across the engines and the directors.
• Register all VMware ESXi host initiators with a Host Type of default.
• For single converged systems, ensure that only 50 percent of the available VPLEX front-end ports are connected as standard for each engine: A0-FC00, A0-FC01, B0-FC00, and B0-FC01.
• Use the remaining VPLEX front-end ports, A0-FC02, A0-FC03, B0-FC02, and B0-FC03, in a cross-connect configuration or when connecting a second converged system.
• Ensure that field personnel rebalance the front-end host connectivity when upgrading the engine count (single to dual, dual to quad) on the customer premises.

The following sections describe best practices for configuring VPLEX WAN-COM.

Keep in mind that:

• In a VPLEX configuration, intracluster connectivity means director-to-director communication in the VPLEX cluster.
• In a Metro configuration, intercluster connectivity means communication between the VPLEX clusters. The configuration uses the WAN-COM module on each director.
• Customers must purchase VPLEX Metro with the appropriate WAN-COM module because reconfiguration of the hardware module is not supported after the initial installation.

Consider the following general best practices when configuring Cisco MDS replication VSANs and FC Inter-Switch Links for VPLEX Metro with FC replication.

When you deploy VPLEX Metro with internal intersite connectivity, ensure that:

• Internal intersite connections terminate at the converged system Cisco MDS switches. For example, an ISL to a local DWDM multiplexor (MUX) or termination of a directly connected point-to-point dark fiber link.

External intersite connections are external to the converged systems. VPLEX WAN-COM replication ports connect directly to external switches and no ISLs exist between the converged systems.

• The FC WAN-COM module supports switched fabric, DWDM, and FCIP protocols.
• The FC WAN-COM module does not support using Cisco Inter-VSAN routing for WAN-COM zoning.
• Use independent FC WAN links for redundancy on converged systems.
• Each VPLEX director has two FC WAN ports. You must connect them to separate fabrics to maximize redundancy and fault tolerance.
• Logically isolate replication traffic from other traffic using dedicated VSANs.

Adhere to the following guidelines when configuring Inter-Switch Link (ISL) for trunking and buffer-to-buffer credits and deploying VPLEX Metro with DWDM and SONET.

Adhere to the following best practices:

• Trunking enables interconnected ports to transmit and receive frames in more than one Cisco VSAN over the same physical link using the enhanced ISL (EISL) frame format.
• When trunking mode is disabled, add ISL interfaces to the Cisco VSAN that is being extended before activating the links. Doing so ensures that only the Cisco VSAN is extended between converged systems.
• By default, trunk mode is enabled on all FC interfaces, but it takes effect only when in E-port mode.
• An operational E-port with trunk mode enabled is called a TE port.
• The trunking protocol uses the trunk-allowed Cisco VSANs configured for TE ports to determine the allowed active Cisco VSANs in which frames can be received or transmitted.
• If configuring ISLs with trunking mode enabled, do not add local Cisco VSANs to the trunk-allowed Cisco VSAN list.
• Enable trunking only when multiple Cisco VSANs must be extended between converged systems.
• On the Cisco MDS 9000 switches, set the primary converged system end of the trunk setting to “on” and the secondary converged system end to “auto.”

For more information about how to configure inter-switch links between converged systems, see the following documentation:

• Configuring Cisco MDS VSANs
• Fibre Channel Inter-Switch Links (ISL)
• Tier-3 Logical Build Guide. Look for the guide on the Dell Technologies Support website.

Adhere to the following best practices:

• FC uses buffer-to-buffer credits (BB_Credits) as a mechanism for hardware-based flow control, so it is not necessary to switch hardware to discard frames caused by high congestion.
• Standard FC flow control and BB_Credit value are adequate for most short-haul deployments. Extra buffering and WAN-optimized flow control are often needed for longer distances.
• Determining sufficient BB_Credits before use is crucial because miscalculations might lead to performance degradation due to credit starvation.
• Add 20 percent to the calculated BB_Credit value to account for spikes in traffic.
• Credit starvation occurs when the number of available credits reaches zero, preventing all forms of FC transmissions. This condition triggers a timeout value, causing the ISL link to reinitialize.
• Depending on the distance between ISL end-points and the MDS switch or switching module being used, BB_Credits must be used to ensure optimal operation.

The following table provides guidelines for determining how many BB_Credits are required based on the distance and speed of the ISL. If the calculated value exceeds the default value, adjust the ISL interface configuration.

The following table shows the default and maximum values for the buffer-to-buffer credit configuration settings per ISL.

If using DWDM or SONET connectivity between converged systems, ensure that the two rings have diverse pathing and that latency is measured for both paths.

The following behavior is expected:

• VPLEX directors load-balance (implement round-robin) between the two paths so that any large discrepancy in latency causes VPLEX to operate at speeds based on the slower path.
• If there is a large discrepancy, VPLEX issues call home events but does not act.

Before you configure Cisco MDS Inter-VSAN Routing (IVR), review the following information about licensing and the Cisco MDS Enterprise Package part numbers.

Consider that:

• The Cisco MDS Enterprise Package must enable Inter-VSAN (IVR) routing on each converged system FC switch. IVR is a feature of Advanced Traffic-Engineering.
• IVR allows a selective transfer of data traffic between specific initiators and targets on different VSANs. This feature eliminates the requirement to merge VSANs into a single logical fabric.
• IVR facilitates resource sharing across VSANs without compromising the VSAN benefits of scalability, reliability, availability, or network security.
• IVR works across WANs using FCIP. VPLEX Metro supports FCIP.
• The Cisco MDS Enterprise Package enables zone-based Quality of Service (QoS) to complement the standard QoS that is already available. It also enables extended buffer-to-buffer credits to increase the distance for SAN extension.
• In addition to Advanced Traffic-Engineering, the Cisco MDS Enterprise Package enables Enhanced Network Security with the following features:
  • Cisco TrustSec FC Link Encryption
  • Switch-switch and host-switch authentication with the FC Security Protocol (FCSP)
  • Diffie-Hellman Challenge Handshake Authentication Protocol (DH-CHAP)
  • Port security that locks mappings of entities to switch ports.
  • VSAN-based access control
  • IP Security (IPsec) for FCIP
  • Digital certificates and fabric binding for open systems

The following table shows the part numbers to use when ordering a Cisco MDS Enterprise Package.

Adhere to the following best practices.

• Connect the IP WAN ports to the Cisco Nexus 9000. The IP WAN ports are optical 10 Gbps and do not automatically negotiate down to slower speeds. They must connect to 10 Gbps SFPs.
• Provide 10 Gbps connectivity locally where the IP WAN-COM ports attach to the network. Other network segments can run at speeds other than 10 Gbps.
• Assign all IP WAN ports in port-group 0 to the same VLAN in each site.
• Assign all IP WAN ports in port-group 1 to the same VLAN in each site.

The IP WAN ports do not support 802.1Q tagging.

• Use different VLANs for port-group 0 and port-group 1.
• Configure the switch interfaces as access ports.
• The Maximum Transmission Unit (MTU) size attribute affects IP WAN-COM performance. The default IPv4 MTU size for network switches is 1,500.
  • Increasing the size of the MTU increases performance over the WAN.
  • VPLEX supports a maximum MTU size of 9,000. Use the highest MTU size that is supported on the network.
  • Configure every network switch in the path between the VPLEX clusters to support jumbo frames. Otherwise, the frame is fragmented into multiple smaller frames with an MTU of 1500, which negatively affects performance.
• When jumbo frames are used with IPv6, the routers do not fragment the packet on behalf of the source. Instead, they drop the packet and send back an error message.
• Set the correct socket buffer size according to your anticipated workload. The following values are suggested for starting your base lining process with a specified socket buffer size:
  • 1 MB5 MB is optimal for an MTU of 1,500 with an RTT of 1 millisecond.
  • 5 MB is optimal for an MTU of 1,500 with an RTT of 10 milliseconds.
  • 5 MB is optimal for an MTU of 9,000 with an RTT of 1 millisecond and 10 milliseconds.

For instructions on how to change the MTU size or the socket buffer size, see docu58234 VPLEX IP Networking: Implementation Planning and Best Practices on the Dell Technologies Support website.

Before you configure the VPLEX Cluster Witness Server, consider that:

• VPLEX Cluster Witness Server is a VPLEX component that is provisioned as a VM on a VMware ESXi host. VPLEX Witness is typically deployed in a third site or failure domain to enforce isolation from failures that could potentially affect the VPLEX clusters at either site.
• Deploying a VPLEX Metro solution with VPLEX Witness provides continuous availability to the storage volumes if a site failure or intercluster link failure (WAN partition) occurs.
• VPLEX Witness failure-handling semantics apply only to the distributed vVols in a CG.
• VPLEX Witness server is a mandatory requirement for cross-connect.
• VPLEX Witness server is recommended for noncross-connect.

Dell Technologies recommends that you deploy an external encryption solution such as IPsec. Such encryption enables data confidentiality and end-point authentication over IP WAN-COM links between VPLEX clusters on converged systems.

When configuring VPLEX solutions, apply the recommended guidelines for port usage and firewall rules.

Look for the VPLEX Security GeoSynchrony Configuration Guide on the Dell Support website.

Dell Technologies recommends the following Metro node design best practices guidance.

On converged systems, consider that:

• Each Metro node director must have redundant physical connections to the back-end storage array across dual fabrics.
• Metro node systems require that 100 percent of all front-end and back-end ports are connected.
• Metro node does not support vVols.
• Moving the ESXi host boot LUN behind Metro node is not supported. Boot from SAN LUNs must be provided to the host directly from the storage array, and not from the Metro node.

Metro node zoning is based on the following design principles:

• Zone Metro node director A ports to one group of four storage array ports.
• Zone Metro node director B ports to a different group of four storage array ports.
• Each Metro node director is treated as a VMware ESXi cluster (host). Follow the host connectivity guidelines in the Tier-3 Logical Build Guide. Look for the document on the Dell Support website. In a Metro node environment, the same array ports are zoned to Metro node director-A, with two active paths per fabric. The storage array ports that are reserved for the second VMware ESXi cluster are zoned to Metro node director-B.
• The even back-end port (IO-02) connects to Fabric A. The odd back-end port (IO-03) connects to Fabric B.

See the Dell Integrated Data Protection Zoning for Metro node spreadsheet for back-end zoning examples with Dell Unity XT and PowerStore arrays. To obtain a copy of the spreadsheet, contact your Dell Technologies sales representative.

Adhere to the following best practices for specific storage arrays.

As an Asymmetric Logical Unit Access (ALUA) array, the Unity XT array requires the four active and four passive connectivity rules. Each director node of the Metro node cluster must have four active and four passive paths to all LUNs.

Each Metro node director has two back-end ports. For compliance, each Metro node director back-end port must be zoned to two ports on each Unity XT storage processor (SP). The SP that owns the LUN provides the active paths to the LUN, while the second SP provides the passive paths.

When configuring back-end connectivity with Unity and Unity XT storage arrays, ensure that:

• Each Metro node director has logical and physical connectivity to storage controllers.
• The minimum supported configuration is eight paths (four active and four passive) per Metro node director for any given LUN.

For more information, see the Dell Integrated Data Protection Zoning for Metro node spreadsheet on the Dell Technologies Support website.

When configuring back-end connectivity with PowerStore storage arrays, ensure that:

• Each PowerStore node has two IOM slots.
• Each node has at least one FC IOM.
• The even FC ports (0 and 2) are cabled to the Cisco MDS switch A fabric.
• The odd FC ports (1 and 3) are cabled to the Cisco MDS switch B fabric.
• Each Metro node engine is zoned to a port group on each appliance: FC00 is zoned to two Fabric A ports, and FC01 is zoned to two Fabric B ports.
• Each metro node engine is zoned to all appliances in the array cluster using the same FC ports. Each LUN is owned by one appliance, enabling LUN migration between appliances in the same array cluster.
• Each Metro node director must have logical and physical connectivity to storage controllers.
• The minimum supported configuration is four paths per Metro node director for any given LUN.

When configuring front-end zoning for Metro node on converged systems, ensure that:

• The front-end I/O module on each Metro node director has a minimum of two physical connections, one to each fabric.
• Even port IO-00 connects to Fabric A, odd port IO-01) connects to Fabric B.
• Each host has at least one path to Metro node director-A and one path to Metro node director B on each fabric, totaling four logical paths.
• Four paths are configured from each host to the Metro node cluster to complete an NDU.
• All VMware ESXi host initiators are registered with a Host Type of default.

In a Metro node configuration:

• Intracluster connectivity means director-to-director communication within the Metro node cluster.
• Intercluster connectivity means communication between the Metro node clusters.

Connect the IP WAN ports to the Cisco Nexus 9000 switches.

Consider that:

• Each of the four Metro node directors has two WAN ports that are configured into two port groups: WC-00 and WC-01.
• Dell Technologies recommends independent WAN links for redundancy.
• Two separate non-vPC VLANs for the WC-00 port group must be defined on Switch A and for the WC-01 port group on Switch B.
• IP WAN-COM ports cannot participate in a port channel, so attaching to a member switch in a vPC pair results in orphan ports. The vPC peer-link between Cisco Nexus switches in a converged system is not sized to account for Metro node Metro traffic.
• Separate uplinks are configured for the non-vPC VLANs. You can attach IP WAN-COM ports directly to the customer dark fiber network switches.
• As optical 10 Gbps ports, the IP WAN-COM ports do not automatically negotiate down to slower speeds and must connect to 10 Gbps SFPs. Provide 10 Gbps connectivity locally where the IP WAN-COM ports attach to the network. Other network segments can run at speeds other than 10 Gbps.
• All WC-00 ports are assigned to the same VLAN in each site.
• All WC-01 ports are assigned to the same VLAN in each site.
• The IP WAN ports do not support 802.1Q (VLAN) tagging. Configure the switch interfaces as access ports.
• The MTU size attribute affects IP WAN-COM performance.
• The default IPv4 MTU size for network switches is 1,500.
• Increasing the size of the MTU increases performance over the WAN.
• Metro node supports a maximum MTU size of 9,000. Use the highest MTU size that is supported on the network.
• Every network switch in the path between the Metro node clusters is configured to support jumbo frames. Otherwise, the frame is fragmented into multiple smaller frames with an MTU of 1,500, which negatively affects performance.
• The supported network round-trip latency is less than or equal to 10 milliseconds.
• The correct socket buffer size (socket-buf-size) is set for your anticipated workload.

To start your base lining process, use the following socket buffer size values:

• 1 MB for an MTU of 1,500 with an RTT of 1 millisecond
• 5 MB for an MTU of 1,500 with an RTT of 10 milliseconds
• 5 MB for an MTU of 9,000 with an RTT of 1 millisecond and 10 milliseconds

For information about changing the MTU size or the socket buffer size, look for the Metro node best practices document on the Dell Technologies Support website.

When configuring a host Cross-Connect for Metro node Metro, consider that:

• The maximum round-trip latency for Cross-Connect is 1 millisecond with all Metro node Metro configurations.
• The maximum round-trip latency for Cross-Connect is 5 milliseconds with VMware vSphere.
• Metro node Witness server is mandatory for Cross-Connect.
• Dell Technologies does not support Cross-Connect for converged systems that do not include Cisco MDS FC switches. This scenario is known as the separated networking option.
• Host Cross-Connect requires front-end SAN connectivity between converged systems. This configuration ensures that hosts in the primary converged system can communicate with front-end ports of the Metro node cluster that is connected to the secondary converged system, and conversely.
• The autoresume feature must be set to true for all Metro node Metro CGs.
• The thin-rebuild feature must be set to true for all Metro node Metro distributed volumes.
• Cross-Connect zoning follows the rules that are defined in Metro node front-end (FE) logical port groups.
• Storage array cross-connect (back-end SAN connectivity) is sometimes implemented for adding protection to system volumes.

Dell Technologies recommends using the components that are shown in the following table to provide data protection backup for the 3-Tier platform.

Integrate these components directly within the 3-Tier platform to protect its workloads. This way, you can isolate the backup traffic and contain it within the 3-Tier platform, preventing the traffic from moving to and from the external core network.

Proper sizing of a data protection backup system for a 3-Tier Platform requires an understanding of:

• The workloads
• The amount of front-end data to be protected
• The available backup window
• The required Recovery Time Objective (RTO) and Recovery Point Objective (RPO)

Proper sizing also includes current backup requirements with a plan to seamlessly scale up to meet the capacity requirements for three to five years. Selecting the right components requires the use of appropriate sizing tools and assistance from an experienced Dell Technologies data protection professional.

PowerProtect Data Manager software is an enterprise backup solution that provides software-defined data protection, deduplication, operational agility, self-service, and IT governance.

PowerProtect Data Manager key features include:

• Software-defined data protection with integrated deduplication, replication, and reuse.
• Data backup and recovery self-service operations from native applications that are combined with central IT governance.
• Multicloud optimization with integrated Cloud Tiering.
• SaaS-based monitoring and reporting.
• Modern services-based architecture for ease of deployment, scaling, and updating.

PowerProtect Data Manager integrates multiple data protection products within the Dell Technologies Data Protection portfolio to enable data protection as a service, providing the following benefits:

• Enables the data protection team to create data paths with provisioning, automation, and scheduling to embed protection engines into the infrastructure for high-performance backup and recovery.
• Enables backup administrators of large-scale environments to schedule backups for the following asset types from a central location on the PowerProtect Data Manager server:
  • VMware virtual machines (VMs)
  • File systems
  • VMAX storage groups
  • PowerStore Block Volumes
  • Kubernetes clusters
  • Microsoft Exchange and SQL databases
  • Oracle databases
  • SAP HANA databases
  • Network attached storage (NAS) shares
• Uses an agent-based approach to discover the protected and unprotected databases on an application server.
• Enables governed self-service and centralized protection by:
  • Monitoring Service Level Objectives (SLOs)
  • Identifying violations of RPOs)
• Supports deploying an external VM Direct appliance to move data with a VM Direct Engine. The PowerProtect Data Manager software comes prebundled with an embedded VM Direct Engine. The VM Direct Engine is automatically used as a fallback proxy for performing backup and restore operations when the external VM Direct Engines fail or are disabled. Dell Technologies recommends that you deploy external VM Direct Engines, because the embedded VM Direct Engine has limited capacity for performing backup streams. The embedded VM Direct Engine is sufficient, however, for VM crash-consistent protection policies that use the Transparent Snapshot Data Mover (TSDM) protection mechanism.
• Supports the vRealize Automation DP extension, which enables provisioning of VMs with PowerProtect Data Manager protection, manual backup, and restore to the original or a new location. For more information, see the vRealize Automation Data Protection Extension for PowerProtect Data Manager Installation and Administration Guide . Look for the guide on Dell Technologies Support.
• Supports integration of Dell Cloud Disaster Recovery (Cloud DR), including workflows for Cloud DR deployment, protection, and recovery operations in the AWS or Azure cloud.
• Supports PowerProtect Search, which enables backup administrators to quickly search for and restore VM file copies. The Search Service can be enabled by adding a search node to the configurable search engine that is automatically deployed during the PowerProtect Data Manager deployment.
• Provides a RESTful interface that allows the user to monitor, configure, and orchestrate PowerProtect Data Manager. Customers can use the APIs to integrate their own automation framework or quickly write new scripts with the help of easy-to-follow tutorials.
• Integrates with Dell Technologies PowerProtect Cloud Snapshot Manager to view PowerProtect Cloud Snapshot Manager jobs, alerts, and reports from a consolidated PowerProtect Data Manager dashboard.

PowerProtect Data Manager deploys as a VM from an OVA in a VMware vSphere environment, and stores all backups and data on a PowerProtect Data Domain. The Data Domain Boost (DD Boost) protocol provides advanced integration with backup and enterprise applications for increased performance. DD Boost distributes parts of the deduplication process to the backup server or application clients, enabling client-side deduplication for faster and more efficient backup and recovery.

PowerProtect Data Manager appliance provides the following features for Converged Systems:

• Deployable as a virtual appliance on the 3-Tier platform management solution.
• Ability to back up a wide variety of physical and virtual compute environments, applications, and database, at image and file level.
• VMware vSphere Web-Client plug-in.
• Instant Access VM restore from Data Domain. For information about the features and limitations of the Instant Access process, see the PowerProtect Data Manager documentation on Dell Technologies Support website.

See the latest PowerProtect Data Manager Deployment Guide for more information about:

• Scalability information about the number of vCenter servers
• External VM Direct Engines
• PowerProtect Data Domain systems
• VMs supported per PowerProtect Data Manager server
• Limits for network latency

See the latest PowerProtect Data Manager Deployment Guide for information about resource requirements such as vCPUs, RAM, Storage, and vNICs for:

• PowerProtect Data Manager
• External VM Direct Engine
• PowerProtect Search Engine

When configuring PowerProtect Data Manager, do not use an IP address in the 172.24.0.192 /26 subnet. IP addresses from 172.24.0.192 through 172.24.0.255 are reserved for the private Docker network.

The following table shows the targeted performance and scalability numbers for the PowerProtect Data Manager software.

PowerProtect Data Domain provides for in-flight data encryption as a default for new installations.

Although you can disable encryption, Dell Technologies strongly recommends leaving it enabled. If encryption is enabled later, the cleaning cycle is CPU-intensive. The cycle can also be lengthy.

Encryption for replication is also enabled by default, but may be disabled.

For more information about encryption, see the latest Dell PowerProtect Data Manager Administrator and Security Configuration Guides on the Dell Technologies Support website.

Through integration with vSphere vCenter, PowerProtect Data Manager manages, protects, and recovers VMs. PowerProtect Data Manager automates data retention SLA compliance by ensuring that the correct number of backup copies are stored in the correct location, at the required protection level.

To be able to protect VMs, the first step is to add vSphere vCenter to the PowerProtect Data Manager UI. The system discovers the vCenter and installs the PowerProtect Data Manager plug-in for vSphere. After vCenter is discovered, deploy the external Protection Engine in vCenter as a best practice. The Protection Engine enables you to perform VM snapshot backups and move the backup data to the target Data Domain.

HotAdd is a VMware feature where devices can be added “hot” while a VM is running. The Protection Engine takes a snapshot and creates a linked clone of the production VM. It then attaches and reads VMDKs of the linked clone for backup. For more information, see the VMware document HotAdd Transport.

The Protection Engine also supports Change Block Tracking (CBT). CBT allows PowerProtect Data Manager to determine the changes (or deltas) that have occurred since the previous backup. At the next incremental backup, the deltas are transferred to Data Domain through the Protection Engine.

During the first full backup of a VM, the Protection Engine reads the entire content of all VMDKs. It uses DD Boost, which uses global deduplication, to transfer the data to Data Domain. The next backup only reads the changes that have occurred since the previous backup. It overlays them on a copy of that backup to create a new full backup, while transferring only the incremental changes.

PowerProtect Data Manager supports image-level backup and recovery of VMs on both VMFS and VMware vVols datastores when integrated with a 3-Tier Platform.

For more information about support for vVols datastores with PowerProtect Data Manager, see the Dell PowerProtect Data Manager Administration and User Guide on the Dell Technologies Support website.

When using PowerProtect Data Manager with the external VM Direct Engine:

• Install VMware Tools on every VM.
• Enable CBT on every VM.
• Deploy one or more External Protection Engines to enable you to use the HotAdd transport mode. It enables faster backups and restores while putting less pressure on the network, routers, and firewalls.
• Do not use special characters in the vSphere Data Center name, datastore names, or VMs. Special characters include: {, }, %, &, $, #, @, !, /, \, <, >, :, and so on. Using such characters causes backup problems with the VMware Protection Engine.
• For VMs with vFlash enabled disks protected with the HotAdd transport mode, ensure that the ESXi host where the Protection Engine is located has vFlash resources that are configured to be greater than or equal to the resources of the VM. If there are not enough resources, the backup of any vFlash enabled VMDK fails.
• PowerProtect Data Manager does not support the following disk types:
  • Independent disks (persistent and nonpersistent)
  • RDM Independent – Virtual Compatibility Mode
  • RDM Physical Compatibility Mode
  • First class disks
  • SCSI disks that are configured to the SCSI controller with SCSI Bus Sharing set as Physical or Virtual
• Avoid deploying VMs with IDE due to degradation of backup performance. Use SCSI virtual disks whenever possible.

PowerProtect Data Domain systems provide a data protection backup storage target for PowerProtect Data Manager on 3-Tier platforms.

PowerProtect Data Domain supports deduplication of backup data before writing it to storage, thus minimizing storage requirements by storing only unique data.

Dell Technologies recommends using the following PowerProtect Data Domain models for the 3-Tier Platform:

• DD3300
• DD6900
• DD9400
• DD9900

Dell Technologies recommends the following PowerProtect Data Domain configurations.

Add the Network Card option of an Intel® X710 Dual Port 10 GbE SFP+ PCIe Full Height Adapter.

You can configure these models in any way, including for HA, following these recommendations:

• Include DD Boost licensing for integration with PowerProtect Data Manager.
• Include replication licensing to support Data Domain Replication.
• Add the Network Daughter Card – DD 10GSFP I/O Module NDC Intel with four 10GbE SR SFP Transceivers.
• Add one of the following options for DPE Input Output Cards and Transceivers:
  • Two 10GSFP ENET 4 PT INTEL with eight 10GbE SR SFP Transceivers
  • One 25GSFPENET 2PT INTEL and one 10/25G SFP ENET 2P Low Profile INTEL with four 25GbE SR SFP-NOF Transceivers

Configure this model in any way, including for high availability, following these recommendations.

• Include DD Boost licensing for integration with PowerProtect Data Manager.
• Include Replication licensing to support Data Domain Replication.
• Add a network daughter card – DD 10GSFP I/O Module NDC Intel with four 10 GbE SR SFP transceivers.
• Add one of the following options for DPE input-output cards and transceivers:
• Two 10 G SFP ENET four-port INTEL with eight 10 GbE SR SFP transceivers
• One 25 G SFP ENET two-port INTEL and one 10/25 G SFP ENET two-port Low Profile INTEL with four 25 GbE SR SFP-NOF transceivers
• Two 100 G QSFP I/O Module Mellanox with four 100 GbE SR4 SFP-NOR transceivers

Data Domain Boost (DD Boost) extends the optimization capabilities of Data Domain solutions.

DD Boost also simplifies disaster recovery procedures and increases performance by distributing parts of the deduplication process to the backup server or application clients. It provides faster and more efficient backup and recovery.

For more information about DD Boost, see the following documents on the Dell Technologies Support website:

• Dell DDOS Administration Guide
• Dell DD Boost for Partner Integration Administration Guide
• Dell DD Boost for OpenStorage Administration Guide
• Dell DD Boost FS for Linux Configuration Guide
• Dell DD Boost FS for Windows Configuration Guide

Replication is performed for disaster recovery and multisite backup and archive consolidation.

Data Domain replication provides automated, policy-based, network-efficient, encrypted replication, replicating compressed, deduplicated data over a wide area network asynchronously. Data Domain replication requires a minimum of two Data Domain systems, both of which require replication licensing.

Data Domain supports the following replication topologies:

• One to One
• One to One bi-directional
• One to Many
• Many to One
• Cascaded

For additional information, see the latest Data Domain DDOS Administration Guide.

The Data Domain Metadata on Flash feature creates caches for the Data Domain file system metadata.

The caches are created on solid state drives (SSD) and accelerate access to metadata and backup data through low latency and high IOPS. This feature improves performance for both traditional and random workloads.

The DD9900 model includes a mandatory 2U SSD storage shelf, called the FS25 SSD Shelf. It counts towards the total number of shelves that are connected to the Data Domain system. This shelf supports 10 SSD drives.

The DD9400 controller contains five SSDs in the front-facing 2.5" bays for support of the Metadata on Flash feature. The DD6900 controller contains two SSDs in the front facing 2.5" bays for support of the Metadata on Flash feature.

Data Domain offers a high availability option which supports all current Data Domain features and functionality and integrates with all Data Domain-supported converged infrastructure systems.

Data Domain HA is supported on the following Data Domain systems:

• DD6900
• DD9400
• DD9900

In a Data Domain HA configuration, two identical Data Domain controllers (nodes) are configured as an Active-Standby pair. This pair allows for redundancy if there is a system failure, such as loss of power, or a system crash of the Active node.

Data Domain HA provides the following benefits:

• Automatic failover requiring no user intervention
• Fully redundant design with no single point of failure within the system
• No loss of performance on failover
• Failover within 10 minutes for most operations (DD Boost applications may take longer. CIFS, DD VTL, and NDMP must be manually restarted)
• Ease of management and configuration through DD OS CLIs
• Alerting for hardware failures and malfunctions
• Single-node performance and scalability within an HA configuration in both a healthy and degraded state
• Support of the same feature set as single-node Data Domain systems (except for Data Domain Extended Retention and vDisk)
• Support for systems with all SAS drives.
• No impact to the ability to scale the Data Domain system.
• Support for nondisruptive software updates and upgrades

Data Domain HA functionality is supported with IP. To ensure HA, the Active and Standby nodes must have access to the same IP networks.

For IP connectivity, the Data Domain HA system uses a floating IP address. This address ensures data access to the Data Domain HA system, regardless of which physical node is the Active node.

The onboard Integrated Dell Remote Access Controller (iDRAC) is configured for system upgrades and maintenance operations.

You can also use iDRAC to change security settings, and to remotely power the system on and off. Do not use the iDRAC to change the storage configuration, system settings, or BIOS settings, as doing so impacts system functionality. Only Data Domain technical support should perform changes to these settings.

PowerProtect DD Management Center (DDMC) is a scalable solution for centralized management of multiple Data Domain systems. Dell Technologies highly recommends DDMC, a browser-based VMware virtual system, for installation on the management platform of the 3-Tier platform. However. If a management platform is not available, you can install DDMC on another system that is IP-accessible. DDMC includes the following features:

• Provides current and historical data for all managed systems with subject presentation ranging from site-wide summaries to granular detail for a selected object.
• Projects system capacity and availability, capacity threshold health, and a compression factor.
• Monitors storage on multiple systems with DD Boost backup and replication.

The 3-Tier Platform Data Protection backup system uses either the 3-Tier Platform top-of-rack or a dedicated pair of network switches for backup traffic.

Dell Technologies recommends using the 3-Tier platform top-of-rack switches unless there are not enough available ports to support the backup system. In that scenario, allocate a dedicated pair of switches for the backup system.

With dedicated network switches for the backup system, the 3-Tier Platform network is extended into the backup system network with uplinks. This configuration allows only backup traffic and management traffic to pass. Dell Technologies recommends using Cisco Nexus 93180YC-FX network switches for dedicated data protection backup.

Dell Technologies recommends that out of band management traffic, in particular iDRAC for PowerProtect Data Domain, connects into the 3-Tier Platform management platform network switches.

Use the following table to determine the number of ports that are required for:

• OOB management on the management platform network switches
• In-band management or backup data on the 3-Tier platform or dedicated data protection switches

The following figure shows the 3-Tier Platform data protection backup network connectivity without dedicated network switches:

If you are using the recommended top-of-rack network switch for the 3-Tier Platform, breakout connectivity may be required for 10/25 GbE connections. See the Network Layer chapter in this guide for more information.

The following figure shows the 3-Tier Platform data protection backup network connectivity with dedicated network switches.

Dell Technologies recommends the VLANs for 3-Tier Platform data protection backup.

The ability to protect critical data and to manage and view system health, performance, and efficiency is imperative in an enterprise environment.

Dell Technologies offers various solutions to make managing your data protection systems easier. You can integrate the following data protection management solutions with a 3-Tier Platform:

• Data Protection Central
• Data Protection Advisor

Data Protection Central is a management console for PowerProtect Data Domain, PowerProtect Data Manager, and Data Protection Advisor.

Data Protection Central provides the following features:

• Ability to launch the following administrator consoles from a central location:
  • PowerProtect Data Manager
  • PowerProtect DD System Manager
  • Data Protection Advisor Console
• Supports single sign-on authentication for the following:
  • PowerProtect Data Manager
  • PowerProtect Data Domain
• Comprehensive and customizable Dashboards for information for PowerProtect Data Manager, PowerProtect Data Domain, and Data Protection Advisor details:
  • PowerProtect Data Manager
• Launch PowerProtect Data Manager UI using Single Sign-On.
• Monitor system health status and any alerts from the system.
• View and tag assets that are added to PowerProtect Data Manager.
• Monitor backup and replication activities at the asset and job level:
  • PowerProtect Data Domain
• Launch PowerProtect DD System Manager using Single Sign-On.
• Monitor system health status and any alerts from the system.
• Monitor Storage capacity usage:
  • Data Protection Advisor
• Launch Data Protection Advisor Web Console using Single Sign-on.
• Ability to run most used Data Protection Advisor reports for PowerProtect Data Manager and PowerProtect Data Domain.
• Reporting capabilities through integration with Data Protection Advisor

Data Protection Central requires the following resources:

Dell Technologies recommends that Data Protection Central be deployed to the 3-Tier management platform as a VM. Once it is deployed, the Data Protection Central console is accessible through a web browser.

Through the console, you can perform the following actions:

• Add and launch the following systems for management:
  • PowerProtect Data Manager
  • PowerProtect DD System Manager
  • Data Protection Advisor
• Access the following monitors:
  • Activity monitor
  • Alert Monitor
  • Capacity monitor
• Generate reports for PowerProtect Data Manager and PowerProtect Data Domain when Data Protection Advisor is in the environment.

Data Protection Advisor is a platform for performing reporting and analytics in a data protection environment.

This solution can provide full visibility into the health and effectiveness of the data protection strategy. It can monitor technologies including backup software, storage array replication, servers, databases, and virtual infrastructures.

The Data Protection Advisor reporting engine provides highly customizable reports for:

• Highlighting issues within the environment
• Capacity management
• Service level
• Chargeback
• Change management

The Data Protection Advisor Predictive Analysis Engine provides:

• Early warnings about potential issues
• Alerts to allow for early resolution of potential issues
• A reduction in negative business impacts

The following systems are discovered and monitored in the initial release of Data Protection Advisor:

• PowerProtect Data Manager
• PowerProtect Data Domain
• VPLEX
• RecoverPoint
• RecoverPoint for Virtual Machines

You can add other hardware or software that is listed in the Data Protection Advisor Software Compatibility Guide for monitoring after the initial deployment.

A Data Protection Advisor deployment includes the following installations:

• A Data Protection Advisor datastore server on a single virtual machine
• A Data Protection Advisor application server on a single virtual machine

To determine the sizing for the Data Protection Advisor datastore server and application server, use Solution Builder by engaging a Dell Technologies professional.

The sizing estimator uses the following input to size the Data Protection Advisor environment:

• Total number of backup servers to be monitored.
• Total number of backup jobs per day.
• Storage replication capacity.
• Reports and analysis rules that are run per day.
• Expected annual growth.

The sizing estimator provides the following output for both the datastore server and application server using this input:

• Minimum storage requirement
• Minimum CPU requirement
• Minimum memory requirement

The sizing estimator is not exact and may only provide a general recommendation that is based on assumptions and expectations. It is a best practice is to virtualize each of these components for flexible system resource expansion.

Dell Technologies recommends that the Data Protection Advisor datastore and application servers be installed on Windows Server VMs that are hosted on the 3-Tier management platform.

The following resource settings are applied to the memory, CPU, virtual disk, and NIC of each of the VMs.

Dell Technologies recommends that the datastore be exported to a flat file on a regular schedule in order to safely back up the Data Protection datastore. Back this up using a backup product such as Avamar or NetWorker. This technique is the only supported method for backing up the Data Protection Advisor datastore.

The Data Protection Advisor (DPA) agents perform data collection from objects that DPA monitors. For objects such as PowerProtect Data Manager and Data Domain, deploy a DPA proxy agent on a designated host, other than the datastore and application servers, for data collection services.

When monitoring objects at remote sites, deploy DPA proxy agents at each site for collection duties. The agents typically communicate with monitored objects by using CLI, SNMP, SSH, or direct database connection. The DPA proxy agents may be sensitive to latency, which might cause data collection to fail.

No limit applies to the number of objects that a single DPA agent can monitor. There is no limit on the number of DPA agents that may be deployed. There are also no additional licensing costs for the DPA agents themselves. However, additional licensing costs may apply for the operating systems on which the agents operate.

The management solution uses workloads to adapt and respond to evolving management capabilities for one or more 3-Tier Platforms. The solution runs the workloads as collections of core applications, optional Dell Technologies applications, and ecosystem applications to provide:

• Health monitoring
• Performance and capacity management
• Fault domain isolation

The following principles are central to the design and architecture of the 3-Tier Platform:

• No single point of failure: The critical aspects of the 3-Tier platform system incorporate redundancy, including server HA features, redundant networking, and multipath storage.
• Integrated management: When the VMware hypervisor is deployed, the 3-Tier Platform includes integrated management by using Cisco UCS C-Series servers and a Dell Unity XT Hybrid Array.
• Hardware configuration for virtualization: The 3-Tier platform is designed for general-purpose virtualized workloads. Each server is equipped with the processing power, memory, and converged network adapters that support Ethernet and FC, as required for virtualization.
• Best practices: The 3-Tier platform design incorporates storage, networking, and hypervisor best practices to ensure availability and serviceability.
• Flexible configuration: The 3-Tier platform is configurable to suit requirements for a specific virtualized infrastructure. The system supports flexible options to suit customer needs, including:
  • Rack and blade server models
  • Compute server quantity
  • Processor model
  • Memory capacity
  • Storage array types and capacity

The following management solutions are supported on the 3-Tier Platform:

• Single system management
• Multisystem management

Single system management uses one VMware vCenter Server to operate core applications for the management and workload domain on the 3-Tier Platform. Single system management uses Cisco ToR and management switches for network connectivity.

Multisystem management contains servers and shared storage that manage multiple 3-Tier Platforms in a data center. The 3-Tier Platform multisystem management solution uses Cisco ToR and management switches for network connectivity and supports up to 14 production domain VMware vCenter servers.

The following table shows the available configurations:

Single-system and multisystem management solutions use installed software components that are based on the LCM release for the deployed solution version.

The following table shows the hardware and software components of the management solution: