PowerScale OneFS 9.8.0.0 Web Administration Guide

• A PowerScale OneFS cluster contains an external (front-end) network over which clients can move data in and out of the cluster. The cluster also has an internal (back-end) network over which the nodes communicate with each other. The back-end network is isolated from devices that are not in the cluster.
• The Amazon Virtual Private Cloud (VPC) must have sufficient IPv4 address space to host OneFS internal and external networks, and any additional clients that are using the deployed cluster. For details on planning the network, see the Isilon OneFS External Network Connectivity Guide version 8.10.

• IPv4 is supported for primary IPs in APEX File Storage for AWS.
• IPv6 is not supported for primary IPs in APEX File Storage for AWS.

• All cloud providers require a "primary" IP on each instance type. Primary IPs are allocated by the cloud provider and are tied to the lifetime of the interface.
• There is a limit on the maximum number of IPs depending on the cloud provider instance type. AWS has a limit for the number of IPs that can be configured in each network interface based on the instance type. If this limit is exceeded, AWS does not allow configuration of the IPs.
• The number of IPs used in the cluster that each node serves must not exceed the maximum number of IPs allowed for the instance type.
• APEX File Storage for AWS prevents most instances of IP oversubscription during configuration time to ensure availability during a cluster outage. OneFS is unable to account for unevenly allocated dynamic IPs, so it cannot prevent all instances of IP oversubscription.
• For more information, see: Elastic Network Interfaces and How Amazon VPC works .

• Limited DHCP support is added in APEX File Storage for AWS for cloud deployments only. DHCP cannot be enabled for on-premises deployments.
• The DHCP service does not configure network interfaces because some settings are managed by the cloud provider, such as ifaces and IP ranges.
• Additional settings are not supported by the cloud provider, such as link aggregation and RDMA. Other externally managed IP configurations are not supported in APEX File Storage for AWS, such as IPv6.
• The dhclient service dhclient-ext-1 is integrated with SmartConnect.
• The DHCP leased IP never changes, however the leases have an expiration of an hour. If APEX File Storage for AWS is unable to reach the DHCP server to renew the lease, the Primary IPs may expire. OneFS writes a CELOG alert before a primary leased AWS IP is set to expire. Administrators can troubleshoot by running the isi event view 32 command.
• In cloud deployments, another allocation method exists: ExternallyManaged. This allocation method was designed to allow cloud providers to dictate the placement of Primary IPs. Pools of this allocation method are created and managed by SmartConnect, and thus cannot be edited or changed from ExternallyManaged. Externally managed network pools can only be created by the system. Pools cannot be changed to be externally managed, and pools cannot be changed from externally managed. This configuration is to prevent accidental misconfigurations.
• If you are adding new IPs to the front-end subnet on the cloud provider, you must extend the range in OneFS using the --force parameter.
• Administrators can modify the IPs in an externally managed network pool using the isi network pool modify subnet0.pool0 --force --add-ranges command.
• Administrators can view the allocation method setting by running the isi network pools view command.

• One network pool is created by default for client connections. The network pool name is groupnet0.subnet0.pool0. Each node in the cluster is assigned one IP address from this pool.
• The IP address assignments for each network interface are on Amazon EC2.
• The Primary IP Pool (groupnet0.subnet0.pool0) is now always ExternallyManaged. Therefore the allocation method cannot be changed, nor can the IPs be reassigned.
  • NOTE:Removing the IP address for a node from the default network pool could cause cluster inaccessibility.
• One IP address per NIC is the primary address and cannot be deleted, changed, or reassigned.
• One IP is used for groupnet0.subnet0.pool0 on each node.
  • The IP addresses used in the pool groupnet0.subnet0.pool0 are the AWS primary addresses that cannot be moved from one node to another.
  • The IP addresses used in the pool groupnet0.subnet0.pool0 cannot be a dynamic pool and cannot be changed to a dynamic pool.
  • NOTE:Mishandling of pool0 or any of the IP addresses in it can render the cluster inaccessible.

• Once a cluster is deployed, users are allowed to create additional network pools. These new pools can use static or dynamic allocation. The remaining IP addresses can be used after the cluster deployment for creating additional pools.

• Every node in a cluster monitors maintenance events from the AWS Instance Meta Data Service (IMDS) through the external network.
• If a node cannot connect to the IMDS through the external network for two minutes or more, the node is set to read-only.

• When configuring a OneFS cluster in AWS, you must allocate two ranges of IP addresses in different AWS subnets, with one for each of the back-end and front-end networks. You can create two dedicated subnets for each OneFS cluster in an existing VPC.
• The internal subnet must be reserved exclusively for use by a single OneFS cluster. The cluster must contain enough free IP addresses to assign one IP address for each instance in the cluster.
• Nodes in a cluster are created with a network interface for external client connections.
  • The external network interface is named 1ni-name ext-1 and nic-name ena1.
  • The external subnet must have at least one free IP address for each node in the OneFS cluster. This subnet can be shared with other clients.
  • AWS reserves the first four addresses in subnet Classless Inter-Domain Routing (CIDR). The first address is used as the default gateway address. One IP is used for groupnet0.subnet0.pool0 on each node. You can use any remaining IPs from the external subnet CIDR range after the cluster deployment to create additional pools.

Administrators create subnets on AWS cloud deployments as follows:

• Create a dedicated subnet for the OneFS cluster internal (back-end) network interfaces.
  • Create one internal network interface IP address for each node.
  • Do not share this subnet with other EC2 instances.
• Create a dedicated subnet for the OneFS cluster external (front-end) network interfaces.
  • Create one external network interface IP address for each node.
  • You can share the subnet with other EC2 instances.
• NOTE:Each NIC in AWS can belong to only one subnet groupnet0.subnet0 and all network pools must belong to groupnet0.subnet0.

• The cluster moves the front-end dynamic IP addresses between nodes during network failover. For on-premises clusters, nodes send GARP packets immediately after the IP move, and the IP reassignment is nearly instantaneous. However, on AWS, the cluster calls the cloud provider API to reassign the IP address which can take approximately 20 s–40 s.
• The back-end network in AWS uses a single network (int-a), and the infrastructure is fully managed by the cloud provider. It uses the AWS primary address of the network interface and must not be modified.
• An additional dynamic pool must be created from the remaining addresses in the external subnet after deployment to use network failover.
• Network failover is slower on AWS and can take 30 s–40 s compared to a few seconds on a PowerScale OneFS on-premises cluster.
• IP addresses in dynamic pools on AWS cannot be changed in the software by the running instance without also going through EC2, which requires authorization.
• The AWS IAM role and policy that you provide to the cluster at deployment time allows the IAM role to unassign and assign IP addresses and describe network interfaces.
• AWS cloud calls that are triggered during normal IP failover flow through the OneFS isi_cloud_net library.

• OneFS clusters running in the Cloud support configuring multiple network pools. When you create a cluster, it creates a default network pool that is known as groupnet0.subnet0.pool0 automatically during the initial cluster deployment. One IP address for each node from the external subnet address range is used in this pool. These IPs are the AWS primary addresses of the external network interfaces. Any remaining unused addresses from the AWS subnet CIDR can be used to create additional network pools.
• OneFS allows both static and dynamic allocation policies for the new pools. You can use the OneFS CLI, OneFS Platform APIs, or the OneFS WebUI to create network pools.
• When an IP address is assigned to an interface on a node, the node makes an API call to the AWS EC2 server to associate the IP address to the network interface. OneFS does not recommend adding elastic IP addresses to the nodes to contact EC2 servers. It is recommended that you create a VPC interface endpoint for nodes to connect directly to AWS EC2 services using private IP addresses, as if the EC2 service is hosted in the cluster VPC.
• The VPC endpoint can be created through the AWS VPC console or by using the AWS CLI. See the procedures in the APEX File Storage for AWS Deployment Guide to create an interface VPC endpoint that connects to an AWS EC2 service.
• Also see Access an AWS service using an interface VPC endpoint.

• The OneFS SmartConnect DNS feature depends on the ability of the DNS server to perform delegation.
• You have the option of using either private DNS servers or the AWS-provided Route53. The default DNS server on AWS, known as the Route53 Resolver, does not support DNS delegation, although it does support forwarding rules for resolution. Therefore, forwarding rules must be set up on Route53 to use the SmartConnect DNS feature.
• Administrators set up Route53 Resolver endpoints, which then forward requests to the SmartConnect IP. For more information, see: Getting started with Route 53 Resolver.

• Cluster resizing, by changing the number of drives in a node or by changing the size of the drives in a node, is not supported. Cluster capacity can only be changed by adding nodes or by smartfailing and deleting nodes.
• Adding a node that was previously removed by a Smartfail operation is not supported. The preferred alternative is to destroy and create instances in AWS.
• Reformatting a node with a new configuration that changes the externally managed IP addresses from the original configuration is not supported in APEX File Storage for AWS.

• A security group must be applied to the external interfaces in the cluster. The details of this group depend on your planned use case. For more information about creating an external security group in OneFS, see the PowerScale OneFS Security Configuration Guide.

• All data drives are NVMe (nonvolatile memory express) types.
  • Bays that are mapped by the PCI bus device function have a maximum of 25 bays (0-24).
  • Drives do not support hard drive or SSD detection interfaces. The EBS type that is used by the cluster is provided at deployment time in the instance user data.

• The boot drive is EBS gp3 (nvd0)
• EBS data drives are EBS st1 or gp3.
  • All data drives in the cluster must be the same EBS type
  • nvd1-nvdN
  • Drive serial number starts with "AWS," for example AWS38335C1FBB24403E3

• Local instance store contents are persistent through reboot.
• Local instance store contents are not migrated with a virtual machine (VM) from host to host.
  • VM host migration is possible on any power-off event.
  • Standard reboots and panics do not cause or enable VM migration.
• Amazon Elastic Block Store (EBS) is remote (off-rack) storage.
• EBS is performance-capped and I/O to local instance stores do not count against this limit.

The OneFS software journal is on the first Local instance store drive in bay 0.

• The remaining Local instance store drives are at the top of the bay map.
• PowerScale nodes on AWS with st1 (hard drive) hard drives use the remaining Local instance stores for metadata read cache.
• Local instance store drives are unused if the cluster EBS type is gp3.
• Local instance store drives are used for metadata that is read if the cluster EBS type is st1 (hard drive) hard drive and displays as data drives.
• EBS data drives are in the remaining bays.

The OneFS software journal is saved during orderly shutdowns as follows:

• The AWS EC2 Management Console and CLI Instance Stop operations cause orderly shutdowns.
• AWS maintenance events cause orderly shutdowns.
• Subsequent boot restores the software journal from a saved copy.

Local instance stores are provisioned from Local NVMe types as follows:

• Two for 8xlarge and 12xlarge, four for 16xlarge and 24xlarge.
• nvd (N+1)-nvd (N+(2 or 4)).
• Volume serial number starts with "vol," for example vol00ea30e97ded5ff9f.

Local instance store data preserved

All instances rebooted

Local instance store data preserved

Host may lose network connectivity during scheduled time

Host may lose power at the scheduled time