FSx

Launch 3rd part filesystems in a fully managed way.

Provided filesystems:

  • Windows File Server

  • Lustre

  • NetApp ONTAP

  • OpenZFS

FSx for Windows Server

Fully managed native Windows file share service. It’s designed for integrating with Windows environments and it’s usually available at \\fs-xxxxxxxx.mycompany.org\sharename.

It can integrate with AWS Managed directory services or on-prem self-managed ones directly.

It can be deployes in single or Multi-AZ mode within a VPC and uses ENI to accept connection. Even in single AZ mode it provides redundancy in that AZ to resist hardware failure. Data can be backed up daily on S3.

FSx provides built-in On-demand and Scheduled backups.

Performance:

  • up to 20 GB/s

  • n000.000 IOPS

  • < 1 ms latency

Backing hardware:

  • SSD

  • HDD

Accessible using:

  • VPC

  • VPC Peering

  • Site-to-site VPN

  • Direct Connect

It can integrate with AWS Workspaces (that REQUIRES a directory service somewhere), so it provides a shared space.

It also supports:

  • Deduplication

  • Distributed File System (DFS)

  • KMS at-rest encryption

  • Enforced in-transit encryption

  • Volume shadow copies for file level versioning

  • User quotas

Unique features

  • VSS - User-driven restores: allow restoring file version with no admin intervention.

  • SMB access

  • Natively uses Windows permissions model

  • DFS (Distributed File System)

  • Integrates with managed or self-hosted directory service

FSx for Lustre

High performance computing (HPC) file system that provides a Managed Lustre implementation for Linux POSIX-compliant (it uses POSIX-style permissions) clients.

It is single AZ resilient. You get a single ENI in one AZ.

Performance:

  • n00 GB/s throughput

  • n000.000 IOPS

  • < 1 ms latency

  • Increases with size: min 1.2 TiB with increments of 2.4 TiB

  • Baseline performance: can burst up to 1.3 GiB/s per TiB (Credit System).

    • Scratch: 200 MB/s per TiB

    • Persistent: 50 MB/s, 100 MB/s, 200 MB/s

FSx for Lustre is deployed in a VPC and is accessed via private networking from on-prem with Site-to-site VPN and Direct connect.

The filesystem is where data lies while being used but, you can associate a repository like S3. In this case data is lazy-loaded from the repository to the filesystem upon first access. When you’re done processing you can use the hsm_archive command to export data back to S3. The filesystem is completely SEPARATE from the repository, sync does not happen automatically, it has to be requested.

Lustre splits data in several parts:

  • Metadata (filenames, permissions) ⇒ Metadata Targets (MST)

  • Data ⇒ Multiple Object Storage Targets (OSTs) of 1.12 TiB each (that’s why performance is so high)

Accessible using:

  • VPC

  • VPC Peering

  • Site-to-site VPN

  • Direct Connect

Deployment Types

Scratch

  • NOT replicated NOR HA

  • For short term data storage

  • For pure performance

  • More data means more server, so more disks, which increases the chances of failure

Persistent

  • SSD or HDD

  • Longer term storage

  • Single AZ High Availability and replication only

  • Self-healing

Backups

You can back up to S3.

  • Automatic: 0-35 days retention

  • Manual

Use cases

  • ML

  • Big Data

  • Financial Modeling

  • Video Processing

  • Electronic Design Automation

  • SageMaker

FSx for NetApp ONTAP

Ideal for moving filesystems that run on ONTAP or NAS to AWS.

Protocols:

  • NFS

  • SMB

  • iSCSI

Compatible with:

  • Linux

  • Windows

  • macOS

  • VMWare Cloud on AWS

  • Amazon Workspaces & AppStream 2.0

  • EC2, ECS, EKS

Features:

  • Storage shrinks or grows automatically

  • Snapshots

  • Deduplication

  • Compression

  • Replication

  • low-cost

  • Point-in-time clone and restore, useful for testing workloads using clones

FSx for OpenZFS

Managed ZSF filesystem. Compatible with NFS (v3, v4, v4.1, v4.2)

Up to 1.000.000 IOPS with < 0.5 ms latency.

Compatible with:

  • Linux

  • Windows

  • macOS

  • VMWare Cloud on AWS

  • Amazon Workspaces & AppStream 2.0

  • EC2, ECS, EKS

Features:

  • Snapshots

  • Compression

  • low-cost

  • Point-in-time clone and restore, useful for testing workloads using clones