FSx Data Repository Association (DRA) Setup Guide

FSx Data Repository Association (DRA) Setup Guide

Overview

Data Repository Associations (DRAs) link FSx Lustre filesystems to S3 buckets, enabling automatic import/export of data between the filesystem and S3. This guide covers setting up DRAs for cross-account S3 bucket access.

Key Learnings

Important: When creating DRAs to remote S3 buckets (especially cross-account), you must grant S3 and KMS permissions to BOTH:

  1. The current user’s IAM role that will execute the DRA creation command
  2. The FSx service linked role that will actually access the S3bucket

However, the FSx service linked role is only created after the first successful DRA is created. This creates a chicken-and-egg problem:

  • You can’t grant permissions to the service linked role until it exists
  • The service linked role won’t exist until you successfully create the first DRA

Solution: Create a temporary DRA to any accessible S3 bucket first to trigger service linked role creation. Once the role is created:

  1. Grant it S3/KMS permissions in the remote account
  2. Delete the temporary DRA
  3. Create the actual DRAs to the remote buckets

The initial DRA creation command validates access using the current user’s credentials, while the actual data operations use the service linked role.

Architecture

When you create an FSx filesystem with DRAs configured, AWS automatically creates a service linked role:

  • Role Name: AWSServiceRoleForFSxS3Access_<filesystem-id>
  • Service: s3.data-source.lustre.fsx.amazonaws.com
  • Purpose: Allows FSx to access S3 buckets for import/export operations

Permission Requirements

1. Current User Role Permissions

Your user role (e.g., ODINDevOps) needs S3 permissions to validate bucket access during DRA creation:

{
  "Effect": "Allow",
  "Action": [
    "s3:GetObject",
    "s3:ListBucket",
    "s3:PutObject"
  ],
  "Resource": [
    "arn:aws:s3:::bucket-name",
    "arn:aws:s3:::bucket-name/*"
  ]
}

2. FSx Service Linked Role Permissions (Bucket Policy)

The remote bucket must have a bucket policy granting the FSx service linked role access. Add this to the bucket policy in the remote account:

{
  "Sid": "AllowFSxDataRepositoryAccess",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::YOUR_ACCOUNT:role/aws-service-role/s3.data-source.lustre.fsx.amazonaws.com/AWSServiceRoleForFSxS3Access_fs-XXXXXXXXX"
  },
  "Action": [
    "s3:GetObject",
    "s3:PutObject",
    "s3:DeleteObject",
    "s3:ListBucket",
    "s3:GetBucketLocation",
    "s3:GetBucketAcl",
    "s3:GetBucketNotification",
    "s3:PutBucketNotification",
    "s3:AbortMultipartUpload",
    "s3:ListMultipartUploadParts",
    "s3:ListBucketMultipartUploads"
  ],
  "Resource": [
    "arn:aws:s3:::bucket-name",
    "arn:aws:s3:::bucket-name/*"
  ]
}

3. KMS Permissions (if bucket is encrypted)

If the S3 bucket uses KMS encryption, the FSx service linked role needs decrypt/encrypt permissions. Add this to the KMS key policy in the remote account:

{
  "Sid": "AllowFSxKMSAccess",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::YOUR_ACCOUNT:role/aws-service-role/s3.data-source.lustre.fsx.amazonaws.com/AWSServiceRoleForFSxS3Access_fs-XXXXXXXXX"
  },
  "Action": [
    "kms:Decrypt",
    "kms:DescribeKey",
    "kms:GenerateDataKey"
  ],
  "Resource": "*"
}

Setup Process

Step 1: Get FSx Filesystem ID

If creating a new FSx filesystem, you’ll need the filesystem ID after it’s created:

aws fsx describe-file-systems --query 'FileSystems[?Tags[?Key=="Name" && contains(Value, "your-fsx-name")]].FileSystemId' --output text

Step 2: Identify the Service Linked Role ARN

Once the FSx filesystem exists, the service linked role will be created when you attempt the first DRA:

aws iam list-roles --query "Roles[?RoleName=='AWSServiceRoleForFSxS3Access_<filesystem-id>'].Arn" --output text

Example output:

arn:aws:iam::455037549247:role/aws-service-role/s3.data-source.lustre.fsx.amazonaws.com/AWSServiceRoleForFSxS3Access_fs-0e79e6793fec3276b

Step 3: Remote Account Setup

In the remote account (where the S3 bucket is located):

  1. Add bucket policy - Grant the FSx service linked role permissions (see Permission Requirements section above)
  2. Update KMS key policy - If bucket is encrypted, grant decrypt/encrypt permissions (see Permission Requirements section above)
  3. Verify permissions - After policy updates, wait a few moments for propagation

Step 4: Create DRA via Terraform

Define DRAs in your Terraform configuration:

module "my_fsx_lustre" {
  source = "path/to/victoria-lustre-module"
  
  # ... other configuration ...
  
  dras = [
    {
      file_system_path         = "/my-mount-path"
      data_repository_path     = "s3://remote-bucket-name/prefix/"
      imported_file_chunk_size = 1024
      auto_import_events       = ["NEW", "CHANGED", "DELETED"]
      auto_export_events       = ["NEW", "CHANGED", "DELETED"]
    }
  ]
}

Then apply with your user profile:

export AWS_PROFILE=odin
terraform apply -target=module.my_fsx_lustre

Step 5: Create DRA via AWS CLI

Alternatively, create DRAs using the AWS CLI:

export AWS_PROFILE=odin

aws fsx create-data-repository-association \
  --file-system-id fs-XXXXXXXXX \
  --file-system-path /mount-path \
  --data-repository-path "s3://bucket-name/prefix/" \
  --s3 'AutoImportPolicy={Events=[NEW,CHANGED,DELETED]},AutoExportPolicy={Events=[NEW,CHANGED,DELETED]}' \
  --imported-file-chunk-size 1024

Troubleshooting

Error: “Amazon FSx is unable to validate access to the S3 bucket”

This error indicates permission issues. Verify:

  1. User role has S3 permissions - Your current user role can list and access the bucket
  2. Service linked role exists - Check that the role was created: 
    aws iam list-roles | grep "AWSServiceRoleForFSxS3Access"
  3. Bucket policy is correct - Verify the exact service linked role ARN in the policy
  4. KMS permissions - If bucket is encrypted, check KMS key policy grants decrypt access
  5. Policy propagation - Wait a few moments after policy changes before retrying

Checking Service Linked Role Creation

The service linked role is created when the first DRA creation attempt is made (even if it fails). Monitor its creation:

aws iam list-roles --query "Roles[?contains(RoleName, 'AWSServiceRoleForFSxS3Access')].RoleName" --output table

Example: Cross-Account Setup

For the Odin-to-Kamino data migration scenario:

Odin Account (455037549247):

  • FSx Lustre filesystem: fs-0e79e6793fec3276b
  • Service linked role: AWSServiceRoleForFSxS3Access_fs-0e79e6793fec3276b
  • User role: ODINDevOps

Kamino Account (where buckets are):

  • Add bucket policy granting Odin’s service linked role full S3 access
  • Add KMS key policy granting the role decrypt/encrypt permissions
  • Ensure both the service linked role ARN and account ID are correctly specified

References