Linux Data Manager

Linux Data Manager

The Linux Data Manager provides Samba gateway access to FSx Lustre storage for GxP-compliant data management.

Overview

Property Value
Hostname data-manager-linux.odin.cluster.local
Instance Type t3.medium
OS Ubuntu 22.04
Purpose Samba gateway providing SMB access to FSx Lustre

SSH Access

ssh data-manager-linux

Or from the jump host:

ssh YOUR_USERNAME@data-manager-linux.odin.cluster.local

FSx Mounts

The Linux Data Manager has access to multiple FSx Lustre filesystems:

Mount Point Size Description
/mnt/odin 12TB Main HPC data with legacy buckets
/mnt/qcs 1.2TB QCS Navify ingest/output buckets
/mnt/gxp 1.2TB GxP-compliant data storage
/mnt/ingest 9.6TB Kamino data migration ingress buckets

Samba Shares

The Linux Data Manager exports FSx filesystems via Samba for Windows access:

Share Name FSx Path Size Description
\\...\gxp /mnt/gxp 1.2TB GxP-compliant data storage
\\...\odin /mnt/odin 12TB Main HPC data with legacy buckets
\\...\qcs /mnt/qcs 1.2TB QCS Navify ingest/output buckets
\\...\ingest /mnt/ingest 9.6TB Kamino data migration ingress buckets

Full UNC paths:

\\data-manager-linux.odin.cluster.local\gxp
\\data-manager-linux.odin.cluster.local\odin
\\data-manager-linux.odin.cluster.local\qcs
\\data-manager-linux.odin.cluster.local\ingest

Data Flow Architecture

graph TB
    S3[(S3 Buckets)] <-->|Auto Sync| FSX[FSx Lustre]
    FSX -->|Native Mount| DML[Linux Data Manager]
    DML -->|Samba SMB| DMW[Windows Data Manager]
    DMW -->|Mounted Drive| User[Windows User]

Users can access FSx data through:

  1. Direct FSx mount on login nodes, headnode, and compute nodes
  2. Samba share from Linux data-manager for Windows access
  3. SSH/SCP to login nodes or data-manager instances for file transfers

Common Operations

List Available Data

# List main HPC data
ls /mnt/odin/

# List QCS Navify data
ls /mnt/qcs/

# List GxP data
ls /mnt/gxp/

# List Kamino ingress data
ls /mnt/ingest/

Transfer Files via SCP

From your local machine:

# Upload to data manager
scp myfile.txt data-manager-linux:/mnt/gxp/

# Download from data manager
scp data-manager-linux:/mnt/gxp/myfile.txt ./

Check Samba Service Status

ssh data-manager-linux "sudo systemctl status smbd"

Next Steps