HYPERSCALERS LeoFS Storage Appliance

The Hyperscalers LeoFS Storage applianceHyperscalers and LeoFS have partnered, leveraging their combined expertise of hardware with file system to introduce an innovative storage solution that decisively breaks away from

The Hyperscalers LeoFS Storage appliance

Hyperscalers and LeoFS have partnered, leveraging their combined expertise of hardware with file system to introduce an innovative storage solution that decisively breaks away from the confinements inherent with legacy offerings.

LeoFS is market proven parallel file system for different types of I/O intensive workloads, fully POSIX-Compliant and compatible with all software applications, x86 based servers and IP networks. It is used in almost all industries and has hundreds of customers:

• AI Machine Learning ad Big Data
• Scientific Computing

1. Life Science
2. Biopharmaceutics
3. Genomics
4. Cryo-electron Microscopy  
5. Satellite Imaginary/Observatory
6. Geographical Data and Mapping  
7. Meteorology/Climate 

• Oil and Gas
• Video Surveillance 
• Media and Entertainment
• Higher Education 
• Telecom and Internet

SOLUTION ARCHITECTURE

LeoFS Structure

LeoFS is a full POSIX compliant storage. It comes with native clients for Linux, Windows and macOS, all kernel modules that do not require any patches, using only commodity hardware.

  

With LeoFS, data files are transparently distributed over multiple nodes.  By simply increase the number of servers and disks in the cluster, you can seamlessly scale the file system’s throughput and capacity to the needed level, all being aggregated in a single namespace.

LeoFS was written to break away limitations from traditional storage solutions.  It does not include concepts from legacy algorithms or other open source coding.  The file system provides high performance for all workloads including big and small files, intensive reads and writes (random or sequential) and metadata heavy.  Below is single cluster threshold.

LeoFS is software defined storage system, It doesn’t depend on specific hardware or devices, so all storage server of hyperscalers can be used to build LeoFS system. 

CHARACTERISTICS

No Bottleneck

No controllers, gateways, nor distributors is required in  LeoFS system and data files are transparently distributed over multiple nodes. All client applications communicate directly with all storage nodes.

      

Customer on-site 7.5PB LeoFS system, consists of 102 nodes of 4U 24-bay storage servers, with totally 1,880 drives of 4TB SATA HDDs. Dual 10GbE network is used and provides an aggregated I/O close to 150GB/s. Average single drive write 75MB/s.

Enterprise Features

• Load balance switch, hardware evenly share system workload
• Runs on platforms such as x86, OpenPOWER, ARM, and Xeon Phi 
• Provides access interfaces like Samba, NFS, FTP, HTTP, iSCSI, S3 etc. 
• Support for group/user ACLs and quota
• Fully active network with automatic failure detection
• Supports Infiniband, GigE, multiple subnet and bonding
• Cold data sanity check, automatic repair, no downtime
• WORM directory, avoid modification of saved data

TEST ENVIRONMENT 

Test Environment Diagram

     

Hardware Configuration

Dashboard

RECOMMENDED HARDWARE

S6P 3.5" | S24P-5U

Support up to 1.8 Petabytes with 84x PMR HDD within 5U form factor. Thermal and RVI optimized mechanical design, future proof for 50TB+ HAMR generation. Flexible option for single node / dual node MB to meet different workload needs. Optimized topology with dual SAS card to achieve enhanced performance. Serviceability built in design for painless data center deployment and maintenance.

S5XQ 2.5" | D53XQ-2U

2 CPU Sockets for up to 80 cores using Intel® Xeon® Platinum 8380 Processor 40cores each. 32 Memory slots for up to 8TB DIMM or Up to 12TB DIMM+PMEM (200 series. Storage Option A - 24 Front Storage drive bays 2.5" hot-plug SATA/SAS. Storage Option B - 16 x SATA or SAS and 8 x U.2 NVMe. Storage Option C - 24 x U.2 NVMe. 2 Rear Storage drive bays 2.5" hot-plug NVMe/SATA/SAS drives (optional). 10 PCIe 4.0 Expansion Slots for Network Interface Cards. 2 Dual width accelerators (GPU or FPGA)

       

S6Q | D54Q-2U

Powered by 4th Gen Intel® Xeon® Scalable processors with up to 350W TDP. PCIe 5 ready, supports up to 400GbE networking bandwidth. DDR5 platform. Up to 2x DW accelerators in a 2U chassis for AI inference workloads. Enhanced serviceability with tool-less, hot-swap designs. NEBS compliant for Telco/5G datacenter deployment. Liquid cooling supported.

https://www.hyperscalers.com/storage/storage-servers/S6Q-D54Q-2U-intel-gen4-sapphire-rapids-pcie5-densest-hyperscale-server-nvme-drives-buy

S5P | T22P-4U

Available with either 1 or 2 server nodes(*1). Extremely high density up to 78 HDDs. Optional 2 hot-swappable NVMe for caching. 1 M.2 on board per node for OS redundancy. Dual SAS path design to release the bottle neck of SAS card.

https://www.hyperscalers.com/storage/storage-servers/S5P-T22P-4U

S2PL | D51PL-4U

102 x 3.5" HDD Storage Capacity. Extreme Computing Performance. Screw-less HDD Tray for Efficient Deployment. Flexible and Versatile I/O Scalability. Proprietary 7mm SSD for Fast Booting.

https://www.hyperscalers.com/storage/storage-servers/hyper-converged-D51PL-4U-densest-storage-server-qct-hyper-scale-buy-hyperscale-quanta

S2P | T21P-4U

Single or Dual Server Node Design. HDD/SSD Individually power off. Lower Operating Costs Screwless HDD/SSD. Record Storage Capacity up to 2.34 PB IN 4U. Flexible networking options up to 12x25G Ports

https://www.hyperscalers.com/storage/storage-servers/Quanta-QCT-storage-server-QuantaPlex-T21P-4U-T21P4U-buy-distribution-usa-s2p-tc-hp-apollo-4510-gen10-dell-XA90-super-storage-6047r-compare

LeoFS can be built with almost all storage servers on Hyperscalers website. For interested on high density storage, you can just chose those with as many disk slots as possible. Please note that LeoFS system should be built with all hot swappable disks as online system for production often will have disk defects and disk replacements. The system must be stopped if disk defects occurred in a storage server with non-hot-swappable data disks or metadata disks.  This is something we need to avoid.  In addition, no JBOD is recommended.

Read more about SUCCESSFUL CASES in the Reference Guide in Download Section