The Parascale Virtual Storage Network (VSN) achieves scalability through parallelism and allows administrators to scale file storage and bandwidth independently. Parascale delivers scalability in disk capacity because each server manages its own direct-attached storage and the CPU-intensive block management that it entails. It also delivers scalability in bandwidth because multiple applications access multiple Parascale Storage Nodes in parallel.

A simple deployment architecture is shown above. See the bottom of this page for a discussion of deployment strategies and an example of a high-availability deployment.

Because the Linux file system is mature, it delivers rock-solid stability and support on a broad range of hardware. By leveraging Linux's proven XFS file system, Parascale is free to focus on features that XFS cannot provide. It takes years for a new file system to mature and to be trusted by the user community. Using the latest, proven technologies, the Parascale team has focused its efforts on bringing the best possible solution to the challenges of fast-growing digital content storage.

Like the Google File System, the Parascale VSN employs an asymmetric architecture. One server, the Control Node, is a traffic cop. The Control Node tells Storage Nodes where to put files and manages the system metadata, keeping track of the location of files and replicas, as well as file-locking, file versioning, and access control criteria. No file contents pass through the Control Node.

The efficiency of maintaining all metadata on one node greatly outweighs any risk of a single point of failure. The alternative is to distribute metadata across a clustered system. Distributed metadata incurs increased file-access latency and high cache coherency network traffic, both of which limit scalability and performance. The risk of a single point of failure can be efficiently addressed by hardening just the control node with high-availability hardware, SAN storage, and server failover. See below for a comparison to clustered file systems.

Compared to Control Nodes, Storage Nodes (SNs) need only a basic hardware configuration are often inexpensive x86 servers fitted with inexpensive SATA disk. Of course, exact configurations of CPU, memory, NIC, and disk configuration are selected consistent with application requirements.

In file systems, including Linux XFS, over 90% of the CPU load is block management; the remaining CPU load is related to metadata management. In the Parascale VSN, all file system block management is distributed across the SNs, greatly lightening the load on the CN.

Parascale does not depend on Redundant Arrays of Inexpensive Disk (RAID) for data availability. Large collections of files may require dozens or hundreds of Storage Nodes (SNs), exposing the storage system not only to disk failure, but also to server failure due to motherboard, power supply, or NIC failure. Because RAID only protects against disk failure, it is insufficient for large deployments. Rather than harden every SN to ward off server failure, Parascale is designed to accommodate SN failure without compromising data availability or performance.

Parascale guarantees file availability by replicating files across servers. The number of replicas is specified by the Parascale administrator. When a disk or server does fail, replication also allows recovery to be parallelized. If a file is maintained on servers A, B, and C, the loss of server A will prompt half the files to be replicated from B while the other half are replicated from C. This strategy halves the time needed to restore a lost server (or disk).

While redundancy across servers provides superior resilience compared to single-SN RAID solutions for redundancy, striping (e.g., RAID-0, RAID-5, RAID-6) across SNs is an appropriate solution to improve per-SN bandwidth. Overall system read bandwidth is also improved by replication. Frequently-accessed files, like the most popular movies in a video-on-demand application, should be replicated more often.

The same underlying mechanisms that Parascale uses to provide transparent, automatic file replication, are used to balance file load across servers. For example, if a new server is added to a Parascale deployment to expand storage capacity, Parascale will automatically begin to migrate existing files to the new server. As a result, new files are written across all servers and disks and the new server does not become a bottleneck in system performance.

This "hotspot" problem is common with multi-appliance NAS deployments for digital content. The problem commonly manifests itself in applications, like image archiving, where storage capacity requirements grow progressively and additional storage is provided "as needed." The most recently provisioned server becomes the bottleneck for overall system performance.

Third-party Linux applications can run on Storage Node servers. These applications can run alongside Parascale because the Parascale software runs as application code, not as kernel code. This configuration is appropriate for data processing applications like indexing, compression, or analyses that can take advantage of local disk access and spare CPU cycles.

If applications are deployed on Storage Nodes, these servers should be configured appropriately. Parascale recommends that no third-party applications be deployed on the Control Node server.

Multiple client applications write to or read from multiple SNs in parallel over an IP network (LAN or WAN), using HTTP, NFS, or FTP protocols. (The VSN is architected to additionally support CIFS, though support for this protocol is not provided in the current release.) No Parascale software is required on any application platform; standard protocol clients are used. The VSN presents a single namespace to all applications, yet parallelizes both reads and writes. The parallelization varies with the capabilities of the transfer protocol.

Protocol	Storage Node (SN) allocation	Storage Node data transfer
HTTP	CN redirects to SN, once per file	Direct to/from SN
FTP	Bound statically to an SN	Direct to/from or indirect through SN
NFS v2/3	Mount request made to CN; redirected transparently to an SN	Direct to/from or indirect through SN

HTTP simplifies parallelization of reads and writes because both operations can be "redirected." The original request is made to an IP address of the CN and then redirected to the optimum SN—one that both has the requested data on its hard drives (or the space to write the data) and that is least busy. This permits the CN to serve as a sophisticated load balancer, taking into account much more than simply server load.

Parascale's patented technology enables redirection of NFS operations as well. For the NFS protocol, a specific SN is allocated to a client at mount time. Data writes are usually performed on the disk drives local to that SN. Data reads that cannot be satisfied on the local SN are relayed and fulfilled over the Back-side network's non-blocking switch. This relaying applies to all protocols. Active file migration progressively relocates non-local data to local SNs. Parascale plans to support pNFS to allow direct and parallel client access to storage nodes, thus improving scalability and further reducing Back-side traffic.

We describe the Parascale software as a virtual storage network or virtual storage grid, not a storage cluster. Like storage clusters, the Parascale Virtual Storage Network (VSN) is designed for scalability in storage capacity. However, storage clusters are unlike the Parascale technology in so many ways that we have adopted a different moniker—one more descriptive, if less familiar.

The term "storage" groups both file- and block-oriented systems. Parascale is focused exclusively on files. The term "cluster" has traditionally been used to describe tightly bound systems and implies a symmetric architecture where compute nodes are not specialized. Parascale nodes, by comparison, are loosely bound and we employ two types of specialized nodes: a Control Node and a Storage Node.

	Storage Cluster	Virtual Storage Network (VSN)
Architecture	Symmetric	Asymmetric
Node binding	Tight	Loose
Data topology	Shared	Shared nothing
Scalability	Dozens of nodes	Thousands of nodes
Software mode	Kernel software	Application software
Interconnect	Custom, high-speed (e.g., Fibre Channel)	Standard (e.g. Ethernet)
Examples	Red Hat GFS, PolyServe, Oracle Clustered Filesystem	Parascale, Amazon S3, Google File System

The Parascale data topology is critical to scalability in the number of storage nodes. Because Parascale employs a shared-nothing topology, one node cannot interrupt access by another node to a disk. Shared-nothing topologies greatly reduce file locking and cache coherency issues that limit the scalability of clusters beyond 16 or 32 nodes.

Finally, the Parascale VSN differs from storage clusters because its implementation is as an application—not as low-level kernel code that modifies basic operations within the file system and operating system. Implementing the software as an application means that we leverage the operating system and file system "as is." Parascale greatly reduces the risk of security exploits and data loss by allowing systems administrators to patch the underlying systems as necessary.

In the 20 years since the first introduction of storage clusters, a series of advances in technology have altered the IT environment, including high-performance commodity computers, inexpensive memory, high-bandwidth commodity interconnect, and solid open-source operating systems. By leveraging many of these advances, Parascale has been able to address a new problem ("digital content"), with a new solution (VSN) that adopts a modern network-centric architecture. With companies like Google and Amazon employing similar architectures, we're in good company.

At a minimum, all Control Node deployments should use RAID for metadata availability. For example, RAID-6 protects effectively against concurrent failure of two disk drives. High-availability deployment of the Parascale Control Node should protect against server failure, as well.

Parascale recommends active/passive failover servers with SAN storage to protect database integrity. Because Parascale leverages an IP network, these dual Control Node servers need not be co-located and may be separated by a low-latency channel on a wide-area network (WAN).