Storage

From DreamFactory
Jump to: navigation, search
m (Layout and Structure)
m
Line 1: Line 1:
 
== Instance Storage ==
 
== Instance Storage ==
When DreamFactory Enterprise™ provisions a new instance, a directory is created for it to use as local storage. The location of this directory depends on the **mount** assigned to the server on which the instance was provisioned.  
+
When DreamFactory Enterprise™ provisions a new instance, a directory is created for it to use as local storage. The location of this directory depends on the '''mount''' assigned to the server on which the instance was provisioned.  
  
 
Mounts can be local or remote storage. As long as the storage space is accessible via the file system of the server, it can be utilized for storage.
 
Mounts can be local or remote storage. As long as the storage space is accessible via the file system of the server, it can be utilized for storage.
  
By default, DreamFactory Enterprise™ defines the mount point `/data/storage` for your initial cluster. This directory contains a subdirectory for each cluster. The initial installation defaults to **local** as the storage area. This exists on your system in `/data/storage/local`. When you create new clusters, you can configure their storage areas in a different subdirectory, or different mount point altogether. It is designed to be flexible and map to most storage scenarios.
+
By default, DreamFactory Enterprise&trade; defines the mount point <pre>/data/storage</pre> for your initial cluster. This directory contains a subdirectory for each cluster. The initial installation defaults to '''local''' as the storage area. This exists on your system in <pre>/data/storage/local</pre>. When you create new clusters, you can configure their storage areas in a different subdirectory, or different mount point altogether. It is designed to be flexible and map to most storage scenarios.
  
 
== Layout and Structure ==
 
== Layout and Structure ==
Line 10: Line 10:
  
 
[[File:storage-layout.png]]
 
[[File:storage-layout.png]]
 +
 +
The <pre>/data</pre> tree contains self-explanatory subdirectories. Under <pre>/data/storage</pre> is where it begins to get interesting.
  
 
=== Lightweight Striping ===
 
=== Lightweight Striping ===
 +
To ''spread'' the storage load, as it were, of thousands of virtual instances, the instance storage tree implements a simplistic hashed directory tree. Each [[DFE/Dashboard|Dashboard]] user is assigned a unique storage ID in the form of a GUID (<pre>[[DFE/Architecture/Database/User|User.storage_id_text]]</pre>). This GUID is hashed, and with it and the first four bytes are used to create a three-level nested subdirectory. Example:
 +
 +
User id #1 has a storage ID of <pre>00933DBC-C7A9-DE0E-5D38-FB23B574A580</pre>.
 +
Applying a SHA1 hash gives us <pre>9414f5052eead73c262d5ad89922541426208763ce86c2b64cc6a6680914a915</pre>.
 +
 +
  
 
== Preventative Maintenance ==
 
== Preventative Maintenance ==
 
=== Automated ===
 
=== Automated ===
 
=== Manual ===
 
=== Manual ===

Revision as of 22:20, 14 December 2015

Instance Storage

When DreamFactory Enterprise™ provisions a new instance, a directory is created for it to use as local storage. The location of this directory depends on the mount assigned to the server on which the instance was provisioned.

Mounts can be local or remote storage. As long as the storage space is accessible via the file system of the server, it can be utilized for storage.

By default, DreamFactory Enterprise™ defines the mount point
/data/storage
for your initial cluster. This directory contains a subdirectory for each cluster. The initial installation defaults to local as the storage area. This exists on your system in
/data/storage/local
. When you create new clusters, you can configure their storage areas in a different subdirectory, or different mount point altogether. It is designed to be flexible and map to most storage scenarios.

Layout and Structure

Below is a diagram of the default layout for storage.

Storage-layout.png

The
/data
tree contains self-explanatory subdirectories. Under
/data/storage
is where it begins to get interesting.

Lightweight Striping

To spread the storage load, as it were, of thousands of virtual instances, the instance storage tree implements a simplistic hashed directory tree. Each Dashboard user is assigned a unique storage ID in the form of a GUID (
[[DFE/Architecture/Database/User|User.storage_id_text]]
). This GUID is hashed, and with it and the first four bytes are used to create a three-level nested subdirectory. Example: User id #1 has a storage ID of
00933DBC-C7A9-DE0E-5D38-FB23B574A580
. Applying a SHA1 hash gives us
9414f5052eead73c262d5ad89922541426208763ce86c2b64cc6a6680914a915
.


Preventative Maintenance

Automated

Manual