https://www.aerospike.com/docs/architecture/storage.html


Hybrid Storage
The Hybrid Memory System contains indexes and data stored in each node, 
handles interaction with the physical storage, 
contains modules for automatically removing old data from the database, 
and defragments physical storage to optimize disk usage.

混合存储
索引 数据    处理与物理存储的交互
自动删除旧数据
物理存储碎片整理，优化磁盘使用

Storage Media
Aerospike can store data in DRAM, SSDs, and traditional spinning media. 
Each namespace is separately configurable, 
which allows developers to put small, frequently accessed namespaces in DRAM, 
and put larger namespaces in less expensive storage such as SSD. 
Aerospike optimizes data storage on SSDs by bypassing the file system, 
which takes advantage of low-level SSD read and write patterns.

存储介质 
DRAM SSD 传统的旋转介质 （机械硬盘）
每个 namespace独立配置
小的热点数据放在内存
较大的放在成本较低的SSD中
ssd优化， 绕过文件系统，利用低级SSD读写模式。

Record data is stored together.
The default storage size for one row is 1 MB.
Storage is copy-on-write.
Free space is reclaimed during defragmentation.
Each namespace has a fixed amount of storage, 
and each node must have the same namespaces on each server, 
which requires the same amount of storage for each namespace.


记录数据一起存储。
一行的默认存储大小为1 MB。
存储 copy-on-write
在碎片整理期间回收空闲空间。
每个名称空间都有固定的存储空间，每个节点必须具有相同的名称空间，每个名称空间具有相同的存储空间。


DRAM without persistance, DRAM with storage persistance, or using Flash storage (SSDs). 
Persistent storage can also be a file on any storage device.


DRAM

Pure DRAM storage—without persistance
纯内存模式

Aerospike allocates data using JEMalloc, allows allocation into different pools. 
Long-term allocation such as for the storage layer, is separately allocatable. 
JEMalloc has exceptionally low fragmentation properties.

使用JEMalloc分配数据，允许分配到不同的池中。
JEMalloc具有非常低的碎片属性。


achieves high reliabliltiy by using multiple copies of DRAM. 
automatically reshards and replicates data on failure or during cluster node management
k-safety is obtained at a high level. 
When a node returns online, its data automatically populates from a copy.


Data on SSD/Flash

When a write ( update or insert ) has been received from the client, 
a latch is taken on the row to avoid two conflicting writes on the same record for this cluster

行锁， 避免写冲突

in the case of network partition, conflicting writes may be taken to provide availability, which are resolved later )
在网络分区的情况下，可能会使用冲突写来提供可用性，稍后会解决这个问题


 In some cluster states, data may also need to be read from other nodes and conflicts resolved. 
 在某些集群状态下，可能还需要从其他节点读取数据并解决冲突。
 
 After write validation, in-memory representations of the record update on the master. 
 The data to be written to a device is placed in a buffer for writing. 
 When the write buffer is full, it is queued to disk. 
 Write buffer size (which is the same as the maximum row size) 
 and write throughput determine the risk of uncommitted data, 
 and configuration parameters allow flushing of these buffers to limit potential data loss. 
 Replicas and their in-memory indexes then update. 
 After all in-memory copies update, the result returns to the client.
 
写验证之后，主记录更新完成
写缓冲区 ，缓冲区满了之后，排队写入磁盘
写缓冲区大小与最大行大小一致
刷新缓冲区，避免数据丢失
更新副本及其内存索引， 所有内存副本更新知乎，返回给客户端

支持的数据类型


integers
doubles
strings
blobs
lists
maps
geoJSON
natively serialized types

The name of the column is stored only once, 
and only 32K concurrent unique bin names are allowed in a namespace

If you require more bin names, use maps. 
Map allow you to store an arbitrary set of key-value pairs, 
and access those values in UDFs for efficiency.

如果需要多个bin，请使用MAP。Map存储任意键值对，在udf中访问这些值以提高效率。

Integers are stored as 8-byte quantities, which limits integer values in the current version. 
The Aerospike network protocol allows integers of variable size, which allows expansion. 
Aerospike stores strings as UTF-8. UTF-8 is more compact than unicode for many strings. 
To allow cross-language compatibility, 
client libraries convert the native character unicode to UTF-8, and back.

整数存储为8字节的数量。允许可变大小的整数，这允许扩展。
将字符串存储为UTF-8。对于许多字符串，UTF-8比unicode更紧凑。
为了允许跨语言兼容性，客户端库将本机字符unicode转换为UTF-8，然后再转换回来。

Binary objects (blobs) are most efficient in that they are size-limited only by the size of the total record. 
Some customers will use their own serializer, and possibly compress the object and store it directly. 
This means that data cannot be easily accessed using UDFs, 
nor can atomic operations for List and Map be used. 
Aerospike's storage compression feature avoids this side-effect of application-side compression.

二进制对象(blob)的效率最高，因为它们的大小仅受总记录大小的限制。
一些客户将使用自己的序列化器，并可能压缩对象并直接存储它。
这意味着不能使用udf轻松访问数据，也不能使用List和Map的原子操作。
Aerospike的存储压缩特性避免了应用程序端压缩的这种副作用。


Flash Optimization
The Aeropsike Defragmenter tracks the number of active records on each block on disk, 
and reclaims blocks that fall below a minimum level of use. 
The defragmentation system continually scans available blocks, 
and looks for blocks with a certain amount of free space.

Flash优化
碎片整理程序跟踪磁盘上每个块上的活动记录数量，并回收低于最低使用级别的块。
碎片整理系统不断地扫描可用的块，并寻找具有一定空闲空间的块。


Eviction Based on Storage
The Aerospike Evictor removes references to expired records, 
and reclaims memory when the system gets beyond a set high-water mark. 
When configuring a namespace, specify the maximum amount of DRAM avaliable to it 
and the default time-to-live (TTL) or expiration for its data.

回收器删除对过期记录的引用，并在系统超过设置的高水位时回收内存
在配置名称空间时，指定最大内存使用量 默认的生存时间(TTL) 或 过期时间。

The Evictor searches for expired data, frees the in-memory index, and releases the record on disk. 
It also tracks memory used by the namespace and if memory exceeds the set high-water mark, 
releases older although not necessarily expired records. 
By allowing the Evictor to remove old data when the system hits memory limitations, 
Aerospike can effectively be used as an LRU cache. 
Note that the age of a record is measured from the last time it was modified, 
and that the Application can override the default lifetime any time it writes data to the record. 
The Application may also tell the system that a particular record should never be automatically evicted.

回收器搜索过期数据，释放内存索引，并释放磁盘上的记录。
跟踪名称空间使内存使用，如果内存超过设置的高水位标记，则释放旧的记录，但不一定是过期的记录。
通过允许回收器在系统遇到内存限制时删除旧数据，可以有效地用作LRU缓存。
注意，记录的年龄是从最后一次修改记录时开始测量的，
应用程序可以在写入数据时覆盖默认的生存期。
应用程序还可以告诉系统，不应该自动删除特定的记录。