aerospike数据分布
所属分类 aerospike
浏览量 1573
Data Distribution
https://www.aerospike.com/docs/architecture/data-distribution.html
Shared-Nothing architecture
Every node in the Aerospike cluster is identical.
All nodes are peers.
There is no single point of failure.
节点对等,无单点失败
Smart Partitions algorithm, data distributes evenly across all nodes in the cluster.
智能分区算法,数据均匀分布在集群中的所有节点上。
Each namespace is divided into 4096 logical partitions, which are evenly distributed between the cluster nodes.
每个命名空间被分成4096个逻辑分区,均匀分布在集群节点上
https://www.aerospike.com/products/features/dynamic-cluster-management/
keys RipeMD160 digests mod4096 partitions partition-assignment partitionMap
uses a deterministic hash process to consistently map a record to a single partition.
使用确定性哈希一致地将记录映射到单个分区。
To determine record assingment to a partition,
the record's key (of any size) is hashed into a 20-byte fixed-length digest using RIPEMD160.
Using 12 bits of this digest, the partition ID of the record is determined.
使用RIPEMD160散列把key 映射成 20字节的摘要值
使用摘要的12位,确定记录的分区ID
RIPEMD160, which is a field-tested, extremely random hash function,
ensures that records distribute very evenly on a partition by partition basis.
RIPEMD160是一个经过实地测试的非常随机的散列函数,它确保记录在分区的基础上非常均匀地分布。
Partition Distribution to Cluster Nodes
uses a random hashing method to ensure that partitions distribute evenly to the cluster nodes.
There is no need for manual sharding.
All of the nodes in the cluster are peers .
there is no single database master node that can fail and take the whole database down.
When nodes are added or removed,
a new cluster will form and its nodes will coordinate to evenly divide partitions between themselves.
The cluster will then automatically re-balance.
分区均匀分布在集群节点上,不需要手工分布
节点对等, 不会单点失败导致整个数据库挂掉
新增或删除节点,自动协调均匀划分分区,自动平衡
uses a random hashing method to ensure that partitions distribute evenly to the cluster nodes.
There is no need for manual sharding.
All of the nodes in the cluster are peers .
there is no single database master node that can fail and take the whole database down.
When nodes are added or removed,
a new cluster will form and its nodes will coordinate to evenly divide partitions between themselves.
The cluster will then automatically re-balance.
Data Replication and Synchronization
replicates partitions on one or more nodes.
分区复制 副本
One node becomes the data master for reads and writes for a partition,
while other nodes store its replica partitions.
由分区master节点处理读写
The replication factor is configurable;
however, it cannot exceed the number of nodes in the cluster.
More replicas equals better reliability,
but creates higher cluster demand as write requests must go to all replicas.
Most deployments use replication factor of 2 (one master copy and one replica).
副本因子可以配置,不过不能超过 节点数
一般建议2个副本
Synchronous replication provides a higher level of correctness in the face of no network faults.
A write transaction propagates to all replicas before committing the data and returning results to the client.
In rare cases during cluster reconfiguration when the Smart Client
may have sent the request to the wrong node because it is briefly out of date,
the Smart Cluster transparently proxys the request to the right node.
When a cluster is recovering from partitioning,
there may be writes which have been applied in conflict to different partitions.
In this case, Aerospike applies a heuristic to choose the most likely version,
which is it resolves any conflicts that occurred between different copies of the data.
By default, the version with the largest number of changes ( highest generation count ) is chosen,
although the version with the most recently modified time can be chosen.
The correct choice will be determined by the data model.
在没有网络故障的情况下,同步复制提供更高级别的正确性。
写入传播到所有副本才会返回给客户端
在集群重新配置期间,极少数情况下,当智能客户机可能将请求发送到错误的节点(因为请求已经过期)时,
智能集群会透明地将请求代理到正确的节点。
写冲突 启发式选择最可能的版本
默认选择更新次数最大的版本
可以选择更改时间最近的版本。正确的选择将由数据模型决定。
Aerospike Cluster with No Replication
replication factor = 1
Smart Client is location-aware.
It knows where each partition is located so that the data retrieval is achieved in a single hop.
Every read and write request is sent to the data master for processing.
智能客户端是位置感知的。知道每个分区的位置,以便在一个单跳中实现数据检索。
每个读写请求都发送到数据主服务器进行处理。
When a node receives a write request,
it saves the data and forwards the write request to the replica node.
Once the replica node confirms a successful write and the node writes the data itself,
a confirmation returns to the client.
Automatic Rebalancing
The transaction algorithms integrated with the data distribution system
ensure that there is one consensus vote to coordinate a cluster change.
Voting per cluster change, instead of per transaction,
provides higher performance while maintaining shared-nothing simplicity.
与数据分布系统集成的事务算法确保有一个一致的投票来协调集群更改。
对于每个集群更改而不是每个事务进行投票,可以提供更高的性能,同时保持无共享的简单性。
Aerospike allows configuration options to specify how fast rebalance proceeds.
Temporarily slowing transactions heals the cluster more quickly.
If you need to maintain transactional speed and volume, the cluster rebalances more slowly.
重新平衡的速度可配置 ,暂时减缓事务处理速度可以更快地修复集群
During rebalance, Aerospike does not retain full replication factors of all partitions.
Some in-transit partitions temporarily become single replica,
to provide maximal memory and storage availability as the cluster rebalances to new stability.
重新平衡期间,不会保留所有分区的完整副本。
一些正在传输的分区暂时变成单个副本,以便在集群重新平衡到新的稳定性时提供最大的内存和存储可用性。
By not requiring operator intervention,
the cluster self-heals even at the most demanding times.
For example, in one customer deployment a rack circuit breaker tripped,
and one node of an 8-node cluster went down. No operator intervention was required.
After several hours the fault was corrected and the rack came back online.
Operators never had to take special steps to maintain the Aerospike cluster.
由于不需要操作员的干预,集群即使在最需要的时候也能自我修复。
In Aerospike, capacity planning and system monitoring manage virtually any failure with no loss of service.
You can configure and provision your hardware capacity,
and set up the replication/synchronization policies
so that the database recovers from failures without affecting users.
容量规划和系统监控几乎可以管理任何故障,而不会造成服务损失。
可以配置和提供硬件容量,并设置复制/同步策略,以便数据库在不影响用户的情况下从故障中恢复。
Traffic Saturation Management
The Aerospike Database monitoring tools let you evaluate bottlenecks.
Network bottlenecks decrease database throughput capacity, making requests slow.
流量饱和度管理
利用监视工具评估瓶颈
网络瓶颈降低了数据库吞吐量,使请求变慢。
Capacity Overflows
On storage overflow, the Aerospike stop-write limit prevents new record writes.
Replica and migration writes, as well as reads, continue processing.
So, even beyond optimal capacity, the database does not stop handling requests.
It continues to do as much as possible to continue processing user requests.
存储溢出时,stop-write限制 阻止新记录的写入。
复制和迁移读写操作继续处理。
因此,即使超出了最佳容量,数据库也不会停止处理请求。
它继续尽可能地继续处理用户请求。
上一篇
下一篇
aerospike写块大小设置FAQ
五大最佳开源java性能监控工具
轻量级web容器undertow
图论术语表
zookeeper的一些缺点
neo4j编译与安装记录