aerospike3.8及以上版本的数据淘汰机制
所属分类 aerospike
浏览量 1579
根据原文翻译整理
Eviction mechanisms in Aerospike
https://discuss.aerospike.com/t/eviction-mechanisms-in-aerospike/2854
In Aerospike 3.8 and onwards there are two signficant changes to the eviction mechanism. These are as follows:
The number of eviction histogram buckets is much greater and can be configured by the user.
This is possible by seperating the granularity of expiration TTL histogram and the eviction algorithm.
Buckets will not be partially evicted during eviction
淘汰机制的两个重要变化
驱逐桶的数量要大得多,可以由用户配置。可分别配置过期TTL分布粒度 和 回收算法 。
桶将不会在驱逐期间被部分驱逐
Histogram 直方图 柱状图 分布图
The default number of buckets for the eviction algorithm is 10,000 although this can be configured between 100 and 10,000,000.
The parameter controlling the number of eviction histogram buckets is evict-hist-buckets and it can be set as follows:
asinfo -v 'set-config:context=namespace;id=test;evict-hist-buckets=20000'
驱逐算法桶的默认数量是10000,可以在100到1000万之间进行配置。
控制驱逐桶数的参数为 evict-hist-buckets ,可以用一下命令进行设置
asinfo -v 'set-config:context=namespace;id=test;evict-hist-buckets=20000'
It should be noted that each eviction histogram bucket requires 4 bytes of memory and therefore if the maximum 10,000,000 buckets were set, the resultant eviction histogram would consume 40 MB of memory.
每个驱逐桶需要4字节的内存,因此,如果设置了最大的10,000,000桶,那么最终的驱逐桶将消耗40mb内存。
When evicting during a cold start process, the cluster will configure either 100,000 buckets or use the evict-hist-buckets ,
the highest value of the two will be chosen. This will only be valid for cold start eviction, when the cluster is running, evict-hist-buckets will be the dominant parameter.
在冷启动过程中进行回收时,集群将配置100,000个存储桶或使用 evict-hist-buckets ,将选择这两个中的最大值。这只对冷启动回收有效,当集群运行时,使用 evict-hist-buckets 参数。
When evicting during a cold start process, the cluster will configure either 100,000 buckets or use the evict-hist-buckets ,
the highest value of the two will be chosen. This will only be valid for cold start eviction, when the cluster is running, evict-hist-buckets will be the dominant parameter.
在冷启动过程中进行回收时,集群将配置100,000个存储桶或使用 evict-hist-buckets ,将选择这两个中的最大值。这只对冷启动回收有效,当集群运行时,使用 evict-hist-buckets 参数。
驱逐日志信息
Apr 07 2016 13:42:17 GMT: INFO (nsup): (thr_nsup.c:1079)
{test} found 12345 records eligible for eviction
In some circumstances, no records will be evicted. This could be for a number of reasons and these will be shown in the logs.
在某些情况下,没有记录将被驱逐。这可能有许多原因,这些原因将显示在日志中。
There are situations where evictions will not be possible,
these situations will be logged in a clear and understandable manner.
驱逐没有生效的情况会以清晰易懂的方式记录下来。
Only records with a non-zero TTL will be eligable for eviction (meaning that records without a TTL or with a TTL of 0 will not be considered for eviction).
只有TTL为非零的记录才可用于驱逐(这意味着没有TTL或TTL为0的记录将不被考虑用于驱逐)。
As discussed earlier, only buckets below the threshold bucket will be considered for eviction.
If the threshold bucket is the first bucket, this means that no buckets can be evicted.
The following log message will indicate this (here 200346037 is the void time of the threshold bucket)
如前所述,只有低于阈值的桶才会被考虑驱逐。如果阈值桶是第一个桶,这意味着不能驱逐任何桶。
如果桶中的记录都具有相同的TTL,那么不管分辨率如何,它们都将保持在相同的桶中。
增加 evict-hist-buckets 的影响可能小于增加 evict_tenths_pct。
Void time is the absolute time when a record should be removed from the database (expired).
Void time is measured from the “aerospike epoch”, which is the number of seconds since 1 January 2010 (midnight GMT) or 40 years after the UNIX epoch (1 January 1970 at midnight GMT).
void-time 是一个绝对时间,距离 2010年1月1日 的 秒数
TTL (time to live) is the difference between the current time and the void time.
evict-tenths-pct Maximum 1/10th percentage of objects to be deleted during each cycle of NSUP.
在NSUP的每个周期中,要删除的对象的最大1/10百分比。
https://www.aerospike.com/docs/reference/configuration/#evict-tenths-pct
asinfo -v "set-config:context=namespace;id=namespaceName;evict-tenths-pct=10"
nsup-period defines the amount of time in seconds for NSUP to sleep between cycles (default 120s)
NSUP 线程 运行间隔时间 默认 120s
NSUP namespace supervisor sub-system
evict-hist-buckets defines the amount of buckets used in the eviction algorithm
回收算法桶数
Expiration TTL histogram bucket width will be maximum-record-ttl/100
TTL桶宽 最大记录TTL /100
Eviction histogram bucked width will be maximum-record-ttl/evict-hist-buckets
驱逐桶宽 maximum-record-ttl/evict-hist-buckets
Cold start will use a minimum of 100,000 buckets for the eviction histogram
冷启动时最小的驱逐桶数 100000
上一篇
下一篇
2018年简单总结
Aerospike3.8之前版本的TTL和驱逐机制
人工智能AI真香!!!
aerospike中的过期驱逐和停止写入机制
aerospike数据逐出与停止写入相关参数
2018年度最让人尴尬的新闻