lucene flush commit与elasticsearch的refresh flush
所属分类 lucene
浏览量 1135
参考源码
elasticsearch5.6.12 lucene6.6.1
lucene
flush 将内存中的segment写到磁盘但不执行fileChannel.force一部分数据会在buffer中
commit 调用force,将buffer中的数据写到磁盘
elasticsearch
refresh 调用lucene的flush
flush 调用lucene的commit
lucene flush
DocumentsWriter.doFlush
DocumentsWriterPerThread.flush
DefaultIndexingChain.flush
// nvd nvm文件
writeNorms
writeDocValues
// dii dim文件
writePoints
// fdt, fdx文件
storedFieldsConsumer.flush
// doc, pos, tim, tip文件
termsHash.flush
// fnm文件
docWriter.codec.fieldInfosFormat().write
// cfs, cfe, si, liv 文件
DocumentsWriterPerThread.sealFlushedSegment
// 删除cfs, cfe, si, liv 之外的文件
IndexWriter.doAfterFlush
flush 相关配置
maxBufferedDocs
ramBufferSizeMB
perThreadHardLimitMB
Lucene commit
commit执行fileChannel.force,将buffer中的数据写到磁盘
flush all segments
执行合并 maybeMerge,满足merge条件则合并
IndexWriter.commit
IndexWriter.commitInternal
IndexWriter.prepareCommitInternal
// flush segments
DocumentsWriter.flushAllThreads
// sync
IndexWriter.startCommit
Directory.sync
IOUtils.fsync
FileChannel.force
FileChannelImpl.force
// 更新commit信息segments_n,删除旧的segments_n-1
IndexWriter.finishCommit
// 满足merge条件则merge
IndexWriter.maybeMerge
Lucene maybeMerge
merge 异步执行
合并小段 ,大段不合并
IndexWriter.maybeMerge
IndexWriter.updatePendingMerges
// 查找可merge的segments
TieredMergePolicy.findMerges
// 执行merge
ConcurrentMergeScheduler.merge
// 异步执行merge
MergeThread
es refresh
refresh_interval
indices.memory.index_buffer_size
InternalEngine.refresh
// lucene refresh
ReferenceManager.maybeRefreshBlocking
DirectoryReader.openIfChanged
StandardDirectoryReader.doOpenIfChanged
IndexWriter.getReader
// flush segments
DocumentsWriter.flushAllThreads
IndexWriter.maybeMerge
es flush
translog.ckp
translog-n.tlog
InternalEngine.flush
// prepareCommit translog
Translog.prepareCommit
// es commit index writer
InternalEngine.commitIndexWriter
// lucene commit
IndexWriter.commit
InternalEngine.refresh
Translog.commit
elasticsearch中refresh和flush区别
上一篇
下一篇
Lucene6索引文件格式
Lucene7 Index File Formats
lucene二十年
Lucene核心知识点
《清单革命》读书笔记
Lucene 查询类型及实例