PheonixLSM:高性能低空间开销的分布式键值存储本地引擎
作者:
作者单位:

作者简介:

通讯作者:

基金项目:

伦理声明:



PheonixLSM: Local Key-Value Engine for Distributed Key-Value Storage with High Performance and Low Spatial Consumption
Author:
Ethical statement:

Affiliation:

Funding:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    分布式键值存储将数据复制到多个存储服务器的本地引擎中,并通过一致性协议保证各副本数据的一致性。其中,以日志结构合并树为核心数据结构的实现方式最为常见。然而,面向通用业务模式设计的日志结构合并树,并不适合一致性逻辑的特殊业务模式,会引发增删改性能的降低,并在全量修复过程中造成空间放大。针对上述问题,该文提出了一种新型本地引擎 PheonixLSM,通过增加增删改操作和回刷操作的约束,消除了分布式键值存储增删改流程中的双写问题,提升了引擎性能。通过重构日志结构合并树底层的 SST 文件布局,支持删除实时回收空间,消除了全量修复时的额外空间放大。实验结果显示,与原生本地引擎相比,使用 PheonixLSM 的分布式键值存储系统,增删改性能提升 90.7%,全量修复的空间放大从 65.6% 降至 6.4%,并减少了 72.3% 的修复时间。

    Abstract:

    Distributed key-value storage is a key component in the distributed storage system, which replicates the key-value pairs to local engines in different storage servers and uses the consensus algorithm to keep replicas consistent. The log-structured merge tree based local key-value engine is the most popular storage algorithm that designed for general purpose. However, the original LSM-tree structure is not suitable for the specific workload of the upper-layer consensus logic. It usually causes performance loss in the write operations and extra spatial amplification during full-node repair. To solve this problem, a local engine named PheonixLSM is designed for distributed key-value storage. PheonixLSM boosts performance by eliminating the double-sync problem for write operations. It also reorganizes the SST file layout to eliminate extra write amplification during full-node repair. Experimental results showed that, compared with distributed key-value storage using default local engine, that PheonixLSM can achieve up to 90.7% write performance gain and reduce the extra write amplification from 65.6% to 6.4%, and the repair time also can be reduced by 72.3%.

    参考文献
    相似文献
    引证文献
引用本文

引文格式
李润辉,古亮,喻之斌. PheonixLSM:高性能低空间开销的分布式键值存储本地引擎 [J].集成技术,2022,11(2):28-40

Citing format
LI Runhui, GU Liang, YU Zhibin. PheonixLSM: Local Key-Value Engine for Distributed Key-Value Storage with High Performance and Low Spatial Consumption[J]. Journal of Integration Technology,2022,11(2):28-40

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2022-03-22
  • 出版日期: