PheonixLSM: Local Key-Value Engine for Distributed Key-Value Storage with High Performance and Low Spatial Consumption
Author:
Affiliation:

Funding:

Ethical statement:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
    Abstract:

    Distributed key-value storage is a key component in the distributed storage system, which replicates the key-value pairs to local engines in different storage servers and uses the consensus algorithm to keep replicas consistent. The log-structured merge tree based local key-value engine is the most popular storage algorithm that designed for general purpose. However, the original LSM-tree structure is not suitable for the specific workload of the upper-layer consensus logic. It usually causes performance loss in the write operations and extra spatial amplification during full-node repair. To solve this problem, a local engine named PheonixLSM is designed for distributed key-value storage. PheonixLSM boosts performance by eliminating the double-sync problem for write operations. It also reorganizes the SST file layout to eliminate extra write amplification during full-node repair. Experimental results showed that, compared with distributed key-value storage using default local engine, that PheonixLSM can achieve up to 90.7% write performance gain and reduce the extra write amplification from 65.6% to 6.4%, and the repair time also can be reduced by 72.3%.

    Reference
    Related
    Cited by
Get Citation

LI Runhui, GU Liang, YU Zhibin. PheonixLSM: Local Key-Value Engine for Distributed Key-Value Storage with High Performance and Low Spatial Consumption[J]. Journal of Integration Technology,2022,11(2):28-40

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
History
  • Received:
  • Revised:
  • Adopted:
  • Online: March 22,2022
  • Published: