| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 | |
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 | |
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 | |
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 | |
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 | |
[CI] Add spell check process to CI (#165) | 1 年前 | |
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 | |
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 | |
[MooncakeStore] Introduce MooncakeStore: A new Distributed Object Store for XpYd PD disaggregation (#126) Mooncake Store is a high-performance **distributed key-value (KV) cache storage engine** designed specifically for LLM inference scenarios. Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the former, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be garbage collected). Mooncake Store provides low-level object storage and management capabilities, while specific caching strategies (e.g., eviction policies) are left to upper-layer frameworks (like vLLM) or users for implementation, offering higher flexibility and customizability. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including Put, Get, and Remove operations. - **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - **Eventual consistency**: Mooncake Store ensures that Get operations read complete and correct data, but does not guarantee the latest written data. This eventual consistency model ensures high performance while simplifying system design. - **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - **Dynamic resource scaling**: Mooncake Store supports dynamically adding and removing nodes to flexibly handle changes in system load, achieving elastic resource management (to be refined in future versions). --------- Co-authored-by: Feng Ren <alogfans@gmail.com> Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com> Co-authored-by: susun <751080330@qq.com> Co-authored-by: liusy58 <liusy58@linux.alibaba.com> | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 |