memcache/example/python · Ascend/memcache - AtomGit

chenyz6feat: support meta setup config api

文件	最后提交记录	最后更新时间
README.md	feat: support meta setup config api Co-authored-by: shilinlee<836160610@qq.com> # message auto-generated for no-merge-commit merge: !191 merge setup_meta into develop feat: support meta setup config api Created-by: shilinlee_com Commit-by: shilinlee Merged-by: chenyz6 Description: # Pull Request Template ## Description feat: support meta setup config api ## Related Issues <!-- Replace with actual issue numbers. Example: Closes #123, Related to #456 --> ## Type of Change - [ ] Bug fix - [x] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing ```python from memcache_hybrid import MetaConfig, MetaService def main(): config = MetaConfig() config.meta_service_url = "tcp://127.0.0.1:5000" config.log_level = "error" print(config) meta_service = MetaService assert meta_service.setup(config) == 0, "Meta service setup failed" meta_service.main() if __name__ == "__main__": main() ``` ![image.png](https://raw.gitcode.com/user-images/assets/7672915/9bf16a31-53e6-4c3d-85fe-be642d4a22d7/image.png 'image.png') ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!191	1 个月前
interactive_app.py	format: reformat all source files with clang-format Co-authored-by: j00808874<jiangchanghong3@huawei.com> # message auto-generated for no-merge-commit merge: !70 merge mr into develop format: reformat all source files with clang-format Created-by: j00808874 Commit-by: j00808874 Merged-by: yrewzjsx Description: # Pull Request Template ## Description format: reformat all source files with clang-format Ensure that future MRs show only semantic diffs, not formatting noise ## Related Issues <!-- Replace with actual issue numbers. Example: Closes #123, Related to #456 --> ## Type of Change - [ ] Bug fix - [ ] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing <!-- Describe how the changes were tested. Include: - Unit/integration test coverage - Manual validation steps - Links to CI reports (e.g., CIDA) if available - Note any pending tests --> ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!70	4 个月前
test_mmc_batch.py	format: reformat all source files with clang-format Co-authored-by: j00808874<jiangchanghong3@huawei.com> # message auto-generated for no-merge-commit merge: !70 merge mr into develop format: reformat all source files with clang-format Created-by: j00808874 Commit-by: j00808874 Merged-by: yrewzjsx Description: # Pull Request Template ## Description format: reformat all source files with clang-format Ensure that future MRs show only semantic diffs, not formatting noise ## Related Issues <!-- Replace with actual issue numbers. Example: Closes #123, Related to #456 --> ## Type of Change - [ ] Bug fix - [ ] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing <!-- Describe how the changes were tested. Include: - Unit/integration test coverage - Manual validation steps - Links to CI reports (e.g., CIDA) if available - Note any pending tests --> ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!70	4 个月前
test_mmc_demo.py	format: reformat all source files with clang-format Co-authored-by: j00808874<jiangchanghong3@huawei.com> # message auto-generated for no-merge-commit merge: !70 merge mr into develop format: reformat all source files with clang-format Created-by: j00808874 Commit-by: j00808874 Merged-by: yrewzjsx Description: # Pull Request Template ## Description format: reformat all source files with clang-format Ensure that future MRs show only semantic diffs, not formatting noise ## Related Issues <!-- Replace with actual issue numbers. Example: Closes #123, Related to #456 --> ## Type of Change - [ ] Bug fix - [ ] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing <!-- Describe how the changes were tested. Include: - Unit/integration test coverage - Manual validation steps - Links to CI reports (e.g., CIDA) if available - Note any pending tests --> ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!70	4 个月前
test_mmc_layers.py	支持用户copy时自动检测传输方向 Co-authored-by: y30060514<yangjianhong2@h-partners.com> # message auto-generated for no-merge-commit merge: !87 merge develop into develop 支持用户copy时自动检测传输方向 Created-by: CYangJH Commit-by: y30060514 Merged-by: chenyz6 Description: # Pull Request Template ## Description 支持用户做copy时不用传入传输方向，在mf的代码内部自行判断 ## Related Issues N/A ## Type of Change - [ ] Bug fix - [x] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing python example/python/test_mmc_layers.py ## Checklist - [x] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [x] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!87	4 个月前
test_mmc_layers_batch.py	适配用户批量copy时自动监测方向 Co-authored-by: y30060514<yangjianhong2@h-partners.com> # message auto-generated for no-merge-commit merge: !126 merge develop into develop 适配用户批量copy时自动监测方向 Created-by: CYangJH Commit-by: y30060514 Merged-by: chenyz6 Description: # Pull Request Template ## Description 适配用户批量copy时自动监测方向 ## Related Issues N/A ## Type of Change - [ ] Bug fix - [x] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing <!-- Describe how the changes were tested. Include: - Unit/integration test coverage - Manual validation steps - Links to CI reports (e.g., CIDA) if available - Note any pending tests --> ## Checklist - [x] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [x] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!126	3 个月前
test_mmc_meta_service.py	format: reformat all source files with clang-format Co-authored-by: j00808874<jiangchanghong3@huawei.com> # message auto-generated for no-merge-commit merge: !70 merge mr into develop format: reformat all source files with clang-format Created-by: j00808874 Commit-by: j00808874 Merged-by: yrewzjsx Description: # Pull Request Template ## Description format: reformat all source files with clang-format Ensure that future MRs show only semantic diffs, not formatting noise ## Related Issues <!-- Replace with actual issue numbers. Example: Closes #123, Related to #456 --> ## Type of Change - [ ] Bug fix - [ ] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing <!-- Describe how the changes were tested. Include: - Unit/integration test coverage - Manual validation steps - Links to CI reports (e.g., CIDA) if available - Note any pending tests --> ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!70	4 个月前
test_mmc_start_meta_service_and_simple_test.py	feat: support meta setup config api Co-authored-by: shilinlee<836160610@qq.com> # message auto-generated for no-merge-commit merge: !191 merge setup_meta into develop feat: support meta setup config api Created-by: shilinlee_com Commit-by: shilinlee Merged-by: chenyz6 Description: # Pull Request Template ## Description feat: support meta setup config api ## Related Issues <!-- Replace with actual issue numbers. Example: Closes #123, Related to #456 --> ## Type of Change - [ ] Bug fix - [x] New feature - [ ] Performance optimization - [ ] Documentation update - [ ] Other (please describe):_______________ ## Testing ```python from memcache_hybrid import MetaConfig, MetaService def main(): config = MetaConfig() config.meta_service_url = "tcp://127.0.0.1:5000" config.log_level = "error" print(config) meta_service = MetaService assert meta_service.setup(config) == 0, "Meta service setup failed" meta_service.main() if __name__ == "__main__": main() ``` ![image.png](https://raw.gitcode.com/user-images/assets/7672915/9bf16a31-53e6-4c3d-85fe-be642d4a22d7/image.png 'image.png') ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. See merge request: Ascend/memcache!191	1 个月前

代码实现介绍

本样例简单验证了memcache_hybrid相关python接口

本样例需要在npu环境下编译运行

如果编译选择CANN依赖

首先,请在环境上提前安装NPU固件驱动和CANN包(环境安装参考链接)

HDK固件驱动需要使用25.0.RC1 及以上版本(社区版HDK下载链接)

安装完成后需要配置CANN环境变量 (参考安装Toolkit开发套件包的第三步配置环境变量)

运行样例前请先编译安装 memfabric_hybrid的run包，默认安装路径为/usr/local/,然后source安装路径下的set_env.sh

memfabric_hybrid参考安装命令

bash memfabric_hybrid-1.0.0_linux_aarch64.run # 修改为实际的安装包名
bash memcache_hybrid-1.0.0_linux_aarch64.run  # 修改为实际的安装包名
source /usr/local/memfabric_hybrid/set_env.sh
source /usr/local/memcache_hybrid/set_env.sh

启动元数据服务

export MMC_META_CONFIG_PATH=/usr/local/memcache_hybrid/latest/config/mmc-meta.conf;
python3
>>> from memcache_hybrid import MetaService
>>> MetaService.main()

也可以直接在 Python 中配置启动参数（无需 MMC_META_CONFIG_PATH）：

from memcache_hybrid import MetaConfig, MetaService

config = MetaConfig()
config.meta_service_url = "tcp://192.168.1.1:5000"
config.config_store_url = "tcp://192.168.1.2:6000"
config.metrics_url = "http://192.168.1.1:8000"
config.ha_enable = False
config.log_level = "info"

MetaService.setup(config)
MetaService.main()

配置大页

注：仅device rdma/host rdma等protocol需要设置

 cat /proc/meminfo
 echo 2048 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

执行脚本

选择脚本，直接执行，以test_mmc_start_meta_service_and_simple_test.py为例，会在一个进程里面启动meta服务和localService并且完成put，get等测试（不需要前面步骤单独的启动meta服务进程），如需修改参数，请调整MetaConfig、LocalConfig相关代码。

python python/test_mmc_start_meta_service_and_simple_test.py