feat(spdiag): 为 vLLM 调用路径添加 PerfPoint 打点与 MC_LOG 日志 - #42
Merged
zchuango merged 7 commits intoAug 31, 2026
Conversation
fix(tracing): 添加缺失的 span 结束调用
依据 vllm_spdiag_logging_plan.md，按 Q1b/Q2A/Q3b 方案为 Mooncake Store vLLM 调用链添加 SpDiag 性能打点与日志输出： - mooncake_perf_points.def: 追加 25 个新 PerfKey，覆盖 store_py 入口 层（STORE_PY_*）、RealClient 核心层（RC_*）、Client 服务层 （CLIENT_BATCH_QUERY）。 - store_py.cpp: 为 setup/register_buffer/batch_put_from_multi_buffers/ batch_get_into_multi_buffers/batchIsExist/remove_all/close 等 8 个 Python 绑定方法添加 PerfPoint（KEY_MODULE），remove_all 与 close 额外输出 MC_LOG 汇总行。 - real_client.cpp: 为 setup_real/batchIsExist/batch_put_from_multi_ buffers/batch_get_into_multi_buffers 等 9 个方法添加 PerfPoint； 下沉层 _internal 方法用 MODULE 级别，并按 Q1b 输出 MC_LOG 汇总行 + per-key 多行（包含 success/key/size/replica/endpoint 字段， 缺失字段不输出）。 - client_service.cpp: 在 L1117 BatchQuery 实际实现中添加 CLIENT_BATCH_QUERY PerfPoint（Q3b，仅实际实现，避免重复统计）。
SpDiag::PerfPoint v1.0.0 仅暴露 Start()/End()/Abandon() 三个方法， 没有 ElapsedMicros()。服务器构建报错： real_client.cpp:1218: error: 'class SpDiag::PerfPoint' has no member named 'ElapsedMicros' 修复方式：在 setup_real / batchIsExist / register_buffer 以及 store_py.cpp 的 remove_all / close 入口层用 std::chrono::steady_clock 手动测量 elapsed_us，与下沉层 batch_get_into_multi_buffers_internal 已有的 t0/t1/total_us 风格保持一致。
现有 stress_cluster_bench 只调用老接口 batch_get_into，无法触发 vllm 调用路径的新打点（batch_put_from_multi_buffers / batchIsExist / batch_get_into_multi_buffers）。 按方案文档第 6 章实现 store_connector_bench.cpp： - 4 个 scenario: write / is_exist / get / all - 模拟 vllm 多 layer KV cache 场景（每请求 num_layers 个 key + buffer） - 调用 RealClient::batch_put_from_multi_buffers / batchIsExist / batch_get_into_multi_buffers，触发 STORE_PY_* + RC_* + CLIENT_BATCH_QUERY SpDiag 打点 + MC_LOG 汇总日志 - 输出带宽 + 时延分位数（P50/P90/P99） - 同步追加 CMakeLists.txt 编译目标，链接库参考 stress_cluster_bench
malloc 返回的内存不保证页对齐和 NUMA 本地性，UB/RDMA driver 会拒绝注册为 DMA 内存，导致 register_buffer 失败： Failed to register segment ... : Success [0] UbTransport: cannot register LocalMemory 参考 stress_cluster_bench 的做法，改用 numa_alloc_local (mmap + page-aligned + NUMA-aware)，匹配 UB driver 的要求。 析构同步改用 numa_free。
batchIsExist 返回 1=存在(成功),0=不存在(失败) batch_get_into_multi_buffers 返回 >0=字节数(成功),<0=错误码(失败) 原代码统一用 v!=0 判断失败,导致 IS_EXIST 和 GET 全部误判为失败 修复:按 phase 分别判断 - IS_EXIST: v != 1 为失败 - GET: v <= 0 为失败 - WRITE: v != 0 为失败(保持不变)
zchuango
approved these changes
Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
改动概述
为 vLLM 调用 Mooncake Store 的路径添加 SpDiag 性能打点和 MC_LOG 日志输出，覆盖 Python 绑定层、RealClient 核心层、Client 服务层三层调用链。
改动内容
1. PerfKey 定义（mooncake-integration/store/mooncake_perf_points.def）
新增 25 个 PerfKey，分三层覆盖 vLLM 调用路径：
*_STORE_PY_*keyRC_*keyPUT_BATCH_FULLkey2. PerfPoint 打点实现
mooncake-integration/store/store_py.cppmooncake-store/src/real_client.cppmooncake-store/src/client_service.cppClient::BatchPut方法添加PUT_BATCH_FULL打点3. 设计决策（Q1b / Q2A / Q3b）
KEY_MODULE级别，下沉_internal方法使用MODULE级别，与现有代码风格保持一致4. Benchmark 工具
新增
mooncake-store/benchmarks/store_connector_bench.cpp，专测 vLLM 调用路径打点：numa_alloc_local分配内存（支持 RDMA 注册）5. MC_LOG 日志格式示例
汇总行 + per-key 行（缺失字段不输出）：[batch_put_from_multi_buffers] elapsed_us=1849 num_keys=8 total_bytes=8388608
success=1 key=layer.0.req_0 size=1048576
success=1 key=layer.1.req_0 size=1048576
...
[batch_is_exist] elapsed_us=484 num_keys=8
success=1 key=layer.0.req_0
success=1 key=layer.1.req_0
...
[batch_get_into_multi_buffers] elapsed_us=158237 num_keys=8
success=1 key=layer.0.req_0 size=1048576 replica=memory endpoint=141.61.84.149:14024
success=0 key=layer.2.req_0 size=1048576
...