Critical UTF-8 Corruption Issue (Replacement Character �) — Exclusive to NIM Deployment Serving Inkling; Not Reproducible on Direct Inkling Endpoints

AI & Data Science Computer Vision & Image Processing
, ,
1

Dear NIM Engineering Team,
尊敬的 NIM 工程团队：

We are reporting a reproducible encoding corruption issue that only occurs when Inkling is served through NIM (NVIDIA Inference Microservice). It does not manifest on direct native Inkling endpoints, nor does it appear on the same NIM cluster running other models. This confirms the fault is isolated to the Inkling-specific inference pipeline layer within NIM.
我们报告一个可复现的编码损坏问题：仅在通过 NIM 部署运行 Inkling 时触发（NVIDIA Inference Microservice）。在直接原生 Inkling 端点上未出现，在同一 NIM 集群运行其他模型时也未出现。这确认故障被隔离至 NIM 内部针对 Inkling 的推理管道层。

1. Problem Description / 问题描述

When generating long-form Chinese text (~1,500 characters, high-density with sensitive characters such as 绷, 子, 弦, 紫, 粗, 饥, 串, 械) via Inkling served on NIM, the output randomly contains Unicode Replacement Characters (U+FFFD, displayed as �).
在通过 NIM 部署的 Inkling 生成约 1,500 字的长中文文本时（高密度包含敏感字如 绷、子、弦、紫、粗、饥、串、械），输出会随机出现 Unicode 替代字符（U+FFFD，显示为 �）。

Critical exclusivity check / 关键排他性验证：

  • Direct Inkling endpoint (bypass NIM): Identical prompt, temperature, max_tokens, and decoding config produce perfect UTF-8 with zero �. / 直接 Inkling 端点（绕过 NIM）： 相同提示、温度、max_tokens 与解码配置产生完全正常的 UTF-8，无任何 �。
  • Same NIM cluster, other LLMs: Equivalent long-form Chinese generation loads on other models deployed on the identical NIM instance do not trigger byte-level corruption. / 同一 NIM 集群，其他大模型： 在同一 NIM 实例上部署的其他模型，执行等效的中文长文生成负载，未触发字节级损坏。
  • Model scope: Corruption is strictly tied to the NIM + Inkling combination. / 模型范围： 损坏严格绑定于 NIM + Inkling 组合。

Confirmed corrupted instances in our test:
我们测试中确认的损坏实例（仅 NIM + Inkling 路径）：

  • � → � (context: “�得越来越紧”)
  • 粗 → � (context: “手指关节�大”)
  • 饥 → � (context: “诉说�饿”)
  • 弦/身 → � (context: “琴�”)
  • 穗/在 → � (context: “高粱�在他身后”)

2. Reproduction Conditions / 复现条件

  • Environment: NIM deployment serving Inkling only; not reproducible on direct Inkling API or on NIM with non-Inkling models. / 环境：仅 NIM 部署运行 Inkling 时；直接 Inkling API 或 NIM 运行非 Inkling 模型时无法复现。
  • Modes: Both streaming (SSE) and non-streaming trigger the bug via the NIM Inkling endpoint. / 模式： 通过 NIM Inkling 端点的流式（SSE）与非流式均触发。
  • Content: High-intensity Chinese prose mixing common characters, punctuation (full-width/half-width), numerals, and Latin letters. / 高强度中文散文，混合常用汉字、标点（全角/半角）、数字与拉丁字母。
  • Stability: The corruption is intermittent but reproducible under long-context generation; the same character (�) renders correctly earlier in the text but fails later within the same NIM Inkling session. / 乱码间歇性但可复现，长上下文中同一字符（如 绷）在同一 NIM Inkling 会话中前文正常、后文损坏。

3. Key Finding: Isolated to NIM + Inkling / 关键发现：问题孤立于 NIM + Inkling 组合

We have verified that this is not a core Inkling model weight or standard tokenizer issue. The identical prompt and decoding configuration work correctly on the direct native Inkling endpoint (outside NIM) and on other models within the same NIM cluster. The problem must lie in NIM-specific inference pipeline layers specific to the Inkling container/configuration (e.g., Inkling-specific detokenizer mapping, container locale, serialization path, or output filter integration).
我们已验证：这不是 Inkling 核心模型权重或标准分词器（Tokenizer）的问题。相同提示与解码配置在直接原生 Inkling 端点（NIM 之外）以及同一 NIM 集群内的其他模型上运行正常。问题必然位于 NIM 针对 Inkling 的特定推理服务管道层（如 Inkling 专属解码器映射、容器 locale、序列化路径或输出过滤集成）。

This is not a linguistic word-segmentation error (e.g., Jieba/THULAC). Modern LLMs use subword tokenization (BPE/SentencePiece); corruption occurs during Token ID → UTF-8 byte reconstruction or downstream serialization within the NIM Inkling wrapper.
这不是语言学分词错误（如 Jieba/THULAC 失效）。现代大模型使用子词分词（BPE/SentencePiece）；损坏发生在 Token ID → UTF-8 字节重建 或 NIM Inkling 封装内的下游序列化 阶段。

4. Technical Analysis / 技术分析

Probable root causes specific to the NIM deployment of Inkling:
可能根因（NIM 部署 Inkling 特定）：

  1. UTF-8 byte-boundary truncation in NIM serialization: Multi-byte Chinese characters (e.g., 绷 = E7 BB B7; 粗 = E7 B2 97) may be split across byte buffers in the NIM Inkling stream/JSON encoder, causing invalid continuation bytes that are replaced by EF BF BD (�). / NIM 序列化中的 UTF-8 字节边界截断：多字节汉字（如 绷 = E7 BB B7）可能在 NIM Inkling 的流式/JSON 编码缓冲区被切断，导致无效续字节被替换为 EF BF BD（�）。
  2. Inkling-specific detokenizer/tokenizer config mismatch in NIM container: The NIM-side decoder for Inkling may load a mismatched tokenizer.json / tokenizer_config.json or apply a different SentencePiece/BPE decode path than the native endpoint. Certain Token combinations valid in native Inkling may be reconstructed with partial byte sequences inside NIM. / NIM 容器内 Inkling 专属 Tokenizer/Detokenizer 配置不匹配：NIM 侧针对 Inkling 的解码器可能加载了不匹配的 tokenizer.json / tokenizer_config.json，或应用了与原生端点不同的 SentencePiece/BPE 解码路径。某些在原生 Inkling 中有效的 Token 组合，在 NIM 内可能被重建为部分字节序列。
  3. Output guardrail/filter byte-level replacement in NIM Inkling path: Content safety filters integrated specifically for the Inkling NIM deployment may misidentify valid UTF-8 byte patterns and inject replacement characters at the byte-stream level rather than rejecting the full request. / NIM Inkling �径中输出过滤层的字节级替换：专门为 Inkling NIM 部署集成的内容安全过滤，可能误判合法 UTF-8 字节模式，并在字节流层直接插入替代字符，而非拒绝整个请求。
  4. Container locale/encoding pollution (LC_ALL, PYTHONIOENCODING) in Inkling container: If the Inkling-specific NIM container defaults to C/POSIX without UTF-8, Python serialization may apply replace error handling to non-ASCII characters, emitting �. / Inkling 容器的 Locale/编码污染：若 NIM 中 Inkling 专属容器未正确设置 LC_ALL=C.UTF-8 或 PYTHONIOENCODING=utf-8，Python 序列化可能对非 ASCII 字符应用 replace 错误处理，输出 �。
  5. Shared serialization library defect exposed only by Inkling token patterns: Since both streaming and non-streaming fail, inspect the shared JSON encoder / gRPC serializer used by both paths—certain Inkling token-ID-to-byte mappings may trigger a boundary bug not present with other models’ token distributions. / 仅由 Inkling Token 模式暴露的共用序列化库缺陷：由于流式与非流式均失败，请检查两路径共用的 JSON 编码器 / gRPC 序列化器------某些 Inkling 特有的 Token-ID 到字节映射可能触发边界缺陷，而其他模型的 Token 分布不会触发。

5. Recommended Debugging Actions / 建议排查方向

To isolate the layer, please check specifically for the Inkling-on-NIM pipeline:
为隔离问题层，请针对 NIM 上的 Inkling 管道 检查：

  • Raw byte capture on NIM Inkling endpoint: Inspect the raw HTTP body / SSE stream bytes before client rendering from the NIM Inkling URL. Confirm whether the byte sequence is already EF BF BD (proving server-side injection in NIM) versus correct UTF-8 returned by the direct Inkling endpoint. / NIM Inkling 端点的原始字节抓取：在客户端渲染前检查来自 NIM Inkling URL 的 HTTP body / SSE 流原始字节，确认是否已为 EF BF BD（证明 NIM 服务端注入），对比直接 Inkling 端点返回的正确 UTF-8。
  • Token ID comparison (NIM Inkling vs. Native Inkling): Capture the raw Token ID sequence returned by NIM Inkling and decode it with the standard native Inkling tokenizer (tiktoken/sentencepiece) to identify where invalid byte reconstruction begins. / Token ID 对比（NIM Inkling vs. 原生 Inkling）：捕获 NIM Inkling 返回的原始 Token ID 序列，并用标准原生 Inkling 分词器（tiktoken/sentencepiece）解码，定位非法字节重建起点。
  • Cross-model control test on same NIM node: Run the identical Chinese long-form prompt on a different model served by the same NIM instance/container host. If corruption is absent, the defect is not in the host OS/JSON library but in the Inkling-specific container/config. / 同 NIM 节点的跨模型对照测试：在同一 NIM 实例/容器主机上，用相同中文长文提示测试不同模型。若损坏消失，则缺陷不在主机 OS/JSON 库，而在 Inkling 专属容器/配置。
  • Environment variables in Inkling container: Verify PYTHONIOENCODING=utf-8 and LC_ALL=C.UTF-8 specifically inside the Inkling NIM container, not just the host. / Inkling 容器内环境变量：专门验证 Inkling NIM 容器内（而非仅主机）的 PYTHONIOENCODING=utf-8 与 LC_ALL=C.UTF-8。
  • Filter isolation test: Temporarily disable output filtering/guardrails in the NIM Inkling config (if configurable) to determine if the filter layer is injecting �. / 过滤隔离测试：暂时关闭 NIM Inkling 配置中的输出过滤/防护（若可配置），判断是否由过滤层注入 �。
  • Streaming vs. non-streaming buffer analysis for Inkling: Since both modes fail on NIM Inkling, inspect the shared serialization library (JSON encoder / gRPC serializer / SSE formatter) used specifically by the Inkling service path. / Inkling 流式与非流式缓冲分析：由于两种模式在 NIM Inkling 上均失败，请检查专门用于 Inkling 服务路径的共用序列化库（JSON 编码器 / gRPC 序列化器 / SSE 格式化器）。

6. Conclusion / 结论

This is a NIM deployment-layer UTF-8 integrity bug exclusive to the Inkling service pipeline, not an inherent Inkling model generation failure, nor a generic linguistic tokenization error, nor a client-side rendering issue. The intermittent corruption of high-frequency common characters (绷, 粗, 饥) in long Chinese contexts—present only when served via NIM Inkling, and absent on direct Inkling endpoints and other NIM models—strongly points to byte-level encoding pipeline errors within the NIM inference service specific to Inkling’s container, tokenizer mapping, or output serialization layer.
这是一项 NIM 部署层、仅针对 Inkling 服务管道的 UTF-8 完整性缺陷，而非 Inkling 模型本身的生成缺陷，也非通用语言学分词错误，亦非客户端渲染问题。高频常用字（绷、粗、饥）在长中文上下文中的间歇损坏------仅在通过 NIM Inkling 服务时出现，而在直接 Inkling 端点及其他 NIM 模型上缺失------强烈指向 NIM 推理服务内部、针对 Inkling 容器、Tokenizer 映射或输出序列化层的字节级编码管道错误。

We request priority investigation into the NIM Inkling tokenizer/decoder configuration, container locale/encoding settings, and output serialization pipeline.
我们请求优先调查 NIM Inkling 的 Tokenizer/解码器配置、容器 locale/编码设置与输出序列化管道。

Thank you for your support.
感谢支持。

Best regards,
此致

Platform Note / 平台备注: Issue reproduced exclusively on NIM serving Inkling; direct Inkling endpoint and other NIM-hosted models verified clean. / 仅在 NIM 运行 Inkling 时复现；直接 Inkling 端点与其他 NIM 托管模型已验证无问题。