w8a16

Here is 1 public repository matching this topic...

用于学习模型加载、CUDA 算子、KV Cache 与 token 生成的轻量 C++ 推理运行时

cmake cpp cuda nvidia transformer quantization kv-cache tensor-core llm-inference w8a16

Add a description, image, and links to the w8a16 topic page so that developers can more easily learn about it.

To associate your repository with the w8a16 topic, visit your repo's landing page and select "manage topics."