GLM-4.7 needs about 354 GB of GPU memory for FP8 weights. The smallest fitting setup at 32k context is 4× NVIDIA B200. It supports 46 concurrent conversations with SGLang defaults.
GPU requirements by context length
8k context
GPU
GPUs
Concurrent users
NVIDIA B200
4
64
NVIDIA H200 SXM
4
64
NVIDIA H100 SXM
8
64
NVIDIA H100 NVL
8
64
NVIDIA A100 80GB SXM
8
64
NVIDIA RTX PRO 6000 Blackwell Server Edition
8
64
NVIDIA H20
8
64
NVIDIA L40S
16 (verify server configuration)
64
32k context
GPU
GPUs
Concurrent users
NVIDIA B200
4
46
NVIDIA H200 SXM
4
18
NVIDIA H100 SXM
8
33
NVIDIA H100 NVL
8
50
NVIDIA A100 80GB SXM
8
27
NVIDIA RTX PRO 6000 Blackwell Server Edition
8
52
NVIDIA H20
8
52
NVIDIA L40S
16 (verify server configuration)
41
128k context
GPU
GPUs
Concurrent users
NVIDIA B200
4
11
NVIDIA H200 SXM
4
4
NVIDIA H100 SXM
8
8
NVIDIA H100 NVL
8
12
NVIDIA A100 80GB SXM
8
6
NVIDIA RTX PRO 6000 Blackwell Server Edition
8
13
NVIDIA H20
8
13
NVIDIA L40S
16 (verify server configuration)
10
FP8 weights, SGLang defaults, estimates.
Weights by precision
Weight memory
Precision
Weights
Smallest fitting setup at 32k
BF16
about 706 GB
8x NVIDIA B200
FP8
about 354 GB
4x NVIDIA B200
INT4 (AWQ)
about 195 GB
2x NVIDIA B200
Model notes
Attention
Grouped-query attention (GQA)
Architecture
Mixture-of-Experts
Layers
92
Hidden size
5,120
Vocabulary
151,552
Native precision
bfloat16
At 32k context, 44% of this card's usable memory is working as conversation cache.
Frequently asked questions
Will zai-org/GLM-4.7 run on a single H100?
No, a single H100 SXM cannot hold zai-org/GLM-4.7 at 32k context with FP8 weights. The model requires multiple GPUs.
What is the cheapest GPU setup for zai-org/GLM-4.7?
The cheapest fitting setup at 32k context is 4x NVIDIA B200 with FP8 weights and SGLang defaults.
How much VRAM does zai-org/GLM-4.7 need at 128k context?
At 128k context on the cheapest fitting setup, zai-org/GLM-4.7 uses about 89 GB of VRAM per GPU for FP8 weights and about 6.2 GB per conversation for the KV cache. The total VRAM needed depends on the GPU count and parallelism configuration.
Can I run zai-org/GLM-4.7 with INT4 quantization?
Yes, zai-org/GLM-4.7 has INT4 weight figures. Its weights take about 195 GB in INT4 versus about 354 GB in FP8. INT4 can make the model fit on fewer GPUs, but check inference quality for your workload.
Want a different context length, GPU, or quantization? The calculator runs the same numbers live, preloaded with this model.