RUST_MODULE_PATH: /home/pigeon/dev/duck-llm/target/release/libduck_llm.so DPDK initialized successfully. Found 4 ports. Port 0 driver: net_ice Port 0 device: 0000:01:00.0 Port 0 bus info: vendor_id=8086, device_id=1593 Port 1 driver: net_ice Port 1 device: 0000:01:00.1 Port 1 bus info: vendor_id=8086, device_id=1593 Port 2 driver: net_ice Port 2 device: 0000:01:00.2 Port 2 bus info: vendor_id=8086, device_id=1593 Port 3 driver: net_ice Port 3 device: 0000:01:00.3 Port 3 bus info: vendor_id=8086, device_id=1593 Thread bound to lcore 4 Thread bound to lcore 6 Thread bound to lcore 8 Thread bound to lcore 2 Port 1 [host_ip 0a150201, bcast_ip 0a1502ff] initialized with MAC: 9C:69:B4:68:64:41 Port 2 [host_ip 0a150301, bcast_ip 0a1503ff] initialized with MAC: 9C:69:B4:68:64:42 Port 3 [host_ip 0a150401, bcast_ip 0a1504ff] initialized with MAC: 9C:69:B4:68:64:43 Port 0 [host_ip 0a150101, bcast_ip 0a1501ff] initialized with MAC: 9C:69:B4:68:64:40 [Worker port 1] [init] link: up=1 speed=25000Mbps duplex=full autoneg=1 [Worker port 1] [link-wait] link became up after 2ms [Worker port 1] [post-wait] link: up=1 speed=25000Mbps duplex=full autoneg=1 [Worker port 2] [init] link: up=0 speed=0Mbps duplex=half autoneg=0 [Worker port 3] [init] link: up=0 speed=0Mbps duplex=half autoneg=0 [Worker port 0] [init] link: up=0 speed=0Mbps duplex=half autoneg=0 [Worker port 2] [link-wait] link became up after 460ms [Worker port 3] [link-wait] link became up after 460ms [Worker port 2] [post-wait] link: up=1 speed=25000Mbps duplex=full autoneg=1 [Worker port 3] [post-wait] link: up=1 speed=25000Mbps duplex=full autoneg=1 [Worker port 0] [link-wait] link became up after 458ms [Worker port 0] [post-wait] link: up=1 speed=25000Mbps duplex=full autoneg=1 Initializing DeepseekV32OfficialFp8EagerForCausalLM with config: DeepseekV32Config( attention_bias: False attention_dropout: 0.0 bos_token_id: 0 eos_token_id: 1 ep_size: 1 first_k_dense_replace: 3 hidden_act: silu hidden_size: 7168 index_head_dim: 128 index_n_heads: 64 index_topk: 2048 initializer_range: 0.02 intermediate_size: 18432 kv_lora_rank: 512 max_position_embeddings: 163840 model_path: /home/pigeon/models/DeepSeek-V3.2 model_type: deepseek_v32 moe_intermediate_size: 2048 n_group: 8 n_routed_experts: 256 n_shared_experts: 1 norm_topk_prob: True num_attention_heads: 128 num_experts_per_tok: 8 num_hidden_layers: 61 num_key_value_heads: 128 num_nextn_predict_layers: 1 pad_token_id: None q_lora_rank: 1536 qk_nope_head_dim: 128 qk_rope_head_dim: 64 quantization_config: {'activation_scheme': 'dynamic', 'fmt': 'e4m3', 'quant_method': 'fp8', 'scale_fmt': 'ue8m0', 'weight_block_size': [128, 128]} rms_norm_eps: 1e-06 rope_scaling: {'beta_fast': 32, 'beta_slow': 1, 'factor': 40, 'mscale': 1.0, 'mscale_all_dim': 1.0, 'original_max_position_embeddings': 4096, 'type': 'yarn'} rope_theta: 10000 routed_scaling_factor: 2.5 scoring_func: sigmoid tie_word_embeddings: False topk_group: 4 torch_dtype: torch.bfloat16 use_cache: True v_head_dim: 128 vocab_size: 129280 dequantize_fp8: False use_duck: True mock_duck: False async_duck_init: True duck_tp_size: 32 duck_weight_cache: True duck_weight_cache_force_invalidate: False fp9_gemv_multiple_of: 64 flattened_decode: True v32_attention_impl: official_fp8 v32_full_attn_max_seq_len: 2048 v32_static_graph_max_model_len: 2048 v32_fp8_kv_cache_block_size: 128 v32_fp8_kv_cache_round_scale_pow2: True v32_triton_fp8_kv_cache_append: False v32_triton_fp8_decode_context: False v32_mtp1_companion: False v32_mtp1_online: True v32_mtp1_track_accuracy: True v32_mtp1_print_each_step: True v32_first_k_dense_cuda: True device: cuda qk_head_dim: 192 ) duck-llm global options: DUCK_TRITON_FP8_LINEAR=1 (default=0): Enable Triton-backed CUDA quantized Linear for eligible bf16@fp8 paths. DUCK_CUDA_MEMORY_TRACE=0 (default=0): Enable CUDA memory trace JSONL logging. DUCK_CUDA_MEMORY_SNAPSHOT_ON_OOM=0 (default=0): Dump CUDA allocator snapshot on OOM. cuda_mem_stats label=DeepseekV32OfficialFp8EagerForCausalLM.load_weights.before device=cuda allocated_mib=329.0 reserved_mib=382.0 active_mib=329.0 inactive_split_mib=13.0 free_mib=29385.6 total_mib=32111.5 max_allocated_mib=366.2 max_reserved_mib=382.0 rust_mlp.init_cached layer=0 cache_slot=0: py_prepack=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.040ms, total=0.045ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=576 Qwen3MLPDuck load_weights layer=0: weights_load=0.000ms, rust_mlp_init=0.058ms, cache_hit=True, total=0.058ms rust_mlp.init_cached layer=1 cache_slot=1: py_prepack=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.039ms, total=0.044ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=576 Qwen3MLPDuck load_weights layer=1: weights_load=0.000ms, rust_mlp_init=0.057ms, cache_hit=True, total=0.058ms rust_mlp.init_cached layer=2 cache_slot=2: py_prepack=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.042ms, total=0.046ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=576 Qwen3MLPDuck load_weights layer=2: weights_load=0.000ms, rust_mlp_init=0.066ms, cache_hit=True, total=0.066ms rust_moe.moe_init_cached layer=3 cache_slot=3: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.028ms, total=0.031ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=3: weights_load=0.000ms, rust_moe_init=0.044ms, cache_hit=True, total=0.044ms rust_moe.moe_init_cached layer=4 cache_slot=4: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.034ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=4: weights_load=0.000ms, rust_moe_init=0.046ms, cache_hit=True, total=0.046ms rust_moe.moe_init_cached layer=5 cache_slot=5: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.028ms, total=0.031ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=5: weights_load=0.000ms, rust_moe_init=0.042ms, cache_hit=True, total=0.042ms rust_moe.moe_init_cached layer=6 cache_slot=6: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.032ms, total=0.035ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=6: weights_load=0.000ms, rust_moe_init=0.045ms, cache_hit=True, total=0.045ms rust_moe.moe_init_cached layer=7 cache_slot=7: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.028ms, total=0.030ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=7: weights_load=0.000ms, rust_moe_init=0.042ms, cache_hit=True, total=0.042ms rust_moe.moe_init_cached layer=8 cache_slot=8: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=8: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.044ms rust_moe.moe_init_cached layer=9 cache_slot=9: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.034ms, total=0.036ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=9: weights_load=0.000ms, rust_moe_init=0.047ms, cache_hit=True, total=0.047ms rust_moe.moe_init_cached layer=10 cache_slot=10: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.034ms, total=0.036ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=10: weights_load=0.000ms, rust_moe_init=0.046ms, cache_hit=True, total=0.047ms rust_moe.moe_init_cached layer=11 cache_slot=11: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.033ms, total=0.036ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=11: weights_load=0.000ms, rust_moe_init=0.048ms, cache_hit=True, total=0.048ms rust_moe.moe_init_cached layer=12 cache_slot=12: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.033ms, total=0.036ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=12: weights_load=0.000ms, rust_moe_init=0.050ms, cache_hit=True, total=0.050ms rust_moe.moe_init_cached layer=13 cache_slot=13: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=13: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=14 cache_slot=14: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=14: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=15 cache_slot=15: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.036ms, total=0.039ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=15: weights_load=0.000ms, rust_moe_init=0.050ms, cache_hit=True, total=0.051ms rust_moe.moe_init_cached layer=16 cache_slot=16: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.035ms, total=0.037ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=16: weights_load=0.000ms, rust_moe_init=0.049ms, cache_hit=True, total=0.049ms rust_moe.moe_init_cached layer=17 cache_slot=17: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=17: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=18 cache_slot=18: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.028ms, total=0.030ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=18: weights_load=0.000ms, rust_moe_init=0.041ms, cache_hit=True, total=0.041ms rust_moe.moe_init_cached layer=19 cache_slot=19: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.033ms, total=0.035ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=19: weights_load=0.000ms, rust_moe_init=0.056ms, cache_hit=True, total=0.056ms rust_moe.moe_init_cached layer=20 cache_slot=20: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.034ms, total=0.036ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=20: weights_load=0.000ms, rust_moe_init=0.048ms, cache_hit=True, total=0.048ms rust_moe.moe_init_cached layer=21 cache_slot=21: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.037ms, total=0.040ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=21: weights_load=0.000ms, rust_moe_init=0.053ms, cache_hit=True, total=0.053ms rust_moe.moe_init_cached layer=22 cache_slot=22: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.033ms, total=0.036ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=22: weights_load=0.001ms, rust_moe_init=0.049ms, cache_hit=True, total=0.050ms rust_moe.moe_init_cached layer=23 cache_slot=23: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.037ms, total=0.040ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=23: weights_load=0.000ms, rust_moe_init=0.052ms, cache_hit=True, total=0.053ms rust_moe.moe_init_cached layer=24 cache_slot=24: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.032ms, total=0.035ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=24: weights_load=0.000ms, rust_moe_init=0.046ms, cache_hit=True, total=0.046ms rust_moe.moe_init_cached layer=25 cache_slot=25: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=25: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=26 cache_slot=26: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.034ms, total=0.037ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=26: weights_load=0.000ms, rust_moe_init=0.048ms, cache_hit=True, total=0.049ms rust_moe.moe_init_cached layer=27 cache_slot=27: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.028ms, total=0.031ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=27: weights_load=0.000ms, rust_moe_init=0.041ms, cache_hit=True, total=0.041ms rust_moe.moe_init_cached layer=28 cache_slot=28: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.033ms, total=0.037ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=28: weights_load=0.000ms, rust_moe_init=0.048ms, cache_hit=True, total=0.049ms rust_moe.moe_init_cached layer=29 cache_slot=29: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.039ms, total=0.042ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=29: weights_load=0.000ms, rust_moe_init=0.054ms, cache_hit=True, total=0.054ms rust_moe.moe_init_cached layer=30 cache_slot=30: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=30: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=31 cache_slot=31: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.032ms, total=0.034ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=31: weights_load=0.000ms, rust_moe_init=0.045ms, cache_hit=True, total=0.046ms rust_moe.moe_init_cached layer=32 cache_slot=32: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.028ms, total=0.031ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=32: weights_load=0.000ms, rust_moe_init=0.041ms, cache_hit=True, total=0.042ms rust_moe.moe_init_cached layer=33 cache_slot=33: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.037ms, total=0.040ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=33: weights_load=0.000ms, rust_moe_init=0.051ms, cache_hit=True, total=0.051ms rust_moe.moe_init_cached layer=34 cache_slot=34: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.033ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=34: weights_load=0.000ms, rust_moe_init=0.045ms, cache_hit=True, total=0.045ms rust_moe.moe_init_cached layer=35 cache_slot=35: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.034ms, total=0.037ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=35: weights_load=0.000ms, rust_moe_init=0.048ms, cache_hit=True, total=0.049ms rust_moe.moe_init_cached layer=36 cache_slot=36: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.031ms, total=0.034ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=36: weights_load=0.000ms, rust_moe_init=0.047ms, cache_hit=True, total=0.047ms rust_moe.moe_init_cached layer=37 cache_slot=37: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.032ms, total=0.035ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=37: weights_load=0.000ms, rust_moe_init=0.047ms, cache_hit=True, total=0.048ms rust_moe.moe_init_cached layer=38 cache_slot=38: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.033ms, total=0.036ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=38: weights_load=0.000ms, rust_moe_init=0.049ms, cache_hit=True, total=0.049ms rust_moe.moe_init_cached layer=39 cache_slot=39: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.038ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=39: weights_load=0.000ms, rust_moe_init=0.050ms, cache_hit=True, total=0.050ms rust_moe.moe_init_cached layer=40 cache_slot=40: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.035ms, total=0.038ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=40: weights_load=0.000ms, rust_moe_init=0.049ms, cache_hit=True, total=0.049ms rust_moe.moe_init_cached layer=41 cache_slot=41: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=41: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=42 cache_slot=42: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.031ms, total=0.033ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=42: weights_load=0.000ms, rust_moe_init=0.044ms, cache_hit=True, total=0.045ms rust_moe.moe_init_cached layer=43 cache_slot=43: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.033ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=43: weights_load=0.000ms, rust_moe_init=0.045ms, cache_hit=True, total=0.045ms rust_moe.moe_init_cached layer=44 cache_slot=44: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.042ms, total=0.045ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=44: weights_load=0.000ms, rust_moe_init=0.056ms, cache_hit=True, total=0.056ms rust_moe.moe_init_cached layer=45 cache_slot=45: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=45: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=46 cache_slot=46: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.028ms, total=0.031ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=46: weights_load=0.000ms, rust_moe_init=0.040ms, cache_hit=True, total=0.041ms rust_moe.moe_init_cached layer=47 cache_slot=47: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.033ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=47: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.044ms rust_moe.moe_init_cached layer=48 cache_slot=48: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.033ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=48: weights_load=0.000ms, rust_moe_init=0.045ms, cache_hit=True, total=0.045ms rust_moe.moe_init_cached layer=49 cache_slot=49: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.034ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=49: weights_load=0.000ms, rust_moe_init=0.046ms, cache_hit=True, total=0.046ms rust_moe.moe_init_cached layer=50 cache_slot=50: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.031ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=50: weights_load=0.000ms, rust_moe_init=0.042ms, cache_hit=True, total=0.042ms rust_moe.moe_init_cached layer=51 cache_slot=51: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.033ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=51: weights_load=0.000ms, rust_moe_init=0.044ms, cache_hit=True, total=0.045ms rust_moe.moe_init_cached layer=52 cache_slot=52: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=52: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=53 cache_slot=53: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.030ms, total=0.033ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=53: weights_load=0.000ms, rust_moe_init=0.049ms, cache_hit=True, total=0.049ms rust_moe.moe_init_cached layer=54 cache_slot=54: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=54: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=55 cache_slot=55: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=55: weights_load=0.000ms, rust_moe_init=0.042ms, cache_hit=True, total=0.042ms rust_moe.moe_init_cached layer=56 cache_slot=56: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.032ms, total=0.038ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=56: weights_load=0.000ms, rust_moe_init=0.049ms, cache_hit=True, total=0.050ms rust_moe.moe_init_cached layer=57 cache_slot=57: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.029ms, total=0.032ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=57: weights_load=0.000ms, rust_moe_init=0.043ms, cache_hit=True, total=0.043ms rust_moe.moe_init_cached layer=58 cache_slot=58: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.027ms, total=0.029ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=58: weights_load=0.000ms, rust_moe_init=0.040ms, cache_hit=True, total=0.041ms rust_moe.moe_init_cached layer=59 cache_slot=59: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.032ms, total=0.035ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=59: weights_load=0.000ms, rust_moe_init=0.045ms, cache_hit=True, total=0.045ms rust_moe.moe_init_cached layer=60 cache_slot=60: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.032ms, total=0.035ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=60: weights_load=0.000ms, rust_moe_init=0.045ms, cache_hit=True, total=0.046ms rust_moe.moe_init_cached layer=61 cache_slot=61: py_prepack=0.000ms, py_concat=0.000ms, rust_wait_prev=0.000ms, rust_plan=0.038ms, total=0.041ms, fp9_gemv_multiple_of=64, hidden_size=7168, local_intermediate_size_padded=64, num_experts=256 DeepseekV32ExpertsDuck load_weights layer=61: weights_load=0.000ms, rust_moe_init=0.054ms, cache_hit=True, total=0.055ms === Linear Load Stats: DeepseekV32OfficialFp8EagerForCausalLM === totals: linear_count=498 source_total_mib=16642.2 (weight=16638.6, scale=3.6) resident_total_mib=17634.6 (weight=17630.6, scale=3.3, aux=0.6) device count src_total res_total res_w res_scale res_aux --------------------------------------------------------------------------------- cuda:0 498 16642.2 17634.6 17630.6 3.3 0.6 by device + category: device category count src_total res_total res_w res_scale res_aux ------------------------------------------------------------------------------------------------------------------------- cuda:0 fp8_scaled_kept_quantized_triton 434 13686.5 13687.1 13683.1 3.3 0.6 cuda:0 fp8_scaled_dequantized_no_triton_to_bf16 62 992.2 1984.0 1984.0 0.0 0.0 cuda:0 source_bf16_loaded_as_bf16 2 1963.5 1963.5 1963.5 0.0 0.0 samples: cuda:0 / fp8_scaled_kept_quantized_triton: - model.layers.0.mlp.down_proj shape=7168x18432 src=fp8_e4m3fn loaded=fp8_e4m3fn reason=official_fp8_kept_quantized device=cuda:0 resident_mib=126.0 - model.layers.0.mlp.gate_proj shape=18432x7168 src=fp8_e4m3fn loaded=fp8_e4m3fn reason=official_fp8_kept_quantized device=cuda:0 resident_mib=126.0 - model.layers.0.mlp.up_proj shape=18432x7168 src=fp8_e4m3fn loaded=fp8_e4m3fn reason=official_fp8_kept_quantized device=cuda:0 resident_mib=126.0 cuda:0 / fp8_scaled_dequantized_no_triton_to_bf16: - model.layers.0.self_attn.kv_b_proj shape=32768x512 src=fp8_e4m3fn loaded=bf16 reason=official_fp8_decode_predequantized_bf16 device=cuda:0 resident_mib=32.0 - model.layers.1.self_attn.kv_b_proj shape=32768x512 src=fp8_e4m3fn loaded=bf16 reason=official_fp8_decode_predequantized_bf16 device=cuda:0 resident_mib=32.0 - model.layers.2.self_attn.kv_b_proj shape=32768x512 src=fp8_e4m3fn loaded=bf16 reason=official_fp8_decode_predequantized_bf16 device=cuda:0 resident_mib=32.0 cuda:0 / source_bf16_loaded_as_bf16: - lm_head shape=129280x7168 src=bf16 loaded=bf16 reason=official_lm_head_bf16_storage_fp32_out device=cuda:0 resident_mib=1767.5 - model.layers.61.eh_proj shape=7168x14336 src=bf16 loaded=bf16 reason=official_dense_bf16 device=cuda:0 resident_mib=196.0 cuda_mem_stats label=DeepseekV32OfficialFp8EagerForCausalLM.load_weights.after device=cuda allocated_mib=20159.0 reserved_mib=20504.0 active_mib=20159.0 inactive_split_mib=345.0 free_mib=9263.6 total_mib=32111.5 max_allocated_mib=20173.0 max_reserved_mib=20504.0 DeepseekV32OfficialFp8EagerForCausalLM static graph prebuild: max_model_len=2048 DeepseekV32OfficialFp8Eager prepare_generation_cache: required_total_length=2048 per_layer_cache_mib=1.3 total_cache_mib=78.2 cuda_mem_stats label=DeepseekV32OfficialFp8Eager.prepare_generation_cache device=cuda allocated_mib=20237.2 reserved_mib=20582.0 active_mib=20237.2 inactive_split_mib=344.8 free_mib=9185.6 total_mib=32111.5 max_allocated_mib=20237.2 max_reserved_mib=20582.0 === Duck Load Stats: DeepseekV32OfficialFp8EagerForCausalLM === totals: module_count=62 source_total_mib=635657.2 theory_per_rank_mib=19864.3 rust_buf_per_rank_mib=22351.6 rust_buf_all_ranks_mib=715250.6 weights_buffer_used_mib=22351.6 group count src_total theory/rk rust/rk rust/all ----------------------------------------------------------------------------------- duck_moe_cache_hit 59 634522.9 19828.8 22311.7 713973.8 duck_mlp_cache_hit 3 1134.3 35.4 39.9 1276.9 samples: duck_moe_cache_hit: - model.layers.3.mlp.experts layer=3 cache_slot=3 tp=32 source_total_mib=10754.6 source_per_rank_mib=336.1 rust_buf_per_rank_mib=378.2 - model.layers.4.mlp.experts layer=4 cache_slot=4 tp=32 source_total_mib=10754.6 source_per_rank_mib=336.1 rust_buf_per_rank_mib=378.2 - model.layers.5.mlp.experts layer=5 cache_slot=5 tp=32 source_total_mib=10754.6 source_per_rank_mib=336.1 rust_buf_per_rank_mib=378.2 duck_mlp_cache_hit: - model.layers.0.mlp layer=0 cache_slot=0 tp=32 source_total_mib=378.1 source_per_rank_mib=11.8 rust_buf_per_rank_mib=13.3 - model.layers.1.mlp layer=1 cache_slot=1 tp=32 source_total_mib=378.1 source_per_rank_mib=11.8 rust_buf_per_rank_mib=13.3 - model.layers.2.mlp layer=2 cache_slot=2 tp=32 source_total_mib=378.1 source_per_rank_mib=11.8 rust_buf_per_rank_mib=13.3 DeepseekV32OfficialFp8EagerForCausalLM wait_pending_init=0.000ms DeepseekV32OfficialFp8EagerForCausalLM initialized 请输入您的问题（输入exit退出）：<｜begin▁of▁sentence｜><｜User｜>104857601是质数吗<｜Assistant｜> {'input_ids': tensor([[ 0, 128803, 7163, 27521, 19698, 389, 1703, 996, 3467, 128804, 128798]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])} tensor([[ 0, 128803, 7163, 27521, 19698, 389, 1703, 996, 3467, 128804, 128798]]) DeepseekV32OfficialFp8Eager prepare_generation_cache: required_total_length=1035 per_layer_cache_mib=1.3 total_cache_mib=78.2 cuda_mem_stats label=DeepseekV32OfficialFp8Eager.prepare_generation_cache device=cuda allocated_mib=20501.5 reserved_mib=21354.0 active_mib=20501.5 inactive_split_mib=850.5 free_mib=8303.6 total_mib=32111.5 max_allocated_mib=20501.7 max_reserved_mib=21354.0 我们被问到 "104857601是质数吗？" 我们需要检查104857601是否是质数。 首先，注意到104857601这个数字可能具有某种特殊形式。1048576是2^20（因为2^10=1024，2^20=1024^2=1048576）。所以104857601 = 1048576 * 100 + 1？实际上，104857600 = 1048576 * 100，所以104857601 = 104857600 + 1 = 100 * 2^20 + 1。但更直接地：104857601 = 2^20 * 100 + 1？不，2^20 = 1048576，所以104857601 = 104857600 + 1 = 100*1048576 + 1 = 100 * 2^20 + 1。但也许它也可以写成其他形式，比如Fermat数的变体？Fermat数是2^(2^n)+1。这里指数是20，不是2的幂。 我们可以尝试检查小质数因子。首先，我们可以计算104857601 mod一些小质数。 让我们计算104857601除以2：它是奇数，所以不能被2整除。 除以3：数字和：1+0+4+8+5+7+6+0+1 = 32，32 mod 3 = 2，所以不能被3整除。 除以5：最后一位是1，所以不能被5整除。 除以7：104857601 ÷ 7？计算104857601 mod 7。我们可以逐步计算：104857601 = 1048576*100 + 1，但也许直接模运算更好。或者我们可以使用计算器，但既然这是文本，我们手动计算。 计算104857601 mod 7： 一种方法：10 mod 7 = 3，所以我们可以迭代数字。但也许更简单：104857601 = 1048576 * 100 + 1。先计算1048576 mod 7，然后乘以100 mod 7，加1。 计算1048576 mod 7：注意到2^20 = 1048576。计算2^n mod 7的周期：2^1=2 mod 7=2，2^2=4 mod 7=4，2^3=8 mod 7=1，2^4=16 mod 7=2，所以周期为3？实际上，2^3 ≡ 1 (mod 7)，所以2^(3k) ≡ 1 mod 7。20除以3余2，所以2^20 ≡ 2^2 = 4 mod 7。所以1048576 ≡ 4 mod 7。 那么100 mod 7 = 2（因为98是7的倍数，100-98=2）。所以1048576*100 ≡ 4*2 = 8 ≡ 1 mod 7。然后加1：1+1=2 mod 7。所以104857601 ≡ 2 mod 7，不为0。所以不能被7整除。 除以11：计算104857601 mod 11。使用交替和：从右边开始，数字：1,0,6,7,5,8,4,0,1（注意数字顺序：104857601，数字位：1（个位），0（十位），6（百位），7（千位），5（万位），8（十万位），4（百万位），0（千万位），1（亿位），0（十亿位）？等等，数字是104857601，共9位？实际上是9位：1 0 4 8 5 7 6 0 1。从右边开始：个位：1，十位：0，百位：6，千位：7，万位：5，十万位：8，百万位：4，千万位：0，亿位：1。所以交替和（从右边开始奇数位减偶数位）：奇数位：个位1，百位6，万位5，百万位4，亿位1 → 和 = 1+6+5+4+1 = 17。偶数位：十位0，千位7，十万位8，千万位0 → 和 = 0+7+8+0 = 15。差：17-15=2。2 mod 11不为0，所以不能被11整除。 除以13：计算104857601 mod 13。我们可以使用类似的方法。也许注意到104857601 = 2^20 * 100 + 1。计算2^20 mod 13。2的模13的阶：2^1=2，2^2=4，2^3=8，2^4=16 mod 13=3，2^5=6，2^6=12，2^7 Generation stats: time 67.714s, input_len 11, output_len 1024, ttft 0.857s, tpot 0.065s, prefill 12.830 tok/s, decode 15.301 tok/s 请输入您的问题（输入exit退出）：