请使用下面的模版提问（创建话题后勾选相应的选项）：
Jetson 模组
Jetson AGX Orin
Jetson Orin NX
Jetson Orin Nano
Jetson AGX Xavier
Jetson Xavier NX
Jetson TX 系列
Jetson Nano
Jetson AGX Thor
Jetson 软件
JetPack 5.1.3
JetPack 5.1.4
JetPack 6.0
JetPack 6.1
JetPack 6.2
JetPack 7.1
DeepStream SDK
NVIDIA Isaac
SDK Manager 管理工具版本
2.3.0
2.2.0
2.1.0
其他
问题描述
我在Jetson Thor(Jetpack 7.1)上使用TensorRT Edge LLM编译qwen3.5 35BA3B模型，在编译选项{max_input_len, max_kv_capacity}为{16k,16k}时可以正常编译出engine并完成运行，但是当我切换为{32k,32k}时，编译过程中报错[ERROR] [TensorRT] Unexpected exception stoi，此时虽然有engine产物落盘，但无法运行，运行时仍报错[ERROR] [TensorRT] Unexpected exception stoi
错误日志
[14:09:40.457] [INFO] [llmBuilder.cpp:282:build] Using TRT_VERSION=10.13.3
[14:09:40.458] [INFO] [llmBuilder.cpp:286:build] Using __LUNOWUD=-peep:match_dual_gemm=off -peep:fc_h_fusion=off
[14:09:40.458] [INFO] [trtUtils.h:71:loadEdgellmPluginLib] EDGELLM_PLUGIN_PATH: /home/robot/Documents/jhzhang54/trt101-35b/TensorRT-Edge-LLM-35b/build/libNvInfer_edgellm_plugin.so
[14:09:40.464] [INFO] [llmBuilder.cpp:643:parseConfig] Heterogeneous head sizes from kv_layer_configs: 10 layers
[14:09:40.616] [INFO] [TensorRT] [MemUsageChange] Init CUDA: CPU +22, GPU +0, now: CPU 30, GPU 82258 (MiB)
[14:09:41.908] [INFO] [TensorRT] [MemUsageChange] Init builder kernel library: CPU +1227, GPU +1226, now: CPU 1459, GPU 83696 (MiB)
[14:09:41.908] [INFO] [llmBuilder.cpp:338:build] Parsing ONNX model: /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/onnx_mtp_trt101_fix_v_1537744/llm/model.onnx
[14:09:41.987] [INFO] [TensorRT] ----------------------------------------------------------------
[14:09:41.987] [INFO] [TensorRT] Input filename: /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/onnx_mtp_trt101_fix_v_1537744/llm/model.onnx
[14:09:41.987] [INFO] [TensorRT] ONNX IR version: 0.0.10
[14:09:41.987] [INFO] [TensorRT] Opset version: 24
[14:09:41.987] [INFO] [TensorRT] Producer name: pytorch
[14:09:41.987] [INFO] [TensorRT] Producer version: 2.11.0+cu130
[14:09:41.987] [INFO] [TensorRT] Domain:
[14:09:41.987] [INFO] [TensorRT] Model version: 0
[14:09:41.987] [INFO] [TensorRT] Doc string:
[14:09:41.987] [INFO] [TensorRT] ----------------------------------------------------------------
[14:09:41.987] [WARNING] [TensorRT] ModelImporter.cpp:653: Make sure input last_token_ids has Int64 binding.
[14:09:43.170] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.170] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.170] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.170] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.170] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.171] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.171] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.172] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.172] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.172] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.172] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.172] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.172] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.172] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.172] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.172] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.172] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.173] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.173] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.173] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.173] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.173] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.173] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.173] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.174] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.174] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.174] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.174] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.174] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.174] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.174] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.174] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.174] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.174] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.175] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.175] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.175] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.175] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.175] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.175] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.175] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.175] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.176] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.176] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.176] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.176] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.176] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.176] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.177] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.177] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.341] [INFO] [pluginJitCompiler.cpp:138:compilePluginJitKernel] Compiled XQA kernel for SM101, dtype=1, kv_dtype=1, head_dim=256, q_heads_per_kv=8, tokens_per_page=128, sliding_window=0, spec_decode=0 with NVRTC (132736 bytes, 163 ms)
[14:09:43.476] [INFO] [pluginJitCompiler.cpp:138:compilePluginJitKernel] Compiled XQA kernel for SM101, dtype=1, kv_dtype=1, head_dim=256, q_heads_per_kv=8, tokens_per_page=128, sliding_window=0, spec_decode=1 with NVRTC (175496 bytes, 133 ms)
[14:09:43.477] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.478] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.478] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.478] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.478] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.479] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.479] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.479] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.479] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.479] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.479] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.479] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.479] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.479] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.480] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.480] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.480] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.480] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.480] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.480] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.480] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.481] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.481] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.481] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.481] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.481] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.481] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.481] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.481] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.481] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.481] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.482] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.482] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.482] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.482] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.482] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.482] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.482] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.482] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.483] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.483] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.483] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.483] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.483] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.483] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.484] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.484] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.484] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.484] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.484] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.484] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.484] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.484] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.484] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.484] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.485] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.485] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.485] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.485] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.485] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.485] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.485] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.485] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.485] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.486] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.486] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.486] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.486] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.486] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.486] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.486] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.486] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.487] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.487] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.487] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.487] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.487] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.487] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.487] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.487] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.488] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.488] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.488] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.488] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.488] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.488] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.488] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.488] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.488] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.488] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.488] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.488] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.489] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.489] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.489] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.489] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.489] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.489] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.489] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.489] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.490] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.490] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.490] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.490] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.490] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.490] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.491] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.491] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.491] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.491] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.491] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.491] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.491] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.491] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.491] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.491] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.491] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.491] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.492] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.492] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.492] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.492] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.492] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.492] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.492] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.492] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.493] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.493] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.493] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.493] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.493] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.493] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.493] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.493] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.493] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.493] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.494] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.494] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.494] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.494] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.494] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.494] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.494] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.494] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.494] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.494] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.494] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.495] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.495] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.495] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.495] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.495] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.495] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.496] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.496] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.496] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.496] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.496] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.496] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.496] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.496] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.497] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.497] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.497] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.497] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.497] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.497] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.497] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.497] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.498] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.498] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.498] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.498] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.498] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.498] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.498] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.498] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.498] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.498] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.498] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.498] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.499] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.499] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.499] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.499] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.499] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.499] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.499] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.499] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.499] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.499] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.499] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.500] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.500] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.500] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.500] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.500] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.501] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.501] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.501] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.501] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.501] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.502] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.502] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.502] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.502] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.502] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.502] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.502] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.502] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.502] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.503] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.503] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.503] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.504] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.504] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.504] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.504] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.504] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.504] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.504] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.504] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.504] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.504] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.504] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.505] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.505] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.505] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.505] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.505] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.505] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.505] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.505] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.505] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.505] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.505] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.506] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.506] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.506] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.506] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.506] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.506] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.506] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.506] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.507] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.507] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.507] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.507] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.507] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.507] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.507] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.507] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.507] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.507] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.507] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.508] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.508] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.508] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.508] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.508] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.508] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.509] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.509] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.509] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.509] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.509] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.509] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.509] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.509] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.509] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.509] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.509] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.510] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.510] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.510] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.510] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.510] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.510] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.510] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.510] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.510] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.510] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.511] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.511] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.511] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.511] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.511] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.511] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.511] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.512] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.512] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.512] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.512] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.512] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.512] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.512] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.512] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.512] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.512] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.513] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.513] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.513] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.513] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.513] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.513] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.513] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.513] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.513] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.514] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.514] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.514] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.514] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.514] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.514] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.514] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.514] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.515] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.515] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.515] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.515] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.515] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.515] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.515] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.515] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.516] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.516] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.516] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.516] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.516] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.516] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.516] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.516] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.516] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.516] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.516] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.516] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.517] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.517] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.517] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.517] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.517] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.517] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.517] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.517] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.518] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.518] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.518] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.518] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.518] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.518] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.519] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.519] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.519] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.519] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.519] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.519] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.519] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.519] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.519] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.519] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.520] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.520] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.520] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.520] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.520] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.520] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.520] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.520] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.520] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.521] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.521] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.521] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.521] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.521] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.521] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.521] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.521] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.521] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.521] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.521] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.522] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.522] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.522] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.522] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.522] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.522] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.522] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.522] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.522] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.523] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.523] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.523] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.523] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.523] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.523] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.523] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.523] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.524] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.524] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.524] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.524] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.524] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.524] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.524] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.524] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.525] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.525] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.525] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.525] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.525] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.525] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.526] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.526] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.526] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.526] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.526] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.526] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.526] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.526] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.526] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.526] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.527] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.527] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.527] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.527] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.527] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.527] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.527] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.527] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.527] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.528] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.528] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.528] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.528] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.528] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.528] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.528] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.528] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.529] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.529] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.529] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.529] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.529] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.529] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.530] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.530] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.530] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.530] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.530] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.530] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.530] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.530] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.530] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.530] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.530] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.531] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.531] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.531] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.531] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.531] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.531] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.531] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.531] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.531] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.532] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.532] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.532] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.532] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.532] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.533] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.533] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.533] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.533] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.533] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.533] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.533] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.533] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.533] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.533] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.533] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.534] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.534] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.534] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.534] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.534] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.534] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.535] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.535] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.535] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.535] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.535] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.535] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.535] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.535] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.535] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.536] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.536] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.536] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.536] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.537] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.537] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.537] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.537] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.537] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.537] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.538] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.538] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.538] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.538] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.538] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.538] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.538] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.538] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.539] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.539] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.539] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.539] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.539] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.539] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.541] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.541] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.542] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.542] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.542] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.542] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.542] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.542] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.542] [INFO] [TensorRT] Searching for plugin: causal_conv1d, plugin_version: 1, plugin_namespace:
[14:09:43.542] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.542] [INFO] [TensorRT] Successfully created plugin: causal_conv1d
[14:09:43.543] [INFO] [TensorRT] Searching for plugin: gated_delta_net, plugin_version: 1, plugin_namespace:
[14:09:43.543] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute use_ddtree not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.543] [INFO] [TensorRT] Successfully created plugin: gated_delta_net
[14:09:43.543] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.543] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.543] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.543] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.543] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.543] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.544] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.544] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.544] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.544] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.544] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.544] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.544] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.544] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.545] [INFO] [TensorRT] Searching for plugin: AttentionPlugin, plugin_version: 1, plugin_namespace:
[14:09:43.545] [WARNING] [TensorRT] onnxOpImporters.cpp:6854: Attribute xqa_jit_kernels not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[14:09:43.545] [INFO] [TensorRT] Successfully created plugin: AttentionPlugin
[14:09:43.545] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.545] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.545] [INFO] [TensorRT] Searching for plugin: Int4MoePlugin, plugin_version: 1, plugin_namespace:
[14:09:43.545] [INFO] [TensorRT] Successfully created plugin: Int4MoePlugin
[14:09:43.545] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.545] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.545] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.545] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.546] [INFO] [TensorRT] Searching for plugin: Int4GroupwiseGemmPluginV2, plugin_version: 1, plugin_namespace:
[14:09:43.546] [INFO] [TensorRT] Successfully created plugin: Int4GroupwiseGemmPluginV2
[14:09:43.579] [INFO] [llmBuilder.cpp:364:build] Profiling verbosity set to DETAILED
[14:09:43.896] [INFO] [TensorRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[14:09:46.699] [INFO] [TensorRT] Compiler backend is used during engine build.
[14:11:54.602] [INFO] [TensorRT] Detected 79 inputs and 132 output network tensors.
[14:11:55.588] [INFO] [TensorRT] Total Host Persistent Memory: 80 bytes
[14:11:55.588] [INFO] [TensorRT] Total Device Persistent Memory: 0 bytes
[14:11:55.588] [INFO] [TensorRT] Max Scratch Memory: 3095007744 bytes
[14:11:55.588] [INFO] [TensorRT] [BlockAssignment] Started assigning block shifts. This will take 1 steps to complete.
[14:11:55.588] [INFO] [TensorRT] [BlockAssignment] Algorithm ShiftNTopDown took 0.00563ms to assign 1 blocks to 1 nodes requiring 3095007744 bytes.
[14:11:55.588] [INFO] [TensorRT] Total Activation Memory: 3095007744 bytes
[14:15:58.787] [INFO] [TensorRT] Detected 79 inputs and 132 output network tensors.
[14:15:59.783] [INFO] [TensorRT] Total Host Persistent Memory: 80 bytes
[14:15:59.783] [INFO] [TensorRT] Total Device Persistent Memory: 0 bytes
[14:15:59.783] [INFO] [TensorRT] Max Scratch Memory: 67400704 bytes
[14:15:59.783] [INFO] [TensorRT] [BlockAssignment] Started assigning block shifts. This will take 1 steps to complete.
[14:15:59.783] [INFO] [TensorRT] [BlockAssignment] Algorithm ShiftNTopDown took 0.005472ms to assign 1 blocks to 1 nodes requiring 67400704 bytes.
[14:15:59.783] [INFO] [TensorRT] Total Activation Memory: 67400704 bytes
[14:16:00.334] [INFO] [TensorRT] Total Weights Memory: 19905817360 bytes
[14:16:00.351] [INFO] [TensorRT] Compiler backend is used during engine execution.
[14:16:00.352] [INFO] [TensorRT] Engine generation completed in 376.457 seconds.
[14:16:00.586] [ERROR] [TensorRT] Unexpected exception stoi
[14:16:00.586] [INFO] [TensorRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 1 MiB, GPU 18984 MiB
[14:16:40.128] [INFO] [TensorRT] [MemUsageStats] Peak memory usage during Engine building and serialization: CPU: 44341 MiB
[14:19:59.405] [INFO] [builderUtils.cpp:384:buildAndSerializeEngine] Engine saved to /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/engines_32k/base/spec_base.engine
[14:20:00.751] [INFO] [llmBuilder.cpp:1835:copyConfig] Copied config.json with builder config to /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/engines_32k/base/base_config.json
[14:20:00.757] [INFO] [fileUtils.cpp:48:copyFile] Successfully copied /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/onnx_mtp_trt101_fix_v_1537744/llm/tokenizer_config.json to /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/engines_32k/base/tokenizer_config.json
[14:20:00.757] [INFO] [llmBuilder.cpp:1866:copyTokenizerFiles] Copied tokenizer file: tokenizer_config.json
[14:20:01.063] [INFO] [fileUtils.cpp:48:copyFile] Successfully copied /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/onnx_mtp_trt101_fix_v_1537744/llm/tokenizer.json to /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/engines_32k/base/tokenizer.json
[14:20:01.063] [INFO] [llmBuilder.cpp:1866:copyTokenizerFiles] Copied tokenizer file: tokenizer.json
[14:20:01.067] [INFO] [fileUtils.cpp:48:copyFile] Successfully copied /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/onnx_mtp_trt101_fix_v_1537744/llm/processed_chat_template.json to /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/engines_32k/base/processed_chat_template.json
[14:20:01.067] [INFO] [llmBuilder.cpp:1866:copyTokenizerFiles] Copied tokenizer file: processed_chat_template.json
[14:20:15.795] [INFO] [fileUtils.cpp:48:copyFile] Successfully copied /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/onnx_mtp_trt101_fix_v_1537744/llm/embedding.safetensors to /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/engines_32k/base/embedding.safetensors
[14:20:15.795] [INFO] [llmBuilder.cpp:2068:copyEmbeddingFile] Copied embedding.safetensors to /home/robot/Documents/share/jhzhang54/model/weight_gptq_moe_self_attn_0825_trt101/engines_32k/base/embedding.safetensors
[14:20:16.606] [INFO] [llm_build.cpp:380:main] [TP rank 0/1] LLM engine built successfully.