nohup: ignoring input [W Context.cpp:69] Warning: torch.set_deterministic is in beta, and its design and functionality may change in the future. (function operator()) [15 03:44:50 :39] WRN migrate all stand-alone args into a single task [15 03:44:50 :159] set task task_0's 0th input model path as /root/axera/axera-quan-hjj/model/yolov8n-pose.onnx [15 03:44:50 :178] set task task_0's 0th output model path as /root/axera/axera-quan-hjj/joint/yolov8n-pose.joint [15 03:44:50 :297] set task task_0's pulsar_conf.output_dir as /root/axera/axera-quan-hjj /opt/venv/lib/python3.6/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.) return torch._C._cuda_getDeviceCount() > 0 [15 03:44:51 :17] planning task task_0 [15 03:44:51 :459] WRN affine_preprocess at QAT model compiling is deprecated, insert enforce_integers at front, please use scale_to_integers instead. [15 03:44:51 :340] ################## Running task task_0 ################## [15 03:44:51 :30] python3 /root/python_modules/super_pulsar/super_pulsar/toolchain_wrappers/wrapper_neuwizard.py --config /tmp/tmppybjgzd8.prototxt [15 03:44:52 :36] DBG [neuwizard] [W Context.cpp:69] Warning: torch.set_deterministic is in beta, and its design and functionality may change in the future. (function operator()) [15 03:45:20 :36] DBG [neuwizard] ONNX Model Version 12 for "/root/axera/axera-quan-hjj/model/yolov8n-pose.onnx" [15 03:45:20 :36] DBG [neuwizard] Planning load step finished; elapsed time: 0.01s [15 03:45:20 :36] DBG [neuwizard] Planning transform step finished; elapsed time: 0.01s [15 03:45:20 :36] DBG [neuwizard] Planning transform step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning transform step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "native" finished; elapsed time: 0.18s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "native_no_bn" finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "pretransformed" finished; elapsed time: 0.33s [15 03:45:20 :36] DBG [neuwizard] Planning calibrate step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "transformed" finished; elapsed time: 21.52s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "posttransformed" finished; elapsed time: 1.51s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "magma" finished; elapsed time: 0.45s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "magma_validified" finished; elapsed time: 0.12s [15 03:45:20 :36] DBG [neuwizard] Planning transform step to "lava_with_rtv" finished; elapsed time: 2.70s [15 03:45:20 :36] DBG [neuwizard] Planning dump_joint_model step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning evaluate step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning ir_bit_macs step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning ir_bit_float_params step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Planning ir_bit_quantized_params step finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Loading model finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Transforming to "onnx_step_1" finished; elapsed time: 0.01s [15 03:45:20 :36] DBG [neuwizard] Transforming to "onnx_step_2" finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Transforming to "onnx" finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Transforming to "native" finished; elapsed time: 0.08s [15 03:45:20 :36] DBG [neuwizard] Transforming to "native_no_bn" finished; elapsed time: 0.00s [15 03:45:20 :36] DBG [neuwizard] Transforming to "pretransformed" finished; elapsed time: 0.12s [15 03:45:20 :36] DBG [neuwizard] /opt/venv/lib/python3.6/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.) [15 03:45:20 :36] DBG [neuwizard] return torch._C._cuda_getDeviceCount() > 0 [15 03:47:55 :36] DBG [neuwizard] Calibrating finished; elapsed time: 154.35s [15 03:47:55 :36] DBG [neuwizard] Recalibrating for op_267_FeatureQuantization_Global_Backward, op_266:cat is missing [15 04:21:16 :36] DBG [neuwizard] Recalibrating for op_311_FeatureQuantization_Global_Backward, op_310:cat is missing [15 05:12:19 :36] DBG [neuwizard] Recalibrating for op_333_FeatureQuantization_Global_Backward, op_332:cat is missing [15 06:02:36 :36] DBG [neuwizard] Recalibrating for op_380_FeatureQuantization_Global_Backward, op_379:cat is missing [15 06:53:01 :36] DBG [neuwizard] Recalibrating for op_402_FeatureQuantization_Global_Backward, op_401:cat is missing [15 07:44:43 :36] DBG [neuwizard] Transforming to "transformed" finished; elapsed time: 14206.47s [15 07:44:43 :36] DBG [neuwizard] Transforming to "posttransformed" finished; elapsed time: 0.63s [15 07:44:43 :36] DBG [neuwizard] Transforming to "magma" finished; elapsed time: 0.21s [15 07:44:43 :36] DBG [neuwizard] Transforming to "magma_validified" finished; elapsed time: 0.04s [15 07:44:43 :36] DBG [neuwizard] Transforming to "lava_with_rtv" finished; elapsed time: 1.31s [15 07:44:43 :36] DBG [neuwizard] Dynamically planning transform step to "lava" finished; elapsed time: 0.01s [15 07:44:43 :36] DBG [neuwizard] Transforming to "lava" dynamically finished; elapsed time: 0.02s [15 07:44:43 :36] DBG [neuwizard] Dynamically planning transform step to "lava_onnx" finished; elapsed time: 0.13s [15 07:44:43 :36] DBG [neuwizard] Transforming to "lava_onnx" dynamically finished; elapsed time: 0.18s [15 07:44:43 :36] DBG [neuwizard] Dynamically planning transform step to "lava_onnx_axe" finished; elapsed time: 0.01s [15 07:44:43 :36] DBG [neuwizard] Transforming to "lava_onnx_axe" dynamically finished; elapsed time: 0.02s [15 07:44:51 :36] DBG [neuwizard] /root/python_modules/neuwizard-latest/neuwizard/operators/lava/AX620/Conv2d.py:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:141.) [15 07:44:51 :36] DBG [neuwizard] Joint model dumpped as "/root/axera/axera-quan-hjj/joint/model.lava_joint" [15 07:44:51 :36] DBG [neuwizard] Dumping Joint Model finished; elapsed time: 6.19s [15 07:44:51 :36] DBG [neuwizard] Evaluation is not performed. [15 07:44:51 :36] DBG [neuwizard] Evaluating finished; elapsed time: 0.00s [15 07:44:51 :36] DBG [neuwizard] Overview Table of Bit MACs [15 07:44:51 :36] DBG [neuwizard] | Domain | native | pretransformed | transformed | posttransformed | magma | lava | [15 07:44:51 :36] DBG [neuwizard] |----------|----------|------------------|---------------|-------------------|---------|--------| [15 07:44:51 :36] DBG [neuwizard] | Bit MACs | 294G | 299G | 303G | 340G | 317G | 328G | [15 07:44:51 :36] DBG [neuwizard] Bit MACs measurement for each domain finished; elapsed time: 0.12s [15 07:44:51 :36] DBG [neuwizard] Overview Table of parameter size [15 07:44:51 :36] DBG [neuwizard] | Domain | native | pretransformed | transformed | posttransformed | magma | lava | [15 07:44:51 :36] DBG [neuwizard] |----------------------|----------|------------------|---------------|-------------------|---------|--------| [15 07:44:51 :36] DBG [neuwizard] | Parameter Size(bits) | 105M | 107M | 108M | 115M | 115M | 144M | [15 07:44:51 :36] DBG [neuwizard] Float Parameter size measurement for each domain finished; elapsed time: 0.20s [15 07:44:51 :36] DBG [neuwizard] Overview Table of parameter size [15 07:44:51 :36] DBG [neuwizard] | Domain | native | pretransformed | transformed | posttransformed | magma | lava | [15 07:44:51 :36] DBG [neuwizard] |----------------------|----------|------------------|---------------|-------------------|---------|--------| [15 07:44:51 :36] DBG [neuwizard] | Parameter Size(bits) | 26M | 27M | 27M | 29M | 28M | 29M | [15 07:44:51 :36] DBG [neuwizard] Quantized Parameter size measurement for each domain finished; elapsed time: 0.40s [15 07:44:55 :535] DBG working in "/root/tmp_45ri310" [15 07:44:55 :227] python3 pulsar.py gen /root/tmp_45ri310/part_0.lava/part_0.lava env/ax620a_virtual_111_config.ini -b 1 -pe 16 --times_thres 0 --job_stealing 3 --checkall --param_compress --continuous_input --no_sim --hyper_params run_cf.wait_mode=True [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] inference_report.log: [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:-------------------------|:-----------------------------|:-------------|:-------------------| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:---------------|:-----------------|:--------------|:--------------|:------------|:--------------|:----------------|:-------------------|:----------------| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:-----------------------|:-----------------|:------------------|:------------------|:--------------------| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:-----------------------|:-----------------|:--------------------|:--------------------|:--------------------|:------------------|:-----------------|:--------------|:-------------| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] profile stream EU: ld/st_ratio might include ringbuf/linebuf/feature_swap parts; mv_ratio migth have ringbuf part. [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:--------------------|-----------:|:-------------|:-----------------|:---------------|:----------------------|:---------------| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:---------------------------------------------|:----------------|:-----------------------| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:-----------|-----------:|----------:|:--------|-------:|----------:| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] inference: 16.8 ms, 59.59 fps [15 07:49:01 :250] DBG [pulsar] qps = fps * batch_size = 59.59 [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] simulated fps is based on DDR_BW: 1.59 GB/s [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] DDR IO stats: [15 07:49:01 :250] DBG [pulsar] ideal_input_data_size: 4441088 Byte, [15 07:49:01 :250] DBG [pulsar] ideal_output_data_size: 3897600 Byte, [15 07:49:01 :250] DBG [pulsar] extra_mid_io_data_size: 5366528 Byte, [15 07:49:01 :250] DBG [pulsar] total_io_data_size: 13705216 Byte [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] MAC per inference: 5119759360 MAC@int8 [15 07:49:01 :250] DBG [pulsar] MAC utils: 33.10 % [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] commit_id: [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] |:----------------------------------------------|-----------:|:-------------| [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] subgraph num: 5 [15 07:49:01 :250] DBG [pulsar]  [15 07:49:01 :250] DBG [pulsar] pulsar.py totally used 245s [15 07:49:20 :582] File saved: /root/axera/axera-quan-hjj/joint/yolov8n-pose.joint [15 07:49:20 :587] DBG cleared /root/tmp_45ri310 | Pre_alloc OCM | linebuffer(may SWAP later) | ringbuffer | parameter | | size(ratio of whole OCM) | 72960(3.5)% | 0(0.0)% | 112128(5.3)% | | range | (1912064, 1985024) | (None, None) | (1985024, 2097152) | | profile conv | work_cyc | linebuf | warmup_tail | core_idle | io_idle | stride2_idle | standalone_fetch | MAC | | ratio in conv | 6977493 (100.0%) | 167731 (2.4%) | 260365 (3.7%) | 0 (0.0%) | 180533 (2.6%) | 1499023 (21.5%) | 285 (0.0%) | 4444235 (63.7%) | | profile ideal DDR_IO | min_io_sum | min_params_read | min_inputs_read | min_outputs_write | | DDR IO size (Byte) | 8338688 (100.0%) | 4031488 (48.3%) | 409600 (4.9%) | 3897600 (46.7%) | | profile extra DDR_IO | extra_ddr_io | extra_params_read | extra_inputs_read | extra_outputs_wrt | extra_swap_read | extra_swap_wrt | ddr_rb_read | ddr_rb_wrt | | DDR IO size (Byte) | 5366528 (100.0%) | 2499328 (46.6%) | 1484800 (27.7%) | 1382400 (25.8%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | | profile stream EU | work_cyc | ld_ratio | ld_param_ratio | mv_ratio | mv_linebuffer_ratio | st_ratio | | teng | 11685635 | 958075(8.2%) | 3473475(29.7%) | 4462406(38.2%) | 126068(1.1%) | 2665610(22.8%) | | breakdown of mv_ratio in profile stream EU | teng | all_eus with mv-cmds | | total_cyc_num | 4462406(100.0%) | 4462406(100.0%) | | mv,affine,unpack_lsb | 2099343(47.0%) | 2099343(47.0%) | | weight0_mode,convNxM,mode23,nopad | 729952(16.4%) | 729952(16.4%) | | mv,subtensor | 541404(12.1%) | 541404(12.1%) | | mv,concat_c | 291234(6.5%) | 291234(6.5%) | | mv_patch | 262377(5.9%) | 262377(5.9%) | | weight0_mode,convNxM,mode20,nopad | 214922(4.8%) | 214922(4.8%) | | mv,padding_ch | 175951(3.9%) | 175951(3.9%) | | mv,upsample | 125316(2.8%) | 125316(2.8%) | | mv,padding | 21678(0.5%) | 21678(0.5%) | | revert_split | 229(0.0%) | 229(0.0%) | | EU | work_cyc | tot_cyc | ratio | fps | fps_bnd | | conv-1core | 6977494 | 13425465 | 51.0% | 59.590 | 114.650 | | teng | 11685635 | 13425465 | 87.0% | 59.590 | 68.460 | | breakdown of cmds_num for each op | cmds_num | percentage | | mv,affine,unpack_lsb | 1478 | 28.63% | | weight0_mode,mode23 | 1394 | 27.00% | | weight0_mode,mode23,nopad | 398 | 7.71% | | weight0_mode,convNxM,mode23,nopad | 328 | 6.35% | | pool,weight0_mode,mode20,conv_align,subtensor | 288 | 5.58% | | weight0_mode,mode20 | 220 | 4.26% | | mv,subtensor | 211 | 4.09% | | weight0_mode,convNxM,mode20,nopad | 144 | 2.79% | | revert_split | 138 | 2.67% | | weight0_mode,convNxM,mode26,nopad | 132 | 2.56% | | mv,concat_c | 99 | 1.92% | | pool,weight0_mode,mode20,subtensor | 96 | 1.86% | | weight0_mode,mode20,nopad | 94 | 1.82% | | mv,upsample | 49 | 0.95% | | mv,padding_ch | 47 | 0.91% | | mv_patch | 30 | 0.58% | | yuv42xto444,yuv2bgr0 | 11 | 0.21% | | mv,padding | 6 | 0.12% | | total_num | 5163 | 100% | 07:49:19 [I]final_check: FINAL_CHECK: output op_1119 check successed! 07:49:19 [I]final_check: FINAL_CHECK: output op_1120 check successed! 07:49:19 [I]final_check: FINAL_CHECK: output op_1121 check successed! 07:49:19 [I]final_check: FINAL_CHECK: output op_1500 check successed! 07:49:19 [I]final_check: FINAL_CHECK: output op_1502 check successed! 07:49:19 [I]final_check: FINAL_CHECK: output op_1504 check successed!