INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:51382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55876 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:36:13,358 - lava_callback - INFO - Received callback for node 69cc68708cd9cbcdc9459734 2026-04-01 00:36:13,365 - lava_callback - INFO - Starting async job submit for node 69cc68708cd9cbcdc9459734 INFO: 172.20.0.1:42004 - "POST /node/69cc68708cd9cbcdc9459734 HTTP/1.0" 202 Accepted 2026-04-01 00:36:13,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:13,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68708cd9cbcdc9459734 HTTP/1.1" 200 None 2026-04-01 00:36:13,396 - lava_callback - DEBUG - Processing node 69cc68708cd9cbcdc9459734: baseline-arm 2026-04-01 00:36:13,402 - lava_callback - WARNING - Failed to upload log 2026-04-01 00:36:13,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:13,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:13,540 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68708cd9cbcdc9459734/lava-logs.yaml 2026-04-01 00:36:13,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:13,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:13,659 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68708cd9cbcdc9459734/lava_callback.json.gz 2026-04-01 00:36:13,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:13,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68708cd9cbcdc9459734 HTTP/1.1" 200 None 2026-04-01 00:36:13,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:13,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:36:13,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:13,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68708cd9cbcdc9459734 HTTP/1.1" 200 None 2026-04-01 00:36:13,762 - lava_callback - INFO - Completed processing callback for node 69cc68708cd9cbcdc9459734 INFO: 172.20.0.1:42006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60494 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:36:43,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:43,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56762 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:36:48,581 - lava_callback - INFO - Received callback for node 69cc68748cd9cbcdc9459740 INFO: 172.20.0.1:56764 - "POST /node/69cc68748cd9cbcdc9459740 HTTP/1.0" 202 Accepted 2026-04-01 00:36:48,630 - lava_callback - INFO - Starting async job submit for node 69cc68748cd9cbcdc9459740 2026-04-01 00:36:48,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:48,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68748cd9cbcdc9459740 HTTP/1.1" 200 None 2026-04-01 00:36:48,664 - lava_callback - DEBUG - Processing node 69cc68748cd9cbcdc9459740: baseline-arm 2026-04-01 00:36:48,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:48,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:48,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68748cd9cbcdc9459740/log.txt.gz 2026-04-01 00:36:48,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:49,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:49,195 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68748cd9cbcdc9459740/lava-logs.yaml 2026-04-01 00:36:49,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:49,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:49,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68748cd9cbcdc9459740/lava_callback.json.gz 2026-04-01 00:36:49,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:49,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68748cd9cbcdc9459740 HTTP/1.1" 200 None 2026-04-01 00:36:49,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:49,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:36:49,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:49,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68748cd9cbcdc9459740 HTTP/1.1" 200 None 2026-04-01 00:36:49,400 - lava_callback - INFO - Completed processing callback for node 69cc68748cd9cbcdc9459740 2026-04-01 00:36:49,777 - lava_callback - INFO - Received callback for node 69cc68718cd9cbcdc9459737 INFO: 172.20.0.1:39442 - "POST /node/69cc68718cd9cbcdc9459737 HTTP/1.0" 202 Accepted 2026-04-01 00:36:49,826 - lava_callback - INFO - Starting async job submit for node 69cc68718cd9cbcdc9459737 2026-04-01 00:36:49,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:49,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68718cd9cbcdc9459737 HTTP/1.1" 200 None 2026-04-01 00:36:49,861 - lava_callback - DEBUG - Processing node 69cc68718cd9cbcdc9459737: baseline-arm 2026-04-01 00:36:49,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:50,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:50,133 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68718cd9cbcdc9459737/log.txt.gz 2026-04-01 00:36:50,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:50,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:50,382 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68718cd9cbcdc9459737/lava-logs.yaml 2026-04-01 00:36:50,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:36:50,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:36:50,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68718cd9cbcdc9459737/lava_callback.json.gz 2026-04-01 00:36:50,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:50,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68718cd9cbcdc9459737 HTTP/1.1" 200 None 2026-04-01 00:36:50,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:50,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:36:50,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:36:50,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68718cd9cbcdc9459737 HTTP/1.1" 200 None 2026-04-01 00:36:50,582 - lava_callback - INFO - Completed processing callback for node 69cc68718cd9cbcdc9459737 INFO: 172.20.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57340 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:37:13,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:13,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49410 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:37:17,515 - lava_callback - INFO - Received callback for node 69cc68738cd9cbcdc945973d INFO: 172.20.0.1:49424 - "POST /node/69cc68738cd9cbcdc945973d HTTP/1.0" 202 Accepted 2026-04-01 00:37:17,521 - lava_callback - INFO - Starting async job submit for node 69cc68738cd9cbcdc945973d 2026-04-01 00:37:17,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:17,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68738cd9cbcdc945973d HTTP/1.1" 200 None 2026-04-01 00:37:17,570 - lava_callback - DEBUG - Processing node 69cc68738cd9cbcdc945973d: baseline-arm 2026-04-01 00:37:17,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:17,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:17,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68738cd9cbcdc945973d/log.txt.gz 2026-04-01 00:37:18,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:18,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:18,259 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68738cd9cbcdc945973d/lava-logs.yaml 2026-04-01 00:37:18,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:18,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:18,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68738cd9cbcdc945973d/lava_callback.json.gz 2026-04-01 00:37:18,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:18,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68738cd9cbcdc945973d HTTP/1.1" 200 None 2026-04-01 00:37:18,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:18,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:37:18,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:18,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68738cd9cbcdc945973d HTTP/1.1" 200 None 2026-04-01 00:37:18,534 - lava_callback - INFO - Completed processing callback for node 69cc68738cd9cbcdc945973d 2026-04-01 00:37:21,700 - lava_callback - INFO - Received callback for node 69cc686f8cd9cbcdc9459731 INFO: 172.20.0.1:56946 - "POST /node/69cc686f8cd9cbcdc9459731 HTTP/1.0" 202 Accepted 2026-04-01 00:37:21,707 - lava_callback - INFO - Starting async job submit for node 69cc686f8cd9cbcdc9459731 2026-04-01 00:37:21,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:21,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc686f8cd9cbcdc9459731 HTTP/1.1" 200 None 2026-04-01 00:37:21,767 - lava_callback - DEBUG - Processing node 69cc686f8cd9cbcdc9459731: baseline-arm 2026-04-01 00:37:22,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:22,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:22,140 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc686f8cd9cbcdc9459731/log.txt.gz 2026-04-01 00:37:22,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:22,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:22,435 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc686f8cd9cbcdc9459731/lava-logs.yaml 2026-04-01 00:37:22,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:22,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:22,591 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc686f8cd9cbcdc9459731/lava_callback.json.gz 2026-04-01 00:37:22,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:22,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc686f8cd9cbcdc9459731 HTTP/1.1" 200 None 2026-04-01 00:37:22,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:22,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:37:22,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:22,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc686f8cd9cbcdc9459731 HTTP/1.1" 200 None 2026-04-01 00:37:22,719 - lava_callback - INFO - Completed processing callback for node 69cc686f8cd9cbcdc9459731 INFO: 172.20.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55106 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:37:37,641 - lava_callback - INFO - Received callback for node 69cc68948cd9cbcdc9459b50 INFO: 172.20.0.1:55108 - "POST /node/69cc68948cd9cbcdc9459b50 HTTP/1.0" 202 Accepted 2026-04-01 00:37:37,689 - lava_callback - INFO - Starting async job submit for node 69cc68948cd9cbcdc9459b50 2026-04-01 00:37:37,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:37,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68948cd9cbcdc9459b50 HTTP/1.1" 200 None 2026-04-01 00:37:37,730 - lava_callback - DEBUG - Processing node 69cc68948cd9cbcdc9459b50: baseline-arm-pengutronix 2026-04-01 00:37:37,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:38,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:38,039 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68948cd9cbcdc9459b50/log.txt.gz 2026-04-01 00:37:38,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:38,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:38,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68948cd9cbcdc9459b50/lava-logs.yaml 2026-04-01 00:37:38,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:38,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:38,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68948cd9cbcdc9459b50/lava_callback.json.gz 2026-04-01 00:37:38,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:38,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68948cd9cbcdc9459b50 HTTP/1.1" 200 None 2026-04-01 00:37:38,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:38,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:37:38,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:38,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68948cd9cbcdc9459b50 HTTP/1.1" 200 None 2026-04-01 00:37:38,503 - lava_callback - INFO - Completed processing callback for node 69cc68948cd9cbcdc9459b50 2026-04-01 00:37:42,211 - lava_callback - INFO - Received callback for node 69cc68928cd9cbcdc9459b4a INFO: 172.20.0.1:54858 - "POST /node/69cc68928cd9cbcdc9459b4a HTTP/1.0" 202 Accepted 2026-04-01 00:37:42,217 - lava_callback - INFO - Starting async job submit for node 69cc68928cd9cbcdc9459b4a 2026-04-01 00:37:42,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:42,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68928cd9cbcdc9459b4a HTTP/1.1" 200 None 2026-04-01 00:37:42,251 - lava_callback - DEBUG - Processing node 69cc68928cd9cbcdc9459b4a: baseline-arm-pengutronix 2026-04-01 00:37:42,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:42,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:42,538 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68928cd9cbcdc9459b4a/log.txt.gz 2026-04-01 00:37:42,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:42,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:42,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68928cd9cbcdc9459b4a/lava-logs.yaml 2026-04-01 00:37:42,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:42,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:42,942 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68928cd9cbcdc9459b4a/lava_callback.json.gz 2026-04-01 00:37:42,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:42,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68928cd9cbcdc9459b4a HTTP/1.1" 200 None 2026-04-01 00:37:42,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:42,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:37:43,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:43,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68928cd9cbcdc9459b4a HTTP/1.1" 200 None 2026-04-01 00:37:43,029 - lava_callback - INFO - Completed processing callback for node 69cc68928cd9cbcdc9459b4a 2026-04-01 00:37:43,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:43,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-01 00:37:44,596 - lava_callback - INFO - Received callback for node 69cc68918cd9cbcdc9459b47 INFO: 172.20.0.1:54872 - "POST /node/69cc68918cd9cbcdc9459b47 HTTP/1.0" 202 Accepted 2026-04-01 00:37:44,602 - lava_callback - INFO - Starting async job submit for node 69cc68918cd9cbcdc9459b47 2026-04-01 00:37:44,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:44,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68918cd9cbcdc9459b47 HTTP/1.1" 200 None 2026-04-01 00:37:44,636 - lava_callback - DEBUG - Processing node 69cc68918cd9cbcdc9459b47: baseline-arm-pengutronix 2026-04-01 00:37:44,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:44,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:44,890 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68918cd9cbcdc9459b47/log.txt.gz 2026-04-01 00:37:44,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:45,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:45,154 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68918cd9cbcdc9459b47/lava-logs.yaml 2026-04-01 00:37:45,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:45,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:45,298 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68918cd9cbcdc9459b47/lava_callback.json.gz 2026-04-01 00:37:45,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:45,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68918cd9cbcdc9459b47 HTTP/1.1" 200 None 2026-04-01 00:37:45,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:45,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:37:45,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:45,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68918cd9cbcdc9459b47 HTTP/1.1" 200 None 2026-04-01 00:37:45,388 - lava_callback - INFO - Completed processing callback for node 69cc68918cd9cbcdc9459b47 INFO: 172.20.0.1:54888 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:37:51,807 - lava_callback - INFO - Received callback for node 69cc68878cd9cbcdc9459b02 INFO: 172.20.0.1:58284 - "POST /node/69cc68878cd9cbcdc9459b02 HTTP/1.0" 202 Accepted 2026-04-01 00:37:51,813 - lava_callback - INFO - Starting async job submit for node 69cc68878cd9cbcdc9459b02 2026-04-01 00:37:51,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:51,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68878cd9cbcdc9459b02 HTTP/1.1" 200 None 2026-04-01 00:37:51,854 - lava_callback - DEBUG - Processing node 69cc68878cd9cbcdc9459b02: baseline-arm-clabbe 2026-04-01 00:37:52,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:52,288 - lava_callback - INFO - Received callback for node 69cc68ce8cd9cbcdc9459bd0 2026-04-01 00:37:52,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:52,296 - lava_callback - INFO - Starting async job submit for node 69cc68ce8cd9cbcdc9459bd0 INFO: 172.20.0.1:58294 - "POST /node/69cc68ce8cd9cbcdc9459bd0 HTTP/1.0" 202 Accepted 2026-04-01 00:37:52,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:52,301 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc68878cd9cbcdc9459b02/log.txt.gz 2026-04-01 00:37:52,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ce8cd9cbcdc9459bd0 HTTP/1.1" 200 None 2026-04-01 00:37:52,385 - lava_callback - DEBUG - Processing node 69cc68ce8cd9cbcdc9459bd0: baseline-arm 2026-04-01 00:37:52,432 - lava_callback - WARNING - Failed to upload log 2026-04-01 00:37:52,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:52,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:52,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:52,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:52,644 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68ce8cd9cbcdc9459bd0/lava-logs.yaml 2026-04-01 00:37:52,645 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc68878cd9cbcdc9459b02/lava-logs.yaml 2026-04-01 00:37:52,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:52,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:37:52,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:52,771 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68ce8cd9cbcdc9459bd0/lava_callback.json.gz 2026-04-01 00:37:52,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:52,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ce8cd9cbcdc9459bd0 HTTP/1.1" 200 None 2026-04-01 00:37:52,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:52,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:37:52,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:52,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68ce8cd9cbcdc9459bd0 HTTP/1.1" 200 None 2026-04-01 00:37:52,853 - lava_callback - INFO - Completed processing callback for node 69cc68ce8cd9cbcdc9459bd0 2026-04-01 00:37:52,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:37:52,856 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc68878cd9cbcdc9459b02/lava_callback.json.gz 2026-04-01 00:37:52,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68878cd9cbcdc9459b02 HTTP/1.1" 200 None 2026-04-01 00:37:52,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:52,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:37:52,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:37:52,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68878cd9cbcdc9459b02 HTTP/1.1" 200 None 2026-04-01 00:37:52,993 - lava_callback - INFO - Completed processing callback for node 69cc68878cd9cbcdc9459b02 INFO: 172.20.0.1:58302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:38:13,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:38:13,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52276 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:38:41,934 - lava_callback - INFO - Received callback for node 69cc68ae8cd9cbcdc9459b87 INFO: 172.20.0.1:40798 - "POST /node/69cc68ae8cd9cbcdc9459b87 HTTP/1.0" 202 Accepted 2026-04-01 00:38:41,941 - lava_callback - INFO - Starting async job submit for node 69cc68ae8cd9cbcdc9459b87 2026-04-01 00:38:41,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:38:41,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ae8cd9cbcdc9459b87 HTTP/1.1" 200 None 2026-04-01 00:38:41,981 - lava_callback - DEBUG - Processing node 69cc68ae8cd9cbcdc9459b87: ltp-dio 2026-04-01 00:38:42,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:38:42,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:38:42,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cc68ae8cd9cbcdc9459b87/log.txt.gz 2026-04-01 00:38:42,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:38:43,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:38:43,231 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cc68ae8cd9cbcdc9459b87/lava-logs.yaml 2026-04-01 00:38:43,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:38:43,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:38:43,392 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cc68ae8cd9cbcdc9459b87/lava_callback.json.gz 2026-04-01 00:38:43,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:38:43,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ae8cd9cbcdc9459b87 HTTP/1.1" 200 None 2026-04-01 00:38:43,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:38:43,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:38:43,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:38:43,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68ae8cd9cbcdc9459b87 HTTP/1.1" 200 None 2026-04-01 00:38:43,475 - lava_callback - INFO - Completed processing callback for node 69cc68ae8cd9cbcdc9459b87 2026-04-01 00:38:43,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:38:43,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46950 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:39:18,425 - lava_callback - INFO - Received callback for node 69c9c62a3c3aafdab4be9d46 INFO: 172.20.0.1:46956 - "POST /node/69c9c62a3c3aafdab4be9d46 HTTP/1.0" 202 Accepted 2026-04-01 00:39:18,435 - lava_callback - INFO - Starting async job submit for node 69c9c62a3c3aafdab4be9d46 2026-04-01 00:39:18,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:18,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c62a3c3aafdab4be9d46 HTTP/1.1" 200 None 2026-04-01 00:39:18,468 - lava_callback - DEBUG - Processing node 69c9c62a3c3aafdab4be9d46: baseline-arm-baylibre 2026-04-01 00:39:18,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c62a3c3aafdab4be9d46 2026-04-01 00:39:18,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:18,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c62a3c3aafdab4be9d46 HTTP/1.1" 200 None 2026-04-01 00:39:18,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:18,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 00:39:18,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:18,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c62a3c3aafdab4be9d46 HTTP/1.1" 200 None 2026-04-01 00:39:18,599 - lava_callback - ERROR - Error processing callback for node 69c9c62a3c3aafdab4be9d46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60204 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:39:35,096 - lava_callback - INFO - Received callback for node 69cc688b8cd9cbcdc9459b36 INFO: 172.20.0.1:43700 - "POST /node/69cc688b8cd9cbcdc9459b36 HTTP/1.0" 202 Accepted 2026-04-01 00:39:35,105 - lava_callback - INFO - Starting async job submit for node 69cc688b8cd9cbcdc9459b36 2026-04-01 00:39:35,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:35,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688b8cd9cbcdc9459b36 HTTP/1.1" 200 None 2026-04-01 00:39:35,175 - lava_callback - DEBUG - Processing node 69cc688b8cd9cbcdc9459b36: baseline-arm-clabbe 2026-04-01 00:39:35,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:39:36,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:39:36,028 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc688b8cd9cbcdc9459b36/log.txt.gz 2026-04-01 00:39:36,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:43704 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:39:36,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:39:36,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc688b8cd9cbcdc9459b36/lava-logs.yaml 2026-04-01 00:39:36,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:39:37,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:39:37,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc688b8cd9cbcdc9459b36/lava_callback.json.gz 2026-04-01 00:39:37,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:37,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688b8cd9cbcdc9459b36 HTTP/1.1" 200 None 2026-04-01 00:39:37,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:37,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:39:37,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:37,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc688b8cd9cbcdc9459b36 HTTP/1.1" 200 None 2026-04-01 00:39:37,383 - lava_callback - INFO - Completed processing callback for node 69cc688b8cd9cbcdc9459b36 2026-04-01 00:39:43,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:39:43,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:40:58,408 - lava_callback - INFO - Received callback for node 69c9c6973c3aafdab4be9d86 INFO: 172.20.0.1:32964 - "POST /node/69c9c6973c3aafdab4be9d86 HTTP/1.0" 202 Accepted 2026-04-01 00:40:58,413 - lava_callback - INFO - Starting async job submit for node 69c9c6973c3aafdab4be9d86 2026-04-01 00:40:58,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:40:58,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6973c3aafdab4be9d86 HTTP/1.1" 200 None 2026-04-01 00:40:58,429 - lava_callback - DEBUG - Processing node 69c9c6973c3aafdab4be9d86: baseline-arm-baylibre 2026-04-01 00:40:58,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6973c3aafdab4be9d86 2026-04-01 00:40:58,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:40:58,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6973c3aafdab4be9d86 HTTP/1.1" 200 None 2026-04-01 00:40:58,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:40:58,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 00:40:58,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:40:58,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6973c3aafdab4be9d86 HTTP/1.1" 200 None 2026-04-01 00:40:58,485 - lava_callback - ERROR - Error processing callback for node 69c9c6973c3aafdab4be9d86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54336 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:41:17,970 - lava_callback - INFO - Received callback for node 69cc688c8cd9cbcdc9459b39 INFO: 172.20.0.1:54342 - "POST /node/69cc688c8cd9cbcdc9459b39 HTTP/1.0" 202 Accepted 2026-04-01 00:41:17,976 - lava_callback - INFO - Starting async job submit for node 69cc688c8cd9cbcdc9459b39 2026-04-01 00:41:17,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:18,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688c8cd9cbcdc9459b39 HTTP/1.1" 200 None 2026-04-01 00:41:18,013 - lava_callback - DEBUG - Processing node 69cc688c8cd9cbcdc9459b39: baseline-arm-clabbe 2026-04-01 00:41:18,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:41:18,397 - lava_callback - INFO - Received callback for node 69c9c6a73c3aafdab4be9db3 INFO: 172.20.0.1:54358 - "POST /node/69c9c6a73c3aafdab4be9db3 HTTP/1.0" 202 Accepted 2026-04-01 00:41:18,406 - lava_callback - INFO - Starting async job submit for node 69c9c6a73c3aafdab4be9db3 2026-04-01 00:41:18,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:18,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6a73c3aafdab4be9db3 HTTP/1.1" 200 None 2026-04-01 00:41:18,428 - lava_callback - DEBUG - Processing node 69c9c6a73c3aafdab4be9db3: baseline-arm-baylibre 2026-04-01 00:41:18,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6a73c3aafdab4be9db3 2026-04-01 00:41:18,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:18,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6a73c3aafdab4be9db3 HTTP/1.1" 200 None 2026-04-01 00:41:18,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:18,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:18,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:18,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6a73c3aafdab4be9db3 HTTP/1.1" 200 None 2026-04-01 00:41:18,508 - lava_callback - ERROR - Error processing callback for node 69c9c6a73c3aafdab4be9db3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:18,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:41:18,594 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc688c8cd9cbcdc9459b39/log.txt.gz 2026-04-01 00:41:18,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:41:18,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:41:18,965 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc688c8cd9cbcdc9459b39/lava-logs.yaml 2026-04-01 00:41:18,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:41:19,083 - lava_callback - INFO - Received callback for node 69c9c6a83c3aafdab4be9db6 INFO: 172.20.0.1:54366 - "POST /node/69c9c6a83c3aafdab4be9db6 HTTP/1.0" 202 Accepted 2026-04-01 00:41:19,089 - lava_callback - INFO - Starting async job submit for node 69c9c6a83c3aafdab4be9db6 2026-04-01 00:41:19,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:41:19,102 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cc688c8cd9cbcdc9459b39/lava_callback.json.gz 2026-04-01 00:41:19,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6a83c3aafdab4be9db6 HTTP/1.1" 200 None 2026-04-01 00:41:19,127 - lava_callback - DEBUG - Processing node 69c9c6a83c3aafdab4be9db6: baseline-arm-baylibre 2026-04-01 00:41:19,128 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6a83c3aafdab4be9db6 2026-04-01 00:41:19,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6a83c3aafdab4be9db6 HTTP/1.1" 200 None 2026-04-01 00:41:19,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688c8cd9cbcdc9459b39 HTTP/1.1" 200 None 2026-04-01 00:41:19,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:41:19,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:19,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6a83c3aafdab4be9db6 HTTP/1.1" 200 None 2026-04-01 00:41:19,198 - lava_callback - ERROR - Error processing callback for node 69c9c6a83c3aafdab4be9db6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:19,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc688c8cd9cbcdc9459b39 HTTP/1.1" 200 None 2026-04-01 00:41:19,229 - lava_callback - INFO - Completed processing callback for node 69cc688c8cd9cbcdc9459b39 2026-04-01 00:41:19,766 - lava_callback - INFO - Received callback for node 69c9c6a93c3aafdab4be9db9 INFO: 172.20.0.1:39674 - "POST /node/69c9c6a93c3aafdab4be9db9 HTTP/1.0" 202 Accepted 2026-04-01 00:41:19,774 - lava_callback - INFO - Starting async job submit for node 69c9c6a93c3aafdab4be9db9 2026-04-01 00:41:19,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6a93c3aafdab4be9db9 HTTP/1.1" 200 None 2026-04-01 00:41:19,793 - lava_callback - DEBUG - Processing node 69c9c6a93c3aafdab4be9db9: baseline-arm-baylibre 2026-04-01 00:41:19,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6a93c3aafdab4be9db9 2026-04-01 00:41:19,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6a93c3aafdab4be9db9 HTTP/1.1" 200 None 2026-04-01 00:41:19,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:19,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:19,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6a93c3aafdab4be9db9 HTTP/1.1" 200 None 2026-04-01 00:41:19,864 - lava_callback - ERROR - Error processing callback for node 69c9c6a93c3aafdab4be9db9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:20,452 - lava_callback - INFO - Received callback for node 69c9c6ab3c3aafdab4be9dbc INFO: 172.20.0.1:39688 - "POST /node/69c9c6ab3c3aafdab4be9dbc HTTP/1.0" 202 Accepted 2026-04-01 00:41:20,458 - lava_callback - INFO - Starting async job submit for node 69c9c6ab3c3aafdab4be9dbc 2026-04-01 00:41:20,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:20,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6ab3c3aafdab4be9dbc HTTP/1.1" 200 None 2026-04-01 00:41:20,476 - lava_callback - DEBUG - Processing node 69c9c6ab3c3aafdab4be9dbc: baseline-arm-baylibre 2026-04-01 00:41:20,477 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6ab3c3aafdab4be9dbc 2026-04-01 00:41:20,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:20,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6ab3c3aafdab4be9dbc HTTP/1.1" 200 None 2026-04-01 00:41:20,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:20,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:20,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:20,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6ab3c3aafdab4be9dbc HTTP/1.1" 200 None 2026-04-01 00:41:20,585 - lava_callback - ERROR - Error processing callback for node 69c9c6ab3c3aafdab4be9dbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48652 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:41:38,382 - lava_callback - INFO - Received callback for node 69c9c6ae3c3aafdab4be9dc7 INFO: 172.20.0.1:48664 - "POST /node/69c9c6ae3c3aafdab4be9dc7 HTTP/1.0" 202 Accepted 2026-04-01 00:41:38,387 - lava_callback - INFO - Starting async job submit for node 69c9c6ae3c3aafdab4be9dc7 2026-04-01 00:41:38,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:38,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6ae3c3aafdab4be9dc7 HTTP/1.1" 200 None 2026-04-01 00:41:38,404 - lava_callback - DEBUG - Processing node 69c9c6ae3c3aafdab4be9dc7: baseline-arm-baylibre 2026-04-01 00:41:38,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6ae3c3aafdab4be9dc7 2026-04-01 00:41:38,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:38,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6ae3c3aafdab4be9dc7 HTTP/1.1" 200 None 2026-04-01 00:41:38,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:38,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:38,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:38,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6ae3c3aafdab4be9dc7 HTTP/1.1" 200 None 2026-04-01 00:41:38,474 - lava_callback - ERROR - Error processing callback for node 69c9c6ae3c3aafdab4be9dc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:43,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:43,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49684 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:41:47,042 - lava_callback - INFO - Received callback for node 69c9c6b53c3aafdab4be9dde INFO: 172.20.0.1:49688 - "POST /node/69c9c6b53c3aafdab4be9dde HTTP/1.0" 202 Accepted 2026-04-01 00:41:47,048 - lava_callback - INFO - Starting async job submit for node 69c9c6b53c3aafdab4be9dde 2026-04-01 00:41:47,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6b53c3aafdab4be9dde HTTP/1.1" 200 None 2026-04-01 00:41:47,064 - lava_callback - DEBUG - Processing node 69c9c6b53c3aafdab4be9dde: baseline-arm-clabbe 2026-04-01 00:41:47,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6b53c3aafdab4be9dde 2026-04-01 00:41:47,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6b53c3aafdab4be9dde HTTP/1.1" 200 None 2026-04-01 00:41:47,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:47,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6b53c3aafdab4be9dde HTTP/1.1" 200 None 2026-04-01 00:41:47,127 - lava_callback - ERROR - Error processing callback for node 69c9c6b53c3aafdab4be9dde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:47,693 - lava_callback - INFO - Received callback for node 69c9c6ba3c3aafdab4be9e0d INFO: 172.20.0.1:49692 - "POST /node/69c9c6ba3c3aafdab4be9e0d HTTP/1.0" 202 Accepted 2026-04-01 00:41:47,699 - lava_callback - INFO - Starting async job submit for node 69c9c6ba3c3aafdab4be9e0d 2026-04-01 00:41:47,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6ba3c3aafdab4be9e0d HTTP/1.1" 200 None 2026-04-01 00:41:47,716 - lava_callback - DEBUG - Processing node 69c9c6ba3c3aafdab4be9e0d: baseline-arm-clabbe 2026-04-01 00:41:47,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6ba3c3aafdab4be9e0d 2026-04-01 00:41:47,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6ba3c3aafdab4be9e0d HTTP/1.1" 200 None 2026-04-01 00:41:47,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:47,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:47,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6ba3c3aafdab4be9e0d HTTP/1.1" 200 None 2026-04-01 00:41:47,767 - lava_callback - ERROR - Error processing callback for node 69c9c6ba3c3aafdab4be9e0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:48,355 - lava_callback - INFO - Received callback for node 69c9c6bb3c3aafdab4be9e12 INFO: 172.20.0.1:49704 - "POST /node/69c9c6bb3c3aafdab4be9e12 HTTP/1.0" 202 Accepted 2026-04-01 00:41:48,361 - lava_callback - INFO - Starting async job submit for node 69c9c6bb3c3aafdab4be9e12 2026-04-01 00:41:48,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:48,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6bb3c3aafdab4be9e12 HTTP/1.1" 200 None 2026-04-01 00:41:48,380 - lava_callback - DEBUG - Processing node 69c9c6bb3c3aafdab4be9e12: baseline-arm-clabbe 2026-04-01 00:41:48,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6bb3c3aafdab4be9e12 2026-04-01 00:41:48,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:48,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6bb3c3aafdab4be9e12 HTTP/1.1" 200 None 2026-04-01 00:41:48,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:48,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:48,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:48,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6bb3c3aafdab4be9e12 HTTP/1.1" 200 None 2026-04-01 00:41:48,449 - lava_callback - ERROR - Error processing callback for node 69c9c6bb3c3aafdab4be9e12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:49,045 - lava_callback - INFO - Received callback for node 69c9c6bb3c3aafdab4be9e15 INFO: 172.20.0.1:49720 - "POST /node/69c9c6bb3c3aafdab4be9e15 HTTP/1.0" 202 Accepted 2026-04-01 00:41:49,055 - lava_callback - INFO - Starting async job submit for node 69c9c6bb3c3aafdab4be9e15 2026-04-01 00:41:49,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6bb3c3aafdab4be9e15 HTTP/1.1" 200 None 2026-04-01 00:41:49,086 - lava_callback - DEBUG - Processing node 69c9c6bb3c3aafdab4be9e15: baseline-arm-clabbe 2026-04-01 00:41:49,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6bb3c3aafdab4be9e15 2026-04-01 00:41:49,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6bb3c3aafdab4be9e15 HTTP/1.1" 200 None 2026-04-01 00:41:49,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:49,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6bb3c3aafdab4be9e15 HTTP/1.1" 200 None 2026-04-01 00:41:49,158 - lava_callback - ERROR - Error processing callback for node 69c9c6bb3c3aafdab4be9e15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:41:49,723 - lava_callback - INFO - Received callback for node 69c9c6be3c3aafdab4be9e1e INFO: 172.20.0.1:51782 - "POST /node/69c9c6be3c3aafdab4be9e1e HTTP/1.0" 202 Accepted 2026-04-01 00:41:49,730 - lava_callback - INFO - Starting async job submit for node 69c9c6be3c3aafdab4be9e1e 2026-04-01 00:41:49,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6be3c3aafdab4be9e1e HTTP/1.1" 200 None 2026-04-01 00:41:49,794 - lava_callback - DEBUG - Processing node 69c9c6be3c3aafdab4be9e1e: baseline-arm-clabbe 2026-04-01 00:41:49,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c6be3c3aafdab4be9e1e 2026-04-01 00:41:49,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c6be3c3aafdab4be9e1e HTTP/1.1" 200 None 2026-04-01 00:41:49,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 00:41:49,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:41:49,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c6be3c3aafdab4be9e1e HTTP/1.1" 200 None 2026-04-01 00:41:49,856 - lava_callback - ERROR - Error processing callback for node 69c9c6be3c3aafdab4be9e1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48994 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:42:52,991 - lava_callback - INFO - Received callback for node 69cc69f28cd9cbcdc9459c88 INFO: 172.20.0.1:40646 - "POST /node/69cc69f28cd9cbcdc9459c88 HTTP/1.0" 202 Accepted 2026-04-01 00:42:52,997 - lava_callback - INFO - Starting async job submit for node 69cc69f28cd9cbcdc9459c88 2026-04-01 00:42:53,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:42:53,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc69f28cd9cbcdc9459c88 HTTP/1.1" 200 None 2026-04-01 00:42:53,015 - lava_callback - DEBUG - Processing node 69cc69f28cd9cbcdc9459c88: baseline-arm 2026-04-01 00:42:53,021 - lava_callback - WARNING - Failed to upload log 2026-04-01 00:42:53,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:42:53,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:42:53,178 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc69f28cd9cbcdc9459c88/lava-logs.yaml 2026-04-01 00:42:53,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:42:53,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:42:53,340 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc69f28cd9cbcdc9459c88/lava_callback.json.gz 2026-04-01 00:42:53,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:42:53,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc69f28cd9cbcdc9459c88 HTTP/1.1" 200 None 2026-04-01 00:42:53,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:42:53,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:42:53,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:42:53,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc69f28cd9cbcdc9459c88 HTTP/1.1" 200 None 2026-04-01 00:42:53,405 - lava_callback - INFO - Completed processing callback for node 69cc69f28cd9cbcdc9459c88 INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:43:13,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:43:13,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52394 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:44:58,685 - lava_callback - INFO - Received callback for node 69c9c77b3c3aafdab4be9f0e INFO: 172.20.0.1:52408 - "POST /node/69c9c77b3c3aafdab4be9f0e HTTP/1.0" 202 Accepted 2026-04-01 00:44:58,690 - lava_callback - INFO - Starting async job submit for node 69c9c77b3c3aafdab4be9f0e 2026-04-01 00:44:58,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:44:58,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c77b3c3aafdab4be9f0e HTTP/1.1" 200 None 2026-04-01 00:44:58,707 - lava_callback - DEBUG - Processing node 69c9c77b3c3aafdab4be9f0e: baseline-arm-baylibre 2026-04-01 00:44:58,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c77b3c3aafdab4be9f0e 2026-04-01 00:44:58,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:44:58,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c77b3c3aafdab4be9f0e HTTP/1.1" 200 None 2026-04-01 00:44:58,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:44:58,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 00:44:58,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:44:58,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c77b3c3aafdab4be9f0e HTTP/1.1" 200 None 2026-04-01 00:44:58,770 - lava_callback - ERROR - Error processing callback for node 69c9c77b3c3aafdab4be9f0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52546 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:47:43,554 - lava_callback - INFO - Received callback for node 69cc6b148cd9cbcdc9459cec INFO: 172.20.0.1:34198 - "POST /node/69cc6b148cd9cbcdc9459cec HTTP/1.0" 202 Accepted 2026-04-01 00:47:43,564 - lava_callback - INFO - Starting async job submit for node 69cc6b148cd9cbcdc9459cec 2026-04-01 00:47:43,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:43,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b148cd9cbcdc9459cec HTTP/1.1" 200 None 2026-04-01 00:47:43,619 - lava_callback - DEBUG - Processing node 69cc6b148cd9cbcdc9459cec: baseline-x86 2026-04-01 00:47:43,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:47:44,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:47:44,084 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b148cd9cbcdc9459cec/log.txt.gz 2026-04-01 00:47:44,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:47:44,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:47:44,606 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b148cd9cbcdc9459cec/lava-logs.yaml 2026-04-01 00:47:44,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:47:44,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:47:44,835 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b148cd9cbcdc9459cec/lava_callback.json.gz 2026-04-01 00:47:44,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:44,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b148cd9cbcdc9459cec HTTP/1.1" 200 None 2026-04-01 00:47:44,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:44,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64a48cd9cbcdc9459671 HTTP/1.1" 200 None 2026-04-01 00:47:44,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:44,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b148cd9cbcdc9459cec HTTP/1.1" 200 None 2026-04-01 00:47:44,932 - lava_callback - INFO - Completed processing callback for node 69cc6b148cd9cbcdc9459cec INFO: 172.20.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53518 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:47:58,500 - lava_callback - INFO - Received callback for node 69cc6b128cd9cbcdc9459ce3 INFO: 172.20.0.1:53530 - "POST /node/69cc6b128cd9cbcdc9459ce3 HTTP/1.0" 202 Accepted 2026-04-01 00:47:58,509 - lava_callback - INFO - Starting async job submit for node 69cc6b128cd9cbcdc9459ce3 2026-04-01 00:47:58,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:58,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b128cd9cbcdc9459ce3 HTTP/1.1" 200 None 2026-04-01 00:47:58,583 - lava_callback - DEBUG - Processing node 69cc6b128cd9cbcdc9459ce3: baseline-x86 2026-04-01 00:47:58,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:47:59,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:47:59,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b128cd9cbcdc9459ce3/log.txt.gz 2026-04-01 00:47:59,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:47:59,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:47:59,569 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b128cd9cbcdc9459ce3/lava-logs.yaml 2026-04-01 00:47:59,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:47:59,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:47:59,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b128cd9cbcdc9459ce3/lava_callback.json.gz 2026-04-01 00:47:59,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:59,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b128cd9cbcdc9459ce3 HTTP/1.1" 200 None 2026-04-01 00:47:59,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:59,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64a48cd9cbcdc9459671 HTTP/1.1" 200 None 2026-04-01 00:47:59,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:47:59,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b128cd9cbcdc9459ce3 HTTP/1.1" 200 None 2026-04-01 00:47:59,795 - lava_callback - INFO - Completed processing callback for node 69cc6b128cd9cbcdc9459ce3 INFO: 172.20.0.1:42700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42716 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:48:13,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:13,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:48:36,821 - lava_callback - INFO - Received callback for node 69cc6b138cd9cbcdc9459ce6 INFO: 172.20.0.1:35760 - "POST /node/69cc6b138cd9cbcdc9459ce6 HTTP/1.0" 202 Accepted 2026-04-01 00:48:36,832 - lava_callback - INFO - Starting async job submit for node 69cc6b138cd9cbcdc9459ce6 2026-04-01 00:48:36,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:36,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b138cd9cbcdc9459ce6 HTTP/1.1" 200 None 2026-04-01 00:48:36,892 - lava_callback - DEBUG - Processing node 69cc6b138cd9cbcdc9459ce6: baseline-x86 2026-04-01 00:48:37,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:37,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:37,686 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b138cd9cbcdc9459ce6/log.txt.gz 2026-04-01 00:48:37,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:38,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:38,518 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b138cd9cbcdc9459ce6/lava-logs.yaml 2026-04-01 00:48:38,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:38,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:38,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b138cd9cbcdc9459ce6/lava_callback.json.gz 2026-04-01 00:48:38,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:38,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b138cd9cbcdc9459ce6 HTTP/1.1" 200 None 2026-04-01 00:48:38,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:38,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64a48cd9cbcdc9459671 HTTP/1.1" 200 None 2026-04-01 00:48:38,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:38,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b138cd9cbcdc9459ce6 HTTP/1.1" 200 None 2026-04-01 00:48:38,773 - lava_callback - INFO - Completed processing callback for node 69cc6b138cd9cbcdc9459ce6 2026-04-01 00:48:38,828 - lava_callback - INFO - Received callback for node 69c9c85d3c3aafdab4be9f9b INFO: 172.20.0.1:35766 - "POST /node/69c9c85d3c3aafdab4be9f9b HTTP/1.0" 202 Accepted 2026-04-01 00:48:38,838 - lava_callback - INFO - Starting async job submit for node 69c9c85d3c3aafdab4be9f9b 2026-04-01 00:48:38,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:38,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c85d3c3aafdab4be9f9b HTTP/1.1" 200 None 2026-04-01 00:48:38,870 - lava_callback - DEBUG - Processing node 69c9c85d3c3aafdab4be9f9b: baseline-arm-baylibre 2026-04-01 00:48:38,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c85d3c3aafdab4be9f9b 2026-04-01 00:48:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:38,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c85d3c3aafdab4be9f9b HTTP/1.1" 200 None 2026-04-01 00:48:38,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:38,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 00:48:38,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:38,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c85d3c3aafdab4be9f9b HTTP/1.1" 200 None 2026-04-01 00:48:38,965 - lava_callback - ERROR - Error processing callback for node 69c9c85d3c3aafdab4be9f9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 00:48:43,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:43,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-01 00:48:45,771 - lava_callback - INFO - Received callback for node 69cc6b158cd9cbcdc9459cef INFO: 172.20.0.1:38864 - "POST /node/69cc6b158cd9cbcdc9459cef HTTP/1.0" 202 Accepted 2026-04-01 00:48:45,779 - lava_callback - INFO - Starting async job submit for node 69cc6b158cd9cbcdc9459cef 2026-04-01 00:48:45,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:45,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b158cd9cbcdc9459cef HTTP/1.1" 200 None 2026-04-01 00:48:45,841 - lava_callback - DEBUG - Processing node 69cc6b158cd9cbcdc9459cef: baseline-x86 2026-04-01 00:48:46,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:38874 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:48:46,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:46,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b158cd9cbcdc9459cef/log.txt.gz 2026-04-01 00:48:46,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:47,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:47,200 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b158cd9cbcdc9459cef/lava-logs.yaml 2026-04-01 00:48:47,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:47,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:47,387 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b158cd9cbcdc9459cef/lava_callback.json.gz 2026-04-01 00:48:47,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:47,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b158cd9cbcdc9459cef HTTP/1.1" 200 None 2026-04-01 00:48:47,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:47,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64a48cd9cbcdc9459671 HTTP/1.1" 200 None 2026-04-01 00:48:47,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:47,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b158cd9cbcdc9459cef HTTP/1.1" 200 None 2026-04-01 00:48:47,539 - lava_callback - INFO - Completed processing callback for node 69cc6b158cd9cbcdc9459cef 2026-04-01 00:48:52,846 - lava_callback - INFO - Received callback for node 69cc6b148cd9cbcdc9459ce9 INFO: 172.20.0.1:60384 - "POST /node/69cc6b148cd9cbcdc9459ce9 HTTP/1.0" 202 Accepted 2026-04-01 00:48:52,856 - lava_callback - INFO - Starting async job submit for node 69cc6b148cd9cbcdc9459ce9 2026-04-01 00:48:52,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:52,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b148cd9cbcdc9459ce9 HTTP/1.1" 200 None 2026-04-01 00:48:52,948 - lava_callback - DEBUG - Processing node 69cc6b148cd9cbcdc9459ce9: baseline-x86 2026-04-01 00:48:53,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:53,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:53,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b148cd9cbcdc9459ce9/log.txt.gz 2026-04-01 00:48:53,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:54,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:54,497 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b148cd9cbcdc9459ce9/lava-logs.yaml 2026-04-01 00:48:54,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:48:54,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:48:54,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cc6b148cd9cbcdc9459ce9/lava_callback.json.gz 2026-04-01 00:48:54,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:54,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b148cd9cbcdc9459ce9 HTTP/1.1" 200 None 2026-04-01 00:48:54,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:54,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64a48cd9cbcdc9459671 HTTP/1.1" 200 None 2026-04-01 00:48:54,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:48:54,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b148cd9cbcdc9459ce9 HTTP/1.1" 200 None 2026-04-01 00:48:54,766 - lava_callback - INFO - Completed processing callback for node 69cc6b148cd9cbcdc9459ce9 INFO: 172.20.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:49:14,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:49:14,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35952 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 00:50:15,987 - lava_callback - INFO - Received callback for node 69cc68728cd9cbcdc945973a INFO: 172.20.0.1:35958 - "POST /node/69cc68728cd9cbcdc945973a HTTP/1.0" 202 Accepted 2026-04-01 00:50:15,995 - lava_callback - INFO - Starting async job submit for node 69cc68728cd9cbcdc945973a 2026-04-01 00:50:16,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:50:16,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68728cd9cbcdc945973a HTTP/1.1" 200 None 2026-04-01 00:50:16,084 - lava_callback - DEBUG - Processing node 69cc68728cd9cbcdc945973a: baseline-arm INFO: 172.20.0.1:35964 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:50:16,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:50:16,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:50:16,842 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68728cd9cbcdc945973a/log.txt.gz 2026-04-01 00:50:17,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:50:17,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:50:17,416 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68728cd9cbcdc945973a/lava-logs.yaml 2026-04-01 00:50:17,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:50:17,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:50:17,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc68728cd9cbcdc945973a/lava_callback.json.gz DEBUG: Setup failed for 69cc68728cd9cbcdc945973a. Transitting job node result: incomplete -> fail 2026-04-01 00:50:17,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:50:17,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68728cd9cbcdc945973a HTTP/1.1" 200 None 2026-04-01 00:50:17,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:50:17,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:50:17,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:50:17,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68728cd9cbcdc945973a HTTP/1.1" 200 None 2026-04-01 00:50:17,846 - lava_callback - INFO - Completed processing callback for node 69cc68728cd9cbcdc945973a INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47722 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:50:44,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:50:44,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36880 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:53:59,279 - lava_callback - INFO - Received callback for node 69c9c99f3c3aafdab4bea04a INFO: 172.20.0.1:36886 - "POST /node/69c9c99f3c3aafdab4bea04a HTTP/1.0" 202 Accepted 2026-04-01 00:53:59,284 - lava_callback - INFO - Starting async job submit for node 69c9c99f3c3aafdab4bea04a 2026-04-01 00:53:59,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:53:59,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c99f3c3aafdab4bea04a HTTP/1.1" 200 None 2026-04-01 00:53:59,301 - lava_callback - DEBUG - Processing node 69c9c99f3c3aafdab4bea04a: baseline-arm-baylibre 2026-04-01 00:53:59,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c99f3c3aafdab4bea04a 2026-04-01 00:53:59,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:53:59,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c99f3c3aafdab4bea04a HTTP/1.1" 200 None 2026-04-01 00:53:59,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:53:59,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 00:53:59,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:53:59,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c99f3c3aafdab4bea04a HTTP/1.1" 200 None 2026-04-01 00:53:59,352 - lava_callback - ERROR - Error processing callback for node 69c9c99f3c3aafdab4bea04a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:55:19,193 - lava_callback - INFO - Received callback for node 69c9c9e83c3aafdab4bea074 INFO: 172.20.0.1:36174 - "POST /node/69c9c9e83c3aafdab4bea074 HTTP/1.0" 202 Accepted 2026-04-01 00:55:19,200 - lava_callback - INFO - Starting async job submit for node 69c9c9e83c3aafdab4bea074 2026-04-01 00:55:19,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:19,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c9e83c3aafdab4bea074 HTTP/1.1" 200 None 2026-04-01 00:55:19,217 - lava_callback - DEBUG - Processing node 69c9c9e83c3aafdab4bea074: baseline-arm-baylibre 2026-04-01 00:55:19,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c9e83c3aafdab4bea074 2026-04-01 00:55:19,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:19,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c9e83c3aafdab4bea074 HTTP/1.1" 200 None 2026-04-01 00:55:19,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:19,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 00:55:19,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:19,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c9e83c3aafdab4bea074 HTTP/1.1" 200 None 2026-04-01 00:55:19,282 - lava_callback - ERROR - Error processing callback for node 69c9c9e83c3aafdab4bea074: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42970 - "GET /node/6927d6ae4e6fb3232e272ab1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42972 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:55:36,314 - lava_callback - INFO - Received callback for node 69cc6cf28cd9cbcdc9459d99 INFO: 172.20.0.1:44944 - "POST /node/69cc6cf28cd9cbcdc9459d99 HTTP/1.0" 202 Accepted 2026-04-01 00:55:36,322 - lava_callback - INFO - Starting async job submit for node 69cc6cf28cd9cbcdc9459d99 2026-04-01 00:55:36,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:36,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6cf28cd9cbcdc9459d99 HTTP/1.1" 200 None 2026-04-01 00:55:36,345 - lava_callback - DEBUG - Processing node 69cc6cf28cd9cbcdc9459d99: baseline-arm 2026-04-01 00:55:36,354 - lava_callback - WARNING - Failed to upload log 2026-04-01 00:55:36,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:55:36,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:55:36,700 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc6cf28cd9cbcdc9459d99/lava-logs.yaml 2026-04-01 00:55:36,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:55:36,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:55:36,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc6cf28cd9cbcdc9459d99/lava_callback.json.gz 2026-04-01 00:55:36,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:36,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6cf28cd9cbcdc9459d99 HTTP/1.1" 200 None 2026-04-01 00:55:36,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:36,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:55:36,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:36,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6cf28cd9cbcdc9459d99 HTTP/1.1" 200 None 2026-04-01 00:55:36,954 - lava_callback - INFO - Completed processing callback for node 69cc6cf28cd9cbcdc9459d99 2026-04-01 00:55:44,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:55:44,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59556 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:56:06,538 - lava_callback - INFO - Received callback for node 69cc68938cd9cbcdc9459b4d INFO: 172.20.0.1:59562 - "POST /node/69cc68938cd9cbcdc9459b4d HTTP/1.0" 202 Accepted 2026-04-01 00:56:06,589 - lava_callback - INFO - Starting async job submit for node 69cc68938cd9cbcdc9459b4d 2026-04-01 00:56:06,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:56:06,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68938cd9cbcdc9459b4d HTTP/1.1" 200 None 2026-04-01 00:56:06,659 - lava_callback - DEBUG - Processing node 69cc68938cd9cbcdc9459b4d: baseline-arm-pengutronix 2026-04-01 00:56:06,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:56:07,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:56:07,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68938cd9cbcdc9459b4d/log.txt.gz 2026-04-01 00:56:07,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:59576 - "GET /node/69876b1fe31b58f834f19788%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-01 00:56:07,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:56:07,301 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68938cd9cbcdc9459b4d/lava-logs.yaml 2026-04-01 00:56:07,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 00:56:07,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 00:56:07,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc68938cd9cbcdc9459b4d/lava_callback.json.gz DEBUG: Setup failed for 69cc68938cd9cbcdc9459b4d. Transitting job node result: incomplete -> fail 2026-04-01 00:56:07,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:56:07,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68938cd9cbcdc9459b4d HTTP/1.1" 200 None 2026-04-01 00:56:07,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:56:07,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 00:56:07,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:56:07,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68938cd9cbcdc9459b4d HTTP/1.1" 200 None 2026-04-01 00:56:07,669 - lava_callback - INFO - Completed processing callback for node 69cc68938cd9cbcdc9459b4d 2026-04-01 00:56:14,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:56:14,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53568 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:57:39,484 - lava_callback - INFO - Received callback for node 69c9ca743c3aafdab4bea0b3 INFO: 172.20.0.1:53580 - "POST /node/69c9ca743c3aafdab4bea0b3 HTTP/1.0" 202 Accepted 2026-04-01 00:57:39,490 - lava_callback - INFO - Starting async job submit for node 69c9ca743c3aafdab4bea0b3 2026-04-01 00:57:39,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:57:39,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ca743c3aafdab4bea0b3 HTTP/1.1" 200 None 2026-04-01 00:57:39,508 - lava_callback - DEBUG - Processing node 69c9ca743c3aafdab4bea0b3: baseline-arm-baylibre 2026-04-01 00:57:39,508 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ca743c3aafdab4bea0b3 2026-04-01 00:57:39,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:57:39,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ca743c3aafdab4bea0b3 HTTP/1.1" 200 None 2026-04-01 00:57:39,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:57:39,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 00:57:39,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:57:39,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ca743c3aafdab4bea0b3 HTTP/1.1" 200 None 2026-04-01 00:57:39,569 - lava_callback - ERROR - Error processing callback for node 69c9ca743c3aafdab4bea0b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50590 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:58:19,252 - lava_callback - INFO - Received callback for node 69c9ca983c3aafdab4bea0be INFO: 172.20.0.1:50594 - "POST /node/69c9ca983c3aafdab4bea0be HTTP/1.0" 202 Accepted 2026-04-01 00:58:19,258 - lava_callback - INFO - Starting async job submit for node 69c9ca983c3aafdab4bea0be 2026-04-01 00:58:19,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:19,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ca983c3aafdab4bea0be HTTP/1.1" 200 None 2026-04-01 00:58:19,281 - lava_callback - DEBUG - Processing node 69c9ca983c3aafdab4bea0be: baseline-arm-baylibre 2026-04-01 00:58:19,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ca983c3aafdab4bea0be 2026-04-01 00:58:19,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:19,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ca983c3aafdab4bea0be HTTP/1.1" 200 None 2026-04-01 00:58:19,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:19,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 00:58:19,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:19,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ca983c3aafdab4bea0be HTTP/1.1" 200 None 2026-04-01 00:58:19,525 - lava_callback - ERROR - Error processing callback for node 69c9ca983c3aafdab4bea0be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46092 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:58:59,503 - lava_callback - INFO - Received callback for node 69c9cabe3c3aafdab4bea0d7 INFO: 172.20.0.1:46104 - "POST /node/69c9cabe3c3aafdab4bea0d7 HTTP/1.0" 202 Accepted 2026-04-01 00:58:59,510 - lava_callback - INFO - Starting async job submit for node 69c9cabe3c3aafdab4bea0d7 2026-04-01 00:58:59,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:59,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cabe3c3aafdab4bea0d7 HTTP/1.1" 200 None 2026-04-01 00:58:59,589 - lava_callback - DEBUG - Processing node 69c9cabe3c3aafdab4bea0d7: baseline-arm-baylibre 2026-04-01 00:58:59,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cabe3c3aafdab4bea0d7 2026-04-01 00:58:59,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:59,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cabe3c3aafdab4bea0d7 HTTP/1.1" 200 None 2026-04-01 00:58:59,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:59,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 00:58:59,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:58:59,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cabe3c3aafdab4bea0d7 HTTP/1.1" 200 None 2026-04-01 00:58:59,699 - lava_callback - ERROR - Error processing callback for node 69c9cabe3c3aafdab4bea0d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38880 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:59:19,341 - lava_callback - INFO - Received callback for node 69c9cae33c3aafdab4bea0e6 INFO: 172.20.0.1:38882 - "POST /node/69c9cae33c3aafdab4bea0e6 HTTP/1.0" 202 Accepted 2026-04-01 00:59:19,350 - lava_callback - INFO - Starting async job submit for node 69c9cae33c3aafdab4bea0e6 2026-04-01 00:59:19,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:19,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cae33c3aafdab4bea0e6 HTTP/1.1" 200 None 2026-04-01 00:59:19,382 - lava_callback - DEBUG - Processing node 69c9cae33c3aafdab4bea0e6: baseline-arm-baylibre 2026-04-01 00:59:19,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cae33c3aafdab4bea0e6 2026-04-01 00:59:19,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:19,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cae33c3aafdab4bea0e6 HTTP/1.1" 200 None 2026-04-01 00:59:19,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:19,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 00:59:19,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:19,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cae33c3aafdab4bea0e6 HTTP/1.1" 200 None 2026-04-01 00:59:19,472 - lava_callback - ERROR - Error processing callback for node 69c9cae33c3aafdab4bea0e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56118 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:59:59,366 - lava_callback - INFO - Received callback for node 69c9cb073c3aafdab4bea0f2 INFO: 172.20.0.1:56128 - "POST /node/69c9cb073c3aafdab4bea0f2 HTTP/1.0" 202 Accepted 2026-04-01 00:59:59,372 - lava_callback - INFO - Starting async job submit for node 69c9cb073c3aafdab4bea0f2 2026-04-01 00:59:59,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:59,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb073c3aafdab4bea0f2 HTTP/1.1" 200 None 2026-04-01 00:59:59,394 - lava_callback - DEBUG - Processing node 69c9cb073c3aafdab4bea0f2: baseline-arm-baylibre 2026-04-01 00:59:59,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cb073c3aafdab4bea0f2 2026-04-01 00:59:59,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:59,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb073c3aafdab4bea0f2 HTTP/1.1" 200 None 2026-04-01 00:59:59,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:59,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 00:59:59,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:59:59,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cb073c3aafdab4bea0f2 HTTP/1.1" 200 None 2026-04-01 00:59:59,463 - lava_callback - ERROR - Error processing callback for node 69c9cb073c3aafdab4bea0f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET /node/699356ef523507f3806a8ebc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:00:47,944 - lava_callback - INFO - Received callback for node 69c9cb2f3c3aafdab4bea104 INFO: 172.20.0.1:42986 - "POST /node/69c9cb2f3c3aafdab4bea104 HTTP/1.0" 202 Accepted 2026-04-01 01:00:47,952 - lava_callback - INFO - Starting async job submit for node 69c9cb2f3c3aafdab4bea104 2026-04-01 01:00:47,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:00:47,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb2f3c3aafdab4bea104 HTTP/1.1" 200 None 2026-04-01 01:00:47,975 - lava_callback - DEBUG - Processing node 69c9cb2f3c3aafdab4bea104: baseline-arm-clabbe 2026-04-01 01:00:47,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cb2f3c3aafdab4bea104 2026-04-01 01:00:47,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:00:47,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb2f3c3aafdab4bea104 HTTP/1.1" 200 None 2026-04-01 01:00:47,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:00:48,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 01:00:48,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:00:48,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cb2f3c3aafdab4bea104 HTTP/1.1" 200 None 2026-04-01 01:00:48,047 - lava_callback - ERROR - Error processing callback for node 69c9cb2f3c3aafdab4bea104: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49832 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:01:19,462 - lava_callback - INFO - Received callback for node 69c9cb523c3aafdab4bea10b INFO: 172.20.0.1:49844 - "POST /node/69c9cb523c3aafdab4bea10b HTTP/1.0" 202 Accepted 2026-04-01 01:01:19,468 - lava_callback - INFO - Starting async job submit for node 69c9cb523c3aafdab4bea10b 2026-04-01 01:01:19,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:19,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb523c3aafdab4bea10b HTTP/1.1" 200 None 2026-04-01 01:01:19,490 - lava_callback - DEBUG - Processing node 69c9cb523c3aafdab4bea10b: baseline-arm-baylibre 2026-04-01 01:01:19,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cb523c3aafdab4bea10b 2026-04-01 01:01:19,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:19,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb523c3aafdab4bea10b HTTP/1.1" 200 None 2026-04-01 01:01:19,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:19,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 01:01:19,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:19,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cb523c3aafdab4bea10b HTTP/1.1" 200 None 2026-04-01 01:01:19,569 - lava_callback - ERROR - Error processing callback for node 69c9cb523c3aafdab4bea10b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55674 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:01:59,444 - lava_callback - INFO - Received callback for node 69c9cb763c3aafdab4bea11a INFO: 172.20.0.1:55682 - "POST /node/69c9cb763c3aafdab4bea11a HTTP/1.0" 202 Accepted 2026-04-01 01:01:59,449 - lava_callback - INFO - Starting async job submit for node 69c9cb763c3aafdab4bea11a 2026-04-01 01:01:59,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:59,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb763c3aafdab4bea11a HTTP/1.1" 200 None 2026-04-01 01:01:59,466 - lava_callback - DEBUG - Processing node 69c9cb763c3aafdab4bea11a: baseline-arm-baylibre 2026-04-01 01:01:59,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cb763c3aafdab4bea11a 2026-04-01 01:01:59,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:59,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb763c3aafdab4bea11a HTTP/1.1" 200 None 2026-04-01 01:01:59,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:59,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 01:01:59,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:01:59,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cb763c3aafdab4bea11a HTTP/1.1" 200 None 2026-04-01 01:01:59,522 - lava_callback - ERROR - Error processing callback for node 69c9cb763c3aafdab4bea11a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET /node/6988c1079fc35f969e1edaa9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45404 - "GET /node/693b76b941fb4c9509761c32%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:33900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33902 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:02:19,494 - lava_callback - INFO - Received callback for node 69c9cb983c3aafdab4bea126 INFO: 172.20.0.1:33916 - "POST /node/69c9cb983c3aafdab4bea126 HTTP/1.0" 202 Accepted 2026-04-01 01:02:19,504 - lava_callback - INFO - Starting async job submit for node 69c9cb983c3aafdab4bea126 2026-04-01 01:02:19,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:02:19,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb983c3aafdab4bea126 HTTP/1.1" 200 None 2026-04-01 01:02:19,526 - lava_callback - DEBUG - Processing node 69c9cb983c3aafdab4bea126: baseline-arm-baylibre 2026-04-01 01:02:19,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cb983c3aafdab4bea126 2026-04-01 01:02:19,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:02:19,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cb983c3aafdab4bea126 HTTP/1.1" 200 None 2026-04-01 01:02:19,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:02:19,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:02:19,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:02:19,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cb983c3aafdab4bea126 HTTP/1.1" 200 None 2026-04-01 01:02:19,605 - lava_callback - ERROR - Error processing callback for node 69c9cb983c3aafdab4bea126: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37944 - "GET /node/69877842e31b58f834f1ad97%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:03:08,046 - lava_callback - INFO - Received callback for node 69c9cbc13c3aafdab4bea137 INFO: 172.20.0.1:54108 - "POST /node/69c9cbc13c3aafdab4bea137 HTTP/1.0" 202 Accepted 2026-04-01 01:03:08,056 - lava_callback - INFO - Starting async job submit for node 69c9cbc13c3aafdab4bea137 2026-04-01 01:03:08,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:08,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cbc13c3aafdab4bea137 HTTP/1.1" 200 None 2026-04-01 01:03:08,088 - lava_callback - DEBUG - Processing node 69c9cbc13c3aafdab4bea137: baseline-arm-clabbe 2026-04-01 01:03:08,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cbc13c3aafdab4bea137 2026-04-01 01:03:08,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:08,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cbc13c3aafdab4bea137 HTTP/1.1" 200 None 2026-04-01 01:03:08,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:08,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 01:03:08,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:08,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cbc13c3aafdab4bea137 HTTP/1.1" 200 None 2026-04-01 01:03:08,193 - lava_callback - ERROR - Error processing callback for node 69c9cbc13c3aafdab4bea137: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57638 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:03:48,011 - lava_callback - INFO - Received callback for node 69c9cbe63c3aafdab4bea13f INFO: 172.20.0.1:57652 - "POST /node/69c9cbe63c3aafdab4bea13f HTTP/1.0" 202 Accepted 2026-04-01 01:03:48,020 - lava_callback - INFO - Starting async job submit for node 69c9cbe63c3aafdab4bea13f 2026-04-01 01:03:48,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:48,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cbe63c3aafdab4bea13f HTTP/1.1" 200 None 2026-04-01 01:03:48,052 - lava_callback - DEBUG - Processing node 69c9cbe63c3aafdab4bea13f: baseline-arm-clabbe 2026-04-01 01:03:48,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cbe63c3aafdab4bea13f 2026-04-01 01:03:48,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:48,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cbe63c3aafdab4bea13f HTTP/1.1" 200 None 2026-04-01 01:03:48,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:48,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 01:03:48,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:03:48,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cbe63c3aafdab4bea13f HTTP/1.1" 200 None 2026-04-01 01:03:48,144 - lava_callback - ERROR - Error processing callback for node 69c9cbe63c3aafdab4bea13f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:04:28,076 - lava_callback - INFO - Received callback for node 69c9cc0a3c3aafdab4bea149 INFO: 172.20.0.1:33052 - "POST /node/69c9cc0a3c3aafdab4bea149 HTTP/1.0" 202 Accepted 2026-04-01 01:04:28,082 - lava_callback - INFO - Starting async job submit for node 69c9cc0a3c3aafdab4bea149 2026-04-01 01:04:28,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:04:28,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc0a3c3aafdab4bea149 HTTP/1.1" 200 None 2026-04-01 01:04:28,098 - lava_callback - DEBUG - Processing node 69c9cc0a3c3aafdab4bea149: baseline-arm-clabbe 2026-04-01 01:04:28,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cc0a3c3aafdab4bea149 2026-04-01 01:04:28,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:04:28,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc0a3c3aafdab4bea149 HTTP/1.1" 200 None 2026-04-01 01:04:28,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:04:28,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 01:04:28,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:04:28,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cc0a3c3aafdab4bea149 HTTP/1.1" 200 None 2026-04-01 01:04:28,174 - lava_callback - ERROR - Error processing callback for node 69c9cc0a3c3aafdab4bea149: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:05:08,082 - lava_callback - INFO - Received callback for node 69c9cc303c3aafdab4bea153 INFO: 172.20.0.1:42010 - "POST /node/69c9cc303c3aafdab4bea153 HTTP/1.0" 202 Accepted 2026-04-01 01:05:08,088 - lava_callback - INFO - Starting async job submit for node 69c9cc303c3aafdab4bea153 2026-04-01 01:05:08,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:08,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc303c3aafdab4bea153 HTTP/1.1" 200 None 2026-04-01 01:05:08,104 - lava_callback - DEBUG - Processing node 69c9cc303c3aafdab4bea153: baseline-arm-clabbe 2026-04-01 01:05:08,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cc303c3aafdab4bea153 2026-04-01 01:05:08,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:08,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc303c3aafdab4bea153 HTTP/1.1" 200 None 2026-04-01 01:05:08,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:08,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 01:05:08,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:08,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cc303c3aafdab4bea153 HTTP/1.1" 200 None 2026-04-01 01:05:08,164 - lava_callback - ERROR - Error processing callback for node 69c9cc303c3aafdab4bea153: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60194 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:05:28,153 - lava_callback - INFO - Received callback for node 69c9cc553c3aafdab4bea15d INFO: 172.20.0.1:60196 - "POST /node/69c9cc553c3aafdab4bea15d HTTP/1.0" 202 Accepted 2026-04-01 01:05:28,162 - lava_callback - INFO - Starting async job submit for node 69c9cc553c3aafdab4bea15d 2026-04-01 01:05:28,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:28,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc553c3aafdab4bea15d HTTP/1.1" 200 None 2026-04-01 01:05:28,185 - lava_callback - DEBUG - Processing node 69c9cc553c3aafdab4bea15d: baseline-arm-clabbe 2026-04-01 01:05:28,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cc553c3aafdab4bea15d 2026-04-01 01:05:28,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:28,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc553c3aafdab4bea15d HTTP/1.1" 200 None 2026-04-01 01:05:28,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:28,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:05:28,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:05:28,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cc553c3aafdab4bea15d HTTP/1.1" 200 None 2026-04-01 01:05:28,276 - lava_callback - ERROR - Error processing callback for node 69c9cc553c3aafdab4bea15d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37520 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:06:39,825 - lava_callback - INFO - Received callback for node 69c9cc993c3aafdab4bea172 INFO: 172.20.0.1:45638 - "POST /node/69c9cc993c3aafdab4bea172 HTTP/1.0" 202 Accepted 2026-04-01 01:06:39,831 - lava_callback - INFO - Starting async job submit for node 69c9cc993c3aafdab4bea172 2026-04-01 01:06:39,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:06:39,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc993c3aafdab4bea172 HTTP/1.1" 200 None 2026-04-01 01:06:39,847 - lava_callback - DEBUG - Processing node 69c9cc993c3aafdab4bea172: baseline-arm-baylibre 2026-04-01 01:06:39,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cc993c3aafdab4bea172 2026-04-01 01:06:39,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:06:39,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cc993c3aafdab4bea172 HTTP/1.1" 200 None 2026-04-01 01:06:39,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:06:39,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:06:39,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:06:39,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cc993c3aafdab4bea172 HTTP/1.1" 200 None 2026-04-01 01:06:39,939 - lava_callback - ERROR - Error processing callback for node 69c9cc993c3aafdab4bea172: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33454 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:07:19,747 - lava_callback - INFO - Received callback for node 69c9ccc03c3aafdab4bea17f INFO: 172.20.0.1:53386 - "POST /node/69c9ccc03c3aafdab4bea17f HTTP/1.0" 202 Accepted 2026-04-01 01:07:19,757 - lava_callback - INFO - Starting async job submit for node 69c9ccc03c3aafdab4bea17f 2026-04-01 01:07:19,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:07:19,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ccc03c3aafdab4bea17f HTTP/1.1" 200 None 2026-04-01 01:07:19,777 - lava_callback - DEBUG - Processing node 69c9ccc03c3aafdab4bea17f: baseline-arm-baylibre 2026-04-01 01:07:19,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ccc03c3aafdab4bea17f 2026-04-01 01:07:19,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:07:19,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ccc03c3aafdab4bea17f HTTP/1.1" 200 None 2026-04-01 01:07:19,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:07:19,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:07:19,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:07:19,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ccc03c3aafdab4bea17f HTTP/1.1" 200 None 2026-04-01 01:07:19,861 - lava_callback - ERROR - Error processing callback for node 69c9ccc03c3aafdab4bea17f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:08:11,271 - lava_callback - INFO - Received callback for node 69cc6fda8cd9cbcdc9459e98 INFO: 172.20.0.1:41182 - "POST /node/69cc6fda8cd9cbcdc9459e98 HTTP/1.0" 202 Accepted 2026-04-01 01:08:11,280 - lava_callback - INFO - Starting async job submit for node 69cc6fda8cd9cbcdc9459e98 2026-04-01 01:08:11,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:11,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fda8cd9cbcdc9459e98 HTTP/1.1" 200 None 2026-04-01 01:08:11,340 - lava_callback - DEBUG - Processing node 69cc6fda8cd9cbcdc9459e98: baseline-arm64 2026-04-01 01:08:11,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:11,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:11,979 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fda8cd9cbcdc9459e98/log.txt.gz 2026-04-01 01:08:12,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:12,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:12,580 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fda8cd9cbcdc9459e98/lava-logs.yaml 2026-04-01 01:08:12,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:12,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:12,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fda8cd9cbcdc9459e98/lava_callback.json.gz 2026-04-01 01:08:12,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:12,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fda8cd9cbcdc9459e98 HTTP/1.1" 200 None 2026-04-01 01:08:12,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:12,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:08:12,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:12,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fda8cd9cbcdc9459e98 HTTP/1.1" 200 None 2026-04-01 01:08:12,909 - lava_callback - INFO - Completed processing callback for node 69cc6fda8cd9cbcdc9459e98 2026-04-01 01:08:14,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:14,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41202 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:08:24,846 - lava_callback - INFO - Received callback for node 69cc6fe88cd9cbcdc9459eba INFO: 172.20.0.1:37284 - "POST /node/69cc6fe88cd9cbcdc9459eba HTTP/1.0" 202 Accepted 2026-04-01 01:08:24,855 - lava_callback - INFO - Starting async job submit for node 69cc6fe88cd9cbcdc9459eba 2026-04-01 01:08:24,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:24,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe88cd9cbcdc9459eba HTTP/1.1" 200 None 2026-04-01 01:08:24,953 - lava_callback - DEBUG - Processing node 69cc6fe88cd9cbcdc9459eba: baseline-arm64 2026-04-01 01:08:25,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:25,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:25,740 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe88cd9cbcdc9459eba/log.txt.gz 2026-04-01 01:08:25,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:26,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:26,230 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe88cd9cbcdc9459eba/lava-logs.yaml 2026-04-01 01:08:26,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:37290 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:08:26,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:26,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe88cd9cbcdc9459eba/lava_callback.json.gz 2026-04-01 01:08:26,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:26,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe88cd9cbcdc9459eba HTTP/1.1" 200 None 2026-04-01 01:08:26,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:26,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:08:26,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:26,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe88cd9cbcdc9459eba HTTP/1.1" 200 None 2026-04-01 01:08:26,967 - lava_callback - INFO - Completed processing callback for node 69cc6fe88cd9cbcdc9459eba 2026-04-01 01:08:28,055 - lava_callback - INFO - Received callback for node 69cc6ffb8cd9cbcdc9459efc INFO: 172.20.0.1:37296 - "POST /node/69cc6ffb8cd9cbcdc9459efc HTTP/1.0" 202 Accepted 2026-04-01 01:08:28,064 - lava_callback - INFO - Starting async job submit for node 69cc6ffb8cd9cbcdc9459efc 2026-04-01 01:08:28,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:28,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ffb8cd9cbcdc9459efc HTTP/1.1" 200 None 2026-04-01 01:08:28,093 - lava_callback - DEBUG - Processing node 69cc6ffb8cd9cbcdc9459efc: baseline-nfs-arm64 2026-04-01 01:08:28,096 - lava_callback - WARNING - Failed to upload log 2026-04-01 01:08:28,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:28,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:28,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffb8cd9cbcdc9459efc/lava-logs.yaml 2026-04-01 01:08:28,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:28,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:28,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffb8cd9cbcdc9459efc/lava_callback.json.gz 2026-04-01 01:08:28,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:28,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ffb8cd9cbcdc9459efc HTTP/1.1" 200 None 2026-04-01 01:08:28,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:28,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:08:28,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:28,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ffb8cd9cbcdc9459efc HTTP/1.1" 200 None 2026-04-01 01:08:28,738 - lava_callback - INFO - Completed processing callback for node 69cc6ffb8cd9cbcdc9459efc INFO: 172.20.0.1:43886 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:08:40,406 - lava_callback - INFO - Received callback for node 69cc6fde8cd9cbcdc9459ea5 INFO: 172.20.0.1:37026 - "POST /node/69cc6fde8cd9cbcdc9459ea5 HTTP/1.0" 202 Accepted 2026-04-01 01:08:40,418 - lava_callback - INFO - Starting async job submit for node 69cc6fde8cd9cbcdc9459ea5 2026-04-01 01:08:40,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:40,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fde8cd9cbcdc9459ea5 HTTP/1.1" 200 None 2026-04-01 01:08:40,515 - lava_callback - DEBUG - Processing node 69cc6fde8cd9cbcdc9459ea5: baseline-arm64 2026-04-01 01:08:40,772 - lava_callback - WARNING - Failed to upload log 2026-04-01 01:08:40,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:41,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:41,405 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fde8cd9cbcdc9459ea5/lava-logs.yaml 2026-04-01 01:08:41,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:41,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:41,611 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fde8cd9cbcdc9459ea5/lava_callback.json.gz 2026-04-01 01:08:41,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:41,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fde8cd9cbcdc9459ea5 HTTP/1.1" 200 None 2026-04-01 01:08:41,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:41,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:08:41,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:41,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fde8cd9cbcdc9459ea5 HTTP/1.1" 200 None 2026-04-01 01:08:41,778 - lava_callback - INFO - Completed processing callback for node 69cc6fde8cd9cbcdc9459ea5 2026-04-01 01:08:44,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:44,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37030 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:08:50,796 - lava_callback - INFO - Received callback for node 69cc6fdc8cd9cbcdc9459e9f INFO: 172.20.0.1:55440 - "POST /node/69cc6fdc8cd9cbcdc9459e9f HTTP/1.0" 202 Accepted 2026-04-01 01:08:50,806 - lava_callback - INFO - Starting async job submit for node 69cc6fdc8cd9cbcdc9459e9f 2026-04-01 01:08:50,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:50,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fdc8cd9cbcdc9459e9f HTTP/1.1" 200 None 2026-04-01 01:08:50,885 - lava_callback - DEBUG - Processing node 69cc6fdc8cd9cbcdc9459e9f: baseline-arm64 2026-04-01 01:08:51,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:52,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:52,014 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fdc8cd9cbcdc9459e9f/log.txt.gz 2026-04-01 01:08:52,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:52,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:52,901 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fdc8cd9cbcdc9459e9f/lava-logs.yaml 2026-04-01 01:08:52,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:08:53,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:08:53,069 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fdc8cd9cbcdc9459e9f/lava_callback.json.gz 2026-04-01 01:08:53,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:53,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fdc8cd9cbcdc9459e9f HTTP/1.1" 200 None 2026-04-01 01:08:53,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:53,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:08:53,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:08:53,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fdc8cd9cbcdc9459e9f HTTP/1.1" 200 None 2026-04-01 01:08:53,228 - lava_callback - INFO - Completed processing callback for node 69cc6fdc8cd9cbcdc9459e9f INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41678 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:09:03,424 - lava_callback - INFO - Received callback for node 69cc6fdd8cd9cbcdc9459ea2 INFO: 172.20.0.1:41682 - "POST /node/69cc6fdd8cd9cbcdc9459ea2 HTTP/1.0" 202 Accepted 2026-04-01 01:09:03,431 - lava_callback - INFO - Starting async job submit for node 69cc6fdd8cd9cbcdc9459ea2 2026-04-01 01:09:03,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:03,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fdd8cd9cbcdc9459ea2 HTTP/1.1" 200 None 2026-04-01 01:09:03,469 - lava_callback - DEBUG - Processing node 69cc6fdd8cd9cbcdc9459ea2: baseline-arm64 2026-04-01 01:09:03,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:03,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:03,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fdd8cd9cbcdc9459ea2/log.txt.gz 2026-04-01 01:09:04,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:04,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:04,318 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fdd8cd9cbcdc9459ea2/lava-logs.yaml 2026-04-01 01:09:04,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:04,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:04,476 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fdd8cd9cbcdc9459ea2/lava_callback.json.gz 2026-04-01 01:09:04,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:04,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fdd8cd9cbcdc9459ea2 HTTP/1.1" 200 None 2026-04-01 01:09:04,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:04,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:04,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:04,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fdd8cd9cbcdc9459ea2 HTTP/1.1" 200 None 2026-04-01 01:09:04,569 - lava_callback - INFO - Completed processing callback for node 69cc6fdd8cd9cbcdc9459ea2 2026-04-01 01:09:05,259 - lava_callback - INFO - Received callback for node 69cc6fec8cd9cbcdc9459ede INFO: 172.20.0.1:41694 - "POST /node/69cc6fec8cd9cbcdc9459ede HTTP/1.0" 202 Accepted 2026-04-01 01:09:05,266 - lava_callback - INFO - Starting async job submit for node 69cc6fec8cd9cbcdc9459ede 2026-04-01 01:09:05,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:05,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fec8cd9cbcdc9459ede HTTP/1.1" 200 None 2026-04-01 01:09:05,304 - lava_callback - DEBUG - Processing node 69cc6fec8cd9cbcdc9459ede: baseline-arm64 2026-04-01 01:09:05,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:05,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:05,710 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fec8cd9cbcdc9459ede/log.txt.gz 2026-04-01 01:09:05,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:06,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:06,083 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fec8cd9cbcdc9459ede/lava-logs.yaml 2026-04-01 01:09:06,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:06,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:06,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fec8cd9cbcdc9459ede/lava_callback.json.gz 2026-04-01 01:09:06,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:06,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fec8cd9cbcdc9459ede HTTP/1.1" 200 None 2026-04-01 01:09:06,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:06,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:06,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:06,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fec8cd9cbcdc9459ede HTTP/1.1" 200 None 2026-04-01 01:09:06,327 - lava_callback - INFO - Completed processing callback for node 69cc6fec8cd9cbcdc9459ede INFO: 172.20.0.1:41710 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:09:08,174 - lava_callback - INFO - Received callback for node 69c9cd313c3aafdab4bea1a2 INFO: 172.20.0.1:41722 - "POST /node/69c9cd313c3aafdab4bea1a2 HTTP/1.0" 202 Accepted 2026-04-01 01:09:08,180 - lava_callback - INFO - Starting async job submit for node 69c9cd313c3aafdab4bea1a2 2026-04-01 01:09:08,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:08,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cd313c3aafdab4bea1a2 HTTP/1.1" 200 None 2026-04-01 01:09:08,199 - lava_callback - DEBUG - Processing node 69c9cd313c3aafdab4bea1a2: baseline-arm-clabbe 2026-04-01 01:09:08,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cd313c3aafdab4bea1a2 2026-04-01 01:09:08,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:08,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cd313c3aafdab4bea1a2 HTTP/1.1" 200 None 2026-04-01 01:09:08,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:08,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 01:09:08,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:08,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cd313c3aafdab4bea1a2 HTTP/1.1" 200 None 2026-04-01 01:09:08,250 - lava_callback - ERROR - Error processing callback for node 69c9cd313c3aafdab4bea1a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:09:13,793 - lava_callback - INFO - Received callback for node 69cc6fda8cd9cbcdc9459e9b INFO: 172.20.0.1:56346 - "POST /node/69cc6fda8cd9cbcdc9459e9b HTTP/1.0" 202 Accepted 2026-04-01 01:09:13,800 - lava_callback - INFO - Starting async job submit for node 69cc6fda8cd9cbcdc9459e9b 2026-04-01 01:09:13,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:13,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fda8cd9cbcdc9459e9b HTTP/1.1" 200 None 2026-04-01 01:09:13,842 - lava_callback - DEBUG - Processing node 69cc6fda8cd9cbcdc9459e9b: baseline-arm64 2026-04-01 01:09:14,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:14,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:14,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-01 01:09:14,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:14,222 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fda8cd9cbcdc9459e9b/log.txt.gz 2026-04-01 01:09:14,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:14,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:14,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fda8cd9cbcdc9459e9b/lava-logs.yaml 2026-04-01 01:09:14,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:14,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:14,703 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fda8cd9cbcdc9459e9b/lava_callback.json.gz 2026-04-01 01:09:14,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:14,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fda8cd9cbcdc9459e9b HTTP/1.1" 200 None 2026-04-01 01:09:14,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:14,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:14,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:14,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fda8cd9cbcdc9459e9b HTTP/1.1" 200 None 2026-04-01 01:09:14,812 - lava_callback - INFO - Completed processing callback for node 69cc6fda8cd9cbcdc9459e9b INFO: 172.20.0.1:56358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:09:31,034 - lava_callback - INFO - Received callback for node 69cc6fe08cd9cbcdc9459ea8 INFO: 172.20.0.1:36050 - "POST /node/69cc6fe08cd9cbcdc9459ea8 HTTP/1.0" 202 Accepted 2026-04-01 01:09:31,041 - lava_callback - INFO - Starting async job submit for node 69cc6fe08cd9cbcdc9459ea8 2026-04-01 01:09:31,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:31,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe08cd9cbcdc9459ea8 HTTP/1.1" 200 None 2026-04-01 01:09:31,090 - lava_callback - DEBUG - Processing node 69cc6fe08cd9cbcdc9459ea8: baseline-arm64 2026-04-01 01:09:31,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:31,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:31,420 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe08cd9cbcdc9459ea8/log.txt.gz 2026-04-01 01:09:31,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:31,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:31,696 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe08cd9cbcdc9459ea8/lava-logs.yaml 2026-04-01 01:09:31,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:31,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:31,865 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe08cd9cbcdc9459ea8/lava_callback.json.gz 2026-04-01 01:09:31,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:31,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe08cd9cbcdc9459ea8 HTTP/1.1" 200 None 2026-04-01 01:09:31,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:31,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:31,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:31,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe08cd9cbcdc9459ea8 HTTP/1.1" 200 None 2026-04-01 01:09:32,037 - lava_callback - INFO - Completed processing callback for node 69cc6fe08cd9cbcdc9459ea8 2026-04-01 01:09:34,728 - lava_callback - INFO - Received callback for node 69cc6fed8cd9cbcdc9459ee1 INFO: 172.20.0.1:36060 - "POST /node/69cc6fed8cd9cbcdc9459ee1 HTTP/1.0" 202 Accepted 2026-04-01 01:09:34,738 - lava_callback - INFO - Starting async job submit for node 69cc6fed8cd9cbcdc9459ee1 2026-04-01 01:09:34,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:34,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fed8cd9cbcdc9459ee1 HTTP/1.1" 200 None 2026-04-01 01:09:34,787 - lava_callback - DEBUG - Processing node 69cc6fed8cd9cbcdc9459ee1: baseline-arm64 2026-04-01 01:09:35,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:35,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:35,183 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fed8cd9cbcdc9459ee1/log.txt.gz 2026-04-01 01:09:35,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:35,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:35,583 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fed8cd9cbcdc9459ee1/lava-logs.yaml 2026-04-01 01:09:35,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:35,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:35,734 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fed8cd9cbcdc9459ee1/lava_callback.json.gz 2026-04-01 01:09:35,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:35,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fed8cd9cbcdc9459ee1 HTTP/1.1" 200 None 2026-04-01 01:09:35,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:35,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:35,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:35,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fed8cd9cbcdc9459ee1 HTTP/1.1" 200 None 2026-04-01 01:09:35,844 - lava_callback - INFO - Completed processing callback for node 69cc6fed8cd9cbcdc9459ee1 INFO: 172.20.0.1:36066 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:09:37,536 - lava_callback - INFO - Received callback for node 69cc6ff28cd9cbcdc9459eea INFO: 172.20.0.1:36076 - "POST /node/69cc6ff28cd9cbcdc9459eea HTTP/1.0" 202 Accepted 2026-04-01 01:09:37,544 - lava_callback - INFO - Starting async job submit for node 69cc6ff28cd9cbcdc9459eea 2026-04-01 01:09:37,549 - lava_callback - INFO - Received callback for node 69cc6fe98cd9cbcdc9459ebd INFO: 172.20.0.1:36090 - "POST /node/69cc6fe98cd9cbcdc9459ebd HTTP/1.0" 202 Accepted 2026-04-01 01:09:37,575 - lava_callback - INFO - Starting async job submit for node 69cc6fe98cd9cbcdc9459ebd 2026-04-01 01:09:37,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:37,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:37,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe98cd9cbcdc9459ebd HTTP/1.1" 200 None 2026-04-01 01:09:37,625 - lava_callback - DEBUG - Processing node 69cc6fe98cd9cbcdc9459ebd: baseline-arm64 2026-04-01 01:09:37,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff28cd9cbcdc9459eea HTTP/1.1" 200 None 2026-04-01 01:09:37,669 - lava_callback - DEBUG - Processing node 69cc6ff28cd9cbcdc9459eea: baseline-nfs-arm64 2026-04-01 01:09:38,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:38,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:38,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:38,459 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe98cd9cbcdc9459ebd/log.txt.gz 2026-04-01 01:09:38,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:38,536 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff28cd9cbcdc9459eea/log.txt.gz 2026-04-01 01:09:38,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:38,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:38,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:38,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe98cd9cbcdc9459ebd/lava-logs.yaml 2026-04-01 01:09:38,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:39,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:39,016 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff28cd9cbcdc9459eea/lava-logs.yaml 2026-04-01 01:09:39,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:39,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:39,162 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe98cd9cbcdc9459ebd/lava_callback.json.gz 2026-04-01 01:09:39,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:39,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:39,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff28cd9cbcdc9459eea/lava_callback.json.gz 2026-04-01 01:09:39,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:39,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe98cd9cbcdc9459ebd HTTP/1.1" 200 None 2026-04-01 01:09:39,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff28cd9cbcdc9459eea HTTP/1.1" 200 None 2026-04-01 01:09:39,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:39,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:39,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:39,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:39,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:39,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:39,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ff28cd9cbcdc9459eea HTTP/1.1" 200 None 2026-04-01 01:09:39,288 - lava_callback - INFO - Completed processing callback for node 69cc6ff28cd9cbcdc9459eea 2026-04-01 01:09:39,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe98cd9cbcdc9459ebd HTTP/1.1" 200 None 2026-04-01 01:09:39,293 - lava_callback - INFO - Completed processing callback for node 69cc6fe98cd9cbcdc9459ebd 2026-04-01 01:09:40,658 - lava_callback - INFO - Received callback for node 69cc6fff8cd9cbcdc9459f0c INFO: 172.20.0.1:60312 - "POST /node/69cc6fff8cd9cbcdc9459f0c HTTP/1.0" 202 Accepted 2026-04-01 01:09:40,670 - lava_callback - INFO - Starting async job submit for node 69cc6fff8cd9cbcdc9459f0c 2026-04-01 01:09:40,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:40,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fff8cd9cbcdc9459f0c HTTP/1.1" 200 None 2026-04-01 01:09:40,727 - lava_callback - DEBUG - Processing node 69cc6fff8cd9cbcdc9459f0c: baseline-nfs-arm64 2026-04-01 01:09:41,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:41,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:41,333 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6fff8cd9cbcdc9459f0c/log.txt.gz 2026-04-01 01:09:41,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:41,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:41,743 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6fff8cd9cbcdc9459f0c/lava-logs.yaml 2026-04-01 01:09:41,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:41,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:41,908 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6fff8cd9cbcdc9459f0c/lava_callback.json.gz 2026-04-01 01:09:41,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:41,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fff8cd9cbcdc9459f0c HTTP/1.1" 200 None 2026-04-01 01:09:41,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:41,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:41,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:41,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fff8cd9cbcdc9459f0c HTTP/1.1" 200 None 2026-04-01 01:09:41,987 - lava_callback - INFO - Completed processing callback for node 69cc6fff8cd9cbcdc9459f0c 2026-04-01 01:09:43,222 - lava_callback - INFO - Received callback for node 69cc6fe58cd9cbcdc9459eb4 INFO: 172.20.0.1:60326 - "POST /node/69cc6fe58cd9cbcdc9459eb4 HTTP/1.0" 202 Accepted 2026-04-01 01:09:43,228 - lava_callback - INFO - Starting async job submit for node 69cc6fe58cd9cbcdc9459eb4 2026-04-01 01:09:43,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:43,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe58cd9cbcdc9459eb4 HTTP/1.1" 200 None 2026-04-01 01:09:43,290 - lava_callback - DEBUG - Processing node 69cc6fe58cd9cbcdc9459eb4: baseline-arm64 2026-04-01 01:09:43,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:43,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:43,912 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe58cd9cbcdc9459eb4/log.txt.gz 2026-04-01 01:09:44,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:44,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:44,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-01 01:09:44,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:44,374 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe58cd9cbcdc9459eb4/lava-logs.yaml 2026-04-01 01:09:44,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:44,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:44,544 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe58cd9cbcdc9459eb4/lava_callback.json.gz 2026-04-01 01:09:44,567 - lava_callback - INFO - Received callback for node 69cc6fe48cd9cbcdc9459eb1 2026-04-01 01:09:44,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:60332 - "POST /node/69cc6fe48cd9cbcdc9459eb1 HTTP/1.0" 202 Accepted 2026-04-01 01:09:44,574 - lava_callback - INFO - Starting async job submit for node 69cc6fe48cd9cbcdc9459eb1 2026-04-01 01:09:44,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:44,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe48cd9cbcdc9459eb1 HTTP/1.1" 200 None 2026-04-01 01:09:44,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe58cd9cbcdc9459eb4 HTTP/1.1" 200 None 2026-04-01 01:09:44,616 - lava_callback - DEBUG - Processing node 69cc6fe48cd9cbcdc9459eb1: baseline-arm64 2026-04-01 01:09:44,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:44,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:44,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:44,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe58cd9cbcdc9459eb4 HTTP/1.1" 200 None 2026-04-01 01:09:44,850 - lava_callback - INFO - Completed processing callback for node 69cc6fe58cd9cbcdc9459eb4 2026-04-01 01:09:45,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:45,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:45,165 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe48cd9cbcdc9459eb1/log.txt.gz 2026-04-01 01:09:45,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:45,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:45,615 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe48cd9cbcdc9459eb1/lava-logs.yaml 2026-04-01 01:09:45,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:45,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:45,763 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe48cd9cbcdc9459eb1/lava_callback.json.gz 2026-04-01 01:09:45,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:45,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe48cd9cbcdc9459eb1 HTTP/1.1" 200 None 2026-04-01 01:09:45,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:45,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:45,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:45,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe48cd9cbcdc9459eb1 HTTP/1.1" 200 None 2026-04-01 01:09:45,872 - lava_callback - INFO - Completed processing callback for node 69cc6fe48cd9cbcdc9459eb1 INFO: 172.20.0.1:60340 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:09:47,238 - lava_callback - INFO - Received callback for node 69cc6fe28cd9cbcdc9459eae INFO: 172.20.0.1:60354 - "POST /node/69cc6fe28cd9cbcdc9459eae HTTP/1.0" 202 Accepted 2026-04-01 01:09:47,244 - lava_callback - INFO - Starting async job submit for node 69cc6fe28cd9cbcdc9459eae 2026-04-01 01:09:47,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:47,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe28cd9cbcdc9459eae HTTP/1.1" 200 None 2026-04-01 01:09:47,286 - lava_callback - DEBUG - Processing node 69cc6fe28cd9cbcdc9459eae: baseline-arm64 2026-04-01 01:09:47,466 - lava_callback - INFO - Received callback for node 69cc70268cd9cbcdc9459f7e INFO: 172.20.0.1:60366 - "POST /node/69cc70268cd9cbcdc9459f7e HTTP/1.0" 202 Accepted 2026-04-01 01:09:47,483 - lava_callback - INFO - Starting async job submit for node 69cc70268cd9cbcdc9459f7e 2026-04-01 01:09:47,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:47,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:47,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70268cd9cbcdc9459f7e HTTP/1.1" 200 None 2026-04-01 01:09:47,621 - lava_callback - DEBUG - Processing node 69cc70268cd9cbcdc9459f7e: baseline-arm64-foundriesio 2026-04-01 01:09:47,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:47,770 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe28cd9cbcdc9459eae/log.txt.gz 2026-04-01 01:09:47,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:48,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:48,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:48,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc70268cd9cbcdc9459f7e/log.txt.gz 2026-04-01 01:09:48,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:48,261 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe28cd9cbcdc9459eae/lava-logs.yaml 2026-04-01 01:09:48,286 - lava_callback - INFO - Received callback for node 69c9cd563c3aafdab4bea1af 2026-04-01 01:09:48,293 - lava_callback - INFO - Starting async job submit for node 69c9cd563c3aafdab4bea1af INFO: 172.20.0.1:60376 - "POST /node/69c9cd563c3aafdab4bea1af HTTP/1.0" 202 Accepted 2026-04-01 01:09:48,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:48,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:48,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cd563c3aafdab4bea1af HTTP/1.1" 200 None 2026-04-01 01:09:48,318 - lava_callback - DEBUG - Processing node 69c9cd563c3aafdab4bea1af: baseline-arm-clabbe 2026-04-01 01:09:48,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cd563c3aafdab4bea1af 2026-04-01 01:09:48,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cd563c3aafdab4bea1af HTTP/1.1" 200 None 2026-04-01 01:09:48,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:09:48,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cd563c3aafdab4bea1af HTTP/1.1" 200 None 2026-04-01 01:09:48,393 - lava_callback - ERROR - Error processing callback for node 69c9cd563c3aafdab4bea1af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:09:48,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:48,570 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe28cd9cbcdc9459eae/lava_callback.json.gz 2026-04-01 01:09:48,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe28cd9cbcdc9459eae HTTP/1.1" 200 None 2026-04-01 01:09:48,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:48,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,632 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc70268cd9cbcdc9459f7e/lava-logs.yaml 2026-04-01 01:09:48,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:48,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:48,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe28cd9cbcdc9459eae HTTP/1.1" 200 None 2026-04-01 01:09:48,700 - lava_callback - INFO - Completed processing callback for node 69cc6fe28cd9cbcdc9459eae 2026-04-01 01:09:48,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:48,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc70268cd9cbcdc9459f7e/lava_callback.json.gz 2026-04-01 01:09:48,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70268cd9cbcdc9459f7e HTTP/1.1" 200 None 2026-04-01 01:09:48,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:48,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:48,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70268cd9cbcdc9459f7e HTTP/1.1" 200 None 2026-04-01 01:09:48,977 - lava_callback - INFO - Completed processing callback for node 69cc70268cd9cbcdc9459f7e 2026-04-01 01:09:53,221 - lava_callback - INFO - Received callback for node 69cc70018cd9cbcdc9459f0f INFO: 172.20.0.1:40148 - "POST /node/69cc70018cd9cbcdc9459f0f HTTP/1.0" 202 Accepted 2026-04-01 01:09:53,227 - lava_callback - INFO - Starting async job submit for node 69cc70018cd9cbcdc9459f0f 2026-04-01 01:09:53,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:53,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70018cd9cbcdc9459f0f HTTP/1.1" 200 None 2026-04-01 01:09:53,264 - lava_callback - DEBUG - Processing node 69cc70018cd9cbcdc9459f0f: baseline-nfs-arm64 2026-04-01 01:09:53,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:53,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:53,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc70018cd9cbcdc9459f0f/log.txt.gz 2026-04-01 01:09:53,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:54,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:54,197 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc70018cd9cbcdc9459f0f/lava-logs.yaml 2026-04-01 01:09:54,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:54,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:54,367 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc70018cd9cbcdc9459f0f/lava_callback.json.gz 2026-04-01 01:09:54,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:54,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70018cd9cbcdc9459f0f HTTP/1.1" 200 None 2026-04-01 01:09:54,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:54,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:54,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:54,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70018cd9cbcdc9459f0f HTTP/1.1" 200 None 2026-04-01 01:09:54,486 - lava_callback - INFO - Completed processing callback for node 69cc70018cd9cbcdc9459f0f 2026-04-01 01:09:55,464 - lava_callback - INFO - Received callback for node 69cc6ff98cd9cbcdc9459ef9 INFO: 172.20.0.1:40154 - "POST /node/69cc6ff98cd9cbcdc9459ef9 HTTP/1.0" 202 Accepted 2026-04-01 01:09:55,472 - lava_callback - INFO - Starting async job submit for node 69cc6ff98cd9cbcdc9459ef9 2026-04-01 01:09:55,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:55,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff98cd9cbcdc9459ef9 HTTP/1.1" 200 None 2026-04-01 01:09:55,513 - lava_callback - DEBUG - Processing node 69cc6ff98cd9cbcdc9459ef9: baseline-nfs-arm64 2026-04-01 01:09:56,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:56,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:56,163 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff98cd9cbcdc9459ef9/log.txt.gz 2026-04-01 01:09:56,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:09:56,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:56,760 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff98cd9cbcdc9459ef9/lava-logs.yaml 2026-04-01 01:09:56,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:56,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:56,961 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff98cd9cbcdc9459ef9/lava_callback.json.gz 2026-04-01 01:09:56,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:57,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff98cd9cbcdc9459ef9 HTTP/1.1" 200 None 2026-04-01 01:09:57,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:57,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:57,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:57,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ff98cd9cbcdc9459ef9 HTTP/1.1" 200 None 2026-04-01 01:09:57,082 - lava_callback - INFO - Completed processing callback for node 69cc6ff98cd9cbcdc9459ef9 2026-04-01 01:09:58,230 - lava_callback - INFO - Received callback for node 69cc70218cd9cbcdc9459f6d INFO: 172.20.0.1:40178 - "POST /node/69cc70218cd9cbcdc9459f6d HTTP/1.0" 202 Accepted 2026-04-01 01:09:58,237 - lava_callback - INFO - Starting async job submit for node 69cc70218cd9cbcdc9459f6d 2026-04-01 01:09:58,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:58,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70218cd9cbcdc9459f6d HTTP/1.1" 200 None 2026-04-01 01:09:58,272 - lava_callback - DEBUG - Processing node 69cc70218cd9cbcdc9459f6d: baseline-arm64-pengutronix 2026-04-01 01:09:58,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:58,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:58,645 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70218cd9cbcdc9459f6d/log.txt.gz 2026-04-01 01:09:58,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:58,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:58,935 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70218cd9cbcdc9459f6d/lava-logs.yaml 2026-04-01 01:09:58,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:09:59,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:09:59,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70218cd9cbcdc9459f6d/lava_callback.json.gz 2026-04-01 01:09:59,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:59,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70218cd9cbcdc9459f6d HTTP/1.1" 200 None 2026-04-01 01:09:59,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:59,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:09:59,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:09:59,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70218cd9cbcdc9459f6d HTTP/1.1" 200 None 2026-04-01 01:09:59,162 - lava_callback - INFO - Completed processing callback for node 69cc70218cd9cbcdc9459f6d INFO: 172.20.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:10:06,978 - lava_callback - INFO - Received callback for node 69cc70238cd9cbcdc9459f70 INFO: 172.20.0.1:53180 - "POST /node/69cc70238cd9cbcdc9459f70 HTTP/1.0" 202 Accepted 2026-04-01 01:10:06,984 - lava_callback - INFO - Starting async job submit for node 69cc70238cd9cbcdc9459f70 2026-04-01 01:10:07,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:07,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70238cd9cbcdc9459f70 HTTP/1.1" 200 None 2026-04-01 01:10:07,024 - lava_callback - DEBUG - Processing node 69cc70238cd9cbcdc9459f70: baseline-arm64-pengutronix 2026-04-01 01:10:07,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:07,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:07,359 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70238cd9cbcdc9459f70/log.txt.gz 2026-04-01 01:10:07,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:07,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:07,691 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70238cd9cbcdc9459f70/lava-logs.yaml 2026-04-01 01:10:07,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:07,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:07,867 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70238cd9cbcdc9459f70/lava_callback.json.gz 2026-04-01 01:10:07,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:07,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70238cd9cbcdc9459f70 HTTP/1.1" 200 None 2026-04-01 01:10:07,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:07,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:07,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:07,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70238cd9cbcdc9459f70 HTTP/1.1" 200 None 2026-04-01 01:10:07,973 - lava_callback - INFO - Completed processing callback for node 69cc70238cd9cbcdc9459f70 2026-04-01 01:10:10,881 - lava_callback - INFO - Received callback for node 69cc6ff88cd9cbcdc9459ef6 INFO: 172.20.0.1:52942 - "POST /node/69cc6ff88cd9cbcdc9459ef6 HTTP/1.0" 202 Accepted 2026-04-01 01:10:10,891 - lava_callback - INFO - Starting async job submit for node 69cc6ff88cd9cbcdc9459ef6 2026-04-01 01:10:10,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:10,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff88cd9cbcdc9459ef6 HTTP/1.1" 200 None 2026-04-01 01:10:10,931 - lava_callback - DEBUG - Processing node 69cc6ff88cd9cbcdc9459ef6: baseline-nfs-arm64 2026-04-01 01:10:11,005 - lava_callback - INFO - Received callback for node 69cc6fef8cd9cbcdc9459ee4 INFO: 172.20.0.1:52948 - "POST /node/69cc6fef8cd9cbcdc9459ee4 HTTP/1.0" 202 Accepted 2026-04-01 01:10:11,022 - lava_callback - INFO - Starting async job submit for node 69cc6fef8cd9cbcdc9459ee4 2026-04-01 01:10:11,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:11,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fef8cd9cbcdc9459ee4 HTTP/1.1" 200 None 2026-04-01 01:10:11,158 - lava_callback - DEBUG - Processing node 69cc6fef8cd9cbcdc9459ee4: baseline-arm64 2026-04-01 01:10:11,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:11,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:12,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:12,086 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff88cd9cbcdc9459ef6/log.txt.gz 2026-04-01 01:10:12,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:12,192 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fef8cd9cbcdc9459ee4/log.txt.gz 2026-04-01 01:10:12,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:12,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:12,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:12,778 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff88cd9cbcdc9459ef6/lava-logs.yaml 2026-04-01 01:10:12,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:12,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:12,833 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fef8cd9cbcdc9459ee4/lava-logs.yaml 2026-04-01 01:10:12,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:12,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:12,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff88cd9cbcdc9459ef6/lava_callback.json.gz 2026-04-01 01:10:12,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:12,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:12,979 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fef8cd9cbcdc9459ee4/lava_callback.json.gz 2026-04-01 01:10:13,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:13,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff88cd9cbcdc9459ef6 HTTP/1.1" 200 None 2026-04-01 01:10:13,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:13,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fef8cd9cbcdc9459ee4 HTTP/1.1" 200 None 2026-04-01 01:10:13,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:13,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:13,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:13,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:13,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:13,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ff88cd9cbcdc9459ef6 HTTP/1.1" 200 None 2026-04-01 01:10:13,084 - lava_callback - INFO - Completed processing callback for node 69cc6ff88cd9cbcdc9459ef6 2026-04-01 01:10:13,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fef8cd9cbcdc9459ee4 HTTP/1.1" 200 None 2026-04-01 01:10:13,190 - lava_callback - INFO - Completed processing callback for node 69cc6fef8cd9cbcdc9459ee4 2026-04-01 01:10:13,250 - lava_callback - INFO - Received callback for node 69cc70028cd9cbcdc9459f12 INFO: 172.20.0.1:52952 - "POST /node/69cc70028cd9cbcdc9459f12 HTTP/1.0" 202 Accepted 2026-04-01 01:10:13,261 - lava_callback - INFO - Starting async job submit for node 69cc70028cd9cbcdc9459f12 2026-04-01 01:10:13,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:13,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70028cd9cbcdc9459f12 HTTP/1.1" 200 None 2026-04-01 01:10:13,357 - lava_callback - DEBUG - Processing node 69cc70028cd9cbcdc9459f12: baseline-nfs-arm64 2026-04-01 01:10:13,872 - lava_callback - INFO - Received callback for node 69cc70258cd9cbcdc9459f73 INFO: 172.20.0.1:52962 - "POST /node/69cc70258cd9cbcdc9459f73 HTTP/1.0" 202 Accepted 2026-04-01 01:10:13,881 - lava_callback - INFO - Starting async job submit for node 69cc70258cd9cbcdc9459f73 2026-04-01 01:10:13,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:13,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:13,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70258cd9cbcdc9459f73 HTTP/1.1" 200 None 2026-04-01 01:10:13,929 - lava_callback - DEBUG - Processing node 69cc70258cd9cbcdc9459f73: baseline-arm64-pengutronix 2026-04-01 01:10:14,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:14,185 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc70028cd9cbcdc9459f12/log.txt.gz 2026-04-01 01:10:14,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:14,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:14,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-01 01:10:14,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:14,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:14,491 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70258cd9cbcdc9459f73/log.txt.gz 2026-04-01 01:10:14,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:14,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:14,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc70028cd9cbcdc9459f12/lava-logs.yaml 2026-04-01 01:10:14,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:14,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:14,948 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70258cd9cbcdc9459f73/lava-logs.yaml 2026-04-01 01:10:14,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:15,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:15,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc70028cd9cbcdc9459f12/lava_callback.json.gz 2026-04-01 01:10:15,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:15,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70028cd9cbcdc9459f12 HTTP/1.1" 200 None 2026-04-01 01:10:15,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:15,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:15,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:15,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70028cd9cbcdc9459f12 HTTP/1.1" 200 None 2026-04-01 01:10:15,098 - lava_callback - INFO - Completed processing callback for node 69cc70028cd9cbcdc9459f12 2026-04-01 01:10:15,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:15,144 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cc70258cd9cbcdc9459f73/lava_callback.json.gz INFO: 172.20.0.1:52978 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 01:10:15,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:15,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70258cd9cbcdc9459f73 HTTP/1.1" 200 None 2026-04-01 01:10:15,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:15,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:15,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:15,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70258cd9cbcdc9459f73 HTTP/1.1" 200 None 2026-04-01 01:10:15,244 - lava_callback - INFO - Completed processing callback for node 69cc70258cd9cbcdc9459f73 INFO: 172.20.0.1:52994 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:10:19,040 - lava_callback - INFO - Received callback for node 69cc701d8cd9cbcdc9459f64 INFO: 172.20.0.1:52998 - "POST /node/69cc701d8cd9cbcdc9459f64 HTTP/1.0" 202 Accepted 2026-04-01 01:10:19,047 - lava_callback - INFO - Starting async job submit for node 69cc701d8cd9cbcdc9459f64 2026-04-01 01:10:19,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:19,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701d8cd9cbcdc9459f64 HTTP/1.1" 200 None 2026-04-01 01:10:19,104 - lava_callback - DEBUG - Processing node 69cc701d8cd9cbcdc9459f64: baseline-arm64-clabbe 2026-04-01 01:10:19,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:19,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:19,526 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cc701d8cd9cbcdc9459f64/log.txt.gz 2026-04-01 01:10:19,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:19,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:19,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cc701d8cd9cbcdc9459f64/lava-logs.yaml 2026-04-01 01:10:19,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:20,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:20,039 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cc701d8cd9cbcdc9459f64/lava_callback.json.gz 2026-04-01 01:10:20,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:20,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701d8cd9cbcdc9459f64 HTTP/1.1" 200 None 2026-04-01 01:10:20,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:20,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:20,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:20,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc701d8cd9cbcdc9459f64 HTTP/1.1" 200 None 2026-04-01 01:10:20,145 - lava_callback - INFO - Completed processing callback for node 69cc701d8cd9cbcdc9459f64 INFO: 172.20.0.1:51566 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:10:33,421 - lava_callback - INFO - Received callback for node 69cc6ffc8cd9cbcdc9459eff INFO: 172.20.0.1:40320 - "POST /node/69cc6ffc8cd9cbcdc9459eff HTTP/1.0" 202 Accepted 2026-04-01 01:10:33,428 - lava_callback - INFO - Starting async job submit for node 69cc6ffc8cd9cbcdc9459eff 2026-04-01 01:10:33,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:33,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ffc8cd9cbcdc9459eff HTTP/1.1" 200 None 2026-04-01 01:10:33,471 - lava_callback - DEBUG - Processing node 69cc6ffc8cd9cbcdc9459eff: baseline-nfs-arm64 2026-04-01 01:10:33,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:33,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:33,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffc8cd9cbcdc9459eff/log.txt.gz 2026-04-01 01:10:34,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:34,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:34,367 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffc8cd9cbcdc9459eff/lava-logs.yaml 2026-04-01 01:10:34,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:34,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:34,548 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffc8cd9cbcdc9459eff/lava_callback.json.gz 2026-04-01 01:10:34,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:34,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ffc8cd9cbcdc9459eff HTTP/1.1" 200 None 2026-04-01 01:10:34,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:34,599 - lava_callback - INFO - Received callback for node 69cc6ff58cd9cbcdc9459ef0 INFO: 172.20.0.1:40328 - "POST /node/69cc6ff58cd9cbcdc9459ef0 HTTP/1.0" 202 Accepted 2026-04-01 01:10:34,606 - lava_callback - INFO - Starting async job submit for node 69cc6ff58cd9cbcdc9459ef0 2026-04-01 01:10:34,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:34,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:34,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:34,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff58cd9cbcdc9459ef0 HTTP/1.1" 200 None 2026-04-01 01:10:34,651 - lava_callback - DEBUG - Processing node 69cc6ff58cd9cbcdc9459ef0: baseline-nfs-arm64 2026-04-01 01:10:34,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ffc8cd9cbcdc9459eff HTTP/1.1" 200 None 2026-04-01 01:10:34,670 - lava_callback - INFO - Completed processing callback for node 69cc6ffc8cd9cbcdc9459eff 2026-04-01 01:10:35,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:35,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:35,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff58cd9cbcdc9459ef0/log.txt.gz 2026-04-01 01:10:35,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:35,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:35,699 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff58cd9cbcdc9459ef0/lava-logs.yaml 2026-04-01 01:10:35,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:35,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:35,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff58cd9cbcdc9459ef0/lava_callback.json.gz 2026-04-01 01:10:35,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:35,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff58cd9cbcdc9459ef0 HTTP/1.1" 200 None 2026-04-01 01:10:35,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:35,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:35,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:35,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ff58cd9cbcdc9459ef0 HTTP/1.1" 200 None 2026-04-01 01:10:35,976 - lava_callback - INFO - Completed processing callback for node 69cc6ff58cd9cbcdc9459ef0 INFO: 172.20.0.1:40336 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:10:37,101 - lava_callback - INFO - Received callback for node 69cc6fe78cd9cbcdc9459eb7 INFO: 172.20.0.1:40344 - "POST /node/69cc6fe78cd9cbcdc9459eb7 HTTP/1.0" 202 Accepted 2026-04-01 01:10:37,107 - lava_callback - INFO - Starting async job submit for node 69cc6fe78cd9cbcdc9459eb7 2026-04-01 01:10:37,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:37,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe78cd9cbcdc9459eb7 HTTP/1.1" 200 None 2026-04-01 01:10:37,144 - lava_callback - DEBUG - Processing node 69cc6fe78cd9cbcdc9459eb7: baseline-arm64 2026-04-01 01:10:37,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:37,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:37,733 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe78cd9cbcdc9459eb7/log.txt.gz 2026-04-01 01:10:37,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:38,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:38,215 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe78cd9cbcdc9459eb7/lava-logs.yaml 2026-04-01 01:10:38,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:10:38,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:10:38,475 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe78cd9cbcdc9459eb7/lava_callback.json.gz 2026-04-01 01:10:38,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:38,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe78cd9cbcdc9459eb7 HTTP/1.1" 200 None 2026-04-01 01:10:38,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:38,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:10:38,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:38,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe78cd9cbcdc9459eb7 HTTP/1.1" 200 None 2026-04-01 01:10:38,700 - lava_callback - INFO - Completed processing callback for node 69cc6fe78cd9cbcdc9459eb7 2026-04-01 01:10:44,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:10:44,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:54638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51138 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:11:08,318 - lava_callback - INFO - Received callback for node 69c9cd9d3c3aafdab4bea1c5 INFO: 172.20.0.1:51146 - "POST /node/69c9cd9d3c3aafdab4bea1c5 HTTP/1.0" 202 Accepted 2026-04-01 01:11:08,327 - lava_callback - INFO - Starting async job submit for node 69c9cd9d3c3aafdab4bea1c5 2026-04-01 01:11:08,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:08,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cd9d3c3aafdab4bea1c5 HTTP/1.1" 200 None 2026-04-01 01:11:08,352 - lava_callback - DEBUG - Processing node 69c9cd9d3c3aafdab4bea1c5: baseline-arm-clabbe 2026-04-01 01:11:08,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cd9d3c3aafdab4bea1c5 2026-04-01 01:11:08,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:08,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cd9d3c3aafdab4bea1c5 HTTP/1.1" 200 None 2026-04-01 01:11:08,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:08,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:11:08,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:08,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cd9d3c3aafdab4bea1c5 HTTP/1.1" 200 None 2026-04-01 01:11:08,409 - lava_callback - ERROR - Error processing callback for node 69c9cd9d3c3aafdab4bea1c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36248 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 01:11:16,424 - lava_callback - INFO - Received callback for node 69cc68838cd9cbcdc9459aee INFO: 172.20.0.1:36262 - "POST /node/69cc68838cd9cbcdc9459aee HTTP/1.0" 202 Accepted 2026-04-01 01:11:16,434 - lava_callback - INFO - Starting async job submit for node 69cc68838cd9cbcdc9459aee 2026-04-01 01:11:16,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:16,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68838cd9cbcdc9459aee HTTP/1.1" 200 None 2026-04-01 01:11:16,478 - lava_callback - DEBUG - Processing node 69cc68838cd9cbcdc9459aee: baseline-arm-broonie INFO: 172.20.0.1:36272 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:11:16,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:11:17,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:11:17,405 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68838cd9cbcdc9459aee/log.txt.gz 2026-04-01 01:11:17,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:11:17,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:11:17,804 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68838cd9cbcdc9459aee/lava-logs.yaml 2026-04-01 01:11:17,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:11:17,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:11:17,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68838cd9cbcdc9459aee/lava_callback.json.gz 2026-04-01 01:11:18,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:18,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68838cd9cbcdc9459aee HTTP/1.1" 200 None 2026-04-01 01:11:18,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:18,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 01:11:18,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:18,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68838cd9cbcdc9459aee HTTP/1.1" 200 None 2026-04-01 01:11:18,090 - lava_callback - INFO - Completed processing callback for node 69cc68838cd9cbcdc9459aee INFO: 172.20.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:11:44,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:44,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56894 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:11:48,364 - lava_callback - INFO - Received callback for node 69c9cdc13c3aafdab4bea1d3 INFO: 172.20.0.1:56908 - "POST /node/69c9cdc13c3aafdab4bea1d3 HTTP/1.0" 202 Accepted 2026-04-01 01:11:48,369 - lava_callback - INFO - Starting async job submit for node 69c9cdc13c3aafdab4bea1d3 2026-04-01 01:11:48,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:48,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cdc13c3aafdab4bea1d3 HTTP/1.1" 200 None 2026-04-01 01:11:48,385 - lava_callback - DEBUG - Processing node 69c9cdc13c3aafdab4bea1d3: baseline-arm-clabbe 2026-04-01 01:11:48,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cdc13c3aafdab4bea1d3 2026-04-01 01:11:48,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:48,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cdc13c3aafdab4bea1d3 HTTP/1.1" 200 None 2026-04-01 01:11:48,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:48,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:11:48,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:11:48,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cdc13c3aafdab4bea1d3 HTTP/1.1" 200 None 2026-04-01 01:11:48,437 - lava_callback - ERROR - Error processing callback for node 69c9cdc13c3aafdab4bea1d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40550 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 01:12:16,353 - lava_callback - INFO - Received callback for node 69cc6fe18cd9cbcdc9459eab INFO: 172.20.0.1:40562 - "POST /node/69cc6fe18cd9cbcdc9459eab HTTP/1.0" 202 Accepted 2026-04-01 01:12:16,359 - lava_callback - INFO - Starting async job submit for node 69cc6fe18cd9cbcdc9459eab 2026-04-01 01:12:16,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:16,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe18cd9cbcdc9459eab HTTP/1.1" 200 None 2026-04-01 01:12:16,406 - lava_callback - DEBUG - Processing node 69cc6fe18cd9cbcdc9459eab: baseline-arm64 INFO: 172.20.0.1:40574 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:12:16,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:12:16,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:12:16,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe18cd9cbcdc9459eab/log.txt.gz 2026-04-01 01:12:17,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:12:17,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:12:17,404 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe18cd9cbcdc9459eab/lava-logs.yaml 2026-04-01 01:12:17,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:12:17,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:12:17,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fe18cd9cbcdc9459eab/lava_callback.json.gz 2026-04-01 01:12:17,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:17,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe18cd9cbcdc9459eab HTTP/1.1" 200 None 2026-04-01 01:12:17,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:17,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:12:17,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:17,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe18cd9cbcdc9459eab HTTP/1.1" 200 None 2026-04-01 01:12:17,639 - lava_callback - INFO - Completed processing callback for node 69cc6fe18cd9cbcdc9459eab INFO: 172.20.0.1:33638 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:12:29,814 - lava_callback - INFO - Received callback for node 69cc6ff48cd9cbcdc9459eed INFO: 172.20.0.1:48364 - "POST /node/69cc6ff48cd9cbcdc9459eed HTTP/1.0" 202 Accepted 2026-04-01 01:12:29,822 - lava_callback - INFO - Starting async job submit for node 69cc6ff48cd9cbcdc9459eed 2026-04-01 01:12:29,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:29,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff48cd9cbcdc9459eed HTTP/1.1" 200 None 2026-04-01 01:12:29,866 - lava_callback - DEBUG - Processing node 69cc6ff48cd9cbcdc9459eed: baseline-nfs-arm64 2026-04-01 01:12:30,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:12:30,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:12:30,415 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff48cd9cbcdc9459eed/log.txt.gz 2026-04-01 01:12:30,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:12:30,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:12:30,855 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff48cd9cbcdc9459eed/lava-logs.yaml 2026-04-01 01:12:30,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:12:31,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:12:31,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff48cd9cbcdc9459eed/lava_callback.json.gz 2026-04-01 01:12:31,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:31,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff48cd9cbcdc9459eed HTTP/1.1" 200 None 2026-04-01 01:12:31,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:31,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:12:31,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:31,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ff48cd9cbcdc9459eed HTTP/1.1" 200 None 2026-04-01 01:12:31,174 - lava_callback - INFO - Completed processing callback for node 69cc6ff48cd9cbcdc9459eed INFO: 172.20.0.1:48368 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:12:44,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:44,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:12:48,428 - lava_callback - INFO - Received callback for node 69c9ce0b3c3aafdab4bea1e5 INFO: 172.20.0.1:47076 - "POST /node/69c9ce0b3c3aafdab4bea1e5 HTTP/1.0" 202 Accepted 2026-04-01 01:12:48,437 - lava_callback - INFO - Starting async job submit for node 69c9ce0b3c3aafdab4bea1e5 2026-04-01 01:12:48,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:48,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ce0b3c3aafdab4bea1e5 HTTP/1.1" 200 None 2026-04-01 01:12:48,457 - lava_callback - DEBUG - Processing node 69c9ce0b3c3aafdab4bea1e5: baseline-arm-clabbe 2026-04-01 01:12:48,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ce0b3c3aafdab4bea1e5 2026-04-01 01:12:48,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:48,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ce0b3c3aafdab4bea1e5 HTTP/1.1" 200 None 2026-04-01 01:12:48,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:48,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 01:12:48,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:12:48,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ce0b3c3aafdab4bea1e5 HTTP/1.1" 200 None 2026-04-01 01:12:48,523 - lava_callback - ERROR - Error processing callback for node 69c9ce0b3c3aafdab4bea1e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60444 - "GET /node/6983ed38969566dcb0d29a7c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56750 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:13:47,533 - lava_callback - INFO - Received callback for node 69cc70278cd9cbcdc9459f81 INFO: 172.20.0.1:56758 - "POST /node/69cc70278cd9cbcdc9459f81 HTTP/1.0" 202 Accepted 2026-04-01 01:13:47,542 - lava_callback - INFO - Starting async job submit for node 69cc70278cd9cbcdc9459f81 2026-04-01 01:13:47,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:13:47,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70278cd9cbcdc9459f81 HTTP/1.1" 200 None 2026-04-01 01:13:47,632 - lava_callback - DEBUG - Processing node 69cc70278cd9cbcdc9459f81: baseline-arm64-foundriesio 2026-04-01 01:13:48,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:13:49,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:13:49,063 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc70278cd9cbcdc9459f81/log.txt.gz 2026-04-01 01:13:49,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:13:50,133 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:13:50,136 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc70278cd9cbcdc9459f81/lava-logs.yaml 2026-04-01 01:13:50,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:13:50,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:13:50,390 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc70278cd9cbcdc9459f81/lava_callback.json.gz DEBUG: 69cc70278cd9cbcdc9459f81 Transitting job node result: pass -> incomplete 2026-04-01 01:13:50,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:13:50,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70278cd9cbcdc9459f81 HTTP/1.1" 200 None 2026-04-01 01:13:50,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:13:50,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:13:50,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:13:50,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70278cd9cbcdc9459f81 HTTP/1.1" 200 None 2026-04-01 01:13:50,504 - lava_callback - INFO - Completed processing callback for node 69cc70278cd9cbcdc9459f81 INFO: 172.20.0.1:39014 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:14:01,527 - lava_callback - INFO - Received callback for node 69cc6ff18cd9cbcdc9459ee7 INFO: 172.20.0.1:36080 - "POST /node/69cc6ff18cd9cbcdc9459ee7 HTTP/1.0" 202 Accepted 2026-04-01 01:14:01,534 - lava_callback - INFO - Starting async job submit for node 69cc6ff18cd9cbcdc9459ee7 2026-04-01 01:14:01,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:01,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff18cd9cbcdc9459ee7 HTTP/1.1" 200 None 2026-04-01 01:14:01,587 - lava_callback - DEBUG - Processing node 69cc6ff18cd9cbcdc9459ee7: baseline-arm64 INFO: 172.20.0.1:36092 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:14:02,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:14:02,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:14:02,124 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6ff18cd9cbcdc9459ee7/log.txt.gz 2026-04-01 01:14:02,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:14:02,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:14:02,539 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6ff18cd9cbcdc9459ee7/lava-logs.yaml 2026-04-01 01:14:02,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:14:02,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:14:02,705 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6ff18cd9cbcdc9459ee7/lava_callback.json.gz DEBUG: Setup failed for 69cc6ff18cd9cbcdc9459ee7. Transitting job node result: incomplete -> fail 2026-04-01 01:14:02,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:02,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff18cd9cbcdc9459ee7 HTTP/1.1" 200 None 2026-04-01 01:14:02,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:02,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:14:02,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:02,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ff18cd9cbcdc9459ee7 HTTP/1.1" 200 None 2026-04-01 01:14:02,932 - lava_callback - INFO - Completed processing callback for node 69cc6ff18cd9cbcdc9459ee7 INFO: 172.20.0.1:36100 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:14:14,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:14,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48902 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:14:36,673 - lava_callback - INFO - Received callback for node 69cc68bb8cd9cbcdc9459ba5 INFO: 172.20.0.1:48914 - "POST /node/69cc68bb8cd9cbcdc9459ba5 HTTP/1.0" 202 Accepted 2026-04-01 01:14:36,680 - lava_callback - INFO - Starting async job submit for node 69cc68bb8cd9cbcdc9459ba5 2026-04-01 01:14:36,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:36,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68bb8cd9cbcdc9459ba5 HTTP/1.1" 200 None 2026-04-01 01:14:36,720 - lava_callback - DEBUG - Processing node 69cc68bb8cd9cbcdc9459ba5: ltp-syscalls 2026-04-01 01:14:37,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:14:37,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:14:37,584 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cc68bb8cd9cbcdc9459ba5/log.txt.gz 2026-04-01 01:14:37,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:14:38,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:14:38,228 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cc68bb8cd9cbcdc9459ba5/lava-logs.yaml 2026-04-01 01:14:38,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:14:38,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:14:38,401 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cc68bb8cd9cbcdc9459ba5/lava_callback.json.gz 2026-04-01 01:14:38,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:38,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68bb8cd9cbcdc9459ba5 HTTP/1.1" 200 None 2026-04-01 01:14:38,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:38,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 01:14:38,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:38,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68bb8cd9cbcdc9459ba5 HTTP/1.1" 200 None 2026-04-01 01:14:38,487 - lava_callback - INFO - Completed processing callback for node 69cc68bb8cd9cbcdc9459ba5 2026-04-01 01:14:44,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:14:44,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET /node/68fb69c1c65672e54ce8226d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60958 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:15:20,374 - lava_callback - INFO - Received callback for node 69c9ce9f3c3aafdab4bea290 INFO: 172.20.0.1:59280 - "POST /node/69c9ce9f3c3aafdab4bea290 HTTP/1.0" 202 Accepted 2026-04-01 01:15:20,380 - lava_callback - INFO - Starting async job submit for node 69c9ce9f3c3aafdab4bea290 2026-04-01 01:15:20,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:20,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ce9f3c3aafdab4bea290 HTTP/1.1" 200 None 2026-04-01 01:15:20,399 - lava_callback - DEBUG - Processing node 69c9ce9f3c3aafdab4bea290: baseline-arm64-baylibre 2026-04-01 01:15:20,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ce9f3c3aafdab4bea290 2026-04-01 01:15:20,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:20,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ce9f3c3aafdab4bea290 HTTP/1.1" 200 None 2026-04-01 01:15:20,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:20,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:20,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:20,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ce9f3c3aafdab4bea290 HTTP/1.1" 200 None 2026-04-01 01:15:20,461 - lava_callback - ERROR - Error processing callback for node 69c9ce9f3c3aafdab4bea290: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:21,065 - lava_callback - INFO - Received callback for node 69c9cea03c3aafdab4bea297 INFO: 172.20.0.1:59282 - "POST /node/69c9cea03c3aafdab4bea297 HTTP/1.0" 202 Accepted 2026-04-01 01:15:21,070 - lava_callback - INFO - Starting async job submit for node 69c9cea03c3aafdab4bea297 2026-04-01 01:15:21,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea03c3aafdab4bea297 HTTP/1.1" 200 None 2026-04-01 01:15:21,090 - lava_callback - DEBUG - Processing node 69c9cea03c3aafdab4bea297: baseline-arm64-baylibre 2026-04-01 01:15:21,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cea03c3aafdab4bea297 2026-04-01 01:15:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea03c3aafdab4bea297 HTTP/1.1" 200 None 2026-04-01 01:15:21,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:21,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cea03c3aafdab4bea297 HTTP/1.1" 200 None 2026-04-01 01:15:21,147 - lava_callback - ERROR - Error processing callback for node 69c9cea03c3aafdab4bea297: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:21,732 - lava_callback - INFO - Received callback for node 69c9cea23c3aafdab4bea29a INFO: 172.20.0.1:59296 - "POST /node/69c9cea23c3aafdab4bea29a HTTP/1.0" 202 Accepted 2026-04-01 01:15:21,738 - lava_callback - INFO - Starting async job submit for node 69c9cea23c3aafdab4bea29a 2026-04-01 01:15:21,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea23c3aafdab4bea29a HTTP/1.1" 200 None 2026-04-01 01:15:21,754 - lava_callback - DEBUG - Processing node 69c9cea23c3aafdab4bea29a: baseline-arm64-baylibre 2026-04-01 01:15:21,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cea23c3aafdab4bea29a 2026-04-01 01:15:21,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea23c3aafdab4bea29a HTTP/1.1" 200 None 2026-04-01 01:15:21,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:21,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:21,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cea23c3aafdab4bea29a HTTP/1.1" 200 None 2026-04-01 01:15:21,810 - lava_callback - ERROR - Error processing callback for node 69c9cea23c3aafdab4bea29a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:22,406 - lava_callback - INFO - Received callback for node 69c9cea43c3aafdab4bea2a5 INFO: 172.20.0.1:59310 - "POST /node/69c9cea43c3aafdab4bea2a5 HTTP/1.0" 202 Accepted 2026-04-01 01:15:22,412 - lava_callback - INFO - Starting async job submit for node 69c9cea43c3aafdab4bea2a5 2026-04-01 01:15:22,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:22,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea43c3aafdab4bea2a5 HTTP/1.1" 200 None 2026-04-01 01:15:22,430 - lava_callback - DEBUG - Processing node 69c9cea43c3aafdab4bea2a5: baseline-arm64-baylibre 2026-04-01 01:15:22,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cea43c3aafdab4bea2a5 2026-04-01 01:15:22,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:22,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea43c3aafdab4bea2a5 HTTP/1.1" 200 None 2026-04-01 01:15:22,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:22,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:22,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:22,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cea43c3aafdab4bea2a5 HTTP/1.1" 200 None 2026-04-01 01:15:22,490 - lava_callback - ERROR - Error processing callback for node 69c9cea43c3aafdab4bea2a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:23,092 - lava_callback - INFO - Received callback for node 69c9cea63c3aafdab4bea2aa INFO: 172.20.0.1:59320 - "POST /node/69c9cea63c3aafdab4bea2aa HTTP/1.0" 202 Accepted 2026-04-01 01:15:23,098 - lava_callback - INFO - Starting async job submit for node 69c9cea63c3aafdab4bea2aa 2026-04-01 01:15:23,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:23,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea63c3aafdab4bea2aa HTTP/1.1" 200 None 2026-04-01 01:15:23,114 - lava_callback - DEBUG - Processing node 69c9cea63c3aafdab4bea2aa: baseline-arm64-baylibre 2026-04-01 01:15:23,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cea63c3aafdab4bea2aa 2026-04-01 01:15:23,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:23,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea63c3aafdab4bea2aa HTTP/1.1" 200 None 2026-04-01 01:15:23,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:23,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:23,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:23,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cea63c3aafdab4bea2aa HTTP/1.1" 200 None 2026-04-01 01:15:23,180 - lava_callback - ERROR - Error processing callback for node 69c9cea63c3aafdab4bea2aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59332 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:15:28,572 - lava_callback - INFO - Received callback for node 69c9ce9d3c3aafdab4bea28a INFO: 172.20.0.1:59342 - "POST /node/69c9ce9d3c3aafdab4bea28a HTTP/1.0" 202 Accepted 2026-04-01 01:15:28,578 - lava_callback - INFO - Starting async job submit for node 69c9ce9d3c3aafdab4bea28a 2026-04-01 01:15:28,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:28,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ce9d3c3aafdab4bea28a HTTP/1.1" 200 None 2026-04-01 01:15:28,594 - lava_callback - DEBUG - Processing node 69c9ce9d3c3aafdab4bea28a: baseline-arm-clabbe 2026-04-01 01:15:28,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ce9d3c3aafdab4bea28a 2026-04-01 01:15:28,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:28,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ce9d3c3aafdab4bea28a HTTP/1.1" 200 None 2026-04-01 01:15:28,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:28,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 01:15:28,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:28,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ce9d3c3aafdab4bea28a HTTP/1.1" 200 None 2026-04-01 01:15:28,659 - lava_callback - ERROR - Error processing callback for node 69c9ce9d3c3aafdab4bea28a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47836 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:15:40,221 - lava_callback - INFO - Received callback for node 69c9cea73c3aafdab4bea2ad INFO: 172.20.0.1:44768 - "POST /node/69c9cea73c3aafdab4bea2ad HTTP/1.0" 202 Accepted 2026-04-01 01:15:40,227 - lava_callback - INFO - Starting async job submit for node 69c9cea73c3aafdab4bea2ad 2026-04-01 01:15:40,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:40,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea73c3aafdab4bea2ad HTTP/1.1" 200 None 2026-04-01 01:15:40,246 - lava_callback - DEBUG - Processing node 69c9cea73c3aafdab4bea2ad: baseline-arm64-baylibre 2026-04-01 01:15:40,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cea73c3aafdab4bea2ad 2026-04-01 01:15:40,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:40,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cea73c3aafdab4bea2ad HTTP/1.1" 200 None 2026-04-01 01:15:40,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:40,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:40,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:40,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cea73c3aafdab4bea2ad HTTP/1.1" 200 None 2026-04-01 01:15:40,304 - lava_callback - ERROR - Error processing callback for node 69c9cea73c3aafdab4bea2ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44784 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:15:48,543 - lava_callback - INFO - Received callback for node 69c9ceb73c3aafdab4bea2e7 INFO: 172.20.0.1:44790 - "POST /node/69c9ceb73c3aafdab4bea2e7 HTTP/1.0" 202 Accepted 2026-04-01 01:15:48,548 - lava_callback - INFO - Starting async job submit for node 69c9ceb73c3aafdab4bea2e7 2026-04-01 01:15:48,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:48,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceb73c3aafdab4bea2e7 HTTP/1.1" 200 None 2026-04-01 01:15:48,564 - lava_callback - DEBUG - Processing node 69c9ceb73c3aafdab4bea2e7: baseline-arm64-clabbe 2026-04-01 01:15:48,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ceb73c3aafdab4bea2e7 2026-04-01 01:15:48,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:48,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceb73c3aafdab4bea2e7 HTTP/1.1" 200 None 2026-04-01 01:15:48,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:48,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:48,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:48,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ceb73c3aafdab4bea2e7 HTTP/1.1" 200 None 2026-04-01 01:15:48,617 - lava_callback - ERROR - Error processing callback for node 69c9ceb73c3aafdab4bea2e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:49,223 - lava_callback - INFO - Received callback for node 69c9ceb93c3aafdab4bea2ea INFO: 172.20.0.1:44800 - "POST /node/69c9ceb93c3aafdab4bea2ea HTTP/1.0" 202 Accepted 2026-04-01 01:15:49,228 - lava_callback - INFO - Starting async job submit for node 69c9ceb93c3aafdab4bea2ea 2026-04-01 01:15:49,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceb93c3aafdab4bea2ea HTTP/1.1" 200 None 2026-04-01 01:15:49,245 - lava_callback - DEBUG - Processing node 69c9ceb93c3aafdab4bea2ea: baseline-arm64-clabbe 2026-04-01 01:15:49,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ceb93c3aafdab4bea2ea 2026-04-01 01:15:49,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceb93c3aafdab4bea2ea HTTP/1.1" 200 None 2026-04-01 01:15:49,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:49,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ceb93c3aafdab4bea2ea HTTP/1.1" 200 None 2026-04-01 01:15:49,304 - lava_callback - ERROR - Error processing callback for node 69c9ceb93c3aafdab4bea2ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:49,890 - lava_callback - INFO - Received callback for node 69c9ceba3c3aafdab4bea2ed INFO: 172.20.0.1:35614 - "POST /node/69c9ceba3c3aafdab4bea2ed HTTP/1.0" 202 Accepted 2026-04-01 01:15:49,896 - lava_callback - INFO - Starting async job submit for node 69c9ceba3c3aafdab4bea2ed 2026-04-01 01:15:49,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceba3c3aafdab4bea2ed HTTP/1.1" 200 None 2026-04-01 01:15:49,913 - lava_callback - DEBUG - Processing node 69c9ceba3c3aafdab4bea2ed: baseline-arm64-clabbe 2026-04-01 01:15:49,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ceba3c3aafdab4bea2ed 2026-04-01 01:15:49,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceba3c3aafdab4bea2ed HTTP/1.1" 200 None 2026-04-01 01:15:49,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:49,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:49,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ceba3c3aafdab4bea2ed HTTP/1.1" 200 None 2026-04-01 01:15:49,972 - lava_callback - ERROR - Error processing callback for node 69c9ceba3c3aafdab4bea2ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:50,580 - lava_callback - INFO - Received callback for node 69c9ceba3c3aafdab4bea2f0 INFO: 172.20.0.1:35616 - "POST /node/69c9ceba3c3aafdab4bea2f0 HTTP/1.0" 202 Accepted 2026-04-01 01:15:50,585 - lava_callback - INFO - Starting async job submit for node 69c9ceba3c3aafdab4bea2f0 2026-04-01 01:15:50,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:50,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceba3c3aafdab4bea2f0 HTTP/1.1" 200 None 2026-04-01 01:15:50,602 - lava_callback - DEBUG - Processing node 69c9ceba3c3aafdab4bea2f0: baseline-arm64-clabbe 2026-04-01 01:15:50,603 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ceba3c3aafdab4bea2f0 2026-04-01 01:15:50,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:50,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ceba3c3aafdab4bea2f0 HTTP/1.1" 200 None 2026-04-01 01:15:50,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:50,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:50,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:50,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ceba3c3aafdab4bea2f0 HTTP/1.1" 200 None 2026-04-01 01:15:50,676 - lava_callback - ERROR - Error processing callback for node 69c9ceba3c3aafdab4bea2f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:51,240 - lava_callback - INFO - Received callback for node 69c9cebd3c3aafdab4bea2fd INFO: 172.20.0.1:35628 - "POST /node/69c9cebd3c3aafdab4bea2fd HTTP/1.0" 202 Accepted 2026-04-01 01:15:51,246 - lava_callback - INFO - Starting async job submit for node 69c9cebd3c3aafdab4bea2fd 2026-04-01 01:15:51,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cebd3c3aafdab4bea2fd HTTP/1.1" 200 None 2026-04-01 01:15:51,269 - lava_callback - DEBUG - Processing node 69c9cebd3c3aafdab4bea2fd: baseline-arm64-clabbe 2026-04-01 01:15:51,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cebd3c3aafdab4bea2fd 2026-04-01 01:15:51,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cebd3c3aafdab4bea2fd HTTP/1.1" 200 None 2026-04-01 01:15:51,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:51,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cebd3c3aafdab4bea2fd HTTP/1.1" 200 None 2026-04-01 01:15:51,321 - lava_callback - ERROR - Error processing callback for node 69c9cebd3c3aafdab4bea2fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:15:51,905 - lava_callback - INFO - Received callback for node 69c9cebc3c3aafdab4bea2fa INFO: 172.20.0.1:35642 - "POST /node/69c9cebc3c3aafdab4bea2fa HTTP/1.0" 202 Accepted 2026-04-01 01:15:51,911 - lava_callback - INFO - Starting async job submit for node 69c9cebc3c3aafdab4bea2fa 2026-04-01 01:15:51,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cebc3c3aafdab4bea2fa HTTP/1.1" 200 None 2026-04-01 01:15:51,927 - lava_callback - DEBUG - Processing node 69c9cebc3c3aafdab4bea2fa: baseline-arm64-clabbe 2026-04-01 01:15:51,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cebc3c3aafdab4bea2fa 2026-04-01 01:15:51,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cebc3c3aafdab4bea2fa HTTP/1.1" 200 None 2026-04-01 01:15:51,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 01:15:51,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:15:51,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cebc3c3aafdab4bea2fa HTTP/1.1" 200 None 2026-04-01 01:15:51,997 - lava_callback - ERROR - Error processing callback for node 69c9cebc3c3aafdab4bea2fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48120 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:16:08,592 - lava_callback - INFO - Received callback for node 69c9cec43c3aafdab4bea317 INFO: 172.20.0.1:48124 - "POST /node/69c9cec43c3aafdab4bea317 HTTP/1.0" 202 Accepted 2026-04-01 01:16:08,598 - lava_callback - INFO - Starting async job submit for node 69c9cec43c3aafdab4bea317 2026-04-01 01:16:08,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:08,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cec43c3aafdab4bea317 HTTP/1.1" 200 None 2026-04-01 01:16:08,614 - lava_callback - DEBUG - Processing node 69c9cec43c3aafdab4bea317: baseline-arm-clabbe 2026-04-01 01:16:08,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cec43c3aafdab4bea317 2026-04-01 01:16:08,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:08,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cec43c3aafdab4bea317 HTTP/1.1" 200 None 2026-04-01 01:16:08,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:08,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 01:16:08,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:08,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cec43c3aafdab4bea317 HTTP/1.1" 200 None 2026-04-01 01:16:08,667 - lava_callback - ERROR - Error processing callback for node 69c9cec43c3aafdab4bea317: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54882 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:16:28,523 - lava_callback - INFO - Received callback for node 69cc71d58cd9cbcdc945a173 INFO: 172.20.0.1:54888 - "POST /node/69cc71d58cd9cbcdc945a173 HTTP/1.0" 202 Accepted 2026-04-01 01:16:28,529 - lava_callback - INFO - Starting async job submit for node 69cc71d58cd9cbcdc945a173 2026-04-01 01:16:28,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71d58cd9cbcdc945a173 HTTP/1.1" 200 None 2026-04-01 01:16:28,547 - lava_callback - DEBUG - Processing node 69cc71d58cd9cbcdc945a173: baseline-nfs-arm64 2026-04-01 01:16:28,548 - lava_callback - WARNING - Failed to upload log 2026-04-01 01:16:28,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:16:28,613 - lava_callback - INFO - Received callback for node 69c9cee83c3aafdab4bea35f INFO: 172.20.0.1:54902 - "POST /node/69c9cee83c3aafdab4bea35f HTTP/1.0" 202 Accepted 2026-04-01 01:16:28,636 - lava_callback - INFO - Starting async job submit for node 69c9cee83c3aafdab4bea35f 2026-04-01 01:16:28,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cee83c3aafdab4bea35f HTTP/1.1" 200 None 2026-04-01 01:16:28,661 - lava_callback - DEBUG - Processing node 69c9cee83c3aafdab4bea35f: baseline-arm-clabbe 2026-04-01 01:16:28,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cee83c3aafdab4bea35f 2026-04-01 01:16:28,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cee83c3aafdab4bea35f HTTP/1.1" 200 None 2026-04-01 01:16:28,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 01:16:28,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:16:28,715 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc71d58cd9cbcdc945a173/lava-logs.yaml 2026-04-01 01:16:28,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:16:28,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cee83c3aafdab4bea35f HTTP/1.1" 200 None 2026-04-01 01:16:28,739 - lava_callback - ERROR - Error processing callback for node 69c9cee83c3aafdab4bea35f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 01:16:28,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:16:28,931 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc71d58cd9cbcdc945a173/lava_callback.json.gz 2026-04-01 01:16:28,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71d58cd9cbcdc945a173 HTTP/1.1" 200 None 2026-04-01 01:16:28,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:16:28,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:28,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc71d58cd9cbcdc945a173 HTTP/1.1" 200 None 2026-04-01 01:16:28,992 - lava_callback - INFO - Completed processing callback for node 69cc71d58cd9cbcdc945a173 2026-04-01 01:16:32,600 - lava_callback - INFO - Received callback for node 69cc6e038cd9cbcdc9459ddf INFO: 172.20.0.1:59054 - "POST /node/69cc6e038cd9cbcdc9459ddf HTTP/1.0" 202 Accepted 2026-04-01 01:16:32,606 - lava_callback - INFO - Starting async job submit for node 69cc6e038cd9cbcdc9459ddf 2026-04-01 01:16:32,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:32,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e038cd9cbcdc9459ddf HTTP/1.1" 200 None 2026-04-01 01:16:32,679 - lava_callback - DEBUG - Processing node 69cc6e038cd9cbcdc9459ddf: baseline-arm 2026-04-01 01:16:33,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:16:33,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:16:33,389 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc6e038cd9cbcdc9459ddf/log.txt.gz 2026-04-01 01:16:33,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:16:34,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:16:34,038 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc6e038cd9cbcdc9459ddf/lava-logs.yaml 2026-04-01 01:16:34,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:16:34,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:16:34,224 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc6e038cd9cbcdc9459ddf/lava_callback.json.gz DEBUG: Setup failed for 69cc6e038cd9cbcdc9459ddf. Transitting job node result: incomplete -> fail 2026-04-01 01:16:34,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:34,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e038cd9cbcdc9459ddf HTTP/1.1" 200 None 2026-04-01 01:16:34,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:34,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 01:16:34,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:34,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6e038cd9cbcdc9459ddf HTTP/1.1" 200 None 2026-04-01 01:16:34,515 - lava_callback - INFO - Completed processing callback for node 69cc6e038cd9cbcdc9459ddf INFO: 172.20.0.1:59066 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:16:44,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:16:44,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:17:48,690 - lava_callback - INFO - Received callback for node 69c9cf303c3aafdab4bea3f9 INFO: 172.20.0.1:50176 - "POST /node/69c9cf303c3aafdab4bea3f9 HTTP/1.0" 202 Accepted 2026-04-01 01:17:48,696 - lava_callback - INFO - Starting async job submit for node 69c9cf303c3aafdab4bea3f9 2026-04-01 01:17:48,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:17:48,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cf303c3aafdab4bea3f9 HTTP/1.1" 200 None 2026-04-01 01:17:48,715 - lava_callback - DEBUG - Processing node 69c9cf303c3aafdab4bea3f9: baseline-arm-clabbe 2026-04-01 01:17:48,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9cf303c3aafdab4bea3f9 2026-04-01 01:17:48,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:17:48,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9cf303c3aafdab4bea3f9 HTTP/1.1" 200 None 2026-04-01 01:17:48,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:17:48,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-04-01 01:17:48,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:17:48,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9cf303c3aafdab4bea3f9 HTTP/1.1" 200 None 2026-04-01 01:17:48,777 - lava_callback - ERROR - Error processing callback for node 69c9cf303c3aafdab4bea3f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44942 - "GET /node/69bd7c880e93e2314723e916%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56492 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:19:59,253 - lava_callback - INFO - Received callback for node 69cc6e398cd9cbcdc9459df0 INFO: 172.20.0.1:56500 - "POST /node/69cc6e398cd9cbcdc9459df0 HTTP/1.0" 202 Accepted 2026-04-01 01:19:59,305 - lava_callback - INFO - Starting async job submit for node 69cc6e398cd9cbcdc9459df0 2026-04-01 01:19:59,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:19:59,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e398cd9cbcdc9459df0 HTTP/1.1" 200 None 2026-04-01 01:19:59,387 - lava_callback - DEBUG - Processing node 69cc6e398cd9cbcdc9459df0: baseline-arm-pengutronix 2026-04-01 01:19:59,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:19:59,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:19:59,896 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc6e398cd9cbcdc9459df0/log.txt.gz 2026-04-01 01:19:59,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:20:00,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:20:00,451 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc6e398cd9cbcdc9459df0/lava-logs.yaml 2026-04-01 01:20:00,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:20:00,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:20:00,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc6e398cd9cbcdc9459df0/lava_callback.json.gz DEBUG: Setup failed for 69cc6e398cd9cbcdc9459df0. Transitting job node result: incomplete -> fail 2026-04-01 01:20:00,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:00,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e398cd9cbcdc9459df0 HTTP/1.1" 200 None 2026-04-01 01:20:00,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:00,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 01:20:00,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:00,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6e398cd9cbcdc9459df0 HTTP/1.1" 200 None 2026-04-01 01:20:00,822 - lava_callback - INFO - Completed processing callback for node 69cc6e398cd9cbcdc9459df0 INFO: 172.20.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40258 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:20:14,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:14,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET /node/698f61683013abef95cfa392%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59326 - "GET /node/694aab6f83c67c43d1ede178%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52568 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:20:28,794 - lava_callback - INFO - Received callback for node 69cc72298cd9cbcdc945a1cb INFO: 172.20.0.1:52570 - "POST /node/69cc72298cd9cbcdc945a1cb HTTP/1.0" 202 Accepted 2026-04-01 01:20:28,809 - lava_callback - INFO - Starting async job submit for node 69cc72298cd9cbcdc945a1cb 2026-04-01 01:20:28,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:28,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc72298cd9cbcdc945a1cb HTTP/1.1" 200 None 2026-04-01 01:20:28,905 - lava_callback - DEBUG - Processing node 69cc72298cd9cbcdc945a1cb: baseline-arm64 2026-04-01 01:20:29,123 - lava_callback - WARNING - Failed to upload log 2026-04-01 01:20:29,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:20:29,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:20:29,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc72298cd9cbcdc945a1cb/lava-logs.yaml 2026-04-01 01:20:29,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:20:29,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:20:29,768 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc72298cd9cbcdc945a1cb/lava_callback.json.gz 2026-04-01 01:20:29,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:29,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc72298cd9cbcdc945a1cb HTTP/1.1" 200 None 2026-04-01 01:20:29,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:29,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:20:29,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:29,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc72298cd9cbcdc945a1cb HTTP/1.1" 200 None 2026-04-01 01:20:30,030 - lava_callback - INFO - Completed processing callback for node 69cc72298cd9cbcdc945a1cb INFO: 172.20.0.1:34838 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:20:44,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:20:44,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50660 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:22:20,751 - lava_callback - INFO - Received callback for node 69cc730f8cd9cbcdc945a278 INFO: 172.20.0.1:41272 - "POST /node/69cc730f8cd9cbcdc945a278 HTTP/1.0" 202 Accepted 2026-04-01 01:22:20,758 - lava_callback - INFO - Starting async job submit for node 69cc730f8cd9cbcdc945a278 2026-04-01 01:22:20,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:22:20,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc730f8cd9cbcdc945a278 HTTP/1.1" 200 None 2026-04-01 01:22:20,801 - lava_callback - DEBUG - Processing node 69cc730f8cd9cbcdc945a278: baseline-arm64 2026-04-01 01:22:21,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:22:21,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:22:21,277 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc730f8cd9cbcdc945a278/log.txt.gz 2026-04-01 01:22:21,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:22:21,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:22:21,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc730f8cd9cbcdc945a278/lava-logs.yaml 2026-04-01 01:22:21,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:22:21,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:22:21,862 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc730f8cd9cbcdc945a278/lava_callback.json.gz 2026-04-01 01:22:21,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:22:21,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc730f8cd9cbcdc945a278 HTTP/1.1" 200 None 2026-04-01 01:22:21,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:22:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:22:21,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:22:21,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc730f8cd9cbcdc945a278 HTTP/1.1" 200 None 2026-04-01 01:22:21,969 - lava_callback - INFO - Completed processing callback for node 69cc730f8cd9cbcdc945a278 INFO: 172.20.0.1:41280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35266 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:22:44,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:22:44,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58686 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:23:33,734 - lava_callback - INFO - Received callback for node 69cc72828cd9cbcdc945a217 INFO: 172.20.0.1:58174 - "POST /node/69cc72828cd9cbcdc945a217 HTTP/1.0" 202 Accepted 2026-04-01 01:23:33,745 - lava_callback - INFO - Starting async job submit for node 69cc72828cd9cbcdc945a217 2026-04-01 01:23:33,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:23:33,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc72828cd9cbcdc945a217 HTTP/1.1" 200 None 2026-04-01 01:23:33,792 - lava_callback - DEBUG - Processing node 69cc72828cd9cbcdc945a217: baseline-arm64 2026-04-01 01:23:33,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:23:34,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:23:34,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc72828cd9cbcdc945a217/log.txt.gz 2026-04-01 01:23:34,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:23:34,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:23:34,434 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc72828cd9cbcdc945a217/lava-logs.yaml 2026-04-01 01:23:34,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:23:34,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:23:34,586 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc72828cd9cbcdc945a217/lava_callback.json.gz 2026-04-01 01:23:34,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:23:34,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc72828cd9cbcdc945a217 HTTP/1.1" 200 None 2026-04-01 01:23:34,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:23:34,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:23:34,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:23:34,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc72828cd9cbcdc945a217 HTTP/1.1" 200 None 2026-04-01 01:23:34,753 - lava_callback - INFO - Completed processing callback for node 69cc72828cd9cbcdc945a217 INFO: 172.20.0.1:58178 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:23:44,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:23:44,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57164 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:24:28,214 - lava_callback - INFO - Received callback for node 69cc73678cd9cbcdc945a2c1 INFO: 172.20.0.1:57170 - "POST /node/69cc73678cd9cbcdc945a2c1 HTTP/1.0" 202 Accepted 2026-04-01 01:24:28,223 - lava_callback - INFO - Starting async job submit for node 69cc73678cd9cbcdc945a2c1 2026-04-01 01:24:28,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:24:28,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc73678cd9cbcdc945a2c1 HTTP/1.1" 200 None 2026-04-01 01:24:28,274 - lava_callback - DEBUG - Processing node 69cc73678cd9cbcdc945a2c1: baseline-arm64 2026-04-01 01:24:28,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:24:29,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:24:29,051 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc73678cd9cbcdc945a2c1/log.txt.gz 2026-04-01 01:24:29,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:24:29,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:24:29,622 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc73678cd9cbcdc945a2c1/lava-logs.yaml 2026-04-01 01:24:29,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:24:29,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:24:29,847 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc73678cd9cbcdc945a2c1/lava_callback.json.gz 2026-04-01 01:24:29,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:24:29,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc73678cd9cbcdc945a2c1 HTTP/1.1" 200 None 2026-04-01 01:24:29,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:24:29,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:24:29,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:24:29,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc73678cd9cbcdc945a2c1 HTTP/1.1" 200 None 2026-04-01 01:24:30,007 - lava_callback - INFO - Completed processing callback for node 69cc73678cd9cbcdc945a2c1 INFO: 172.20.0.1:43446 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:24:44,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:24:44,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43614 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:28:47,253 - lava_callback - INFO - Received callback for node 69cc6ff78cd9cbcdc9459ef3 INFO: 172.20.0.1:43620 - "POST /node/69cc6ff78cd9cbcdc9459ef3 HTTP/1.0" 202 Accepted 2026-04-01 01:28:47,261 - lava_callback - INFO - Starting async job submit for node 69cc6ff78cd9cbcdc9459ef3 2026-04-01 01:28:47,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:28:47,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff78cd9cbcdc9459ef3 HTTP/1.1" 200 None 2026-04-01 01:28:47,321 - lava_callback - DEBUG - Processing node 69cc6ff78cd9cbcdc9459ef3: baseline-nfs-arm64 2026-04-01 01:28:48,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:28:48,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:28:48,642 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff78cd9cbcdc9459ef3/log.txt.gz 2026-04-01 01:28:49,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:28:49,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:28:49,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff78cd9cbcdc9459ef3/lava-logs.yaml 2026-04-01 01:28:49,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:28:49,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:28:49,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ff78cd9cbcdc9459ef3/lava_callback.json.gz DEBUG: Setup failed for 69cc6ff78cd9cbcdc9459ef3. Transitting job node result: incomplete -> fail 2026-04-01 01:28:49,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:28:49,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ff78cd9cbcdc9459ef3 HTTP/1.1" 200 None 2026-04-01 01:28:49,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:28:49,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:28:49,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:28:50,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ff78cd9cbcdc9459ef3 HTTP/1.1" 200 None 2026-04-01 01:28:50,103 - lava_callback - INFO - Completed processing callback for node 69cc6ff78cd9cbcdc9459ef3 INFO: 172.20.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53556 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:29:14,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:29:14,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39538 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:31:01,085 - lava_callback - INFO - Received callback for node 69c9d2473c3aafdab4bea4ea INFO: 172.20.0.1:53930 - "POST /node/69c9d2473c3aafdab4bea4ea HTTP/1.0" 202 Accepted 2026-04-01 01:31:01,092 - lava_callback - INFO - Starting async job submit for node 69c9d2473c3aafdab4bea4ea 2026-04-01 01:31:01,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:31:01,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d2473c3aafdab4bea4ea HTTP/1.1" 200 None 2026-04-01 01:31:01,108 - lava_callback - DEBUG - Processing node 69c9d2473c3aafdab4bea4ea: baseline-arm64-baylibre 2026-04-01 01:31:01,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d2473c3aafdab4bea4ea 2026-04-01 01:31:01,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:31:01,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d2473c3aafdab4bea4ea HTTP/1.1" 200 None 2026-04-01 01:31:01,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:31:01,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:31:01,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:31:01,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d2473c3aafdab4bea4ea HTTP/1.1" 200 None 2026-04-01 01:31:01,159 - lava_callback - ERROR - Error processing callback for node 69c9d2473c3aafdab4bea4ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51864 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:32:00,380 - lava_callback - INFO - Received callback for node 69cc74508cd9cbcdc945a353 INFO: 172.20.0.1:58692 - "POST /node/69cc74508cd9cbcdc945a353 HTTP/1.0" 202 Accepted 2026-04-01 01:32:00,390 - lava_callback - INFO - Starting async job submit for node 69cc74508cd9cbcdc945a353 2026-04-01 01:32:00,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:00,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc74508cd9cbcdc945a353 HTTP/1.1" 200 None 2026-04-01 01:32:00,466 - lava_callback - DEBUG - Processing node 69cc74508cd9cbcdc945a353: baseline-arm64-foundriesio 2026-04-01 01:32:01,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:32:01,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:32:01,870 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc74508cd9cbcdc945a353/log.txt.gz INFO: 172.20.0.1:58704 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:32:02,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:32:02,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:32:02,928 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc74508cd9cbcdc945a353/lava-logs.yaml 2026-04-01 01:32:02,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:32:03,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:32:03,166 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc74508cd9cbcdc945a353/lava_callback.json.gz DEBUG: 69cc74508cd9cbcdc945a353 Transitting job node result: pass -> incomplete 2026-04-01 01:32:03,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:03,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc74508cd9cbcdc945a353 HTTP/1.1" 200 None 2026-04-01 01:32:03,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:03,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:32:03,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:03,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc74508cd9cbcdc945a353 HTTP/1.1" 200 None 2026-04-01 01:32:03,299 - lava_callback - INFO - Completed processing callback for node 69cc74508cd9cbcdc945a353 INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:32:14,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:14,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43118 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:32:21,143 - lava_callback - INFO - Received callback for node 69c9d2983c3aafdab4bea4f6 INFO: 172.20.0.1:60760 - "POST /node/69c9d2983c3aafdab4bea4f6 HTTP/1.0" 202 Accepted 2026-04-01 01:32:21,149 - lava_callback - INFO - Starting async job submit for node 69c9d2983c3aafdab4bea4f6 2026-04-01 01:32:21,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:21,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d2983c3aafdab4bea4f6 HTTP/1.1" 200 None 2026-04-01 01:32:21,166 - lava_callback - DEBUG - Processing node 69c9d2983c3aafdab4bea4f6: baseline-arm64-baylibre 2026-04-01 01:32:21,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d2983c3aafdab4bea4f6 2026-04-01 01:32:21,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:21,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d2983c3aafdab4bea4f6 HTTP/1.1" 200 None 2026-04-01 01:32:21,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:21,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:32:21,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:32:21,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d2983c3aafdab4bea4f6 HTTP/1.1" 200 None 2026-04-01 01:32:21,234 - lava_callback - ERROR - Error processing callback for node 69c9d2983c3aafdab4bea4f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39526 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:33:41,203 - lava_callback - INFO - Received callback for node 69c9d2e43c3aafdab4bea505 INFO: 172.20.0.1:55372 - "POST /node/69c9d2e43c3aafdab4bea505 HTTP/1.0" 202 Accepted 2026-04-01 01:33:41,209 - lava_callback - INFO - Starting async job submit for node 69c9d2e43c3aafdab4bea505 2026-04-01 01:33:41,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:33:41,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d2e43c3aafdab4bea505 HTTP/1.1" 200 None 2026-04-01 01:33:41,236 - lava_callback - DEBUG - Processing node 69c9d2e43c3aafdab4bea505: baseline-arm64-baylibre 2026-04-01 01:33:41,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d2e43c3aafdab4bea505 2026-04-01 01:33:41,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:33:41,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d2e43c3aafdab4bea505 HTTP/1.1" 200 None 2026-04-01 01:33:41,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:33:41,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:33:41,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:33:41,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d2e43c3aafdab4bea505 HTTP/1.1" 200 None 2026-04-01 01:33:41,299 - lava_callback - ERROR - Error processing callback for node 69c9d2e43c3aafdab4bea505: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55928 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:35:01,276 - lava_callback - INFO - Received callback for node 69c9d3343c3aafdab4bea514 INFO: 172.20.0.1:40988 - "POST /node/69c9d3343c3aafdab4bea514 HTTP/1.0" 202 Accepted 2026-04-01 01:35:01,282 - lava_callback - INFO - Starting async job submit for node 69c9d3343c3aafdab4bea514 2026-04-01 01:35:01,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:01,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d3343c3aafdab4bea514 HTTP/1.1" 200 None 2026-04-01 01:35:01,303 - lava_callback - DEBUG - Processing node 69c9d3343c3aafdab4bea514: baseline-arm64-baylibre 2026-04-01 01:35:01,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d3343c3aafdab4bea514 2026-04-01 01:35:01,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:01,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d3343c3aafdab4bea514 HTTP/1.1" 200 None 2026-04-01 01:35:01,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:01,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:35:01,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:01,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d3343c3aafdab4bea514 HTTP/1.1" 200 None 2026-04-01 01:35:01,397 - lava_callback - ERROR - Error processing callback for node 69c9d3343c3aafdab4bea514: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:35:14,325 - lava_callback - INFO - Received callback for node 69cc74948cd9cbcdc945a394 INFO: 172.20.0.1:44046 - "POST /node/69cc74948cd9cbcdc945a394 HTTP/1.0" 202 Accepted 2026-04-01 01:35:14,332 - lava_callback - INFO - Starting async job submit for node 69cc74948cd9cbcdc945a394 2026-04-01 01:35:14,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:14,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc74948cd9cbcdc945a394 HTTP/1.1" 200 None 2026-04-01 01:35:14,404 - lava_callback - DEBUG - Processing node 69cc74948cd9cbcdc945a394: baseline-arm64 2026-04-01 01:35:14,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:35:14,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:35:14,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc74948cd9cbcdc945a394/log.txt.gz INFO: 172.20.0.1:44054 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 01:35:15,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:35:15,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:35:15,448 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc74948cd9cbcdc945a394/lava-logs.yaml 2026-04-01 01:35:15,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:35:15,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:35:15,610 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc74948cd9cbcdc945a394/lava_callback.json.gz DEBUG: Setup failed for 69cc74948cd9cbcdc945a394. Transitting job node result: incomplete -> fail 2026-04-01 01:35:15,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:15,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc74948cd9cbcdc945a394 HTTP/1.1" 200 None 2026-04-01 01:35:15,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:15,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:35:15,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:15,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc74948cd9cbcdc945a394 HTTP/1.1" 200 None 2026-04-01 01:35:15,825 - lava_callback - INFO - Completed processing callback for node 69cc74948cd9cbcdc945a394 INFO: 172.20.0.1:44056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:35:44,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:35:44,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54422 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:36:21,339 - lava_callback - INFO - Received callback for node 69c9d3833c3aafdab4bea52b INFO: 172.20.0.1:59090 - "POST /node/69c9d3833c3aafdab4bea52b HTTP/1.0" 202 Accepted 2026-04-01 01:36:21,346 - lava_callback - INFO - Starting async job submit for node 69c9d3833c3aafdab4bea52b 2026-04-01 01:36:21,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:36:21,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d3833c3aafdab4bea52b HTTP/1.1" 200 None 2026-04-01 01:36:21,376 - lava_callback - DEBUG - Processing node 69c9d3833c3aafdab4bea52b: baseline-arm64-baylibre 2026-04-01 01:36:21,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d3833c3aafdab4bea52b 2026-04-01 01:36:21,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:36:21,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d3833c3aafdab4bea52b HTTP/1.1" 200 None 2026-04-01 01:36:21,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:36:21,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:36:21,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:36:21,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d3833c3aafdab4bea52b HTTP/1.1" 200 None 2026-04-01 01:36:21,444 - lava_callback - ERROR - Error processing callback for node 69c9d3833c3aafdab4bea52b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55036 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:37:41,464 - lava_callback - INFO - Received callback for node 69c9d3d03c3aafdab4bea538 INFO: 172.20.0.1:33104 - "POST /node/69c9d3d03c3aafdab4bea538 HTTP/1.0" 202 Accepted 2026-04-01 01:37:41,469 - lava_callback - INFO - Starting async job submit for node 69c9d3d03c3aafdab4bea538 2026-04-01 01:37:41,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:37:41,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d3d03c3aafdab4bea538 HTTP/1.1" 200 None 2026-04-01 01:37:41,492 - lava_callback - DEBUG - Processing node 69c9d3d03c3aafdab4bea538: baseline-arm64-baylibre 2026-04-01 01:37:41,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d3d03c3aafdab4bea538 2026-04-01 01:37:41,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:37:41,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d3d03c3aafdab4bea538 HTTP/1.1" 200 None 2026-04-01 01:37:41,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:37:41,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:37:41,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:37:41,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d3d03c3aafdab4bea538 HTTP/1.1" 200 None 2026-04-01 01:37:41,547 - lava_callback - ERROR - Error processing callback for node 69c9d3d03c3aafdab4bea538: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53958 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:38:49,661 - lava_callback - INFO - Received callback for node 69c9d4253c3aafdab4bea544 INFO: 172.20.0.1:53974 - "POST /node/69c9d4253c3aafdab4bea544 HTTP/1.0" 202 Accepted 2026-04-01 01:38:49,667 - lava_callback - INFO - Starting async job submit for node 69c9d4253c3aafdab4bea544 2026-04-01 01:38:49,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:38:49,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d4253c3aafdab4bea544 HTTP/1.1" 200 None 2026-04-01 01:38:49,683 - lava_callback - DEBUG - Processing node 69c9d4253c3aafdab4bea544: baseline-arm64-clabbe 2026-04-01 01:38:49,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d4253c3aafdab4bea544 2026-04-01 01:38:49,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:38:49,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d4253c3aafdab4bea544 HTTP/1.1" 200 None 2026-04-01 01:38:49,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:38:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:38:49,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:38:49,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d4253c3aafdab4bea544 HTTP/1.1" 200 None 2026-04-01 01:38:49,737 - lava_callback - ERROR - Error processing callback for node 69c9d4253c3aafdab4bea544: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44068 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:40:09,650 - lava_callback - INFO - Received callback for node 69c9d4743c3aafdab4bea54f INFO: 172.20.0.1:44078 - "POST /node/69c9d4743c3aafdab4bea54f HTTP/1.0" 202 Accepted 2026-04-01 01:40:09,656 - lava_callback - INFO - Starting async job submit for node 69c9d4743c3aafdab4bea54f 2026-04-01 01:40:09,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:40:09,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d4743c3aafdab4bea54f HTTP/1.1" 200 None 2026-04-01 01:40:09,674 - lava_callback - DEBUG - Processing node 69c9d4743c3aafdab4bea54f: baseline-arm64-clabbe 2026-04-01 01:40:09,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d4743c3aafdab4bea54f 2026-04-01 01:40:09,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:40:09,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d4743c3aafdab4bea54f HTTP/1.1" 200 None 2026-04-01 01:40:09,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:40:09,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:40:09,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:40:09,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d4743c3aafdab4bea54f HTTP/1.1" 200 None 2026-04-01 01:40:09,730 - lava_callback - ERROR - Error processing callback for node 69c9d4743c3aafdab4bea54f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54760 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:41:29,837 - lava_callback - INFO - Received callback for node 69c9d4c33c3aafdab4bea556 INFO: 172.20.0.1:40842 - "POST /node/69c9d4c33c3aafdab4bea556 HTTP/1.0" 202 Accepted 2026-04-01 01:41:29,846 - lava_callback - INFO - Starting async job submit for node 69c9d4c33c3aafdab4bea556 2026-04-01 01:41:29,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:41:29,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d4c33c3aafdab4bea556 HTTP/1.1" 200 None 2026-04-01 01:41:29,862 - lava_callback - DEBUG - Processing node 69c9d4c33c3aafdab4bea556: baseline-arm64-clabbe 2026-04-01 01:41:29,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d4c33c3aafdab4bea556 2026-04-01 01:41:29,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:41:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d4c33c3aafdab4bea556 HTTP/1.1" 200 None 2026-04-01 01:41:29,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:41:29,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:41:29,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:41:29,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d4c33c3aafdab4bea556 HTTP/1.1" 200 None 2026-04-01 01:41:29,915 - lava_callback - ERROR - Error processing callback for node 69c9d4c33c3aafdab4bea556: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53316 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:42:49,816 - lava_callback - INFO - Received callback for node 69c9d5133c3aafdab4bea562 INFO: 172.20.0.1:54396 - "POST /node/69c9d5133c3aafdab4bea562 HTTP/1.0" 202 Accepted 2026-04-01 01:42:49,822 - lava_callback - INFO - Starting async job submit for node 69c9d5133c3aafdab4bea562 2026-04-01 01:42:49,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:42:49,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5133c3aafdab4bea562 HTTP/1.1" 200 None 2026-04-01 01:42:49,838 - lava_callback - DEBUG - Processing node 69c9d5133c3aafdab4bea562: baseline-arm64-clabbe 2026-04-01 01:42:49,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d5133c3aafdab4bea562 2026-04-01 01:42:49,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:42:49,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5133c3aafdab4bea562 HTTP/1.1" 200 None 2026-04-01 01:42:49,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:42:49,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:42:49,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:42:49,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d5133c3aafdab4bea562 HTTP/1.1" 200 None 2026-04-01 01:42:49,900 - lava_callback - ERROR - Error processing callback for node 69c9d5133c3aafdab4bea562: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59468 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:44:09,883 - lava_callback - INFO - Received callback for node 69c9d5633c3aafdab4bea56d INFO: 172.20.0.1:50016 - "POST /node/69c9d5633c3aafdab4bea56d HTTP/1.0" 202 Accepted 2026-04-01 01:44:09,891 - lava_callback - INFO - Starting async job submit for node 69c9d5633c3aafdab4bea56d 2026-04-01 01:44:09,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:09,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5633c3aafdab4bea56d HTTP/1.1" 200 None 2026-04-01 01:44:09,908 - lava_callback - DEBUG - Processing node 69c9d5633c3aafdab4bea56d: baseline-arm64-clabbe 2026-04-01 01:44:09,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d5633c3aafdab4bea56d 2026-04-01 01:44:09,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:09,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5633c3aafdab4bea56d HTTP/1.1" 200 None 2026-04-01 01:44:09,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:09,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:44:09,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:09,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d5633c3aafdab4bea56d HTTP/1.1" 200 None 2026-04-01 01:44:09,963 - lava_callback - ERROR - Error processing callback for node 69c9d5633c3aafdab4bea56d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42984 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:44:34,610 - lava_callback - INFO - Received callback for node 69cc70178cd9cbcdc9459f4b INFO: 172.20.0.1:48850 - "POST /node/69cc70178cd9cbcdc9459f4b HTTP/1.0" 202 Accepted 2026-04-01 01:44:34,616 - lava_callback - INFO - Starting async job submit for node 69cc70178cd9cbcdc9459f4b 2026-04-01 01:44:34,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:34,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70178cd9cbcdc9459f4b HTTP/1.1" 200 None 2026-04-01 01:44:34,657 - lava_callback - DEBUG - Processing node 69cc70178cd9cbcdc9459f4b: baseline-arm64-broonie 2026-04-01 01:44:34,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:44:35,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:44:35,046 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc70178cd9cbcdc9459f4b/log.txt.gz 2026-04-01 01:44:35,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:44:35,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:44:35,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc70178cd9cbcdc9459f4b/lava-logs.yaml 2026-04-01 01:44:35,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:44:35,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:44:35,541 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc70178cd9cbcdc9459f4b/lava_callback.json.gz 2026-04-01 01:44:35,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:35,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70178cd9cbcdc9459f4b HTTP/1.1" 200 None 2026-04-01 01:44:35,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:35,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:44:35,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:35,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70178cd9cbcdc9459f4b HTTP/1.1" 200 None 2026-04-01 01:44:35,635 - lava_callback - INFO - Completed processing callback for node 69cc70178cd9cbcdc9459f4b INFO: 172.20.0.1:48856 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:44:44,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:44:44,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44772 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:45:29,996 - lava_callback - INFO - Received callback for node 69c9d5b33c3aafdab4bea575 INFO: 172.20.0.1:59054 - "POST /node/69c9d5b33c3aafdab4bea575 HTTP/1.0" 202 Accepted 2026-04-01 01:45:30,002 - lava_callback - INFO - Starting async job submit for node 69c9d5b33c3aafdab4bea575 2026-04-01 01:45:30,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:45:30,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5b33c3aafdab4bea575 HTTP/1.1" 200 None 2026-04-01 01:45:30,018 - lava_callback - DEBUG - Processing node 69c9d5b33c3aafdab4bea575: baseline-arm64-clabbe 2026-04-01 01:45:30,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d5b33c3aafdab4bea575 2026-04-01 01:45:30,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:45:30,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5b33c3aafdab4bea575 HTTP/1.1" 200 None 2026-04-01 01:45:30,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:45:30,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 01:45:30,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:45:30,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d5b33c3aafdab4bea575 HTTP/1.1" 200 None 2026-04-01 01:45:30,072 - lava_callback - ERROR - Error processing callback for node 69c9d5b33c3aafdab4bea575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43298 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:46:42,032 - lava_callback - INFO - Received callback for node 69c9d5f83c3aafdab4bea584 INFO: 172.20.0.1:38128 - "POST /node/69c9d5f83c3aafdab4bea584 HTTP/1.0" 202 Accepted 2026-04-01 01:46:42,037 - lava_callback - INFO - Starting async job submit for node 69c9d5f83c3aafdab4bea584 2026-04-01 01:46:42,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:46:42,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5f83c3aafdab4bea584 HTTP/1.1" 200 None 2026-04-01 01:46:42,054 - lava_callback - DEBUG - Processing node 69c9d5f83c3aafdab4bea584: baseline-arm64-baylibre 2026-04-01 01:46:42,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d5f83c3aafdab4bea584 2026-04-01 01:46:42,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:46:42,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d5f83c3aafdab4bea584 HTTP/1.1" 200 None 2026-04-01 01:46:42,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:46:42,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 01:46:42,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:46:42,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d5f83c3aafdab4bea584 HTTP/1.1" 200 None 2026-04-01 01:46:42,108 - lava_callback - ERROR - Error processing callback for node 69c9d5f83c3aafdab4bea584: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:49:50,002 - lava_callback - INFO - Received callback for node 69cc79ac8cd9cbcdc945a871 INFO: 172.20.0.1:58318 - "POST /node/69cc79ac8cd9cbcdc945a871 HTTP/1.0" 202 Accepted 2026-04-01 01:49:50,008 - lava_callback - INFO - Starting async job submit for node 69cc79ac8cd9cbcdc945a871 2026-04-01 01:49:50,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:49:50,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc79ac8cd9cbcdc945a871 HTTP/1.1" 200 None 2026-04-01 01:49:50,025 - lava_callback - DEBUG - Processing node 69cc79ac8cd9cbcdc945a871: baseline-nfs-arm64 2026-04-01 01:49:50,026 - lava_callback - WARNING - Failed to upload log 2026-04-01 01:49:50,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:49:50,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:49:50,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc79ac8cd9cbcdc945a871/lava-logs.yaml 2026-04-01 01:49:50,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:49:50,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:49:50,346 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc79ac8cd9cbcdc945a871/lava_callback.json.gz 2026-04-01 01:49:50,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:49:50,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc79ac8cd9cbcdc945a871 HTTP/1.1" 200 None 2026-04-01 01:49:50,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:49:50,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:49:50,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:49:50,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc79ac8cd9cbcdc945a871 HTTP/1.1" 200 None 2026-04-01 01:49:50,411 - lava_callback - INFO - Completed processing callback for node 69cc79ac8cd9cbcdc945a871 INFO: 172.20.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:50:14,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:14,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-01 01:50:14,971 - lava_callback - INFO - Received callback for node 69cc67058cd9cbcdc945971b INFO: 172.20.0.1:35786 - "POST /node/69cc67058cd9cbcdc945971b HTTP/1.0" 202 Accepted 2026-04-01 01:50:14,977 - lava_callback - INFO - Starting async job submit for node 69cc67058cd9cbcdc945971b 2026-04-01 01:50:14,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:15,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc67058cd9cbcdc945971b HTTP/1.1" 200 None 2026-04-01 01:50:15,015 - lava_callback - DEBUG - Processing node 69cc67058cd9cbcdc945971b: baseline-arm INFO: 172.20.0.1:35790 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 01:50:15,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:50:15,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:50:15,496 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc67058cd9cbcdc945971b/log.txt.gz 2026-04-01 01:50:15,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:50:15,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:50:15,834 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc67058cd9cbcdc945971b/lava-logs.yaml 2026-04-01 01:50:15,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:50:15,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:50:15,997 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc67058cd9cbcdc945971b/lava_callback.json.gz 2026-04-01 01:50:16,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:16,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc67058cd9cbcdc945971b HTTP/1.1" 200 None 2026-04-01 01:50:16,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:16,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64828cd9cbcdc945964d HTTP/1.1" 200 None 2026-04-01 01:50:16,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:16,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc67058cd9cbcdc945971b HTTP/1.1" 200 None 2026-04-01 01:50:16,099 - lava_callback - INFO - Completed processing callback for node 69cc67058cd9cbcdc945971b INFO: 172.20.0.1:35804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60730 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:50:44,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:44,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40486 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:50:55,226 - lava_callback - INFO - Received callback for node 69cc70138cd9cbcdc9459f46 INFO: 172.20.0.1:45196 - "POST /node/69cc70138cd9cbcdc9459f46 HTTP/1.0" 202 Accepted 2026-04-01 01:50:55,232 - lava_callback - INFO - Starting async job submit for node 69cc70138cd9cbcdc9459f46 2026-04-01 01:50:55,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:55,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70138cd9cbcdc9459f46 HTTP/1.1" 200 None 2026-04-01 01:50:55,272 - lava_callback - DEBUG - Processing node 69cc70138cd9cbcdc9459f46: baseline-arm64-broonie 2026-04-01 01:50:55,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:50:55,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:50:55,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc70138cd9cbcdc9459f46/log.txt.gz 2026-04-01 01:50:55,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:50:56,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:50:56,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc70138cd9cbcdc9459f46/lava-logs.yaml 2026-04-01 01:50:56,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:50:56,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:50:56,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc70138cd9cbcdc9459f46/lava_callback.json.gz 2026-04-01 01:50:56,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:56,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70138cd9cbcdc9459f46 HTTP/1.1" 200 None 2026-04-01 01:50:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:56,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:50:56,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:50:56,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70138cd9cbcdc9459f46 HTTP/1.1" 200 None 2026-04-01 01:50:56,329 - lava_callback - INFO - Completed processing callback for node 69cc70138cd9cbcdc9459f46 INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:51:14,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:51:14,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59880 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 01:53:15,617 - lava_callback - INFO - Received callback for node 69cc7a508cd9cbcdc945a8e5 INFO: 172.20.0.1:59886 - "POST /node/69cc7a508cd9cbcdc945a8e5 HTTP/1.0" 202 Accepted 2026-04-01 01:53:15,623 - lava_callback - INFO - Starting async job submit for node 69cc7a508cd9cbcdc945a8e5 2026-04-01 01:53:15,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:53:15,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7a508cd9cbcdc945a8e5 HTTP/1.1" 200 None 2026-04-01 01:53:15,692 - lava_callback - DEBUG - Processing node 69cc7a508cd9cbcdc945a8e5: baseline-arm64 2026-04-01 01:53:15,826 - lava_callback - WARNING - Failed to upload log 2026-04-01 01:53:15,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:53:16,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:53:16,116 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7a508cd9cbcdc945a8e5/lava-logs.yaml 2026-04-01 01:53:16,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:53:16,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:53:16,420 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7a508cd9cbcdc945a8e5/lava_callback.json.gz 2026-04-01 01:53:16,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:53:16,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7a508cd9cbcdc945a8e5 HTTP/1.1" 200 None 2026-04-01 01:53:16,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:53:16,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:53:16,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:59888 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:53:16,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7a508cd9cbcdc945a8e5 HTTP/1.1" 200 None 2026-04-01 01:53:16,600 - lava_callback - INFO - Completed processing callback for node 69cc7a508cd9cbcdc945a8e5 INFO: 172.20.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50254 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:53:44,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:53:44,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56922 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:55:21,869 - lava_callback - INFO - Received callback for node 69cc7aa98cd9cbcdc945a923 INFO: 172.20.0.1:53492 - "POST /node/69cc7aa98cd9cbcdc945a923 HTTP/1.0" 202 Accepted 2026-04-01 01:55:21,876 - lava_callback - INFO - Starting async job submit for node 69cc7aa98cd9cbcdc945a923 2026-04-01 01:55:21,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:55:21,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7aa98cd9cbcdc945a923 HTTP/1.1" 200 None 2026-04-01 01:55:21,921 - lava_callback - DEBUG - Processing node 69cc7aa98cd9cbcdc945a923: baseline-arm64 2026-04-01 01:55:22,188 - lava_callback - WARNING - Failed to upload log 2026-04-01 01:55:22,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:55:22,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:55:22,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7aa98cd9cbcdc945a923/lava-logs.yaml 2026-04-01 01:55:22,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:55:22,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:55:22,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7aa98cd9cbcdc945a923/lava_callback.json.gz 2026-04-01 01:55:22,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:55:22,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7aa98cd9cbcdc945a923 HTTP/1.1" 200 None 2026-04-01 01:55:22,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:55:22,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:55:22,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:55:22,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7aa98cd9cbcdc945a923 HTTP/1.1" 200 None 2026-04-01 01:55:22,859 - lava_callback - INFO - Completed processing callback for node 69cc7aa98cd9cbcdc945a923 INFO: 172.20.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44528 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:55:44,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:55:44,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58090 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:56:02,487 - lava_callback - INFO - Received callback for node 69c9d8253c3aafdab4bea5e5 INFO: 172.20.0.1:47166 - "POST /node/69c9d8253c3aafdab4bea5e5 HTTP/1.0" 202 Accepted 2026-04-01 01:56:02,501 - lava_callback - INFO - Starting async job submit for node 69c9d8253c3aafdab4bea5e5 2026-04-01 01:56:02,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:02,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d8253c3aafdab4bea5e5 HTTP/1.1" 200 None 2026-04-01 01:56:02,527 - lava_callback - DEBUG - Processing node 69c9d8253c3aafdab4bea5e5: baseline-arm64-baylibre 2026-04-01 01:56:02,527 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d8253c3aafdab4bea5e5 2026-04-01 01:56:02,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:02,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d8253c3aafdab4bea5e5 HTTP/1.1" 200 None 2026-04-01 01:56:02,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:02,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 01:56:02,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:02,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d8253c3aafdab4bea5e5 HTTP/1.1" 200 None 2026-04-01 01:56:02,600 - lava_callback - ERROR - Error processing callback for node 69c9d8253c3aafdab4bea5e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45090 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:56:43,909 - lava_callback - INFO - Received callback for node 69cc7b018cd9cbcdc945a974 INFO: 172.20.0.1:48440 - "POST /node/69cc7b018cd9cbcdc945a974 HTTP/1.0" 202 Accepted 2026-04-01 01:56:43,916 - lava_callback - INFO - Starting async job submit for node 69cc7b018cd9cbcdc945a974 2026-04-01 01:56:43,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:43,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7b018cd9cbcdc945a974 HTTP/1.1" 200 None 2026-04-01 01:56:43,955 - lava_callback - DEBUG - Processing node 69cc7b018cd9cbcdc945a974: baseline-arm64 2026-04-01 01:56:44,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:56:44,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:56:44,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7b018cd9cbcdc945a974/log.txt.gz 2026-04-01 01:56:44,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:56:45,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:56:45,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7b018cd9cbcdc945a974/lava-logs.yaml 2026-04-01 01:56:45,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 01:56:45,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 01:56:45,271 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7b018cd9cbcdc945a974/lava_callback.json.gz 2026-04-01 01:56:45,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:45,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7b018cd9cbcdc945a974 HTTP/1.1" 200 None 2026-04-01 01:56:45,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:45,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 01:56:45,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:56:45,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7b018cd9cbcdc945a974 HTTP/1.1" 200 None 2026-04-01 01:56:45,425 - lava_callback - INFO - Completed processing callback for node 69cc7b018cd9cbcdc945a974 INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:57:14,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:57:14,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46642 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:57:22,532 - lava_callback - INFO - Received callback for node 69c9d87a3c3aafdab4bea5f0 INFO: 172.20.0.1:37458 - "POST /node/69c9d87a3c3aafdab4bea5f0 HTTP/1.0" 202 Accepted 2026-04-01 01:57:22,539 - lava_callback - INFO - Starting async job submit for node 69c9d87a3c3aafdab4bea5f0 2026-04-01 01:57:22,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:57:22,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d87a3c3aafdab4bea5f0 HTTP/1.1" 200 None 2026-04-01 01:57:22,558 - lava_callback - DEBUG - Processing node 69c9d87a3c3aafdab4bea5f0: baseline-arm64-baylibre 2026-04-01 01:57:22,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d87a3c3aafdab4bea5f0 2026-04-01 01:57:22,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:57:22,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d87a3c3aafdab4bea5f0 HTTP/1.1" 200 None 2026-04-01 01:57:22,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:57:22,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 01:57:22,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:57:22,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d87a3c3aafdab4bea5f0 HTTP/1.1" 200 None 2026-04-01 01:57:22,625 - lava_callback - ERROR - Error processing callback for node 69c9d87a3c3aafdab4bea5f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK 2026-04-01 01:58:42,568 - lava_callback - INFO - Received callback for node 69c9d8ca3c3aafdab4bea5fc INFO: 172.20.0.1:45354 - "POST /node/69c9d8ca3c3aafdab4bea5fc HTTP/1.0" 202 Accepted 2026-04-01 01:58:42,574 - lava_callback - INFO - Starting async job submit for node 69c9d8ca3c3aafdab4bea5fc 2026-04-01 01:58:42,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:58:42,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d8ca3c3aafdab4bea5fc HTTP/1.1" 200 None 2026-04-01 01:58:42,591 - lava_callback - DEBUG - Processing node 69c9d8ca3c3aafdab4bea5fc: baseline-arm64-baylibre 2026-04-01 01:58:42,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d8ca3c3aafdab4bea5fc 2026-04-01 01:58:42,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:58:42,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d8ca3c3aafdab4bea5fc HTTP/1.1" 200 None 2026-04-01 01:58:42,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:58:42,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 01:58:42,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 01:58:42,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d8ca3c3aafdab4bea5fc HTTP/1.1" 200 None 2026-04-01 01:58:42,690 - lava_callback - ERROR - Error processing callback for node 69c9d8ca3c3aafdab4bea5fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56786 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:01:02,720 - lava_callback - INFO - Received callback for node 69c9d9493c3aafdab4bea610 INFO: 172.20.0.1:56792 - "POST /node/69c9d9493c3aafdab4bea610 HTTP/1.0" 202 Accepted 2026-04-01 02:01:02,726 - lava_callback - INFO - Starting async job submit for node 69c9d9493c3aafdab4bea610 2026-04-01 02:01:02,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:01:02,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d9493c3aafdab4bea610 HTTP/1.1" 200 None 2026-04-01 02:01:02,742 - lava_callback - DEBUG - Processing node 69c9d9493c3aafdab4bea610: baseline-arm64-baylibre 2026-04-01 02:01:02,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9d9493c3aafdab4bea610 2026-04-01 02:01:02,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:01:02,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9d9493c3aafdab4bea610 HTTP/1.1" 200 None 2026-04-01 02:01:02,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:01:02,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 02:01:02,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:01:02,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9d9493c3aafdab4bea610 HTTP/1.1" 200 None 2026-04-01 02:01:02,830 - lava_callback - ERROR - Error processing callback for node 69c9d9493c3aafdab4bea610: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55576 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:04:37,237 - lava_callback - INFO - Received callback for node 69cc7c7f8cd9cbcdc945aab9 INFO: 172.20.0.1:55588 - "POST /node/69cc7c7f8cd9cbcdc945aab9 HTTP/1.0" 202 Accepted 2026-04-01 02:04:37,246 - lava_callback - INFO - Starting async job submit for node 69cc7c7f8cd9cbcdc945aab9 2026-04-01 02:04:37,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:37,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7c7f8cd9cbcdc945aab9 HTTP/1.1" 200 None 2026-04-01 02:04:37,317 - lava_callback - DEBUG - Processing node 69cc7c7f8cd9cbcdc945aab9: baseline-arm64-foundriesio 2026-04-01 02:04:38,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:04:38,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:04:38,278 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc7c7f8cd9cbcdc945aab9/log.txt.gz 2026-04-01 02:04:38,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:04:39,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:04:39,046 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc7c7f8cd9cbcdc945aab9/lava-logs.yaml 2026-04-01 02:04:39,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:04:39,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:04:39,274 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc7c7f8cd9cbcdc945aab9/lava_callback.json.gz DEBUG: 69cc7c7f8cd9cbcdc945aab9 Transitting job node result: pass -> incomplete 2026-04-01 02:04:39,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:39,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7c7f8cd9cbcdc945aab9 HTTP/1.1" 200 None 2026-04-01 02:04:39,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:39,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:04:39,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:39,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7c7f8cd9cbcdc945aab9 HTTP/1.1" 200 None 2026-04-01 02:04:39,375 - lava_callback - INFO - Completed processing callback for node 69cc7c7f8cd9cbcdc945aab9 2026-04-01 02:04:44,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:44,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:04:50,771 - lava_callback - INFO - Received callback for node 69c9da3f3c3aafdab4bea62e INFO: 172.20.0.1:55446 - "POST /node/69c9da3f3c3aafdab4bea62e HTTP/1.0" 202 Accepted 2026-04-01 02:04:50,777 - lava_callback - INFO - Starting async job submit for node 69c9da3f3c3aafdab4bea62e 2026-04-01 02:04:50,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:50,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9da3f3c3aafdab4bea62e HTTP/1.1" 200 None 2026-04-01 02:04:50,792 - lava_callback - DEBUG - Processing node 69c9da3f3c3aafdab4bea62e: baseline-arm64-clabbe 2026-04-01 02:04:50,792 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9da3f3c3aafdab4bea62e 2026-04-01 02:04:50,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:50,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9da3f3c3aafdab4bea62e HTTP/1.1" 200 None 2026-04-01 02:04:50,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:50,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 02:04:50,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:04:50,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9da3f3c3aafdab4bea62e HTTP/1.1" 200 None 2026-04-01 02:04:50,875 - lava_callback - ERROR - Error processing callback for node 69c9da3f3c3aafdab4bea62e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:05:16,805 - lava_callback - INFO - Received callback for node 69cbfcac8df9359ff48983bf INFO: 172.20.0.1:40856 - "POST /node/69cbfcac8df9359ff48983bf HTTP/1.0" 202 Accepted 2026-04-01 02:05:16,811 - lava_callback - INFO - Starting async job submit for node 69cbfcac8df9359ff48983bf 2026-04-01 02:05:16,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:05:16,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcac8df9359ff48983bf HTTP/1.1" 200 None 2026-04-01 02:05:16,851 - lava_callback - DEBUG - Processing node 69cbfcac8df9359ff48983bf: baseline-arm64 2026-04-01 02:05:17,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:05:17,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:05:17,566 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcac8df9359ff48983bf/log.txt.gz 2026-04-01 02:05:17,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:05:18,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:05:18,070 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcac8df9359ff48983bf/lava-logs.yaml 2026-04-01 02:05:18,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:05:18,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:05:18,233 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcac8df9359ff48983bf/lava_callback.json.gz 2026-04-01 02:05:18,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:05:18,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcac8df9359ff48983bf HTTP/1.1" 200 None 2026-04-01 02:05:18,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:05:18,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 02:05:18,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:05:18,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcac8df9359ff48983bf HTTP/1.1" 200 None 2026-04-01 02:05:18,373 - lava_callback - INFO - Completed processing callback for node 69cbfcac8df9359ff48983bf INFO: 172.20.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54592 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:05:44,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:05:44,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36080 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:06:08,280 - lava_callback - INFO - Received callback for node 69cbfcbc8df9359ff48983e8 INFO: 172.20.0.1:36096 - "POST /node/69cbfcbc8df9359ff48983e8 HTTP/1.0" 202 Accepted 2026-04-01 02:06:08,287 - lava_callback - INFO - Starting async job submit for node 69cbfcbc8df9359ff48983e8 2026-04-01 02:06:08,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:08,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbc8df9359ff48983e8 HTTP/1.1" 200 None 2026-04-01 02:06:08,329 - lava_callback - DEBUG - Processing node 69cbfcbc8df9359ff48983e8: baseline-nfs-arm64 2026-04-01 02:06:08,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:06:08,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:06:08,996 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbc8df9359ff48983e8/log.txt.gz 2026-04-01 02:06:09,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:06:09,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:06:09,510 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbc8df9359ff48983e8/lava-logs.yaml 2026-04-01 02:06:09,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:06:09,705 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:06:09,707 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbc8df9359ff48983e8/lava_callback.json.gz 2026-04-01 02:06:09,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:09,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbc8df9359ff48983e8 HTTP/1.1" 200 None 2026-04-01 02:06:09,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:09,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 02:06:09,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:09,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcbc8df9359ff48983e8 HTTP/1.1" 200 None 2026-04-01 02:06:09,858 - lava_callback - INFO - Completed processing callback for node 69cbfcbc8df9359ff48983e8 2026-04-01 02:06:14,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:14,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47956 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:06:51,085 - lava_callback - INFO - Received callback for node 69c9dab43c3aafdab4bea640 INFO: 172.20.0.1:40180 - "POST /node/69c9dab43c3aafdab4bea640 HTTP/1.0" 202 Accepted 2026-04-01 02:06:51,094 - lava_callback - INFO - Starting async job submit for node 69c9dab43c3aafdab4bea640 2026-04-01 02:06:51,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:51,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dab43c3aafdab4bea640 HTTP/1.1" 200 None 2026-04-01 02:06:51,118 - lava_callback - DEBUG - Processing node 69c9dab43c3aafdab4bea640: baseline-arm64-clabbe 2026-04-01 02:06:51,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dab43c3aafdab4bea640 2026-04-01 02:06:51,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:51,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dab43c3aafdab4bea640 HTTP/1.1" 200 None 2026-04-01 02:06:51,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:51,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 02:06:51,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:06:51,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dab43c3aafdab4bea640 HTTP/1.1" 200 None 2026-04-01 02:06:51,196 - lava_callback - ERROR - Error processing callback for node 69c9dab43c3aafdab4bea640: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42620 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:07:32,148 - lava_callback - INFO - Received callback for node 69cc79fd8cd9cbcdc945a8af INFO: 172.20.0.1:37138 - "POST /node/69cc79fd8cd9cbcdc945a8af HTTP/1.0" 202 Accepted 2026-04-01 02:07:32,155 - lava_callback - INFO - Starting async job submit for node 69cc79fd8cd9cbcdc945a8af 2026-04-01 02:07:32,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:32,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc79fd8cd9cbcdc945a8af HTTP/1.1" 200 None 2026-04-01 02:07:32,228 - lava_callback - DEBUG - Processing node 69cc79fd8cd9cbcdc945a8af: baseline-arm 2026-04-01 02:07:32,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:07:32,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:07:32,970 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc79fd8cd9cbcdc945a8af/log.txt.gz 2026-04-01 02:07:33,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:07:33,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:07:33,630 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc79fd8cd9cbcdc945a8af/lava-logs.yaml 2026-04-01 02:07:33,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:07:33,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:07:33,851 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc79fd8cd9cbcdc945a8af/lava_callback.json.gz DEBUG: Setup failed for 69cc79fd8cd9cbcdc945a8af. Transitting job node result: incomplete -> fail 2026-04-01 02:07:33,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:33,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc79fd8cd9cbcdc945a8af HTTP/1.1" 200 None 2026-04-01 02:07:33,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:33,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 02:07:33,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:34,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc79fd8cd9cbcdc945a8af HTTP/1.1" 200 None 2026-04-01 02:07:34,147 - lava_callback - INFO - Completed processing callback for node 69cc79fd8cd9cbcdc945a8af INFO: 172.20.0.1:37146 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:07:44,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:44,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34418 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:07:55,527 - lava_callback - INFO - Received callback for node 69cc67078cd9cbcdc945971e INFO: 172.20.0.1:37212 - "POST /node/69cc67078cd9cbcdc945971e HTTP/1.0" 202 Accepted 2026-04-01 02:07:55,533 - lava_callback - INFO - Starting async job submit for node 69cc67078cd9cbcdc945971e 2026-04-01 02:07:55,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:55,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc67078cd9cbcdc945971e HTTP/1.1" 200 None 2026-04-01 02:07:55,607 - lava_callback - DEBUG - Processing node 69cc67078cd9cbcdc945971e: ltp-smoke 2026-04-01 02:07:56,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:07:56,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:07:56,318 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc67078cd9cbcdc945971e/log.txt.gz INFO: 172.20.0.1:37220 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:07:56,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:07:56,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:07:56,899 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc67078cd9cbcdc945971e/lava-logs.yaml 2026-04-01 02:07:56,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:07:57,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:07:57,096 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc67078cd9cbcdc945971e/lava_callback.json.gz 2026-04-01 02:07:57,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:57,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc67078cd9cbcdc945971e HTTP/1.1" 200 None 2026-04-01 02:07:57,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:57,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64828cd9cbcdc945964d HTTP/1.1" 200 None 2026-04-01 02:07:57,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:07:57,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc67078cd9cbcdc945971e HTTP/1.1" 200 None 2026-04-01 02:07:57,459 - lava_callback - INFO - Completed processing callback for node 69cc67078cd9cbcdc945971e INFO: 172.20.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:08:10,860 - lava_callback - INFO - Received callback for node 69c9db013c3aafdab4bea650 INFO: 172.20.0.1:41814 - "POST /node/69c9db013c3aafdab4bea650 HTTP/1.0" 202 Accepted 2026-04-01 02:08:10,865 - lava_callback - INFO - Starting async job submit for node 69c9db013c3aafdab4bea650 2026-04-01 02:08:10,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:08:10,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9db013c3aafdab4bea650 HTTP/1.1" 200 None 2026-04-01 02:08:10,881 - lava_callback - DEBUG - Processing node 69c9db013c3aafdab4bea650: baseline-arm64-clabbe 2026-04-01 02:08:10,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9db013c3aafdab4bea650 2026-04-01 02:08:10,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:08:10,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9db013c3aafdab4bea650 HTTP/1.1" 200 None 2026-04-01 02:08:10,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:08:10,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 02:08:10,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:08:10,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9db013c3aafdab4bea650 HTTP/1.1" 200 None 2026-04-01 02:08:10,940 - lava_callback - ERROR - Error processing callback for node 69c9db013c3aafdab4bea650: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 02:08:14,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:08:14,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42562 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:09:30,947 - lava_callback - INFO - Received callback for node 69c9db523c3aafdab4bea660 INFO: 172.20.0.1:57140 - "POST /node/69c9db523c3aafdab4bea660 HTTP/1.0" 202 Accepted 2026-04-01 02:09:30,953 - lava_callback - INFO - Starting async job submit for node 69c9db523c3aafdab4bea660 2026-04-01 02:09:30,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:09:30,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9db523c3aafdab4bea660 HTTP/1.1" 200 None 2026-04-01 02:09:30,970 - lava_callback - DEBUG - Processing node 69c9db523c3aafdab4bea660: baseline-arm64-clabbe 2026-04-01 02:09:30,970 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9db523c3aafdab4bea660 2026-04-01 02:09:30,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:09:30,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9db523c3aafdab4bea660 HTTP/1.1" 200 None 2026-04-01 02:09:30,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:09:31,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 02:09:31,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:09:31,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9db523c3aafdab4bea660 HTTP/1.1" 200 None 2026-04-01 02:09:31,049 - lava_callback - ERROR - Error processing callback for node 69c9db523c3aafdab4bea660: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39692 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:10:50,969 - lava_callback - INFO - Received callback for node 69c9dba23c3aafdab4bea66a INFO: 172.20.0.1:41056 - "POST /node/69c9dba23c3aafdab4bea66a HTTP/1.0" 202 Accepted 2026-04-01 02:10:50,976 - lava_callback - INFO - Starting async job submit for node 69c9dba23c3aafdab4bea66a 2026-04-01 02:10:50,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:10:50,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dba23c3aafdab4bea66a HTTP/1.1" 200 None 2026-04-01 02:10:50,993 - lava_callback - DEBUG - Processing node 69c9dba23c3aafdab4bea66a: baseline-arm64-clabbe 2026-04-01 02:10:50,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dba23c3aafdab4bea66a 2026-04-01 02:10:50,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:10:51,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dba23c3aafdab4bea66a HTTP/1.1" 200 None 2026-04-01 02:10:51,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:10:51,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 02:10:51,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:10:51,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dba23c3aafdab4bea66a HTTP/1.1" 200 None 2026-04-01 02:10:51,050 - lava_callback - ERROR - Error processing callback for node 69c9dba23c3aafdab4bea66a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35214 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:11:21,302 - lava_callback - INFO - Received callback for node 69cc7a318cd9cbcdc945a8dc INFO: 172.20.0.1:47286 - "POST /node/69cc7a318cd9cbcdc945a8dc HTTP/1.0" 202 Accepted 2026-04-01 02:11:21,350 - lava_callback - INFO - Starting async job submit for node 69cc7a318cd9cbcdc945a8dc 2026-04-01 02:11:21,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:11:21,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7a318cd9cbcdc945a8dc HTTP/1.1" 200 None 2026-04-01 02:11:21,410 - lava_callback - DEBUG - Processing node 69cc7a318cd9cbcdc945a8dc: baseline-arm-pengutronix 2026-04-01 02:11:21,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:11:21,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:11:21,737 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc7a318cd9cbcdc945a8dc/log.txt.gz 2026-04-01 02:11:21,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:11:22,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:11:22,037 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc7a318cd9cbcdc945a8dc/lava-logs.yaml 2026-04-01 02:11:22,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:11:22,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:11:22,204 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc7a318cd9cbcdc945a8dc/lava_callback.json.gz DEBUG: Setup failed for 69cc7a318cd9cbcdc945a8dc. Transitting job node result: incomplete -> fail 2026-04-01 02:11:22,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:11:22,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7a318cd9cbcdc945a8dc HTTP/1.1" 200 None 2026-04-01 02:11:22,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:11:22,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 02:11:22,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:11:22,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7a318cd9cbcdc945a8dc HTTP/1.1" 200 None 2026-04-01 02:11:22,402 - lava_callback - INFO - Completed processing callback for node 69cc7a318cd9cbcdc945a8dc INFO: 172.20.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36152 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:11:44,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:11:44,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55900 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:12:11,091 - lava_callback - INFO - Received callback for node 69c9dbf23c3aafdab4bea673 INFO: 172.20.0.1:51448 - "POST /node/69c9dbf23c3aafdab4bea673 HTTP/1.0" 202 Accepted 2026-04-01 02:12:11,099 - lava_callback - INFO - Starting async job submit for node 69c9dbf23c3aafdab4bea673 2026-04-01 02:12:11,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:11,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dbf23c3aafdab4bea673 HTTP/1.1" 200 None 2026-04-01 02:12:11,114 - lava_callback - DEBUG - Processing node 69c9dbf23c3aafdab4bea673: baseline-arm64-clabbe 2026-04-01 02:12:11,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dbf23c3aafdab4bea673 2026-04-01 02:12:11,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:11,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dbf23c3aafdab4bea673 HTTP/1.1" 200 None 2026-04-01 02:12:11,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:11,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 02:12:11,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:11,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dbf23c3aafdab4bea673 HTTP/1.1" 200 None 2026-04-01 02:12:11,166 - lava_callback - ERROR - Error processing callback for node 69c9dbf23c3aafdab4bea673: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56434 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:12:55,828 - lava_callback - INFO - Received callback for node 69cc7dd58cd9cbcdc945abe2 INFO: 172.20.0.1:51506 - "POST /node/69cc7dd58cd9cbcdc945abe2 HTTP/1.0" 202 Accepted 2026-04-01 02:12:55,835 - lava_callback - INFO - Starting async job submit for node 69cc7dd58cd9cbcdc945abe2 2026-04-01 02:12:55,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:55,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7dd58cd9cbcdc945abe2 HTTP/1.1" 200 None 2026-04-01 02:12:55,890 - lava_callback - DEBUG - Processing node 69cc7dd58cd9cbcdc945abe2: baseline-arm64 2026-04-01 02:12:56,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:12:56,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:12:56,474 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7dd58cd9cbcdc945abe2/log.txt.gz INFO: 172.20.0.1:51514 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:12:56,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:12:56,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:12:56,936 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7dd58cd9cbcdc945abe2/lava-logs.yaml 2026-04-01 02:12:56,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:12:57,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:12:57,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc7dd58cd9cbcdc945abe2/lava_callback.json.gz DEBUG: Setup failed for 69cc7dd58cd9cbcdc945abe2. Transitting job node result: incomplete -> fail 2026-04-01 02:12:57,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:57,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7dd58cd9cbcdc945abe2 HTTP/1.1" 200 None 2026-04-01 02:12:57,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:57,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:12:57,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:12:57,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7dd58cd9cbcdc945abe2 HTTP/1.1" 200 None 2026-04-01 02:12:57,395 - lava_callback - INFO - Completed processing callback for node 69cc7dd58cd9cbcdc945abe2 INFO: 172.20.0.1:48192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48202 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:13:14,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:13:14,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:13:31,090 - lava_callback - INFO - Received callback for node 69c9dc423c3aafdab4bea679 INFO: 172.20.0.1:53342 - "POST /node/69c9dc423c3aafdab4bea679 HTTP/1.0" 202 Accepted 2026-04-01 02:13:31,095 - lava_callback - INFO - Starting async job submit for node 69c9dc423c3aafdab4bea679 2026-04-01 02:13:31,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:13:31,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dc423c3aafdab4bea679 HTTP/1.1" 200 None 2026-04-01 02:13:31,111 - lava_callback - DEBUG - Processing node 69c9dc423c3aafdab4bea679: baseline-arm64-clabbe 2026-04-01 02:13:31,112 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dc423c3aafdab4bea679 2026-04-01 02:13:31,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:13:31,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dc423c3aafdab4bea679 HTTP/1.1" 200 None 2026-04-01 02:13:31,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:13:31,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 02:13:31,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:13:31,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dc423c3aafdab4bea679 HTTP/1.1" 200 None 2026-04-01 02:13:31,184 - lava_callback - ERROR - Error processing callback for node 69c9dc423c3aafdab4bea679: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58166 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:15:11,466 - lava_callback - INFO - Received callback for node 69c9dca43c3aafdab4bea685 INFO: 172.20.0.1:54336 - "POST /node/69c9dca43c3aafdab4bea685 HTTP/1.0" 202 Accepted 2026-04-01 02:15:11,472 - lava_callback - INFO - Starting async job submit for node 69c9dca43c3aafdab4bea685 2026-04-01 02:15:11,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:15:11,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dca43c3aafdab4bea685 HTTP/1.1" 200 None 2026-04-01 02:15:11,491 - lava_callback - DEBUG - Processing node 69c9dca43c3aafdab4bea685: baseline-arm64-clabbe 2026-04-01 02:15:11,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dca43c3aafdab4bea685 2026-04-01 02:15:11,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:15:11,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dca43c3aafdab4bea685 HTTP/1.1" 200 None 2026-04-01 02:15:11,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:15:11,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 02:15:11,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:15:11,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dca43c3aafdab4bea685 HTTP/1.1" 200 None 2026-04-01 02:15:11,548 - lava_callback - ERROR - Error processing callback for node 69c9dca43c3aafdab4bea685: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:16:51,263 - lava_callback - INFO - Received callback for node 69c9dd0a3c3aafdab4bea691 INFO: 172.20.0.1:48534 - "POST /node/69c9dd0a3c3aafdab4bea691 HTTP/1.0" 202 Accepted 2026-04-01 02:16:51,268 - lava_callback - INFO - Starting async job submit for node 69c9dd0a3c3aafdab4bea691 2026-04-01 02:16:51,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:51,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dd0a3c3aafdab4bea691 HTTP/1.1" 200 None 2026-04-01 02:16:51,284 - lava_callback - DEBUG - Processing node 69c9dd0a3c3aafdab4bea691: baseline-arm64-clabbe 2026-04-01 02:16:51,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dd0a3c3aafdab4bea691 2026-04-01 02:16:51,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:51,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dd0a3c3aafdab4bea691 HTTP/1.1" 200 None 2026-04-01 02:16:51,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:51,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 02:16:51,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:51,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dd0a3c3aafdab4bea691 HTTP/1.1" 200 None 2026-04-01 02:16:51,374 - lava_callback - ERROR - Error processing callback for node 69c9dd0a3c3aafdab4bea691: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 02:16:53,113 - lava_callback - INFO - Received callback for node 69cc7b608cd9cbcdc945a9cb INFO: 172.20.0.1:48540 - "POST /node/69cc7b608cd9cbcdc945a9cb HTTP/1.0" 202 Accepted 2026-04-01 02:16:53,121 - lava_callback - INFO - Starting async job submit for node 69cc7b608cd9cbcdc945a9cb 2026-04-01 02:16:53,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:53,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7b608cd9cbcdc945a9cb HTTP/1.1" 200 None 2026-04-01 02:16:53,180 - lava_callback - DEBUG - Processing node 69cc7b608cd9cbcdc945a9cb: baseline-nfs-arm64 2026-04-01 02:16:54,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:16:54,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:16:54,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc7b608cd9cbcdc945a9cb/log.txt.gz 2026-04-01 02:16:55,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:16:55,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:16:55,385 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc7b608cd9cbcdc945a9cb/lava-logs.yaml 2026-04-01 02:16:55,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:16:55,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:16:55,602 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc7b608cd9cbcdc945a9cb/lava_callback.json.gz DEBUG: Setup failed for 69cc7b608cd9cbcdc945a9cb. Transitting job node result: incomplete -> fail 2026-04-01 02:16:55,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:55,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7b608cd9cbcdc945a9cb HTTP/1.1" 200 None 2026-04-01 02:16:55,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:55,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:16:55,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:16:55,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7b608cd9cbcdc945a9cb HTTP/1.1" 200 None 2026-04-01 02:16:55,876 - lava_callback - INFO - Completed processing callback for node 69cc7b608cd9cbcdc945a9cb INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44234 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:17:14,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:17:15,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:18:11,327 - lava_callback - INFO - Received callback for node 69c9dd593c3aafdab4bea69f INFO: 172.20.0.1:47430 - "POST /node/69c9dd593c3aafdab4bea69f HTTP/1.0" 202 Accepted 2026-04-01 02:18:11,334 - lava_callback - INFO - Starting async job submit for node 69c9dd593c3aafdab4bea69f 2026-04-01 02:18:11,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:18:11,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dd593c3aafdab4bea69f HTTP/1.1" 200 None 2026-04-01 02:18:11,354 - lava_callback - DEBUG - Processing node 69c9dd593c3aafdab4bea69f: baseline-arm64-clabbe 2026-04-01 02:18:11,354 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dd593c3aafdab4bea69f 2026-04-01 02:18:11,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:18:11,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dd593c3aafdab4bea69f HTTP/1.1" 200 None 2026-04-01 02:18:11,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:18:11,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 02:18:11,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:18:11,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dd593c3aafdab4bea69f HTTP/1.1" 200 None 2026-04-01 02:18:11,411 - lava_callback - ERROR - Error processing callback for node 69c9dd593c3aafdab4bea69f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42760 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:21:11,053 - lava_callback - INFO - Received callback for node 69cc81098cd9cbcdc945adfd INFO: 172.20.0.1:41108 - "POST /node/69cc81098cd9cbcdc945adfd HTTP/1.0" 202 Accepted 2026-04-01 02:21:11,059 - lava_callback - INFO - Starting async job submit for node 69cc81098cd9cbcdc945adfd 2026-04-01 02:21:11,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:21:11,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc81098cd9cbcdc945adfd HTTP/1.1" 200 None 2026-04-01 02:21:11,083 - lava_callback - DEBUG - Processing node 69cc81098cd9cbcdc945adfd: baseline-nfs-arm64 2026-04-01 02:21:11,085 - lava_callback - WARNING - Failed to upload log 2026-04-01 02:21:11,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:21:11,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:21:11,235 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc81098cd9cbcdc945adfd/lava-logs.yaml 2026-04-01 02:21:11,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:21:11,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:21:11,359 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc81098cd9cbcdc945adfd/lava_callback.json.gz 2026-04-01 02:21:11,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:21:11,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc81098cd9cbcdc945adfd HTTP/1.1" 200 None 2026-04-01 02:21:11,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:21:11,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:21:11,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:21:11,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc81098cd9cbcdc945adfd HTTP/1.1" 200 None 2026-04-01 02:21:11,416 - lava_callback - INFO - Completed processing callback for node 69cc81098cd9cbcdc945adfd 2026-04-01 02:21:15,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:21:15,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57070 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:23:09,732 - lava_callback - INFO - Received callback for node 69cc815b8cd9cbcdc945ae32 INFO: 172.20.0.1:46136 - "POST /node/69cc815b8cd9cbcdc945ae32 HTTP/1.0" 202 Accepted 2026-04-01 02:23:09,738 - lava_callback - INFO - Starting async job submit for node 69cc815b8cd9cbcdc945ae32 2026-04-01 02:23:09,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:09,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc815b8cd9cbcdc945ae32 HTTP/1.1" 200 None 2026-04-01 02:23:09,787 - lava_callback - DEBUG - Processing node 69cc815b8cd9cbcdc945ae32: baseline-arm64 2026-04-01 02:23:09,930 - lava_callback - WARNING - Failed to upload log 2026-04-01 02:23:10,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:23:10,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:23:10,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc815b8cd9cbcdc945ae32/lava-logs.yaml 2026-04-01 02:23:10,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:23:10,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:23:10,394 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc815b8cd9cbcdc945ae32/lava_callback.json.gz 2026-04-01 02:23:10,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:10,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc815b8cd9cbcdc945ae32 HTTP/1.1" 200 None 2026-04-01 02:23:10,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:10,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:23:10,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:10,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc815b8cd9cbcdc945ae32 HTTP/1.1" 200 None 2026-04-01 02:23:10,570 - lava_callback - INFO - Completed processing callback for node 69cc815b8cd9cbcdc945ae32 2026-04-01 02:23:15,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:15,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51496 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:23:31,550 - lava_callback - INFO - Received callback for node 69c9de963c3aafdab4bea6ce INFO: 172.20.0.1:40520 - "POST /node/69c9de963c3aafdab4bea6ce HTTP/1.0" 202 Accepted 2026-04-01 02:23:31,558 - lava_callback - INFO - Starting async job submit for node 69c9de963c3aafdab4bea6ce 2026-04-01 02:23:31,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:31,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9de963c3aafdab4bea6ce HTTP/1.1" 200 None 2026-04-01 02:23:31,579 - lava_callback - DEBUG - Processing node 69c9de963c3aafdab4bea6ce: baseline-arm64-clabbe 2026-04-01 02:23:31,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9de963c3aafdab4bea6ce 2026-04-01 02:23:31,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:31,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9de963c3aafdab4bea6ce HTTP/1.1" 200 None 2026-04-01 02:23:31,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:31,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 02:23:31,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:23:31,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9de963c3aafdab4bea6ce HTTP/1.1" 200 None 2026-04-01 02:23:31,633 - lava_callback - ERROR - Error processing callback for node 69c9de963c3aafdab4bea6ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:24:51,614 - lava_callback - INFO - Received callback for node 69c9dee63c3aafdab4bea6dd INFO: 172.20.0.1:59600 - "POST /node/69c9dee63c3aafdab4bea6dd HTTP/1.0" 202 Accepted 2026-04-01 02:24:51,620 - lava_callback - INFO - Starting async job submit for node 69c9dee63c3aafdab4bea6dd 2026-04-01 02:24:51,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:24:51,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dee63c3aafdab4bea6dd HTTP/1.1" 200 None 2026-04-01 02:24:51,637 - lava_callback - DEBUG - Processing node 69c9dee63c3aafdab4bea6dd: baseline-arm64-clabbe 2026-04-01 02:24:51,637 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9dee63c3aafdab4bea6dd 2026-04-01 02:24:51,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:24:51,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9dee63c3aafdab4bea6dd HTTP/1.1" 200 None 2026-04-01 02:24:51,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:24:51,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-04-01 02:24:51,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:24:51,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9dee63c3aafdab4bea6dd HTTP/1.1" 200 None 2026-04-01 02:24:51,702 - lava_callback - ERROR - Error processing callback for node 69c9dee63c3aafdab4bea6dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60324 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:25:05,391 - lava_callback - INFO - Received callback for node 69cc81b88cd9cbcdc945ae6a INFO: 172.20.0.1:60340 - "POST /node/69cc81b88cd9cbcdc945ae6a HTTP/1.0" 202 Accepted 2026-04-01 02:25:05,399 - lava_callback - INFO - Starting async job submit for node 69cc81b88cd9cbcdc945ae6a 2026-04-01 02:25:05,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:25:05,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc81b88cd9cbcdc945ae6a HTTP/1.1" 200 None 2026-04-01 02:25:05,468 - lava_callback - DEBUG - Processing node 69cc81b88cd9cbcdc945ae6a: baseline-arm64 2026-04-01 02:25:05,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:25:05,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:25:05,850 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc81b88cd9cbcdc945ae6a/log.txt.gz 2026-04-01 02:25:05,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:25:06,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:25:06,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc81b88cd9cbcdc945ae6a/lava-logs.yaml 2026-04-01 02:25:06,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:25:06,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:25:06,203 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc81b88cd9cbcdc945ae6a/lava_callback.json.gz 2026-04-01 02:25:06,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:25:06,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc81b88cd9cbcdc945ae6a HTTP/1.1" 200 None 2026-04-01 02:25:06,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:25:06,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:25:06,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:25:06,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc81b88cd9cbcdc945ae6a HTTP/1.1" 200 None 2026-04-01 02:25:06,350 - lava_callback - INFO - Completed processing callback for node 69cc81b88cd9cbcdc945ae6a INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:25:15,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:25:15,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35204 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:27:34,750 - lava_callback - INFO - Received callback for node 69cc82138cd9cbcdc945aea3 INFO: 172.20.0.1:46024 - "POST /node/69cc82138cd9cbcdc945aea3 HTTP/1.0" 202 Accepted 2026-04-01 02:27:34,756 - lava_callback - INFO - Starting async job submit for node 69cc82138cd9cbcdc945aea3 2026-04-01 02:27:34,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:27:34,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc82138cd9cbcdc945aea3 HTTP/1.1" 200 None 2026-04-01 02:27:34,799 - lava_callback - DEBUG - Processing node 69cc82138cd9cbcdc945aea3: baseline-arm64 2026-04-01 02:27:35,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:27:35,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:27:35,391 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc82138cd9cbcdc945aea3/log.txt.gz 2026-04-01 02:27:35,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:27:35,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:27:35,949 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc82138cd9cbcdc945aea3/lava-logs.yaml 2026-04-01 02:27:35,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:27:36,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:27:36,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc82138cd9cbcdc945aea3/lava_callback.json.gz 2026-04-01 02:27:36,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:27:36,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc82138cd9cbcdc945aea3 HTTP/1.1" 200 None 2026-04-01 02:27:36,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:27:36,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:27:36,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:27:36,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc82138cd9cbcdc945aea3 HTTP/1.1" 200 None 2026-04-01 02:27:36,313 - lava_callback - INFO - Completed processing callback for node 69cc82138cd9cbcdc945aea3 INFO: 172.20.0.1:46040 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:27:45,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:27:45,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48774 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:33:32,665 - lava_callback - INFO - Received callback for node 69cc82dd8cd9cbcdc945af08 INFO: 172.20.0.1:38004 - "POST /node/69cc82dd8cd9cbcdc945af08 HTTP/1.0" 202 Accepted 2026-04-01 02:33:32,675 - lava_callback - INFO - Starting async job submit for node 69cc82dd8cd9cbcdc945af08 2026-04-01 02:33:32,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:33:32,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc82dd8cd9cbcdc945af08 HTTP/1.1" 200 None 2026-04-01 02:33:32,763 - lava_callback - DEBUG - Processing node 69cc82dd8cd9cbcdc945af08: baseline-arm64-foundriesio 2026-04-01 02:33:33,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:33:34,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:33:34,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc82dd8cd9cbcdc945af08/log.txt.gz 2026-04-01 02:33:34,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:33:35,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:33:35,110 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc82dd8cd9cbcdc945af08/lava-logs.yaml 2026-04-01 02:33:35,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:33:35,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:33:35,320 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc82dd8cd9cbcdc945af08/lava_callback.json.gz DEBUG: 69cc82dd8cd9cbcdc945af08 Transitting job node result: pass -> incomplete 2026-04-01 02:33:35,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:33:35,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc82dd8cd9cbcdc945af08 HTTP/1.1" 200 None 2026-04-01 02:33:35,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:33:35,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:33:35,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:33:35,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc82dd8cd9cbcdc945af08 HTTP/1.1" 200 None 2026-04-01 02:33:35,478 - lava_callback - INFO - Completed processing callback for node 69cc82dd8cd9cbcdc945af08 INFO: 172.20.0.1:38010 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:33:45,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:33:45,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60030 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:40:45,086 - lava_callback - INFO - Received callback for node 69c9e2a13c3aafdab4bea740 INFO: 172.20.0.1:46710 - "POST /node/69c9e2a13c3aafdab4bea740 HTTP/1.0" 202 Accepted 2026-04-01 02:40:45,092 - lava_callback - INFO - Starting async job submit for node 69c9e2a13c3aafdab4bea740 2026-04-01 02:40:45,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:40:45,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e2a13c3aafdab4bea740 HTTP/1.1" 200 None 2026-04-01 02:40:45,108 - lava_callback - DEBUG - Processing node 69c9e2a13c3aafdab4bea740: baseline-arm-baylibre 2026-04-01 02:40:45,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e2a13c3aafdab4bea740 2026-04-01 02:40:45,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:40:45,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e2a13c3aafdab4bea740 HTTP/1.1" 200 None 2026-04-01 02:40:45,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:40:45,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:40:45,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:40:45,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e2a13c3aafdab4bea740 HTTP/1.1" 200 None 2026-04-01 02:40:45,163 - lava_callback - ERROR - Error processing callback for node 69c9e2a13c3aafdab4bea740: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40048 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:41:24,951 - lava_callback - INFO - Received callback for node 69c9e2c53c3aafdab4bea74a INFO: 172.20.0.1:33922 - "POST /node/69c9e2c53c3aafdab4bea74a HTTP/1.0" 202 Accepted 2026-04-01 02:41:24,957 - lava_callback - INFO - Starting async job submit for node 69c9e2c53c3aafdab4bea74a 2026-04-01 02:41:24,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:41:24,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e2c53c3aafdab4bea74a HTTP/1.1" 200 None 2026-04-01 02:41:24,981 - lava_callback - DEBUG - Processing node 69c9e2c53c3aafdab4bea74a: baseline-arm-baylibre 2026-04-01 02:41:24,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e2c53c3aafdab4bea74a 2026-04-01 02:41:24,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:41:24,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e2c53c3aafdab4bea74a HTTP/1.1" 200 None 2026-04-01 02:41:24,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:41:25,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:41:25,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:41:25,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e2c53c3aafdab4bea74a HTTP/1.1" 200 None 2026-04-01 02:41:25,048 - lava_callback - ERROR - Error processing callback for node 69c9e2c53c3aafdab4bea74a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54910 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:42:05,028 - lava_callback - INFO - Received callback for node 69c9e2e93c3aafdab4bea75a INFO: 172.20.0.1:54926 - "POST /node/69c9e2e93c3aafdab4bea75a HTTP/1.0" 202 Accepted 2026-04-01 02:42:05,034 - lava_callback - INFO - Starting async job submit for node 69c9e2e93c3aafdab4bea75a 2026-04-01 02:42:05,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:05,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e2e93c3aafdab4bea75a HTTP/1.1" 200 None 2026-04-01 02:42:05,050 - lava_callback - DEBUG - Processing node 69c9e2e93c3aafdab4bea75a: baseline-arm-baylibre 2026-04-01 02:42:05,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e2e93c3aafdab4bea75a 2026-04-01 02:42:05,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:05,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e2e93c3aafdab4bea75a HTTP/1.1" 200 None 2026-04-01 02:42:05,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:05,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:42:05,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:05,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e2e93c3aafdab4bea75a HTTP/1.1" 200 None 2026-04-01 02:42:05,112 - lava_callback - ERROR - Error processing callback for node 69c9e2e93c3aafdab4bea75a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38470 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:42:24,983 - lava_callback - INFO - Received callback for node 69c9e30e3c3aafdab4bea766 INFO: 172.20.0.1:48104 - "POST /node/69c9e30e3c3aafdab4bea766 HTTP/1.0" 202 Accepted 2026-04-01 02:42:24,988 - lava_callback - INFO - Starting async job submit for node 69c9e30e3c3aafdab4bea766 2026-04-01 02:42:24,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:25,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e30e3c3aafdab4bea766 HTTP/1.1" 200 None 2026-04-01 02:42:25,004 - lava_callback - DEBUG - Processing node 69c9e30e3c3aafdab4bea766: baseline-arm-baylibre 2026-04-01 02:42:25,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e30e3c3aafdab4bea766 2026-04-01 02:42:25,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:25,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e30e3c3aafdab4bea766 HTTP/1.1" 200 None 2026-04-01 02:42:25,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:25,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:42:25,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:42:25,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e30e3c3aafdab4bea766 HTTP/1.1" 200 None 2026-04-01 02:42:25,064 - lava_callback - ERROR - Error processing callback for node 69c9e30e3c3aafdab4bea766: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49212 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:43:10,698 - lava_callback - INFO - Received callback for node 69cc55a08df9359ff4899f2a INFO: 172.20.0.1:40142 - "POST /node/69cc55a08df9359ff4899f2a HTTP/1.0" 202 Accepted 2026-04-01 02:43:10,705 - lava_callback - INFO - Starting async job submit for node 69cc55a08df9359ff4899f2a 2026-04-01 02:43:10,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:43:10,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc55a08df9359ff4899f2a HTTP/1.1" 200 None 2026-04-01 02:43:10,750 - lava_callback - DEBUG - Processing node 69cc55a08df9359ff4899f2a: baseline-arm64 2026-04-01 02:43:11,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:43:11,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:43:11,401 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc55a08df9359ff4899f2a/log.txt.gz 2026-04-01 02:43:11,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:43:11,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:43:11,953 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc55a08df9359ff4899f2a/lava-logs.yaml 2026-04-01 02:43:11,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:43:12,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:43:12,142 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc55a08df9359ff4899f2a/lava_callback.json.gz 2026-04-01 02:43:12,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:43:12,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc55a08df9359ff4899f2a HTTP/1.1" 200 None 2026-04-01 02:43:12,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:43:12,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 02:43:12,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:43:12,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc55a08df9359ff4899f2a HTTP/1.1" 200 None 2026-04-01 02:43:12,306 - lava_callback - INFO - Completed processing callback for node 69cc55a08df9359ff4899f2a 2026-04-01 02:43:15,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:43:15,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60698 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:46:32,683 - lava_callback - INFO - Received callback for node 69c9e3fd3c3aafdab4bea7ab INFO: 172.20.0.1:38902 - "POST /node/69c9e3fd3c3aafdab4bea7ab HTTP/1.0" 202 Accepted 2026-04-01 02:46:32,688 - lava_callback - INFO - Starting async job submit for node 69c9e3fd3c3aafdab4bea7ab 2026-04-01 02:46:32,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:46:32,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e3fd3c3aafdab4bea7ab HTTP/1.1" 200 None 2026-04-01 02:46:32,726 - lava_callback - DEBUG - Processing node 69c9e3fd3c3aafdab4bea7ab: baseline-arm-clabbe 2026-04-01 02:46:32,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e3fd3c3aafdab4bea7ab 2026-04-01 02:46:32,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:46:32,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e3fd3c3aafdab4bea7ab HTTP/1.1" 200 None 2026-04-01 02:46:32,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:46:32,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:46:32,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:46:32,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e3fd3c3aafdab4bea7ab HTTP/1.1" 200 None 2026-04-01 02:46:32,783 - lava_callback - ERROR - Error processing callback for node 69c9e3fd3c3aafdab4bea7ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58652 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:47:12,501 - lava_callback - INFO - Received callback for node 69c9e4213c3aafdab4bea7b7 INFO: 172.20.0.1:46864 - "POST /node/69c9e4213c3aafdab4bea7b7 HTTP/1.0" 202 Accepted 2026-04-01 02:47:12,507 - lava_callback - INFO - Starting async job submit for node 69c9e4213c3aafdab4bea7b7 2026-04-01 02:47:12,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:12,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4213c3aafdab4bea7b7 HTTP/1.1" 200 None 2026-04-01 02:47:12,523 - lava_callback - DEBUG - Processing node 69c9e4213c3aafdab4bea7b7: baseline-arm-clabbe 2026-04-01 02:47:12,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e4213c3aafdab4bea7b7 2026-04-01 02:47:12,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:12,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4213c3aafdab4bea7b7 HTTP/1.1" 200 None 2026-04-01 02:47:12,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:12,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:47:12,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:12,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e4213c3aafdab4bea7b7 HTTP/1.1" 200 None 2026-04-01 02:47:12,587 - lava_callback - ERROR - Error processing callback for node 69c9e4213c3aafdab4bea7b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42764 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:47:52,570 - lava_callback - INFO - Received callback for node 69c9e4463c3aafdab4bea7c8 INFO: 172.20.0.1:58666 - "POST /node/69c9e4463c3aafdab4bea7c8 HTTP/1.0" 202 Accepted 2026-04-01 02:47:52,576 - lava_callback - INFO - Starting async job submit for node 69c9e4463c3aafdab4bea7c8 2026-04-01 02:47:52,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:52,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4463c3aafdab4bea7c8 HTTP/1.1" 200 None 2026-04-01 02:47:52,592 - lava_callback - DEBUG - Processing node 69c9e4463c3aafdab4bea7c8: baseline-arm-clabbe 2026-04-01 02:47:52,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e4463c3aafdab4bea7c8 2026-04-01 02:47:52,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:52,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4463c3aafdab4bea7c8 HTTP/1.1" 200 None 2026-04-01 02:47:52,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:52,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:47:52,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:47:52,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e4463c3aafdab4bea7c8 HTTP/1.1" 200 None 2026-04-01 02:47:52,652 - lava_callback - ERROR - Error processing callback for node 69c9e4463c3aafdab4bea7c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49974 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:49:32,609 - lava_callback - INFO - Received callback for node 69c9e4b83c3aafdab4bea7ea INFO: 172.20.0.1:57124 - "POST /node/69c9e4b83c3aafdab4bea7ea HTTP/1.0" 202 Accepted 2026-04-01 02:49:32,615 - lava_callback - INFO - Starting async job submit for node 69c9e4b83c3aafdab4bea7ea 2026-04-01 02:49:32,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:49:32,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4b83c3aafdab4bea7ea HTTP/1.1" 200 None 2026-04-01 02:49:32,638 - lava_callback - DEBUG - Processing node 69c9e4b83c3aafdab4bea7ea: baseline-arm-clabbe 2026-04-01 02:49:32,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e4b83c3aafdab4bea7ea 2026-04-01 02:49:32,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:49:32,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4b83c3aafdab4bea7ea HTTP/1.1" 200 None 2026-04-01 02:49:32,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:49:32,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:49:32,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:49:32,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e4b83c3aafdab4bea7ea HTTP/1.1" 200 None 2026-04-01 02:49:32,691 - lava_callback - ERROR - Error processing callback for node 69c9e4b83c3aafdab4bea7ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43334 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:50:09,023 - lava_callback - INFO - Received callback for node 69cc861e8cd9cbcdc945b090 INFO: 172.20.0.1:43348 - "POST /node/69cc861e8cd9cbcdc945b090 HTTP/1.0" 202 Accepted 2026-04-01 02:50:09,031 - lava_callback - INFO - Starting async job submit for node 69cc861e8cd9cbcdc945b090 2026-04-01 02:50:09,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:09,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc861e8cd9cbcdc945b090 HTTP/1.1" 200 None 2026-04-01 02:50:09,093 - lava_callback - DEBUG - Processing node 69cc861e8cd9cbcdc945b090: baseline-arm64 2026-04-01 02:50:09,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:50:09,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:50:09,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc861e8cd9cbcdc945b090/log.txt.gz 2026-04-01 02:50:10,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:50:10,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:50:10,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc861e8cd9cbcdc945b090/lava-logs.yaml 2026-04-01 02:50:10,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:50:10,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:50:10,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc861e8cd9cbcdc945b090/lava_callback.json.gz DEBUG: Setup failed for 69cc861e8cd9cbcdc945b090. Transitting job node result: incomplete -> fail 2026-04-01 02:50:10,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:10,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc861e8cd9cbcdc945b090 HTTP/1.1" 200 None 2026-04-01 02:50:10,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:10,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:50:10,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:10,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc861e8cd9cbcdc945b090 HTTP/1.1" 200 None 2026-04-01 02:50:10,631 - lava_callback - INFO - Completed processing callback for node 69cc861e8cd9cbcdc945b090 2026-04-01 02:50:12,535 - lava_callback - INFO - Received callback for node 69c9e4dc3c3aafdab4bea7f9 INFO: 172.20.0.1:38098 - "POST /node/69c9e4dc3c3aafdab4bea7f9 HTTP/1.0" 202 Accepted 2026-04-01 02:50:12,541 - lava_callback - INFO - Starting async job submit for node 69c9e4dc3c3aafdab4bea7f9 2026-04-01 02:50:12,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:12,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4dc3c3aafdab4bea7f9 HTTP/1.1" 200 None 2026-04-01 02:50:12,559 - lava_callback - DEBUG - Processing node 69c9e4dc3c3aafdab4bea7f9: baseline-arm-clabbe 2026-04-01 02:50:12,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9e4dc3c3aafdab4bea7f9 2026-04-01 02:50:12,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:12,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9e4dc3c3aafdab4bea7f9 HTTP/1.1" 200 None 2026-04-01 02:50:12,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:12,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-04-01 02:50:12,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:12,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9e4dc3c3aafdab4bea7f9 HTTP/1.1" 200 None 2026-04-01 02:50:12,615 - lava_callback - ERROR - Error processing callback for node 69c9e4dc3c3aafdab4bea7f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 02:50:15,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:50:15,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:51:43,091 - lava_callback - INFO - Received callback for node 69cc846d8cd9cbcdc945afc4 INFO: 172.20.0.1:35070 - "POST /node/69cc846d8cd9cbcdc945afc4 HTTP/1.0" 202 Accepted 2026-04-01 02:51:43,098 - lava_callback - INFO - Starting async job submit for node 69cc846d8cd9cbcdc945afc4 2026-04-01 02:51:43,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:51:43,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc846d8cd9cbcdc945afc4 HTTP/1.1" 200 None 2026-04-01 02:51:43,165 - lava_callback - DEBUG - Processing node 69cc846d8cd9cbcdc945afc4: baseline-arm 2026-04-01 02:51:43,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:51:43,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:51:43,973 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc846d8cd9cbcdc945afc4/log.txt.gz 2026-04-01 02:51:44,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:51:44,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:51:44,579 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc846d8cd9cbcdc945afc4/lava-logs.yaml 2026-04-01 02:51:44,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:51:44,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:51:44,967 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc846d8cd9cbcdc945afc4/lava_callback.json.gz DEBUG: Setup failed for 69cc846d8cd9cbcdc945afc4. Transitting job node result: incomplete -> fail 2026-04-01 02:51:45,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:51:45,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc846d8cd9cbcdc945afc4 HTTP/1.1" 200 None 2026-04-01 02:51:45,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:51:45,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 02:51:45,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:51:45,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc846d8cd9cbcdc945afc4 HTTP/1.1" 200 None 2026-04-01 02:51:45,248 - lava_callback - INFO - Completed processing callback for node 69cc846d8cd9cbcdc945afc4 INFO: 172.20.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47390 - "GET /node/69b19ceba26b422765ba396b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47404 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 02:52:15,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:52:15,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43828 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:53:57,755 - lava_callback - INFO - Received callback for node 69cc84958cd9cbcdc945afd7 INFO: 172.20.0.1:43836 - "POST /node/69cc84958cd9cbcdc945afd7 HTTP/1.0" 202 Accepted 2026-04-01 02:53:57,765 - lava_callback - INFO - Starting async job submit for node 69cc84958cd9cbcdc945afd7 2026-04-01 02:53:57,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:53:57,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc84958cd9cbcdc945afd7 HTTP/1.1" 200 None 2026-04-01 02:53:57,863 - lava_callback - DEBUG - Processing node 69cc84958cd9cbcdc945afd7: ltp-smoke 2026-04-01 02:53:58,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:53:58,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:53:58,487 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc84958cd9cbcdc945afd7/log.txt.gz 2026-04-01 02:53:58,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:53:59,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:53:59,111 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc84958cd9cbcdc945afd7/lava-logs.yaml 2026-04-01 02:53:59,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:53:59,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:53:59,307 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc84958cd9cbcdc945afd7/lava_callback.json.gz 2026-04-01 02:53:59,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:53:59,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc84958cd9cbcdc945afd7 HTTP/1.1" 200 None 2026-04-01 02:53:59,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:53:59,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64828cd9cbcdc945964d HTTP/1.1" 200 None 2026-04-01 02:53:59,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:53:59,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc84958cd9cbcdc945afd7 HTTP/1.1" 200 None 2026-04-01 02:53:59,692 - lava_callback - INFO - Completed processing callback for node 69cc84958cd9cbcdc945afd7 INFO: 172.20.0.1:51596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51084 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 02:54:15,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:54:15,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET /node/6983ed86969566dcb0d29dab%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52604 - "GET /node/693af1f7fee8383e92b6a14d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50342 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:55:42,289 - lava_callback - INFO - Received callback for node 69cc56088df9359ff4899f5d INFO: 172.20.0.1:52246 - "POST /node/69cc56088df9359ff4899f5d HTTP/1.0" 202 Accepted 2026-04-01 02:55:42,302 - lava_callback - INFO - Starting async job submit for node 69cc56088df9359ff4899f5d 2026-04-01 02:55:42,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:55:42,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc56088df9359ff4899f5d HTTP/1.1" 200 None 2026-04-01 02:55:42,345 - lava_callback - DEBUG - Processing node 69cc56088df9359ff4899f5d: baseline-nfs-arm64 2026-04-01 02:55:43,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:55:43,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:55:43,139 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc56088df9359ff4899f5d/log.txt.gz 2026-04-01 02:55:43,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:55:43,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:55:43,650 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc56088df9359ff4899f5d/lava-logs.yaml 2026-04-01 02:55:43,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:55:43,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:55:43,838 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc56088df9359ff4899f5d/lava_callback.json.gz 2026-04-01 02:55:43,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:55:43,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc56088df9359ff4899f5d HTTP/1.1" 200 None 2026-04-01 02:55:43,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:55:43,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 02:55:43,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:55:43,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc56088df9359ff4899f5d HTTP/1.1" 200 None 2026-04-01 02:55:43,974 - lava_callback - INFO - Completed processing callback for node 69cc56088df9359ff4899f5d 2026-04-01 02:55:45,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:55:45,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38926 - "GET /node/69bde8c5d54b5122738c6472%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48842 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:56:18,008 - lava_callback - INFO - Received callback for node 69cc6fea8cd9cbcdc9459ec4 INFO: 172.20.0.1:48852 - "POST /node/69cc6fea8cd9cbcdc9459ec4 HTTP/1.0" 202 Accepted 2026-04-01 02:56:18,016 - lava_callback - INFO - Starting async job submit for node 69cc6fea8cd9cbcdc9459ec4 2026-04-01 02:56:18,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:56:18,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fea8cd9cbcdc9459ec4 HTTP/1.1" 200 None 2026-04-01 02:56:18,060 - lava_callback - DEBUG - Processing node 69cc6fea8cd9cbcdc9459ec4: baseline-arm64 2026-04-01 02:56:18,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:56:18,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:56:18,708 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fea8cd9cbcdc9459ec4/log.txt.gz 2026-04-01 02:56:18,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:56:19,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:56:19,159 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fea8cd9cbcdc9459ec4/lava-logs.yaml 2026-04-01 02:56:19,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 02:56:19,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 02:56:19,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc6fea8cd9cbcdc9459ec4/lava_callback.json.gz 2026-04-01 02:56:19,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:56:19,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fea8cd9cbcdc9459ec4 HTTP/1.1" 200 None 2026-04-01 02:56:19,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:56:19,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 02:56:19,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:56:19,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fea8cd9cbcdc9459ec4 HTTP/1.1" 200 None 2026-04-01 02:56:19,469 - lava_callback - INFO - Completed processing callback for node 69cc6fea8cd9cbcdc9459ec4 INFO: 172.20.0.1:47020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42198 - "GET / HTTP/1.0" 200 OK 2026-04-01 02:56:45,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 02:56:45,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51882 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:00:03,819 - lava_callback - INFO - Received callback for node 69cc85a28cd9cbcdc945b057 INFO: 172.20.0.1:51894 - "POST /node/69cc85a28cd9cbcdc945b057 HTTP/1.0" 202 Accepted 2026-04-01 03:00:03,869 - lava_callback - INFO - Starting async job submit for node 69cc85a28cd9cbcdc945b057 2026-04-01 03:00:03,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:00:03,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85a28cd9cbcdc945b057 HTTP/1.1" 200 None 2026-04-01 03:00:03,909 - lava_callback - DEBUG - Processing node 69cc85a28cd9cbcdc945b057: baseline-arm-pengutronix 2026-04-01 03:00:04,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:00:04,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:00:04,333 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc85a28cd9cbcdc945b057/log.txt.gz 2026-04-01 03:00:04,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:00:04,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:00:04,584 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc85a28cd9cbcdc945b057/lava-logs.yaml 2026-04-01 03:00:04,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:00:04,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:00:04,736 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc85a28cd9cbcdc945b057/lava_callback.json.gz DEBUG: Setup failed for 69cc85a28cd9cbcdc945b057. Transitting job node result: incomplete -> fail 2026-04-01 03:00:04,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:00:04,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85a28cd9cbcdc945b057 HTTP/1.1" 200 None 2026-04-01 03:00:04,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:00:04,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 03:00:04,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:00:04,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc85a28cd9cbcdc945b057 HTTP/1.1" 200 None 2026-04-01 03:00:04,898 - lava_callback - INFO - Completed processing callback for node 69cc85a28cd9cbcdc945b057 INFO: 172.20.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43526 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 03:00:15,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:00:15,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET /node/69415f0e459ff5fc64d2db1c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47646 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:04:00,142 - lava_callback - INFO - Received callback for node 69cc86748cd9cbcdc945b0c0 INFO: 172.20.0.1:59588 - "POST /node/69cc86748cd9cbcdc945b0c0 HTTP/1.0" 202 Accepted 2026-04-01 03:04:00,150 - lava_callback - INFO - Starting async job submit for node 69cc86748cd9cbcdc945b0c0 2026-04-01 03:04:00,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:04:00,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc86748cd9cbcdc945b0c0 HTTP/1.1" 200 None 2026-04-01 03:04:00,213 - lava_callback - DEBUG - Processing node 69cc86748cd9cbcdc945b0c0: baseline-nfs-arm64 2026-04-01 03:04:01,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:04:01,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:04:01,523 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc86748cd9cbcdc945b0c0/log.txt.gz 2026-04-01 03:04:02,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:04:02,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:04:02,426 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc86748cd9cbcdc945b0c0/lava-logs.yaml 2026-04-01 03:04:02,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:04:02,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:04:02,649 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc86748cd9cbcdc945b0c0/lava_callback.json.gz DEBUG: Setup failed for 69cc86748cd9cbcdc945b0c0. Transitting job node result: incomplete -> fail 2026-04-01 03:04:02,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:04:02,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc86748cd9cbcdc945b0c0 HTTP/1.1" 200 None 2026-04-01 03:04:02,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:04:02,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 03:04:02,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:04:02,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc86748cd9cbcdc945b0c0 HTTP/1.1" 200 None 2026-04-01 03:04:02,889 - lava_callback - INFO - Completed processing callback for node 69cc86748cd9cbcdc945b0c0 INFO: 172.20.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 03:04:15,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:04:15,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52982 - "GET /node/69bd094205e4bbd6ba635691%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35774 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:08:12,649 - lava_callback - INFO - Received callback for node 69cc6ffd8cd9cbcdc9459f06 INFO: 172.20.0.1:38442 - "POST /node/69cc6ffd8cd9cbcdc9459f06 HTTP/1.0" 202 Accepted 2026-04-01 03:08:12,656 - lava_callback - INFO - Starting async job submit for node 69cc6ffd8cd9cbcdc9459f06 2026-04-01 03:08:12,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:08:12,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ffd8cd9cbcdc9459f06 HTTP/1.1" 200 None 2026-04-01 03:08:12,703 - lava_callback - DEBUG - Processing node 69cc6ffd8cd9cbcdc9459f06: baseline-nfs-arm64 2026-04-01 03:08:13,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:08:13,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:08:13,351 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffd8cd9cbcdc9459f06/log.txt.gz 2026-04-01 03:08:13,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:08:13,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:08:13,853 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffd8cd9cbcdc9459f06/lava-logs.yaml 2026-04-01 03:08:13,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:08:14,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:08:14,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc6ffd8cd9cbcdc9459f06/lava_callback.json.gz 2026-04-01 03:08:14,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:08:14,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ffd8cd9cbcdc9459f06 HTTP/1.1" 200 None 2026-04-01 03:08:14,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:08:14,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 03:08:14,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:08:14,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ffd8cd9cbcdc9459f06 HTTP/1.1" 200 None 2026-04-01 03:08:14,219 - lava_callback - INFO - Completed processing callback for node 69cc6ffd8cd9cbcdc9459f06 INFO: 172.20.0.1:38444 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 03:08:15,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:08:15,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44648 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:09:03,755 - lava_callback - INFO - Received callback for node 69cc83ca8cd9cbcdc945af76 INFO: 172.20.0.1:44656 - "POST /node/69cc83ca8cd9cbcdc945af76 HTTP/1.0" 202 Accepted 2026-04-01 03:09:03,765 - lava_callback - INFO - Starting async job submit for node 69cc83ca8cd9cbcdc945af76 2026-04-01 03:09:03,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:09:03,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc83ca8cd9cbcdc945af76 HTTP/1.1" 200 None 2026-04-01 03:09:03,810 - lava_callback - DEBUG - Processing node 69cc83ca8cd9cbcdc945af76: baseline-nfs-arm64 2026-04-01 03:09:04,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:09:04,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:09:04,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc83ca8cd9cbcdc945af76/log.txt.gz 2026-04-01 03:09:04,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:09:04,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:09:05,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc83ca8cd9cbcdc945af76/lava-logs.yaml 2026-04-01 03:09:05,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:09:05,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:09:05,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc83ca8cd9cbcdc945af76/lava_callback.json.gz 2026-04-01 03:09:05,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:09:05,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc83ca8cd9cbcdc945af76 HTTP/1.1" 200 None 2026-04-01 03:09:05,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:09:05,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 03:09:05,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:09:05,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc83ca8cd9cbcdc945af76 HTTP/1.1" 200 None 2026-04-01 03:09:05,316 - lava_callback - INFO - Completed processing callback for node 69cc83ca8cd9cbcdc945af76 INFO: 172.20.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60074 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 03:09:15,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:09:15,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:13:46,790 - lava_callback - INFO - Received callback for node 69c9ea583c3aafdab4bea877 INFO: 172.20.0.1:38744 - "POST /node/69c9ea583c3aafdab4bea877 HTTP/1.0" 202 Accepted 2026-04-01 03:13:46,796 - lava_callback - INFO - Starting async job submit for node 69c9ea583c3aafdab4bea877 2026-04-01 03:13:46,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:46,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ea583c3aafdab4bea877 HTTP/1.1" 200 None 2026-04-01 03:13:46,822 - lava_callback - DEBUG - Processing node 69c9ea583c3aafdab4bea877: baseline-arm64-baylibre 2026-04-01 03:13:46,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ea583c3aafdab4bea877 2026-04-01 03:13:46,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:46,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ea583c3aafdab4bea877 HTTP/1.1" 200 None 2026-04-01 03:13:46,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:46,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:13:46,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:46,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ea583c3aafdab4bea877 HTTP/1.1" 200 None 2026-04-01 03:13:46,904 - lava_callback - ERROR - Error processing callback for node 69c9ea583c3aafdab4bea877: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35074 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:13:57,714 - lava_callback - INFO - Received callback for node 69cc89548cd9cbcdc945b1d8 INFO: 172.20.0.1:35082 - "POST /node/69cc89548cd9cbcdc945b1d8 HTTP/1.0" 202 Accepted 2026-04-01 03:13:57,721 - lava_callback - INFO - Starting async job submit for node 69cc89548cd9cbcdc945b1d8 2026-04-01 03:13:57,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:57,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc89548cd9cbcdc945b1d8 HTTP/1.1" 200 None 2026-04-01 03:13:57,795 - lava_callback - DEBUG - Processing node 69cc89548cd9cbcdc945b1d8: ltp-smoke 2026-04-01 03:13:58,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:13:58,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:13:58,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc89548cd9cbcdc945b1d8/log.txt.gz 2026-04-01 03:13:58,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:13:59,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:13:59,102 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc89548cd9cbcdc945b1d8/lava-logs.yaml 2026-04-01 03:13:59,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:13:59,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:13:59,291 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc89548cd9cbcdc945b1d8/lava_callback.json.gz 2026-04-01 03:13:59,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:59,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc89548cd9cbcdc945b1d8 HTTP/1.1" 200 None 2026-04-01 03:13:59,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:59,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64828cd9cbcdc945964d HTTP/1.1" 200 None 2026-04-01 03:13:59,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:13:59,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc89548cd9cbcdc945b1d8 HTTP/1.1" 200 None 2026-04-01 03:13:59,626 - lava_callback - INFO - Completed processing callback for node 69cc89548cd9cbcdc945b1d8 INFO: 172.20.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52872 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 03:14:15,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:14:15,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36932 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:15:06,770 - lava_callback - INFO - Received callback for node 69c9eaa73c3aafdab4bea882 INFO: 172.20.0.1:36940 - "POST /node/69c9eaa73c3aafdab4bea882 HTTP/1.0" 202 Accepted 2026-04-01 03:15:06,776 - lava_callback - INFO - Starting async job submit for node 69c9eaa73c3aafdab4bea882 2026-04-01 03:15:06,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:15:06,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eaa73c3aafdab4bea882 HTTP/1.1" 200 None 2026-04-01 03:15:06,794 - lava_callback - DEBUG - Processing node 69c9eaa73c3aafdab4bea882: baseline-arm64-baylibre 2026-04-01 03:15:06,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9eaa73c3aafdab4bea882 2026-04-01 03:15:06,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:15:06,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eaa73c3aafdab4bea882 HTTP/1.1" 200 None 2026-04-01 03:15:06,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:15:06,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:15:06,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:15:06,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9eaa73c3aafdab4bea882 HTTP/1.1" 200 None 2026-04-01 03:15:06,850 - lava_callback - ERROR - Error processing callback for node 69c9eaa73c3aafdab4bea882: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57426 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:16:46,896 - lava_callback - INFO - Received callback for node 69c9eb0e3c3aafdab4bea88e INFO: 172.20.0.1:57438 - "POST /node/69c9eb0e3c3aafdab4bea88e HTTP/1.0" 202 Accepted 2026-04-01 03:16:46,902 - lava_callback - INFO - Starting async job submit for node 69c9eb0e3c3aafdab4bea88e 2026-04-01 03:16:46,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:16:46,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eb0e3c3aafdab4bea88e HTTP/1.1" 200 None 2026-04-01 03:16:46,932 - lava_callback - DEBUG - Processing node 69c9eb0e3c3aafdab4bea88e: baseline-arm64-baylibre 2026-04-01 03:16:46,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9eb0e3c3aafdab4bea88e 2026-04-01 03:16:46,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:16:46,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eb0e3c3aafdab4bea88e HTTP/1.1" 200 None 2026-04-01 03:16:46,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:16:46,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:16:46,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:16:47,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9eb0e3c3aafdab4bea88e HTTP/1.1" 200 None 2026-04-01 03:16:47,019 - lava_callback - ERROR - Error processing callback for node 69c9eb0e3c3aafdab4bea88e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:18:06,940 - lava_callback - INFO - Received callback for node 69c9eb5c3c3aafdab4bea89a INFO: 172.20.0.1:39720 - "POST /node/69c9eb5c3c3aafdab4bea89a HTTP/1.0" 202 Accepted 2026-04-01 03:18:06,946 - lava_callback - INFO - Starting async job submit for node 69c9eb5c3c3aafdab4bea89a 2026-04-01 03:18:06,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:18:06,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eb5c3c3aafdab4bea89a HTTP/1.1" 200 None 2026-04-01 03:18:06,963 - lava_callback - DEBUG - Processing node 69c9eb5c3c3aafdab4bea89a: baseline-arm64-baylibre 2026-04-01 03:18:06,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9eb5c3c3aafdab4bea89a 2026-04-01 03:18:06,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:18:06,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eb5c3c3aafdab4bea89a HTTP/1.1" 200 None 2026-04-01 03:18:06,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:18:07,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:18:07,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:18:07,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9eb5c3c3aafdab4bea89a HTTP/1.1" 200 None 2026-04-01 03:18:07,030 - lava_callback - ERROR - Error processing callback for node 69c9eb5c3c3aafdab4bea89a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49378 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:19:07,034 - lava_callback - INFO - Received callback for node 69c9eba93c3aafdab4bea8ab INFO: 172.20.0.1:49388 - "POST /node/69c9eba93c3aafdab4bea8ab HTTP/1.0" 202 Accepted 2026-04-01 03:19:07,041 - lava_callback - INFO - Starting async job submit for node 69c9eba93c3aafdab4bea8ab 2026-04-01 03:19:07,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:19:07,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eba93c3aafdab4bea8ab HTTP/1.1" 200 None 2026-04-01 03:19:07,061 - lava_callback - DEBUG - Processing node 69c9eba93c3aafdab4bea8ab: baseline-arm64-baylibre 2026-04-01 03:19:07,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9eba93c3aafdab4bea8ab 2026-04-01 03:19:07,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:19:07,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9eba93c3aafdab4bea8ab HTTP/1.1" 200 None 2026-04-01 03:19:07,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:19:07,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:19:07,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:19:07,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9eba93c3aafdab4bea8ab HTTP/1.1" 200 None 2026-04-01 03:19:07,116 - lava_callback - ERROR - Error processing callback for node 69c9eba93c3aafdab4bea8ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45206 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:20:31,439 - lava_callback - INFO - Received callback for node 69cc87828cd9cbcdc945b11a INFO: 172.20.0.1:50676 - "POST /node/69cc87828cd9cbcdc945b11a HTTP/1.0" 202 Accepted 2026-04-01 03:20:31,446 - lava_callback - INFO - Starting async job submit for node 69cc87828cd9cbcdc945b11a 2026-04-01 03:20:31,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:31,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc87828cd9cbcdc945b11a HTTP/1.1" 200 None 2026-04-01 03:20:31,487 - lava_callback - DEBUG - Processing node 69cc87828cd9cbcdc945b11a: baseline-arm64 2026-04-01 03:20:32,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:20:32,134 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:20:32,136 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc87828cd9cbcdc945b11a/log.txt.gz 2026-04-01 03:20:32,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:20:32,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:20:32,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc87828cd9cbcdc945b11a/lava-logs.yaml 2026-04-01 03:20:32,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:20:32,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:20:32,807 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc87828cd9cbcdc945b11a/lava_callback.json.gz 2026-04-01 03:20:32,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:32,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc87828cd9cbcdc945b11a HTTP/1.1" 200 None 2026-04-01 03:20:32,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:32,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 03:20:32,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:32,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc87828cd9cbcdc945b11a HTTP/1.1" 200 None 2026-04-01 03:20:33,020 - lava_callback - INFO - Completed processing callback for node 69cc87828cd9cbcdc945b11a INFO: 172.20.0.1:50684 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:20:45,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:45,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44640 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:20:47,105 - lava_callback - INFO - Received callback for node 69c9ec013c3aafdab4bea8b9 INFO: 172.20.0.1:44646 - "POST /node/69c9ec013c3aafdab4bea8b9 HTTP/1.0" 202 Accepted 2026-04-01 03:20:47,112 - lava_callback - INFO - Starting async job submit for node 69c9ec013c3aafdab4bea8b9 2026-04-01 03:20:47,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:47,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ec013c3aafdab4bea8b9 HTTP/1.1" 200 None 2026-04-01 03:20:47,151 - lava_callback - DEBUG - Processing node 69c9ec013c3aafdab4bea8b9: baseline-arm64-baylibre 2026-04-01 03:20:47,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ec013c3aafdab4bea8b9 2026-04-01 03:20:47,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:47,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ec013c3aafdab4bea8b9 HTTP/1.1" 200 None 2026-04-01 03:20:47,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:47,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:20:47,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:20:47,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ec013c3aafdab4bea8b9 HTTP/1.1" 200 None 2026-04-01 03:20:47,220 - lava_callback - ERROR - Error processing callback for node 69c9ec013c3aafdab4bea8b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43764 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:21:32,111 - lava_callback - INFO - Received callback for node 69cc8a068cd9cbcdc945b225 INFO: 172.20.0.1:38604 - "POST /node/69cc8a068cd9cbcdc945b225 HTTP/1.0" 202 Accepted 2026-04-01 03:21:32,118 - lava_callback - INFO - Starting async job submit for node 69cc8a068cd9cbcdc945b225 2026-04-01 03:21:32,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:21:32,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8a068cd9cbcdc945b225 HTTP/1.1" 200 None 2026-04-01 03:21:32,173 - lava_callback - DEBUG - Processing node 69cc8a068cd9cbcdc945b225: baseline-arm64 2026-04-01 03:21:32,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:21:32,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:21:32,878 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc8a068cd9cbcdc945b225/log.txt.gz 2026-04-01 03:21:33,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:21:33,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:21:33,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc8a068cd9cbcdc945b225/lava-logs.yaml 2026-04-01 03:21:33,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:21:33,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:21:33,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc8a068cd9cbcdc945b225/lava_callback.json.gz 2026-04-01 03:21:33,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:21:33,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8a068cd9cbcdc945b225 HTTP/1.1" 200 None 2026-04-01 03:21:33,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:21:33,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 03:21:33,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:21:33,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc8a068cd9cbcdc945b225 HTTP/1.1" 200 None 2026-04-01 03:21:33,636 - lava_callback - INFO - Completed processing callback for node 69cc8a068cd9cbcdc945b225 INFO: 172.20.0.1:38616 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:21:45,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:21:45,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37594 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:22:14,117 - lava_callback - INFO - Received callback for node 69c9ec533c3aafdab4bea8c2 INFO: 172.20.0.1:42286 - "POST /node/69c9ec533c3aafdab4bea8c2 HTTP/1.0" 202 Accepted 2026-04-01 03:22:14,123 - lava_callback - INFO - Starting async job submit for node 69c9ec533c3aafdab4bea8c2 2026-04-01 03:22:14,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:22:14,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ec533c3aafdab4bea8c2 HTTP/1.1" 200 None 2026-04-01 03:22:14,142 - lava_callback - DEBUG - Processing node 69c9ec533c3aafdab4bea8c2: baseline-arm64-clabbe 2026-04-01 03:22:14,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ec533c3aafdab4bea8c2 2026-04-01 03:22:14,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:22:14,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ec533c3aafdab4bea8c2 HTTP/1.1" 200 None 2026-04-01 03:22:14,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:22:14,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:22:14,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:22:14,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ec533c3aafdab4bea8c2 HTTP/1.1" 200 None 2026-04-01 03:22:14,217 - lava_callback - ERROR - Error processing callback for node 69c9ec533c3aafdab4bea8c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52580 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:23:37,772 - lava_callback - INFO - Received callback for node 69cc68808cd9cbcdc9459ae8 INFO: 172.20.0.1:52594 - "POST /node/69cc68808cd9cbcdc9459ae8 HTTP/1.0" 202 Accepted 2026-04-01 03:23:37,782 - lava_callback - INFO - Starting async job submit for node 69cc68808cd9cbcdc9459ae8 2026-04-01 03:23:37,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:37,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68808cd9cbcdc9459ae8 HTTP/1.1" 200 None 2026-04-01 03:23:37,835 - lava_callback - DEBUG - Processing node 69cc68808cd9cbcdc9459ae8: baseline-arm-broonie 2026-04-01 03:23:38,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:23:38,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:23:38,197 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68808cd9cbcdc9459ae8/log.txt.gz 2026-04-01 03:23:38,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:23:38,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:23:38,531 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68808cd9cbcdc9459ae8/lava-logs.yaml 2026-04-01 03:23:38,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:23:38,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:23:38,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68808cd9cbcdc9459ae8/lava_callback.json.gz 2026-04-01 03:23:38,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:38,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68808cd9cbcdc9459ae8 HTTP/1.1" 200 None 2026-04-01 03:23:38,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:38,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 03:23:38,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:38,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68808cd9cbcdc9459ae8 HTTP/1.1" 200 None 2026-04-01 03:23:38,773 - lava_callback - INFO - Completed processing callback for node 69cc68808cd9cbcdc9459ae8 2026-04-01 03:23:45,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:45,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36034 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:23:54,206 - lava_callback - INFO - Received callback for node 69c9ecbd3c3aafdab4bea8d2 INFO: 172.20.0.1:58478 - "POST /node/69c9ecbd3c3aafdab4bea8d2 HTTP/1.0" 202 Accepted 2026-04-01 03:23:54,213 - lava_callback - INFO - Starting async job submit for node 69c9ecbd3c3aafdab4bea8d2 2026-04-01 03:23:54,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:54,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ecbd3c3aafdab4bea8d2 HTTP/1.1" 200 None 2026-04-01 03:23:54,230 - lava_callback - DEBUG - Processing node 69c9ecbd3c3aafdab4bea8d2: baseline-arm64-clabbe 2026-04-01 03:23:54,230 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ecbd3c3aafdab4bea8d2 2026-04-01 03:23:54,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:54,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ecbd3c3aafdab4bea8d2 HTTP/1.1" 200 None 2026-04-01 03:23:54,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:54,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:23:54,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:23:54,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ecbd3c3aafdab4bea8d2 HTTP/1.1" 200 None 2026-04-01 03:23:54,299 - lava_callback - ERROR - Error processing callback for node 69c9ecbd3c3aafdab4bea8d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33696 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:24:17,294 - lava_callback - INFO - Received callback for node 69cc8b2c8cd9cbcdc945b26c INFO: 172.20.0.1:33704 - "POST /node/69cc8b2c8cd9cbcdc945b26c HTTP/1.0" 202 Accepted 2026-04-01 03:24:17,302 - lava_callback - INFO - Starting async job submit for node 69cc8b2c8cd9cbcdc945b26c 2026-04-01 03:24:17,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:24:17,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8b2c8cd9cbcdc945b26c HTTP/1.1" 200 None 2026-04-01 03:24:17,378 - lava_callback - DEBUG - Processing node 69cc8b2c8cd9cbcdc945b26c: baseline-nfs-arm64 2026-04-01 03:24:18,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:24:18,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:24:18,779 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc8b2c8cd9cbcdc945b26c/log.txt.gz 2026-04-01 03:24:19,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:24:19,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:24:19,729 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc8b2c8cd9cbcdc945b26c/lava-logs.yaml 2026-04-01 03:24:19,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:24:19,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:24:19,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc8b2c8cd9cbcdc945b26c/lava_callback.json.gz DEBUG: Setup failed for 69cc8b2c8cd9cbcdc945b26c. Transitting job node result: incomplete -> fail 2026-04-01 03:24:20,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:24:20,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8b2c8cd9cbcdc945b26c HTTP/1.1" 200 None 2026-04-01 03:24:20,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:24:20,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 03:24:20,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:24:20,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc8b2c8cd9cbcdc945b26c HTTP/1.1" 200 None 2026-04-01 03:24:20,171 - lava_callback - INFO - Completed processing callback for node 69cc8b2c8cd9cbcdc945b26c INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54462 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:24:45,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:24:45,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48538 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:25:14,184 - lava_callback - INFO - Received callback for node 69c9ed0c3c3aafdab4bea8dc INFO: 172.20.0.1:43480 - "POST /node/69c9ed0c3c3aafdab4bea8dc HTTP/1.0" 202 Accepted 2026-04-01 03:25:14,190 - lava_callback - INFO - Starting async job submit for node 69c9ed0c3c3aafdab4bea8dc 2026-04-01 03:25:14,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:25:14,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ed0c3c3aafdab4bea8dc HTTP/1.1" 200 None 2026-04-01 03:25:14,207 - lava_callback - DEBUG - Processing node 69c9ed0c3c3aafdab4bea8dc: baseline-arm64-clabbe 2026-04-01 03:25:14,208 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ed0c3c3aafdab4bea8dc 2026-04-01 03:25:14,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:25:14,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ed0c3c3aafdab4bea8dc HTTP/1.1" 200 None 2026-04-01 03:25:14,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:25:14,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:25:14,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:25:14,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ed0c3c3aafdab4bea8dc HTTP/1.1" 200 None 2026-04-01 03:25:14,297 - lava_callback - ERROR - Error processing callback for node 69c9ed0c3c3aafdab4bea8dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34634 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:27:54,394 - lava_callback - INFO - Received callback for node 69c9edab3c3aafdab4bea8f1 INFO: 172.20.0.1:53016 - "POST /node/69c9edab3c3aafdab4bea8f1 HTTP/1.0" 202 Accepted 2026-04-01 03:27:54,400 - lava_callback - INFO - Starting async job submit for node 69c9edab3c3aafdab4bea8f1 2026-04-01 03:27:54,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:27:54,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9edab3c3aafdab4bea8f1 HTTP/1.1" 200 None 2026-04-01 03:27:54,416 - lava_callback - DEBUG - Processing node 69c9edab3c3aafdab4bea8f1: baseline-arm64-clabbe 2026-04-01 03:27:54,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9edab3c3aafdab4bea8f1 2026-04-01 03:27:54,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:27:54,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9edab3c3aafdab4bea8f1 HTTP/1.1" 200 None 2026-04-01 03:27:54,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:27:54,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:27:54,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:27:54,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9edab3c3aafdab4bea8f1 HTTP/1.1" 200 None 2026-04-01 03:27:54,500 - lava_callback - ERROR - Error processing callback for node 69c9edab3c3aafdab4bea8f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33908 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:29:14,420 - lava_callback - INFO - Received callback for node 69c9edf93c3aafdab4bea8fa INFO: 172.20.0.1:55434 - "POST /node/69c9edf93c3aafdab4bea8fa HTTP/1.0" 202 Accepted 2026-04-01 03:29:14,426 - lava_callback - INFO - Starting async job submit for node 69c9edf93c3aafdab4bea8fa 2026-04-01 03:29:14,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:29:14,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9edf93c3aafdab4bea8fa HTTP/1.1" 200 None 2026-04-01 03:29:14,442 - lava_callback - DEBUG - Processing node 69c9edf93c3aafdab4bea8fa: baseline-arm64-clabbe 2026-04-01 03:29:14,442 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9edf93c3aafdab4bea8fa 2026-04-01 03:29:14,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:29:14,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9edf93c3aafdab4bea8fa HTTP/1.1" 200 None 2026-04-01 03:29:14,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:29:14,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:29:14,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:29:14,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9edf93c3aafdab4bea8fa HTTP/1.1" 200 None 2026-04-01 03:29:14,680 - lava_callback - ERROR - Error processing callback for node 69c9edf93c3aafdab4bea8fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46210 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:31:37,908 - lava_callback - INFO - Received callback for node 69cc8d7a8cd9cbcdc945b30a INFO: 172.20.0.1:46224 - "POST /node/69cc8d7a8cd9cbcdc945b30a HTTP/1.0" 202 Accepted 2026-04-01 03:31:37,915 - lava_callback - INFO - Starting async job submit for node 69cc8d7a8cd9cbcdc945b30a 2026-04-01 03:31:37,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:37,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8d7a8cd9cbcdc945b30a HTTP/1.1" 200 None 2026-04-01 03:31:37,987 - lava_callback - DEBUG - Processing node 69cc8d7a8cd9cbcdc945b30a: ltp-smoke 2026-04-01 03:31:38,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:31:38,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:31:38,586 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc8d7a8cd9cbcdc945b30a/log.txt.gz 2026-04-01 03:31:38,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:31:39,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:31:39,198 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc8d7a8cd9cbcdc945b30a/lava-logs.yaml 2026-04-01 03:31:39,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:31:39,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:31:39,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc8d7a8cd9cbcdc945b30a/lava_callback.json.gz 2026-04-01 03:31:39,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:39,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8d7a8cd9cbcdc945b30a HTTP/1.1" 200 None 2026-04-01 03:31:39,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:39,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64828cd9cbcdc945964d HTTP/1.1" 200 None 2026-04-01 03:31:39,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:39,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc8d7a8cd9cbcdc945b30a HTTP/1.1" 200 None 2026-04-01 03:31:39,680 - lava_callback - INFO - Completed processing callback for node 69cc8d7a8cd9cbcdc945b30a 2026-04-01 03:31:45,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:45,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35900 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:31:54,505 - lava_callback - INFO - Received callback for node 69c9ee9b3c3aafdab4bea90e INFO: 172.20.0.1:51578 - "POST /node/69c9ee9b3c3aafdab4bea90e HTTP/1.0" 202 Accepted 2026-04-01 03:31:54,511 - lava_callback - INFO - Starting async job submit for node 69c9ee9b3c3aafdab4bea90e 2026-04-01 03:31:54,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:54,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ee9b3c3aafdab4bea90e HTTP/1.1" 200 None 2026-04-01 03:31:54,527 - lava_callback - DEBUG - Processing node 69c9ee9b3c3aafdab4bea90e: baseline-arm64-clabbe 2026-04-01 03:31:54,527 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ee9b3c3aafdab4bea90e 2026-04-01 03:31:54,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:54,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ee9b3c3aafdab4bea90e HTTP/1.1" 200 None 2026-04-01 03:31:54,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:54,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-04-01 03:31:54,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:31:54,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ee9b3c3aafdab4bea90e HTTP/1.1" 200 None 2026-04-01 03:31:54,578 - lava_callback - ERROR - Error processing callback for node 69c9ee9b3c3aafdab4bea90e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47422 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:33:03,803 - lava_callback - INFO - Received callback for node 69cc8c288cd9cbcdc945b2a8 INFO: 172.20.0.1:41528 - "POST /node/69cc8c288cd9cbcdc945b2a8 HTTP/1.0" 202 Accepted 2026-04-01 03:33:03,812 - lava_callback - INFO - Starting async job submit for node 69cc8c288cd9cbcdc945b2a8 2026-04-01 03:33:03,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:33:03,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8c288cd9cbcdc945b2a8 HTTP/1.1" 200 None 2026-04-01 03:33:03,863 - lava_callback - DEBUG - Processing node 69cc8c288cd9cbcdc945b2a8: baseline-nfs-arm64 INFO: 172.20.0.1:41542 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:33:04,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:33:04,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:33:04,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc8c288cd9cbcdc945b2a8/log.txt.gz 2026-04-01 03:33:04,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:33:05,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:33:05,129 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc8c288cd9cbcdc945b2a8/lava-logs.yaml 2026-04-01 03:33:05,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:33:05,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:33:05,304 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc8c288cd9cbcdc945b2a8/lava_callback.json.gz 2026-04-01 03:33:05,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:33:05,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8c288cd9cbcdc945b2a8 HTTP/1.1" 200 None 2026-04-01 03:33:05,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:33:05,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 03:33:05,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:33:05,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc8c288cd9cbcdc945b2a8 HTTP/1.1" 200 None 2026-04-01 03:33:05,465 - lava_callback - INFO - Completed processing callback for node 69cc8c288cd9cbcdc945b2a8 INFO: 172.20.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39444 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 03:33:15,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:33:15,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36438 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:34:12,511 - lava_callback - INFO - Received callback for node 69cc8f068cd9cbcdc945b382 INFO: 172.20.0.1:53068 - "POST /node/69cc8f068cd9cbcdc945b382 HTTP/1.0" 202 Accepted 2026-04-01 03:34:12,518 - lava_callback - INFO - Starting async job submit for node 69cc8f068cd9cbcdc945b382 2026-04-01 03:34:12,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:34:12,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8f068cd9cbcdc945b382 HTTP/1.1" 200 None 2026-04-01 03:34:12,559 - lava_callback - DEBUG - Processing node 69cc8f068cd9cbcdc945b382: baseline-arm64 2026-04-01 03:34:13,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:34:13,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:34:13,245 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc8f068cd9cbcdc945b382/log.txt.gz 2026-04-01 03:34:13,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:34:13,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:34:13,725 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc8f068cd9cbcdc945b382/lava-logs.yaml 2026-04-01 03:34:13,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:34:13,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:34:13,903 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc8f068cd9cbcdc945b382/lava_callback.json.gz 2026-04-01 03:34:13,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:34:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc8f068cd9cbcdc945b382 HTTP/1.1" 200 None 2026-04-01 03:34:13,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:34:13,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 03:34:13,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:34:13,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc8f068cd9cbcdc945b382 HTTP/1.1" 200 None 2026-04-01 03:34:14,054 - lava_callback - INFO - Completed processing callback for node 69cc8f068cd9cbcdc945b382 INFO: 172.20.0.1:53078 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 03:34:15,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:34:15,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33210 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:45:23,847 - lava_callback - INFO - Received callback for node 69cc91fb8cd9cbcdc945b4c8 INFO: 172.20.0.1:38406 - "POST /node/69cc91fb8cd9cbcdc945b4c8 HTTP/1.0" 202 Accepted 2026-04-01 03:45:23,854 - lava_callback - INFO - Starting async job submit for node 69cc91fb8cd9cbcdc945b4c8 2026-04-01 03:45:23,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:45:23,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc91fb8cd9cbcdc945b4c8 HTTP/1.1" 200 None 2026-04-01 03:45:23,924 - lava_callback - DEBUG - Processing node 69cc91fb8cd9cbcdc945b4c8: baseline-nfs-arm64 2026-04-01 03:45:24,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:45:24,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:45:24,722 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc91fb8cd9cbcdc945b4c8/log.txt.gz 2026-04-01 03:45:25,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:45:25,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:45:25,235 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc91fb8cd9cbcdc945b4c8/lava-logs.yaml 2026-04-01 03:45:25,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 03:45:25,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 03:45:25,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc91fb8cd9cbcdc945b4c8/lava_callback.json.gz 2026-04-01 03:45:25,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:45:25,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc91fb8cd9cbcdc945b4c8 HTTP/1.1" 200 None 2026-04-01 03:45:25,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:45:25,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 03:45:25,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:45:25,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc91fb8cd9cbcdc945b4c8 HTTP/1.1" 200 None 2026-04-01 03:45:25,550 - lava_callback - INFO - Completed processing callback for node 69cc91fb8cd9cbcdc945b4c8 INFO: 172.20.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK 2026-04-01 03:45:45,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 03:45:45,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41422 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:00:45,466 - lava_callback - INFO - Received callback for node 69cc95118cd9cbcdc945b5fc INFO: 172.20.0.1:35664 - "POST /node/69cc95118cd9cbcdc945b5fc HTTP/1.0" 202 Accepted 2026-04-01 04:00:45,472 - lava_callback - INFO - Starting async job submit for node 69cc95118cd9cbcdc945b5fc 2026-04-01 04:00:45,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:00:45,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc95118cd9cbcdc945b5fc HTTP/1.1" 200 None 2026-04-01 04:00:45,516 - lava_callback - DEBUG - Processing node 69cc95118cd9cbcdc945b5fc: baseline-nfs-arm64 2026-04-01 04:00:46,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:00:46,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:00:46,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc95118cd9cbcdc945b5fc/log.txt.gz 2026-04-01 04:00:46,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:35680 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:00:46,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:00:46,668 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc95118cd9cbcdc945b5fc/lava-logs.yaml 2026-04-01 04:00:46,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:00:46,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:00:46,864 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc95118cd9cbcdc945b5fc/lava_callback.json.gz 2026-04-01 04:00:46,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:00:46,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc95118cd9cbcdc945b5fc HTTP/1.1" 200 None 2026-04-01 04:00:46,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:00:46,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-01 04:00:46,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:00:46,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc95118cd9cbcdc945b5fc HTTP/1.1" 200 None 2026-04-01 04:00:47,049 - lava_callback - INFO - Completed processing callback for node 69cc95118cd9cbcdc945b5fc INFO: 172.20.0.1:36778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60760 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:01:15,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:01:15,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51664 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:40:44,165 - lava_callback - INFO - Received callback for node 69cc687e8cd9cbcdc9459ae3 INFO: 172.20.0.1:60808 - "POST /node/69cc687e8cd9cbcdc9459ae3 HTTP/1.0" 202 Accepted 2026-04-01 04:40:44,171 - lava_callback - INFO - Starting async job submit for node 69cc687e8cd9cbcdc9459ae3 2026-04-01 04:40:44,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:40:44,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc687e8cd9cbcdc9459ae3 HTTP/1.1" 200 None 2026-04-01 04:40:44,212 - lava_callback - DEBUG - Processing node 69cc687e8cd9cbcdc9459ae3: baseline-arm-broonie 2026-04-01 04:40:44,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:40:44,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:40:44,547 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc687e8cd9cbcdc9459ae3/log.txt.gz 2026-04-01 04:40:44,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:40:44,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:40:44,913 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc687e8cd9cbcdc9459ae3/lava-logs.yaml 2026-04-01 04:40:44,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:40:45,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:40:45,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc687e8cd9cbcdc9459ae3/lava_callback.json.gz 2026-04-01 04:40:45,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:40:45,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc687e8cd9cbcdc9459ae3 HTTP/1.1" 200 None 2026-04-01 04:40:45,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:40:45,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:40:45,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:40:45,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc687e8cd9cbcdc9459ae3 HTTP/1.1" 200 None 2026-04-01 04:40:45,141 - lava_callback - INFO - Completed processing callback for node 69cc687e8cd9cbcdc9459ae3 2026-04-01 04:40:45,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:40:45,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:43:44,436 - lava_callback - INFO - Received callback for node 69cc68a68cd9cbcdc9459b64 INFO: 172.20.0.1:58366 - "POST /node/69cc68a68cd9cbcdc9459b64 HTTP/1.0" 202 Accepted 2026-04-01 04:43:44,443 - lava_callback - INFO - Starting async job submit for node 69cc68a68cd9cbcdc9459b64 2026-04-01 04:43:44,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:43:44,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68a68cd9cbcdc9459b64 HTTP/1.1" 200 None 2026-04-01 04:43:44,482 - lava_callback - DEBUG - Processing node 69cc68a68cd9cbcdc9459b64: ltp-cap_bounds 2026-04-01 04:43:44,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:43:45,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:43:45,030 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cc68a68cd9cbcdc9459b64/log.txt.gz 2026-04-01 04:43:45,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:43:45,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:43:45,491 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cc68a68cd9cbcdc9459b64/lava-logs.yaml 2026-04-01 04:43:45,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:43:45,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:43:45,688 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cc68a68cd9cbcdc9459b64/lava_callback.json.gz 2026-04-01 04:43:45,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:43:45,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68a68cd9cbcdc9459b64 HTTP/1.1" 200 None 2026-04-01 04:43:45,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:43:45,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:43:45,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:43:45,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68a68cd9cbcdc9459b64 HTTP/1.1" 200 None 2026-04-01 04:43:45,814 - lava_callback - INFO - Completed processing callback for node 69cc68a68cd9cbcdc9459b64 INFO: 172.20.0.1:58376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:44:04,690 - lava_callback - INFO - Received callback for node 69cc68a88cd9cbcdc9459b7a INFO: 172.20.0.1:41752 - "POST /node/69cc68a88cd9cbcdc9459b7a HTTP/1.0" 202 Accepted 2026-04-01 04:44:04,696 - lava_callback - INFO - Starting async job submit for node 69cc68a88cd9cbcdc9459b7a 2026-04-01 04:44:04,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:04,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68a88cd9cbcdc9459b7a HTTP/1.1" 200 None 2026-04-01 04:44:04,747 - lava_callback - DEBUG - Processing node 69cc68a88cd9cbcdc9459b7a: ltp-containers INFO: 172.20.0.1:41756 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:44:05,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:44:05,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:44:05,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cc68a88cd9cbcdc9459b7a/log.txt.gz 2026-04-01 04:44:05,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:44:05,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:44:05,783 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cc68a88cd9cbcdc9459b7a/lava-logs.yaml 2026-04-01 04:44:05,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:44:05,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:44:05,941 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cc68a88cd9cbcdc9459b7a/lava_callback.json.gz 2026-04-01 04:44:05,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:05,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68a88cd9cbcdc9459b7a HTTP/1.1" 200 None 2026-04-01 04:44:05,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:05,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:44:05,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:06,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68a88cd9cbcdc9459b7a HTTP/1.1" 200 None 2026-04-01 04:44:06,032 - lava_callback - INFO - Completed processing callback for node 69cc68a88cd9cbcdc9459b7a INFO: 172.20.0.1:41762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:44:15,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:15,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37584 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:44:44,571 - lava_callback - INFO - Received callback for node 69cc68a98cd9cbcdc9459b81 INFO: 172.20.0.1:36070 - "POST /node/69cc68a98cd9cbcdc9459b81 HTTP/1.0" 202 Accepted 2026-04-01 04:44:44,578 - lava_callback - INFO - Starting async job submit for node 69cc68a98cd9cbcdc9459b81 2026-04-01 04:44:44,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:44,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68a98cd9cbcdc9459b81 HTTP/1.1" 200 None 2026-04-01 04:44:44,627 - lava_callback - DEBUG - Processing node 69cc68a98cd9cbcdc9459b81: ltp-controllers 2026-04-01 04:44:45,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:44:45,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:44:45,275 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cc68a98cd9cbcdc9459b81/log.txt.gz 2026-04-01 04:44:45,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:44:45,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:44:45,780 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cc68a98cd9cbcdc9459b81/lava-logs.yaml 2026-04-01 04:44:45,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:44:45,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:44:45,959 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cc68a98cd9cbcdc9459b81/lava_callback.json.gz 2026-04-01 04:44:45,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:46,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68a98cd9cbcdc9459b81 HTTP/1.1" 200 None 2026-04-01 04:44:46,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:46,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:44:46,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:46,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68a98cd9cbcdc9459b81 HTTP/1.1" 200 None 2026-04-01 04:44:46,050 - lava_callback - INFO - Completed processing callback for node 69cc68a98cd9cbcdc9459b81 INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:44:51,880 - lava_callback - INFO - Received callback for node 69c9ffb93c3aafdab4bea96a INFO: 172.20.0.1:56120 - "POST /node/69c9ffb93c3aafdab4bea96a HTTP/1.0" 202 Accepted 2026-04-01 04:44:51,886 - lava_callback - INFO - Starting async job submit for node 69c9ffb93c3aafdab4bea96a 2026-04-01 04:44:51,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:51,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ffb93c3aafdab4bea96a HTTP/1.1" 200 None 2026-04-01 04:44:51,902 - lava_callback - DEBUG - Processing node 69c9ffb93c3aafdab4bea96a: baseline-arm-baylibre 2026-04-01 04:44:51,902 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ffb93c3aafdab4bea96a 2026-04-01 04:44:51,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:51,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ffb93c3aafdab4bea96a HTTP/1.1" 200 None 2026-04-01 04:44:51,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:51,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:44:51,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:44:51,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ffb93c3aafdab4bea96a HTTP/1.1" 200 None 2026-04-01 04:44:51,958 - lava_callback - ERROR - Error processing callback for node 69c9ffb93c3aafdab4bea96a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42618 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:45:15,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:15,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42628 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:45:21,488 - lava_callback - INFO - Received callback for node 69cc68ac8cd9cbcdc9459b84 INFO: 172.20.0.1:56598 - "POST /node/69cc68ac8cd9cbcdc9459b84 HTTP/1.0" 202 Accepted 2026-04-01 04:45:21,495 - lava_callback - INFO - Starting async job submit for node 69cc68ac8cd9cbcdc9459b84 2026-04-01 04:45:21,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:21,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ac8cd9cbcdc9459b84 HTTP/1.1" 200 None 2026-04-01 04:45:21,552 - lava_callback - DEBUG - Processing node 69cc68ac8cd9cbcdc9459b84: ltp-cve 2026-04-01 04:45:22,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:45:22,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:45:22,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cc68ac8cd9cbcdc9459b84/log.txt.gz 2026-04-01 04:45:22,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:45:22,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:45:22,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cc68ac8cd9cbcdc9459b84/lava-logs.yaml 2026-04-01 04:45:22,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:45:22,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:45:22,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cc68ac8cd9cbcdc9459b84/lava_callback.json.gz 2026-04-01 04:45:22,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:22,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ac8cd9cbcdc9459b84 HTTP/1.1" 200 None 2026-04-01 04:45:22,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:22,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:45:22,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:22,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68ac8cd9cbcdc9459b84 HTTP/1.1" 200 None 2026-04-01 04:45:22,845 - lava_callback - INFO - Completed processing callback for node 69cc68ac8cd9cbcdc9459b84 INFO: 172.20.0.1:56610 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:45:31,812 - lava_callback - INFO - Received callback for node 69c9ffdd3c3aafdab4bea974 INFO: 172.20.0.1:38702 - "POST /node/69c9ffdd3c3aafdab4bea974 HTTP/1.0" 202 Accepted 2026-04-01 04:45:31,818 - lava_callback - INFO - Starting async job submit for node 69c9ffdd3c3aafdab4bea974 2026-04-01 04:45:31,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:31,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ffdd3c3aafdab4bea974 HTTP/1.1" 200 None 2026-04-01 04:45:31,833 - lava_callback - DEBUG - Processing node 69c9ffdd3c3aafdab4bea974: baseline-arm-baylibre 2026-04-01 04:45:31,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ffdd3c3aafdab4bea974 2026-04-01 04:45:31,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:31,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ffdd3c3aafdab4bea974 HTTP/1.1" 200 None 2026-04-01 04:45:31,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:31,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:45:31,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:31,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ffdd3c3aafdab4bea974 HTTP/1.1" 200 None 2026-04-01 04:45:31,901 - lava_callback - ERROR - Error processing callback for node 69c9ffdd3c3aafdab4bea974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38716 - "GET /node/694c15632f7ca341fdeb0dfc%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-01 04:45:45,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:45:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:46:02,270 - lava_callback - INFO - Received callback for node 69cc68818cd9cbcdc9459aeb INFO: 172.20.0.1:58282 - "POST /node/69cc68818cd9cbcdc9459aeb HTTP/1.0" 202 Accepted 2026-04-01 04:46:02,279 - lava_callback - INFO - Starting async job submit for node 69cc68818cd9cbcdc9459aeb 2026-04-01 04:46:02,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:02,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68818cd9cbcdc9459aeb HTTP/1.1" 200 None 2026-04-01 04:46:02,318 - lava_callback - DEBUG - Processing node 69cc68818cd9cbcdc9459aeb: baseline-arm-broonie 2026-04-01 04:46:02,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:46:02,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:46:02,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68818cd9cbcdc9459aeb/log.txt.gz 2026-04-01 04:46:02,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:46:02,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:46:02,760 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68818cd9cbcdc9459aeb/lava-logs.yaml 2026-04-01 04:46:02,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:46:02,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:46:02,901 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cc68818cd9cbcdc9459aeb/lava_callback.json.gz 2026-04-01 04:46:02,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:02,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68818cd9cbcdc9459aeb HTTP/1.1" 200 None 2026-04-01 04:46:02,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:02,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:46:02,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:02,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68818cd9cbcdc9459aeb HTTP/1.1" 200 None 2026-04-01 04:46:03,032 - lava_callback - INFO - Completed processing callback for node 69cc68818cd9cbcdc9459aeb INFO: 172.20.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58298 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:46:11,883 - lava_callback - INFO - Received callback for node 69ca00033c3aafdab4bea981 INFO: 172.20.0.1:48104 - "POST /node/69ca00033c3aafdab4bea981 HTTP/1.0" 202 Accepted 2026-04-01 04:46:11,889 - lava_callback - INFO - Starting async job submit for node 69ca00033c3aafdab4bea981 2026-04-01 04:46:11,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:11,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca00033c3aafdab4bea981 HTTP/1.1" 200 None 2026-04-01 04:46:11,905 - lava_callback - DEBUG - Processing node 69ca00033c3aafdab4bea981: baseline-arm-baylibre 2026-04-01 04:46:11,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca00033c3aafdab4bea981 2026-04-01 04:46:11,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:11,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca00033c3aafdab4bea981 HTTP/1.1" 200 None 2026-04-01 04:46:11,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:11,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:46:11,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:11,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca00033c3aafdab4bea981 HTTP/1.1" 200 None 2026-04-01 04:46:11,970 - lava_callback - ERROR - Error processing callback for node 69ca00033c3aafdab4bea981: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48116 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:46:15,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:15,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET /node/6990a4e9c45febfe445203a2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54340 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:46:51,893 - lava_callback - INFO - Received callback for node 69ca00283c3aafdab4bea995 INFO: 172.20.0.1:42212 - "POST /node/69ca00283c3aafdab4bea995 HTTP/1.0" 202 Accepted 2026-04-01 04:46:51,899 - lava_callback - INFO - Starting async job submit for node 69ca00283c3aafdab4bea995 2026-04-01 04:46:51,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:51,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca00283c3aafdab4bea995 HTTP/1.1" 200 None 2026-04-01 04:46:51,915 - lava_callback - DEBUG - Processing node 69ca00283c3aafdab4bea995: baseline-arm-baylibre 2026-04-01 04:46:51,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca00283c3aafdab4bea995 2026-04-01 04:46:51,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:51,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca00283c3aafdab4bea995 HTTP/1.1" 200 None 2026-04-01 04:46:51,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:51,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:46:51,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:46:51,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca00283c3aafdab4bea995 HTTP/1.1" 200 None 2026-04-01 04:46:51,973 - lava_callback - ERROR - Error processing callback for node 69ca00283c3aafdab4bea995: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:47:11,952 - lava_callback - INFO - Received callback for node 69ca004d3c3aafdab4bea99e INFO: 172.20.0.1:58036 - "POST /node/69ca004d3c3aafdab4bea99e HTTP/1.0" 202 Accepted 2026-04-01 04:47:11,958 - lava_callback - INFO - Starting async job submit for node 69ca004d3c3aafdab4bea99e 2026-04-01 04:47:11,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:47:11,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca004d3c3aafdab4bea99e HTTP/1.1" 200 None 2026-04-01 04:47:11,974 - lava_callback - DEBUG - Processing node 69ca004d3c3aafdab4bea99e: baseline-arm-baylibre 2026-04-01 04:47:11,974 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca004d3c3aafdab4bea99e 2026-04-01 04:47:11,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:47:11,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca004d3c3aafdab4bea99e HTTP/1.1" 200 None 2026-04-01 04:47:11,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:47:12,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:47:12,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:47:12,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca004d3c3aafdab4bea99e HTTP/1.1" 200 None 2026-04-01 04:47:12,035 - lava_callback - ERROR - Error processing callback for node 69ca004d3c3aafdab4bea99e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54012 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:48:04,627 - lava_callback - INFO - Received callback for node 69cc68b28cd9cbcdc9459b8d INFO: 172.20.0.1:33148 - "POST /node/69cc68b28cd9cbcdc9459b8d HTTP/1.0" 202 Accepted 2026-04-01 04:48:04,636 - lava_callback - INFO - Starting async job submit for node 69cc68b28cd9cbcdc9459b8d 2026-04-01 04:48:04,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:04,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b28cd9cbcdc9459b8d HTTP/1.1" 200 None 2026-04-01 04:48:04,678 - lava_callback - DEBUG - Processing node 69cc68b28cd9cbcdc9459b8d: ltp-fsx 2026-04-01 04:48:05,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33154 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:48:05,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:05,261 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cc68b28cd9cbcdc9459b8d/log.txt.gz 2026-04-01 04:48:05,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:05,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:05,697 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cc68b28cd9cbcdc9459b8d/lava-logs.yaml 2026-04-01 04:48:05,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:05,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:05,868 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cc68b28cd9cbcdc9459b8d/lava_callback.json.gz 2026-04-01 04:48:05,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:05,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b28cd9cbcdc9459b8d HTTP/1.1" 200 None 2026-04-01 04:48:05,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:05,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:48:05,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:05,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68b28cd9cbcdc9459b8d HTTP/1.1" 200 None 2026-04-01 04:48:05,983 - lava_callback - INFO - Completed processing callback for node 69cc68b28cd9cbcdc9459b8d INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54204 - "GET /node/69939abecebe99dd8e0751c2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54214 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:48:15,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:15,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54230 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:48:24,913 - lava_callback - INFO - Received callback for node 69cc68b08cd9cbcdc9459b8a INFO: 172.20.0.1:60732 - "POST /node/69cc68b08cd9cbcdc9459b8a HTTP/1.0" 202 Accepted 2026-04-01 04:48:24,920 - lava_callback - INFO - Starting async job submit for node 69cc68b08cd9cbcdc9459b8a 2026-04-01 04:48:24,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:24,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b08cd9cbcdc9459b8a HTTP/1.1" 200 None 2026-04-01 04:48:24,972 - lava_callback - DEBUG - Processing node 69cc68b08cd9cbcdc9459b8a: ltp-fs_bind 2026-04-01 04:48:25,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:25,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:25,621 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cc68b08cd9cbcdc9459b8a/log.txt.gz 2026-04-01 04:48:25,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:26,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:26,064 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cc68b08cd9cbcdc9459b8a/lava-logs.yaml 2026-04-01 04:48:26,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:26,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:26,225 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cc68b08cd9cbcdc9459b8a/lava_callback.json.gz 2026-04-01 04:48:26,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:26,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b08cd9cbcdc9459b8a HTTP/1.1" 200 None 2026-04-01 04:48:26,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:26,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:48:26,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:26,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68b08cd9cbcdc9459b8a HTTP/1.1" 200 None 2026-04-01 04:48:26,331 - lava_callback - INFO - Completed processing callback for node 69cc68b08cd9cbcdc9459b8a INFO: 172.20.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:48:45,142 - lava_callback - INFO - Received callback for node 69cc68b38cd9cbcdc9459b90 INFO: 172.20.0.1:60480 - "POST /node/69cc68b38cd9cbcdc9459b90 HTTP/1.0" 202 Accepted 2026-04-01 04:48:45,152 - lava_callback - INFO - Starting async job submit for node 69cc68b38cd9cbcdc9459b90 2026-04-01 04:48:45,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:45,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b38cd9cbcdc9459b90 HTTP/1.1" 200 None 2026-04-01 04:48:45,196 - lava_callback - DEBUG - Processing node 69cc68b38cd9cbcdc9459b90: ltp-input 2026-04-01 04:48:45,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:45,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:45,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:45,716 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cc68b38cd9cbcdc9459b90/log.txt.gz 2026-04-01 04:48:45,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-01 04:48:45,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:46,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:46,138 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cc68b38cd9cbcdc9459b90/lava-logs.yaml 2026-04-01 04:48:46,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:48:46,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:48:46,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cc68b38cd9cbcdc9459b90/lava_callback.json.gz 2026-04-01 04:48:46,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:46,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b38cd9cbcdc9459b90 HTTP/1.1" 200 None 2026-04-01 04:48:46,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:46,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:48:46,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:48:46,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68b38cd9cbcdc9459b90 HTTP/1.1" 200 None 2026-04-01 04:48:46,409 - lava_callback - INFO - Completed processing callback for node 69cc68b38cd9cbcdc9459b90 INFO: 172.20.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41812 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:49:15,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:15,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41824 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:49:20,915 - lava_callback - INFO - Received callback for node 69cc68b58cd9cbcdc9459b93 INFO: 172.20.0.1:60344 - "POST /node/69cc68b58cd9cbcdc9459b93 HTTP/1.0" 202 Accepted 2026-04-01 04:49:20,921 - lava_callback - INFO - Starting async job submit for node 69cc68b58cd9cbcdc9459b93 2026-04-01 04:49:20,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:20,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b58cd9cbcdc9459b93 HTTP/1.1" 200 None 2026-04-01 04:49:20,963 - lava_callback - DEBUG - Processing node 69cc68b58cd9cbcdc9459b93: ltp-ipc 2026-04-01 04:49:21,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:49:21,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:49:21,603 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cc68b58cd9cbcdc9459b93/log.txt.gz 2026-04-01 04:49:21,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:49:22,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:49:22,026 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cc68b58cd9cbcdc9459b93/lava-logs.yaml 2026-04-01 04:49:22,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:49:22,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:49:22,195 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cc68b58cd9cbcdc9459b93/lava_callback.json.gz 2026-04-01 04:49:22,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:22,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b58cd9cbcdc9459b93 HTTP/1.1" 200 None 2026-04-01 04:49:22,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:22,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:49:22,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:22,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68b58cd9cbcdc9459b93 HTTP/1.1" 200 None 2026-04-01 04:49:22,284 - lava_callback - INFO - Completed processing callback for node 69cc68b58cd9cbcdc9459b93 INFO: 172.20.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:49:45,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:45,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45204 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:49:57,987 - lava_callback - INFO - Received callback for node 69ca00eb3c3aafdab4bea9cf INFO: 172.20.0.1:45208 - "POST /node/69ca00eb3c3aafdab4bea9cf HTTP/1.0" 202 Accepted 2026-04-01 04:49:57,992 - lava_callback - INFO - Starting async job submit for node 69ca00eb3c3aafdab4bea9cf 2026-04-01 04:49:57,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:58,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca00eb3c3aafdab4bea9cf HTTP/1.1" 200 None 2026-04-01 04:49:58,008 - lava_callback - DEBUG - Processing node 69ca00eb3c3aafdab4bea9cf: baseline-arm-clabbe 2026-04-01 04:49:58,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca00eb3c3aafdab4bea9cf 2026-04-01 04:49:58,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:58,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca00eb3c3aafdab4bea9cf HTTP/1.1" 200 None 2026-04-01 04:49:58,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:58,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:49:58,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:49:58,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca00eb3c3aafdab4bea9cf HTTP/1.1" 200 None 2026-04-01 04:49:58,064 - lava_callback - ERROR - Error processing callback for node 69ca00eb3c3aafdab4bea9cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:50:37,947 - lava_callback - INFO - Received callback for node 69ca01113c3aafdab4bea9db INFO: 172.20.0.1:53976 - "POST /node/69ca01113c3aafdab4bea9db HTTP/1.0" 202 Accepted 2026-04-01 04:50:37,953 - lava_callback - INFO - Starting async job submit for node 69ca01113c3aafdab4bea9db 2026-04-01 04:50:37,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:50:37,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca01113c3aafdab4bea9db HTTP/1.1" 200 None 2026-04-01 04:50:37,968 - lava_callback - DEBUG - Processing node 69ca01113c3aafdab4bea9db: baseline-arm-clabbe 2026-04-01 04:50:37,969 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca01113c3aafdab4bea9db 2026-04-01 04:50:37,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:50:37,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca01113c3aafdab4bea9db HTTP/1.1" 200 None 2026-04-01 04:50:37,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:50:37,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:50:37,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:50:38,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca01113c3aafdab4bea9db HTTP/1.1" 200 None 2026-04-01 04:50:38,035 - lava_callback - ERROR - Error processing callback for node 69ca01113c3aafdab4bea9db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51302 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:51:18,052 - lava_callback - INFO - Received callback for node 69ca01363c3aafdab4bea9ec INFO: 172.20.0.1:51318 - "POST /node/69ca01363c3aafdab4bea9ec HTTP/1.0" 202 Accepted 2026-04-01 04:51:18,057 - lava_callback - INFO - Starting async job submit for node 69ca01363c3aafdab4bea9ec 2026-04-01 04:51:18,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:18,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca01363c3aafdab4bea9ec HTTP/1.1" 200 None 2026-04-01 04:51:18,079 - lava_callback - DEBUG - Processing node 69ca01363c3aafdab4bea9ec: baseline-arm-clabbe 2026-04-01 04:51:18,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca01363c3aafdab4bea9ec 2026-04-01 04:51:18,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:18,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca01363c3aafdab4bea9ec HTTP/1.1" 200 None 2026-04-01 04:51:18,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:18,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:51:18,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:18,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca01363c3aafdab4bea9ec HTTP/1.1" 200 None 2026-04-01 04:51:18,140 - lava_callback - ERROR - Error processing callback for node 69ca01363c3aafdab4bea9ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:51:58,219 - lava_callback - INFO - Received callback for node 69ca015b3c3aafdab4bea9fa INFO: 172.20.0.1:54110 - "POST /node/69ca015b3c3aafdab4bea9fa HTTP/1.0" 202 Accepted 2026-04-01 04:51:58,224 - lava_callback - INFO - Starting async job submit for node 69ca015b3c3aafdab4bea9fa 2026-04-01 04:51:58,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:58,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca015b3c3aafdab4bea9fa HTTP/1.1" 200 None 2026-04-01 04:51:58,240 - lava_callback - DEBUG - Processing node 69ca015b3c3aafdab4bea9fa: baseline-arm-clabbe 2026-04-01 04:51:58,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca015b3c3aafdab4bea9fa 2026-04-01 04:51:58,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:58,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca015b3c3aafdab4bea9fa HTTP/1.1" 200 None 2026-04-01 04:51:58,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:58,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:51:58,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:51:58,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca015b3c3aafdab4bea9fa HTTP/1.1" 200 None 2026-04-01 04:51:58,300 - lava_callback - ERROR - Error processing callback for node 69ca015b3c3aafdab4bea9fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 04:52:04,831 - lava_callback - INFO - Received callback for node 69cc68b68cd9cbcdc9459b96 INFO: 172.20.0.1:48640 - "POST /node/69cc68b68cd9cbcdc9459b96 HTTP/1.0" 202 Accepted 2026-04-01 04:52:04,838 - lava_callback - INFO - Starting async job submit for node 69cc68b68cd9cbcdc9459b96 2026-04-01 04:52:04,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:04,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b68cd9cbcdc9459b96 HTTP/1.1" 200 None 2026-04-01 04:52:04,880 - lava_callback - DEBUG - Processing node 69cc68b68cd9cbcdc9459b96: ltp-pty INFO: 172.20.0.1:48646 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:52:05,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:52:05,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:52:05,553 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cc68b68cd9cbcdc9459b96/log.txt.gz 2026-04-01 04:52:05,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:52:05,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:52:05,979 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cc68b68cd9cbcdc9459b96/lava-logs.yaml 2026-04-01 04:52:05,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:52:06,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:52:06,157 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cc68b68cd9cbcdc9459b96/lava_callback.json.gz 2026-04-01 04:52:06,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:06,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b68cd9cbcdc9459b96 HTTP/1.1" 200 None 2026-04-01 04:52:06,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:06,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:52:06,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:06,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68b68cd9cbcdc9459b96 HTTP/1.1" 200 None 2026-04-01 04:52:06,243 - lava_callback - INFO - Completed processing callback for node 69cc68b68cd9cbcdc9459b96 INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45426 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:52:15,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:15,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53378 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:52:38,082 - lava_callback - INFO - Received callback for node 69ca01813c3aafdab4beaa06 INFO: 172.20.0.1:53388 - "POST /node/69ca01813c3aafdab4beaa06 HTTP/1.0" 202 Accepted 2026-04-01 04:52:38,088 - lava_callback - INFO - Starting async job submit for node 69ca01813c3aafdab4beaa06 2026-04-01 04:52:38,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:38,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca01813c3aafdab4beaa06 HTTP/1.1" 200 None 2026-04-01 04:52:38,104 - lava_callback - DEBUG - Processing node 69ca01813c3aafdab4beaa06: baseline-arm-clabbe 2026-04-01 04:52:38,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca01813c3aafdab4beaa06 2026-04-01 04:52:38,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:38,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca01813c3aafdab4beaa06 HTTP/1.1" 200 None 2026-04-01 04:52:38,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:38,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-01 04:52:38,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:38,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca01813c3aafdab4beaa06 HTTP/1.1" 200 None 2026-04-01 04:52:38,156 - lava_callback - ERROR - Error processing callback for node 69ca01813c3aafdab4beaa06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 04:52:45,512 - lava_callback - INFO - Received callback for node 69cc68ba8cd9cbcdc9459ba2 INFO: 172.20.0.1:59210 - "POST /node/69cc68ba8cd9cbcdc9459ba2 HTTP/1.0" 202 Accepted 2026-04-01 04:52:45,518 - lava_callback - INFO - Starting async job submit for node 69cc68ba8cd9cbcdc9459ba2 2026-04-01 04:52:45,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:45,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ba8cd9cbcdc9459ba2 HTTP/1.1" 200 None 2026-04-01 04:52:45,558 - lava_callback - DEBUG - Processing node 69cc68ba8cd9cbcdc9459ba2: ltp-smoke 2026-04-01 04:52:45,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:52:46,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:52:46,039 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc68ba8cd9cbcdc9459ba2/log.txt.gz 2026-04-01 04:52:46,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:52:46,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:52:46,474 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc68ba8cd9cbcdc9459ba2/lava-logs.yaml 2026-04-01 04:52:46,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:59212 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:52:46,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:52:46,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc68ba8cd9cbcdc9459ba2/lava_callback.json.gz 2026-04-01 04:52:46,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:46,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68ba8cd9cbcdc9459ba2 HTTP/1.1" 200 None 2026-04-01 04:52:46,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:46,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:52:46,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:52:46,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68ba8cd9cbcdc9459ba2 HTTP/1.1" 200 None 2026-04-01 04:52:46,757 - lava_callback - INFO - Completed processing callback for node 69cc68ba8cd9cbcdc9459ba2 INFO: 172.20.0.1:48116 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:53:05,246 - lava_callback - INFO - Received callback for node 69cc68b88cd9cbcdc9459b99 INFO: 172.20.0.1:57138 - "POST /node/69cc68b88cd9cbcdc9459b99 HTTP/1.0" 202 Accepted 2026-04-01 04:53:05,253 - lava_callback - INFO - Starting async job submit for node 69cc68b88cd9cbcdc9459b99 INFO: 172.20.0.1:57144 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:53:05,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:05,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b88cd9cbcdc9459b99 HTTP/1.1" 200 None 2026-04-01 04:53:05,294 - lava_callback - DEBUG - Processing node 69cc68b88cd9cbcdc9459b99: ltp-sched 2026-04-01 04:53:05,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:53:05,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:53:05,980 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cc68b88cd9cbcdc9459b99/log.txt.gz 2026-04-01 04:53:06,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:53:06,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:53:06,391 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cc68b88cd9cbcdc9459b99/lava-logs.yaml 2026-04-01 04:53:06,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:53:06,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:53:06,572 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cc68b88cd9cbcdc9459b99/lava_callback.json.gz 2026-04-01 04:53:06,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:06,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68b88cd9cbcdc9459b99 HTTP/1.1" 200 None 2026-04-01 04:53:06,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:06,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:53:06,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:06,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68b88cd9cbcdc9459b99 HTTP/1.1" 200 None 2026-04-01 04:53:06,653 - lava_callback - INFO - Completed processing callback for node 69cc68b88cd9cbcdc9459b99 INFO: 172.20.0.1:39224 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:53:15,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:15,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39232 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:53:20,957 - lava_callback - INFO - Received callback for node 69cc68bd8cd9cbcdc9459ba8 INFO: 172.20.0.1:60702 - "POST /node/69cc68bd8cd9cbcdc9459ba8 HTTP/1.0" 202 Accepted 2026-04-01 04:53:20,964 - lava_callback - INFO - Starting async job submit for node 69cc68bd8cd9cbcdc9459ba8 2026-04-01 04:53:20,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:21,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68bd8cd9cbcdc9459ba8 HTTP/1.1" 200 None 2026-04-01 04:53:21,006 - lava_callback - DEBUG - Processing node 69cc68bd8cd9cbcdc9459ba8: ltp-syscalls-ipc 2026-04-01 04:53:21,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:53:21,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:53:21,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cc68bd8cd9cbcdc9459ba8/log.txt.gz 2026-04-01 04:53:21,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:53:21,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:53:21,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cc68bd8cd9cbcdc9459ba8/lava-logs.yaml 2026-04-01 04:53:21,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:53:22,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:53:22,092 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cc68bd8cd9cbcdc9459ba8/lava_callback.json.gz 2026-04-01 04:53:22,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:22,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68bd8cd9cbcdc9459ba8 HTTP/1.1" 200 None 2026-04-01 04:53:22,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:22,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:53:22,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:22,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68bd8cd9cbcdc9459ba8 HTTP/1.1" 200 None 2026-04-01 04:53:22,179 - lava_callback - INFO - Completed processing callback for node 69cc68bd8cd9cbcdc9459ba8 INFO: 172.20.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47356 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:53:45,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:53:45,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET /node/694bd890c19e09af58d497c7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55846 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:55:40,851 - lava_callback - INFO - Received callback for node 69cca36f8cd9cbcdc945b7ce INFO: 172.20.0.1:58480 - "POST /node/69cca36f8cd9cbcdc945b7ce HTTP/1.0" 202 Accepted 2026-04-01 04:55:40,858 - lava_callback - INFO - Starting async job submit for node 69cca36f8cd9cbcdc945b7ce 2026-04-01 04:55:40,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:55:40,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca36f8cd9cbcdc945b7ce HTTP/1.1" 200 None 2026-04-01 04:55:40,905 - lava_callback - DEBUG - Processing node 69cca36f8cd9cbcdc945b7ce: baseline-arm-broonie 2026-04-01 04:55:41,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:55:41,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:55:41,282 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cca36f8cd9cbcdc945b7ce/log.txt.gz 2026-04-01 04:55:41,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:55:41,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:55:41,629 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cca36f8cd9cbcdc945b7ce/lava-logs.yaml 2026-04-01 04:55:41,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:55:41,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:55:41,954 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cca36f8cd9cbcdc945b7ce/lava_callback.json.gz 2026-04-01 04:55:41,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:55:41,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca36f8cd9cbcdc945b7ce HTTP/1.1" 200 None 2026-04-01 04:55:41,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:55:42,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:55:42,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:55:42,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca36f8cd9cbcdc945b7ce HTTP/1.1" 200 None 2026-04-01 04:55:42,048 - lava_callback - INFO - Completed processing callback for node 69cca36f8cd9cbcdc945b7ce 2026-04-01 04:55:45,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:55:45,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58716 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:56:04,953 - lava_callback - INFO - Received callback for node 69cc68c08cd9cbcdc9459baf INFO: 172.20.0.1:47046 - "POST /node/69cc68c08cd9cbcdc9459baf HTTP/1.0" 202 Accepted 2026-04-01 04:56:04,959 - lava_callback - INFO - Starting async job submit for node 69cc68c08cd9cbcdc9459baf 2026-04-01 04:56:04,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:04,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68c08cd9cbcdc9459baf HTTP/1.1" 200 None 2026-04-01 04:56:04,999 - lava_callback - DEBUG - Processing node 69cc68c08cd9cbcdc9459baf: ltp-timers INFO: 172.20.0.1:47048 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:56:05,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:56:05,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:56:05,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cc68c08cd9cbcdc9459baf/log.txt.gz 2026-04-01 04:56:05,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:56:06,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:56:06,004 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cc68c08cd9cbcdc9459baf/lava-logs.yaml 2026-04-01 04:56:06,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:56:06,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:56:06,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cc68c08cd9cbcdc9459baf/lava_callback.json.gz 2026-04-01 04:56:06,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:06,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68c08cd9cbcdc9459baf HTTP/1.1" 200 None 2026-04-01 04:56:06,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:06,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:56:06,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:06,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68c08cd9cbcdc9459baf HTTP/1.1" 200 None 2026-04-01 04:56:06,259 - lava_callback - INFO - Completed processing callback for node 69cc68c08cd9cbcdc9459baf INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47076 - "GET /node/694ac27f6c96ed51d316ef59%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50324 - "GET /node/6937e0b3a4c94886b94bf3dd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50336 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:56:15,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:15,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58880 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:56:45,186 - lava_callback - INFO - Received callback for node 69cc68c28cd9cbcdc9459bb2 INFO: 172.20.0.1:55844 - "POST /node/69cc68c28cd9cbcdc9459bb2 HTTP/1.0" 202 Accepted 2026-04-01 04:56:45,193 - lava_callback - INFO - Starting async job submit for node 69cc68c28cd9cbcdc9459bb2 2026-04-01 04:56:45,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:45,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68c28cd9cbcdc9459bb2 HTTP/1.1" 200 None 2026-04-01 04:56:45,233 - lava_callback - DEBUG - Processing node 69cc68c28cd9cbcdc9459bb2: ltp-watchqueue 2026-04-01 04:56:45,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:56:45,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:56:45,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cc68c28cd9cbcdc9459bb2/log.txt.gz 2026-04-01 04:56:45,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:56:46,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:56:46,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cc68c28cd9cbcdc9459bb2/lava-logs.yaml 2026-04-01 04:56:46,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 04:56:46,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 04:56:46,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cc68c28cd9cbcdc9459bb2/lava_callback.json.gz 2026-04-01 04:56:46,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:46,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68c28cd9cbcdc9459bb2 HTTP/1.1" 200 None 2026-04-01 04:56:46,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:56:46,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-01 04:56:46,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:55856 - "GET / HTTP/1.0" 200 OK 2026-04-01 04:56:46,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68c28cd9cbcdc9459bb2 HTTP/1.1" 200 None 2026-04-01 04:56:46,533 - lava_callback - INFO - Completed processing callback for node 69cc68c28cd9cbcdc9459bb2 INFO: 172.20.0.1:50850 - "GET /node/693b083afee8383e92b7123b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35192 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 04:57:15,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 04:57:15,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58242 - "GET /node/6992ff8231166481c8ddf91f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57990 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:04:53,367 - lava_callback - INFO - Received callback for node 69ca04693c3aafdab4beaa6c INFO: 172.20.0.1:40574 - "POST /node/69ca04693c3aafdab4beaa6c HTTP/1.0" 202 Accepted 2026-04-01 05:04:53,372 - lava_callback - INFO - Starting async job submit for node 69ca04693c3aafdab4beaa6c 2026-04-01 05:04:53,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:04:53,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca04693c3aafdab4beaa6c HTTP/1.1" 200 None 2026-04-01 05:04:53,389 - lava_callback - DEBUG - Processing node 69ca04693c3aafdab4beaa6c: baseline-arm64-baylibre 2026-04-01 05:04:53,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca04693c3aafdab4beaa6c 2026-04-01 05:04:53,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:04:53,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca04693c3aafdab4beaa6c HTTP/1.1" 200 None 2026-04-01 05:04:53,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:04:53,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:04:53,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:04:53,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca04693c3aafdab4beaa6c HTTP/1.1" 200 None 2026-04-01 05:04:53,498 - lava_callback - ERROR - Error processing callback for node 69ca04693c3aafdab4beaa6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48378 - "GET /node/6930408b009d5f8c767ee6c8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48388 - "GET /node/694be882fc55ade6371eb03a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48804 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:06:53,036 - lava_callback - INFO - Received callback for node 69ca04e63c3aafdab4beaa79 INFO: 172.20.0.1:47034 - "POST /node/69ca04e63c3aafdab4beaa79 HTTP/1.0" 202 Accepted 2026-04-01 05:06:53,042 - lava_callback - INFO - Starting async job submit for node 69ca04e63c3aafdab4beaa79 2026-04-01 05:06:53,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:06:53,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca04e63c3aafdab4beaa79 HTTP/1.1" 200 None 2026-04-01 05:06:53,058 - lava_callback - DEBUG - Processing node 69ca04e63c3aafdab4beaa79: baseline-arm64-baylibre 2026-04-01 05:06:53,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca04e63c3aafdab4beaa79 2026-04-01 05:06:53,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:06:53,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca04e63c3aafdab4beaa79 HTTP/1.1" 200 None 2026-04-01 05:06:53,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:06:53,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:06:53,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:06:53,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca04e63c3aafdab4beaa79 HTTP/1.1" 200 None 2026-04-01 05:06:53,128 - lava_callback - ERROR - Error processing callback for node 69ca04e63c3aafdab4beaa79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47036 - "GET /node/69934d09523507f3806a5bc7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40038 - "GET /node/6985d61571989fb14149a48b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55824 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:08:53,822 - lava_callback - INFO - Received callback for node 69ca055c3c3aafdab4beaa88 INFO: 172.20.0.1:48880 - "POST /node/69ca055c3c3aafdab4beaa88 HTTP/1.0" 202 Accepted 2026-04-01 05:08:53,832 - lava_callback - INFO - Starting async job submit for node 69ca055c3c3aafdab4beaa88 2026-04-01 05:08:53,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:08:53,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca055c3c3aafdab4beaa88 HTTP/1.1" 200 None 2026-04-01 05:08:53,850 - lava_callback - DEBUG - Processing node 69ca055c3c3aafdab4beaa88: baseline-arm64-baylibre 2026-04-01 05:08:53,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca055c3c3aafdab4beaa88 2026-04-01 05:08:53,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:08:53,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca055c3c3aafdab4beaa88 HTTP/1.1" 200 None 2026-04-01 05:08:53,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:08:53,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:08:53,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:08:53,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca055c3c3aafdab4beaa88 HTTP/1.1" 200 None 2026-04-01 05:08:53,913 - lava_callback - ERROR - Error processing callback for node 69ca055c3c3aafdab4beaa88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57102 - "GET /node/69a6a0db3d518458fb33f095%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET /node/694ba672ae310edbb64664ee%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-01 05:10:13,292 - lava_callback - INFO - Received callback for node 69ca05aa3c3aafdab4beaa96 INFO: 172.20.0.1:34576 - "POST /node/69ca05aa3c3aafdab4beaa96 HTTP/1.0" 202 Accepted 2026-04-01 05:10:13,298 - lava_callback - INFO - Starting async job submit for node 69ca05aa3c3aafdab4beaa96 2026-04-01 05:10:13,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:10:13,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca05aa3c3aafdab4beaa96 HTTP/1.1" 200 None 2026-04-01 05:10:13,320 - lava_callback - DEBUG - Processing node 69ca05aa3c3aafdab4beaa96: baseline-arm64-baylibre 2026-04-01 05:10:13,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca05aa3c3aafdab4beaa96 2026-04-01 05:10:13,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:10:13,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca05aa3c3aafdab4beaa96 HTTP/1.1" 200 None 2026-04-01 05:10:13,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:10:13,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:10:13,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:10:13,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca05aa3c3aafdab4beaa96 HTTP/1.1" 200 None 2026-04-01 05:10:13,373 - lava_callback - ERROR - Error processing callback for node 69ca05aa3c3aafdab4beaa96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57444 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:11:38,823 - lava_callback - INFO - Received callback for node 69ca06003c3aafdab4beaaa1 INFO: 172.20.0.1:57450 - "POST /node/69ca06003c3aafdab4beaaa1 HTTP/1.0" 202 Accepted 2026-04-01 05:11:38,832 - lava_callback - INFO - Starting async job submit for node 69ca06003c3aafdab4beaaa1 2026-04-01 05:11:38,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:11:38,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca06003c3aafdab4beaaa1 HTTP/1.1" 200 None 2026-04-01 05:11:38,855 - lava_callback - DEBUG - Processing node 69ca06003c3aafdab4beaaa1: baseline-arm64-clabbe 2026-04-01 05:11:38,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca06003c3aafdab4beaaa1 2026-04-01 05:11:38,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:11:38,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca06003c3aafdab4beaaa1 HTTP/1.1" 200 None 2026-04-01 05:11:38,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:11:38,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:11:38,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:11:38,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca06003c3aafdab4beaaa1 HTTP/1.1" 200 None 2026-04-01 05:11:38,919 - lava_callback - ERROR - Error processing callback for node 69ca06003c3aafdab4beaaa1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48330 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:12:58,870 - lava_callback - INFO - Received callback for node 69ca064f3c3aafdab4beaab5 INFO: 172.20.0.1:48346 - "POST /node/69ca064f3c3aafdab4beaab5 HTTP/1.0" 202 Accepted 2026-04-01 05:12:58,876 - lava_callback - INFO - Starting async job submit for node 69ca064f3c3aafdab4beaab5 2026-04-01 05:12:58,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:12:58,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca064f3c3aafdab4beaab5 HTTP/1.1" 200 None 2026-04-01 05:12:58,895 - lava_callback - DEBUG - Processing node 69ca064f3c3aafdab4beaab5: baseline-arm64-clabbe 2026-04-01 05:12:58,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca064f3c3aafdab4beaab5 2026-04-01 05:12:58,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:12:58,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca064f3c3aafdab4beaab5 HTTP/1.1" 200 None 2026-04-01 05:12:58,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:12:58,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:12:58,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:12:58,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca064f3c3aafdab4beaab5 HTTP/1.1" 200 None 2026-04-01 05:12:58,954 - lava_callback - ERROR - Error processing callback for node 69ca064f3c3aafdab4beaab5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:14:13,508 - lava_callback - INFO - Received callback for node 69ca06933c3aafdab4beaac0 INFO: 172.20.0.1:59872 - "POST /node/69ca06933c3aafdab4beaac0 HTTP/1.0" 202 Accepted 2026-04-01 05:14:13,516 - lava_callback - INFO - Starting async job submit for node 69ca06933c3aafdab4beaac0 2026-04-01 05:14:13,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:14:13,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca06933c3aafdab4beaac0 HTTP/1.1" 200 None 2026-04-01 05:14:13,533 - lava_callback - DEBUG - Processing node 69ca06933c3aafdab4beaac0: baseline-arm64-baylibre 2026-04-01 05:14:13,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca06933c3aafdab4beaac0 2026-04-01 05:14:13,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:14:13,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca06933c3aafdab4beaac0 HTTP/1.1" 200 None 2026-04-01 05:14:13,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:14:13,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:14:13,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:14:13,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca06933c3aafdab4beaac0 HTTP/1.1" 200 None 2026-04-01 05:14:13,587 - lava_callback - ERROR - Error processing callback for node 69ca06933c3aafdab4beaac0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53722 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:15:39,015 - lava_callback - INFO - Received callback for node 69ca06e73c3aafdab4beaace INFO: 172.20.0.1:53736 - "POST /node/69ca06e73c3aafdab4beaace HTTP/1.0" 202 Accepted 2026-04-01 05:15:39,021 - lava_callback - INFO - Starting async job submit for node 69ca06e73c3aafdab4beaace 2026-04-01 05:15:39,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:15:39,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca06e73c3aafdab4beaace HTTP/1.1" 200 None 2026-04-01 05:15:39,037 - lava_callback - DEBUG - Processing node 69ca06e73c3aafdab4beaace: baseline-arm64-clabbe 2026-04-01 05:15:39,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca06e73c3aafdab4beaace 2026-04-01 05:15:39,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:15:39,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca06e73c3aafdab4beaace HTTP/1.1" 200 None 2026-04-01 05:15:39,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:15:39,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:15:39,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:15:39,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca06e73c3aafdab4beaace HTTP/1.1" 200 None 2026-04-01 05:15:39,088 - lava_callback - ERROR - Error processing callback for node 69ca06e73c3aafdab4beaace: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33788 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:16:39,075 - lava_callback - INFO - Received callback for node 69ca07343c3aafdab4beaae2 INFO: 172.20.0.1:33794 - "POST /node/69ca07343c3aafdab4beaae2 HTTP/1.0" 202 Accepted 2026-04-01 05:16:39,081 - lava_callback - INFO - Starting async job submit for node 69ca07343c3aafdab4beaae2 2026-04-01 05:16:39,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:16:39,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca07343c3aafdab4beaae2 HTTP/1.1" 200 None 2026-04-01 05:16:39,107 - lava_callback - DEBUG - Processing node 69ca07343c3aafdab4beaae2: baseline-arm64-clabbe 2026-04-01 05:16:39,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca07343c3aafdab4beaae2 2026-04-01 05:16:39,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:16:39,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca07343c3aafdab4beaae2 HTTP/1.1" 200 None 2026-04-01 05:16:39,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:16:39,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:16:39,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:16:39,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca07343c3aafdab4beaae2 HTTP/1.1" 200 None 2026-04-01 05:16:39,167 - lava_callback - ERROR - Error processing callback for node 69ca07343c3aafdab4beaae2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60940 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:17:59,106 - lava_callback - INFO - Received callback for node 69ca07803c3aafdab4beaaed INFO: 172.20.0.1:60950 - "POST /node/69ca07803c3aafdab4beaaed HTTP/1.0" 202 Accepted 2026-04-01 05:17:59,111 - lava_callback - INFO - Starting async job submit for node 69ca07803c3aafdab4beaaed 2026-04-01 05:17:59,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:17:59,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca07803c3aafdab4beaaed HTTP/1.1" 200 None 2026-04-01 05:17:59,132 - lava_callback - DEBUG - Processing node 69ca07803c3aafdab4beaaed: baseline-arm64-clabbe 2026-04-01 05:17:59,132 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca07803c3aafdab4beaaed 2026-04-01 05:17:59,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:17:59,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca07803c3aafdab4beaaed HTTP/1.1" 200 None 2026-04-01 05:17:59,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:17:59,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:17:59,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:17:59,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca07803c3aafdab4beaaed HTTP/1.1" 200 None 2026-04-01 05:17:59,196 - lava_callback - ERROR - Error processing callback for node 69ca07803c3aafdab4beaaed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37194 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:19:13,712 - lava_callback - INFO - Received callback for node 69ca07c63c3aafdab4beaafa INFO: 172.20.0.1:54684 - "POST /node/69ca07c63c3aafdab4beaafa HTTP/1.0" 202 Accepted 2026-04-01 05:19:13,718 - lava_callback - INFO - Starting async job submit for node 69ca07c63c3aafdab4beaafa 2026-04-01 05:19:13,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:19:13,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca07c63c3aafdab4beaafa HTTP/1.1" 200 None 2026-04-01 05:19:13,737 - lava_callback - DEBUG - Processing node 69ca07c63c3aafdab4beaafa: baseline-arm64-baylibre 2026-04-01 05:19:13,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca07c63c3aafdab4beaafa 2026-04-01 05:19:13,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:19:13,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca07c63c3aafdab4beaafa HTTP/1.1" 200 None 2026-04-01 05:19:13,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:19:13,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:19:13,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:19:13,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca07c63c3aafdab4beaafa HTTP/1.1" 200 None 2026-04-01 05:19:13,790 - lava_callback - ERROR - Error processing callback for node 69ca07c63c3aafdab4beaafa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56484 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:20:39,286 - lava_callback - INFO - Received callback for node 69ca08193c3aafdab4beab0a INFO: 172.20.0.1:56494 - "POST /node/69ca08193c3aafdab4beab0a HTTP/1.0" 202 Accepted 2026-04-01 05:20:39,292 - lava_callback - INFO - Starting async job submit for node 69ca08193c3aafdab4beab0a 2026-04-01 05:20:39,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:20:39,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca08193c3aafdab4beab0a HTTP/1.1" 200 None 2026-04-01 05:20:39,309 - lava_callback - DEBUG - Processing node 69ca08193c3aafdab4beab0a: baseline-arm64-clabbe 2026-04-01 05:20:39,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca08193c3aafdab4beab0a 2026-04-01 05:20:39,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:20:39,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca08193c3aafdab4beab0a HTTP/1.1" 200 None 2026-04-01 05:20:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:20:39,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-01 05:20:39,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:20:39,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca08193c3aafdab4beab0a HTTP/1.1" 200 None 2026-04-01 05:20:39,385 - lava_callback - ERROR - Error processing callback for node 69ca08193c3aafdab4beab0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:21:53,811 - lava_callback - INFO - Received callback for node 69ca08603c3aafdab4beab1a INFO: 172.20.0.1:44710 - "POST /node/69ca08603c3aafdab4beab1a HTTP/1.0" 202 Accepted 2026-04-01 05:21:53,816 - lava_callback - INFO - Starting async job submit for node 69ca08603c3aafdab4beab1a 2026-04-01 05:21:53,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:21:53,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca08603c3aafdab4beab1a HTTP/1.1" 200 None 2026-04-01 05:21:53,833 - lava_callback - DEBUG - Processing node 69ca08603c3aafdab4beab1a: baseline-arm64-baylibre 2026-04-01 05:21:53,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca08603c3aafdab4beab1a 2026-04-01 05:21:53,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:21:53,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca08603c3aafdab4beab1a HTTP/1.1" 200 None 2026-04-01 05:21:53,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:21:53,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:21:53,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:21:53,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca08603c3aafdab4beab1a HTTP/1.1" 200 None 2026-04-01 05:21:53,887 - lava_callback - ERROR - Error processing callback for node 69ca08603c3aafdab4beab1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45816 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:24:59,514 - lava_callback - INFO - Received callback for node 69ca09173c3aafdab4beab30 INFO: 172.20.0.1:45828 - "POST /node/69ca09173c3aafdab4beab30 HTTP/1.0" 202 Accepted 2026-04-01 05:24:59,520 - lava_callback - INFO - Starting async job submit for node 69ca09173c3aafdab4beab30 2026-04-01 05:24:59,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:24:59,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca09173c3aafdab4beab30 HTTP/1.1" 200 None 2026-04-01 05:24:59,535 - lava_callback - DEBUG - Processing node 69ca09173c3aafdab4beab30: baseline-arm64-clabbe 2026-04-01 05:24:59,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca09173c3aafdab4beab30 2026-04-01 05:24:59,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:24:59,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca09173c3aafdab4beab30 HTTP/1.1" 200 None 2026-04-01 05:24:59,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:24:59,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:24:59,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:24:59,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca09173c3aafdab4beab30 HTTP/1.1" 200 None 2026-04-01 05:24:59,595 - lava_callback - ERROR - Error processing callback for node 69ca09173c3aafdab4beab30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:26:19,625 - lava_callback - INFO - Received callback for node 69ca096b3c3aafdab4beab3d INFO: 172.20.0.1:53496 - "POST /node/69ca096b3c3aafdab4beab3d HTTP/1.0" 202 Accepted 2026-04-01 05:26:19,630 - lava_callback - INFO - Starting async job submit for node 69ca096b3c3aafdab4beab3d 2026-04-01 05:26:19,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:26:19,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca096b3c3aafdab4beab3d HTTP/1.1" 200 None 2026-04-01 05:26:19,647 - lava_callback - DEBUG - Processing node 69ca096b3c3aafdab4beab3d: baseline-arm64-clabbe 2026-04-01 05:26:19,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca096b3c3aafdab4beab3d 2026-04-01 05:26:19,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:26:19,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca096b3c3aafdab4beab3d HTTP/1.1" 200 None 2026-04-01 05:26:19,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:26:19,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:26:19,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:26:19,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca096b3c3aafdab4beab3d HTTP/1.1" 200 None 2026-04-01 05:26:19,709 - lava_callback - ERROR - Error processing callback for node 69ca096b3c3aafdab4beab3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43776 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:27:39,658 - lava_callback - INFO - Received callback for node 69ca09b73c3aafdab4beab4c INFO: 172.20.0.1:43786 - "POST /node/69ca09b73c3aafdab4beab4c HTTP/1.0" 202 Accepted 2026-04-01 05:27:39,666 - lava_callback - INFO - Starting async job submit for node 69ca09b73c3aafdab4beab4c 2026-04-01 05:27:39,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:27:39,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca09b73c3aafdab4beab4c HTTP/1.1" 200 None 2026-04-01 05:27:39,683 - lava_callback - DEBUG - Processing node 69ca09b73c3aafdab4beab4c: baseline-arm64-clabbe 2026-04-01 05:27:39,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca09b73c3aafdab4beab4c 2026-04-01 05:27:39,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:27:39,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca09b73c3aafdab4beab4c HTTP/1.1" 200 None 2026-04-01 05:27:39,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:27:39,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:27:39,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:27:39,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca09b73c3aafdab4beab4c HTTP/1.1" 200 None 2026-04-01 05:27:39,760 - lava_callback - ERROR - Error processing callback for node 69ca09b73c3aafdab4beab4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58526 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:29:59,775 - lava_callback - INFO - Received callback for node 69ca0a4c3c3aafdab4beab65 INFO: 172.20.0.1:51738 - "POST /node/69ca0a4c3c3aafdab4beab65 HTTP/1.0" 202 Accepted 2026-04-01 05:29:59,798 - lava_callback - INFO - Starting async job submit for node 69ca0a4c3c3aafdab4beab65 2026-04-01 05:29:59,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:29:59,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0a4c3c3aafdab4beab65 HTTP/1.1" 200 None 2026-04-01 05:29:59,839 - lava_callback - DEBUG - Processing node 69ca0a4c3c3aafdab4beab65: baseline-arm64-clabbe 2026-04-01 05:29:59,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0a4c3c3aafdab4beab65 2026-04-01 05:29:59,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:29:59,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0a4c3c3aafdab4beab65 HTTP/1.1" 200 None 2026-04-01 05:29:59,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:29:59,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:29:59,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:29:59,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0a4c3c3aafdab4beab65 HTTP/1.1" 200 None 2026-04-01 05:29:59,944 - lava_callback - ERROR - Error processing callback for node 69ca0a4c3c3aafdab4beab65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46272 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:31:14,528 - lava_callback - INFO - Received callback for node 69ca0a8f3c3aafdab4beab6d INFO: 172.20.0.1:57072 - "POST /node/69ca0a8f3c3aafdab4beab6d HTTP/1.0" 202 Accepted 2026-04-01 05:31:14,535 - lava_callback - INFO - Starting async job submit for node 69ca0a8f3c3aafdab4beab6d 2026-04-01 05:31:14,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:31:14,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0a8f3c3aafdab4beab6d HTTP/1.1" 200 None 2026-04-01 05:31:14,555 - lava_callback - DEBUG - Processing node 69ca0a8f3c3aafdab4beab6d: baseline-arm64-baylibre 2026-04-01 05:31:14,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0a8f3c3aafdab4beab6d 2026-04-01 05:31:14,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:31:14,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0a8f3c3aafdab4beab6d HTTP/1.1" 200 None 2026-04-01 05:31:14,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:31:14,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:31:14,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:31:14,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0a8f3c3aafdab4beab6d HTTP/1.1" 200 None 2026-04-01 05:31:14,623 - lava_callback - ERROR - Error processing callback for node 69ca0a8f3c3aafdab4beab6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46652 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:32:34,408 - lava_callback - INFO - Received callback for node 69ca0add3c3aafdab4beab7c INFO: 172.20.0.1:56520 - "POST /node/69ca0add3c3aafdab4beab7c HTTP/1.0" 202 Accepted 2026-04-01 05:32:34,417 - lava_callback - INFO - Starting async job submit for node 69ca0add3c3aafdab4beab7c 2026-04-01 05:32:34,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:32:34,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0add3c3aafdab4beab7c HTTP/1.1" 200 None 2026-04-01 05:32:34,437 - lava_callback - DEBUG - Processing node 69ca0add3c3aafdab4beab7c: baseline-arm64-baylibre 2026-04-01 05:32:34,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0add3c3aafdab4beab7c 2026-04-01 05:32:34,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:32:34,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0add3c3aafdab4beab7c HTTP/1.1" 200 None 2026-04-01 05:32:34,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:32:34,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:32:34,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:32:34,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0add3c3aafdab4beab7c HTTP/1.1" 200 None 2026-04-01 05:32:34,501 - lava_callback - ERROR - Error processing callback for node 69ca0add3c3aafdab4beab7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35622 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:33:59,933 - lava_callback - INFO - Received callback for node 69ca0b323c3aafdab4beab92 INFO: 172.20.0.1:40348 - "POST /node/69ca0b323c3aafdab4beab92 HTTP/1.0" 202 Accepted 2026-04-01 05:33:59,939 - lava_callback - INFO - Starting async job submit for node 69ca0b323c3aafdab4beab92 2026-04-01 05:33:59,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:33:59,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0b323c3aafdab4beab92 HTTP/1.1" 200 None 2026-04-01 05:33:59,956 - lava_callback - DEBUG - Processing node 69ca0b323c3aafdab4beab92: baseline-arm64-clabbe 2026-04-01 05:33:59,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0b323c3aafdab4beab92 2026-04-01 05:33:59,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:33:59,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0b323c3aafdab4beab92 HTTP/1.1" 200 None 2026-04-01 05:33:59,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:33:59,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:33:59,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:34:00,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0b323c3aafdab4beab92 HTTP/1.1" 200 None 2026-04-01 05:34:00,021 - lava_callback - ERROR - Error processing callback for node 69ca0b323c3aafdab4beab92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41802 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:34:54,556 - lava_callback - INFO - Received callback for node 69ca0b773c3aafdab4beab9a INFO: 172.20.0.1:52648 - "POST /node/69ca0b773c3aafdab4beab9a HTTP/1.0" 202 Accepted 2026-04-01 05:34:54,562 - lava_callback - INFO - Starting async job submit for node 69ca0b773c3aafdab4beab9a 2026-04-01 05:34:54,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:34:54,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0b773c3aafdab4beab9a HTTP/1.1" 200 None 2026-04-01 05:34:54,578 - lava_callback - DEBUG - Processing node 69ca0b773c3aafdab4beab9a: baseline-arm64-baylibre 2026-04-01 05:34:54,578 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0b773c3aafdab4beab9a 2026-04-01 05:34:54,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:34:54,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0b773c3aafdab4beab9a HTTP/1.1" 200 None 2026-04-01 05:34:54,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:34:54,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:34:54,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:34:54,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0b773c3aafdab4beab9a HTTP/1.1" 200 None 2026-04-01 05:34:54,650 - lava_callback - ERROR - Error processing callback for node 69ca0b773c3aafdab4beab9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52216 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:37:40,067 - lava_callback - INFO - Received callback for node 69ca0c143c3aafdab4beabb3 INFO: 172.20.0.1:46164 - "POST /node/69ca0c143c3aafdab4beabb3 HTTP/1.0" 202 Accepted 2026-04-01 05:37:40,073 - lava_callback - INFO - Starting async job submit for node 69ca0c143c3aafdab4beabb3 2026-04-01 05:37:40,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:37:40,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0c143c3aafdab4beabb3 HTTP/1.1" 200 None 2026-04-01 05:37:40,095 - lava_callback - DEBUG - Processing node 69ca0c143c3aafdab4beabb3: baseline-arm64-clabbe 2026-04-01 05:37:40,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0c143c3aafdab4beabb3 2026-04-01 05:37:40,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:37:40,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0c143c3aafdab4beabb3 HTTP/1.1" 200 None 2026-04-01 05:37:40,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:37:40,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:37:40,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:37:40,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0c143c3aafdab4beabb3 HTTP/1.1" 200 None 2026-04-01 05:37:40,176 - lava_callback - ERROR - Error processing callback for node 69ca0c143c3aafdab4beabb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34488 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:39:54,836 - lava_callback - INFO - Received callback for node 69ca0ca63c3aafdab4beabcb INFO: 172.20.0.1:44142 - "POST /node/69ca0ca63c3aafdab4beabcb HTTP/1.0" 202 Accepted 2026-04-01 05:39:54,844 - lava_callback - INFO - Starting async job submit for node 69ca0ca63c3aafdab4beabcb 2026-04-01 05:39:54,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:39:54,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0ca63c3aafdab4beabcb HTTP/1.1" 200 None 2026-04-01 05:39:54,861 - lava_callback - DEBUG - Processing node 69ca0ca63c3aafdab4beabcb: baseline-arm64-baylibre 2026-04-01 05:39:54,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0ca63c3aafdab4beabcb 2026-04-01 05:39:54,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:39:54,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0ca63c3aafdab4beabcb HTTP/1.1" 200 None 2026-04-01 05:39:54,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:39:54,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:39:54,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:39:54,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0ca63c3aafdab4beabcb HTTP/1.1" 200 None 2026-04-01 05:39:54,935 - lava_callback - ERROR - Error processing callback for node 69ca0ca63c3aafdab4beabcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58988 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:41:14,840 - lava_callback - INFO - Received callback for node 69ca0cf23c3aafdab4beabde INFO: 172.20.0.1:41902 - "POST /node/69ca0cf23c3aafdab4beabde HTTP/1.0" 202 Accepted 2026-04-01 05:41:14,845 - lava_callback - INFO - Starting async job submit for node 69ca0cf23c3aafdab4beabde 2026-04-01 05:41:14,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:41:14,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0cf23c3aafdab4beabde HTTP/1.1" 200 None 2026-04-01 05:41:14,861 - lava_callback - DEBUG - Processing node 69ca0cf23c3aafdab4beabde: baseline-arm64-baylibre 2026-04-01 05:41:14,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0cf23c3aafdab4beabde 2026-04-01 05:41:14,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:41:14,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0cf23c3aafdab4beabde HTTP/1.1" 200 None 2026-04-01 05:41:14,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:41:14,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:41:14,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:41:14,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0cf23c3aafdab4beabde HTTP/1.1" 200 None 2026-04-01 05:41:14,937 - lava_callback - ERROR - Error processing callback for node 69ca0cf23c3aafdab4beabde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32844 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:42:34,956 - lava_callback - INFO - Received callback for node 69ca0d3e3c3aafdab4beabe8 INFO: 172.20.0.1:55672 - "POST /node/69ca0d3e3c3aafdab4beabe8 HTTP/1.0" 202 Accepted 2026-04-01 05:42:34,961 - lava_callback - INFO - Starting async job submit for node 69ca0d3e3c3aafdab4beabe8 2026-04-01 05:42:34,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:42:35,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0d3e3c3aafdab4beabe8 HTTP/1.1" 200 None 2026-04-01 05:42:35,001 - lava_callback - DEBUG - Processing node 69ca0d3e3c3aafdab4beabe8: baseline-arm64-baylibre 2026-04-01 05:42:35,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0d3e3c3aafdab4beabe8 2026-04-01 05:42:35,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:42:35,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0d3e3c3aafdab4beabe8 HTTP/1.1" 200 None 2026-04-01 05:42:35,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:42:35,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:42:35,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:42:35,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0d3e3c3aafdab4beabe8 HTTP/1.1" 200 None 2026-04-01 05:42:35,052 - lava_callback - ERROR - Error processing callback for node 69ca0d3e3c3aafdab4beabe8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41008 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:43:55,032 - lava_callback - INFO - Received callback for node 69ca0d893c3aafdab4beabfa INFO: 172.20.0.1:50514 - "POST /node/69ca0d893c3aafdab4beabfa HTTP/1.0" 202 Accepted 2026-04-01 05:43:55,037 - lava_callback - INFO - Starting async job submit for node 69ca0d893c3aafdab4beabfa 2026-04-01 05:43:55,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:43:55,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0d893c3aafdab4beabfa HTTP/1.1" 200 None 2026-04-01 05:43:55,053 - lava_callback - DEBUG - Processing node 69ca0d893c3aafdab4beabfa: baseline-arm64-baylibre 2026-04-01 05:43:55,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0d893c3aafdab4beabfa 2026-04-01 05:43:55,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:43:55,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0d893c3aafdab4beabfa HTTP/1.1" 200 None 2026-04-01 05:43:55,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:43:55,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:43:55,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:43:55,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0d893c3aafdab4beabfa HTTP/1.1" 200 None 2026-04-01 05:43:55,126 - lava_callback - ERROR - Error processing callback for node 69ca0d893c3aafdab4beabfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42854 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:45:20,405 - lava_callback - INFO - Received callback for node 69ca0ddc3c3aafdab4beac0c INFO: 172.20.0.1:37876 - "POST /node/69ca0ddc3c3aafdab4beac0c HTTP/1.0" 202 Accepted 2026-04-01 05:45:20,411 - lava_callback - INFO - Starting async job submit for node 69ca0ddc3c3aafdab4beac0c 2026-04-01 05:45:20,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:45:20,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0ddc3c3aafdab4beac0c HTTP/1.1" 200 None 2026-04-01 05:45:20,426 - lava_callback - DEBUG - Processing node 69ca0ddc3c3aafdab4beac0c: baseline-arm64-clabbe 2026-04-01 05:45:20,426 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0ddc3c3aafdab4beac0c 2026-04-01 05:45:20,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:45:20,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0ddc3c3aafdab4beac0c HTTP/1.1" 200 None 2026-04-01 05:45:20,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:45:20,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:45:20,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:45:20,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0ddc3c3aafdab4beac0c HTTP/1.1" 200 None 2026-04-01 05:45:20,481 - lava_callback - ERROR - Error processing callback for node 69ca0ddc3c3aafdab4beac0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 05:46:15,160 - lava_callback - INFO - Received callback for node 69ca0e233c3aafdab4beac1d INFO: 172.20.0.1:38132 - "POST /node/69ca0e233c3aafdab4beac1d HTTP/1.0" 202 Accepted 2026-04-01 05:46:15,166 - lava_callback - INFO - Starting async job submit for node 69ca0e233c3aafdab4beac1d 2026-04-01 05:46:15,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:46:15,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0e233c3aafdab4beac1d HTTP/1.1" 200 None 2026-04-01 05:46:15,182 - lava_callback - DEBUG - Processing node 69ca0e233c3aafdab4beac1d: baseline-arm64-baylibre 2026-04-01 05:46:15,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0e233c3aafdab4beac1d 2026-04-01 05:46:15,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:46:15,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0e233c3aafdab4beac1d HTTP/1.1" 200 None 2026-04-01 05:46:15,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:46:15,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:46:15,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:46:15,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0e233c3aafdab4beac1d HTTP/1.1" 200 None 2026-04-01 05:46:15,243 - lava_callback - ERROR - Error processing callback for node 69ca0e233c3aafdab4beac1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:47:40,495 - lava_callback - INFO - Received callback for node 69ca0e763c3aafdab4beac29 INFO: 172.20.0.1:40120 - "POST /node/69ca0e763c3aafdab4beac29 HTTP/1.0" 202 Accepted 2026-04-01 05:47:40,501 - lava_callback - INFO - Starting async job submit for node 69ca0e763c3aafdab4beac29 2026-04-01 05:47:40,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:47:40,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0e763c3aafdab4beac29 HTTP/1.1" 200 None 2026-04-01 05:47:40,517 - lava_callback - DEBUG - Processing node 69ca0e763c3aafdab4beac29: baseline-arm64-clabbe 2026-04-01 05:47:40,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0e763c3aafdab4beac29 2026-04-01 05:47:40,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:47:40,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0e763c3aafdab4beac29 HTTP/1.1" 200 None 2026-04-01 05:47:40,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:47:40,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:47:40,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:47:40,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0e763c3aafdab4beac29 HTTP/1.1" 200 None 2026-04-01 05:47:40,588 - lava_callback - ERROR - Error processing callback for node 69ca0e763c3aafdab4beac29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:49:00,500 - lava_callback - INFO - Received callback for node 69ca0ec33c3aafdab4beac3e INFO: 172.20.0.1:54560 - "POST /node/69ca0ec33c3aafdab4beac3e HTTP/1.0" 202 Accepted 2026-04-01 05:49:00,506 - lava_callback - INFO - Starting async job submit for node 69ca0ec33c3aafdab4beac3e 2026-04-01 05:49:00,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:49:00,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0ec33c3aafdab4beac3e HTTP/1.1" 200 None 2026-04-01 05:49:00,522 - lava_callback - DEBUG - Processing node 69ca0ec33c3aafdab4beac3e: baseline-arm64-clabbe 2026-04-01 05:49:00,522 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0ec33c3aafdab4beac3e 2026-04-01 05:49:00,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:49:00,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0ec33c3aafdab4beac3e HTTP/1.1" 200 None 2026-04-01 05:49:00,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:49:00,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:49:00,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:49:00,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0ec33c3aafdab4beac3e HTTP/1.1" 200 None 2026-04-01 05:49:00,574 - lava_callback - ERROR - Error processing callback for node 69ca0ec33c3aafdab4beac3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53382 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:50:20,567 - lava_callback - INFO - Received callback for node 69ca0f103c3aafdab4beac57 INFO: 172.20.0.1:34534 - "POST /node/69ca0f103c3aafdab4beac57 HTTP/1.0" 202 Accepted 2026-04-01 05:50:20,573 - lava_callback - INFO - Starting async job submit for node 69ca0f103c3aafdab4beac57 2026-04-01 05:50:20,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:50:20,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0f103c3aafdab4beac57 HTTP/1.1" 200 None 2026-04-01 05:50:20,588 - lava_callback - DEBUG - Processing node 69ca0f103c3aafdab4beac57: baseline-arm64-clabbe 2026-04-01 05:50:20,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0f103c3aafdab4beac57 2026-04-01 05:50:20,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:50:20,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0f103c3aafdab4beac57 HTTP/1.1" 200 None 2026-04-01 05:50:20,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:50:20,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:50:20,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:50:20,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0f103c3aafdab4beac57 HTTP/1.1" 200 None 2026-04-01 05:50:20,651 - lava_callback - ERROR - Error processing callback for node 69ca0f103c3aafdab4beac57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:51:40,629 - lava_callback - INFO - Received callback for node 69ca0f5e3c3aafdab4beac60 INFO: 172.20.0.1:58952 - "POST /node/69ca0f5e3c3aafdab4beac60 HTTP/1.0" 202 Accepted 2026-04-01 05:51:40,638 - lava_callback - INFO - Starting async job submit for node 69ca0f5e3c3aafdab4beac60 2026-04-01 05:51:40,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:51:40,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0f5e3c3aafdab4beac60 HTTP/1.1" 200 None 2026-04-01 05:51:40,661 - lava_callback - DEBUG - Processing node 69ca0f5e3c3aafdab4beac60: baseline-arm64-clabbe 2026-04-01 05:51:40,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0f5e3c3aafdab4beac60 2026-04-01 05:51:40,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:51:40,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0f5e3c3aafdab4beac60 HTTP/1.1" 200 None 2026-04-01 05:51:40,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:51:40,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:51:40,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:51:40,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0f5e3c3aafdab4beac60 HTTP/1.1" 200 None 2026-04-01 05:51:40,733 - lava_callback - ERROR - Error processing callback for node 69ca0f5e3c3aafdab4beac60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52078 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:53:00,890 - lava_callback - INFO - Received callback for node 69ca0faa3c3aafdab4beac77 INFO: 172.20.0.1:40112 - "POST /node/69ca0faa3c3aafdab4beac77 HTTP/1.0" 202 Accepted 2026-04-01 05:53:00,896 - lava_callback - INFO - Starting async job submit for node 69ca0faa3c3aafdab4beac77 2026-04-01 05:53:00,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:00,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0faa3c3aafdab4beac77 HTTP/1.1" 200 None 2026-04-01 05:53:00,947 - lava_callback - DEBUG - Processing node 69ca0faa3c3aafdab4beac77: baseline-arm64-clabbe 2026-04-01 05:53:00,947 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0faa3c3aafdab4beac77 2026-04-01 05:53:00,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:00,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0faa3c3aafdab4beac77 HTTP/1.1" 200 None 2026-04-01 05:53:00,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:00,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:53:00,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:01,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0faa3c3aafdab4beac77 HTTP/1.1" 200 None 2026-04-01 05:53:01,030 - lava_callback - ERROR - Error processing callback for node 69ca0faa3c3aafdab4beac77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50878 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:53:55,583 - lava_callback - INFO - Received callback for node 69ca0fe33c3aafdab4beac80 INFO: 172.20.0.1:59950 - "POST /node/69ca0fe33c3aafdab4beac80 HTTP/1.0" 202 Accepted 2026-04-01 05:53:55,588 - lava_callback - INFO - Starting async job submit for node 69ca0fe33c3aafdab4beac80 2026-04-01 05:53:55,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:55,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0fe33c3aafdab4beac80 HTTP/1.1" 200 None 2026-04-01 05:53:55,604 - lava_callback - DEBUG - Processing node 69ca0fe33c3aafdab4beac80: baseline-arm-baylibre 2026-04-01 05:53:55,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca0fe33c3aafdab4beac80 2026-04-01 05:53:55,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:55,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca0fe33c3aafdab4beac80 HTTP/1.1" 200 None 2026-04-01 05:53:55,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:55,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-01 05:53:55,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:53:55,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca0fe33c3aafdab4beac80 HTTP/1.1" 200 None 2026-04-01 05:53:55,657 - lava_callback - ERROR - Error processing callback for node 69ca0fe33c3aafdab4beac80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47218 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:54:40,817 - lava_callback - INFO - Received callback for node 69ca101a3c3aafdab4beac8c INFO: 172.20.0.1:51108 - "POST /node/69ca101a3c3aafdab4beac8c HTTP/1.0" 202 Accepted 2026-04-01 05:54:40,822 - lava_callback - INFO - Starting async job submit for node 69ca101a3c3aafdab4beac8c 2026-04-01 05:54:40,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:54:40,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca101a3c3aafdab4beac8c HTTP/1.1" 200 None 2026-04-01 05:54:40,838 - lava_callback - DEBUG - Processing node 69ca101a3c3aafdab4beac8c: baseline-arm64-clabbe 2026-04-01 05:54:40,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca101a3c3aafdab4beac8c 2026-04-01 05:54:40,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:54:40,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca101a3c3aafdab4beac8c HTTP/1.1" 200 None 2026-04-01 05:54:40,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:54:40,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:54:40,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:54:40,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca101a3c3aafdab4beac8c HTTP/1.1" 200 None 2026-04-01 05:54:40,894 - lava_callback - ERROR - Error processing callback for node 69ca101a3c3aafdab4beac8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:55:35,669 - lava_callback - INFO - Received callback for node 69ca10553c3aafdab4beac98 INFO: 172.20.0.1:49380 - "POST /node/69ca10553c3aafdab4beac98 HTTP/1.0" 202 Accepted 2026-04-01 05:55:35,675 - lava_callback - INFO - Starting async job submit for node 69ca10553c3aafdab4beac98 2026-04-01 05:55:35,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:55:35,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca10553c3aafdab4beac98 HTTP/1.1" 200 None 2026-04-01 05:55:35,691 - lava_callback - DEBUG - Processing node 69ca10553c3aafdab4beac98: baseline-arm-baylibre 2026-04-01 05:55:35,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca10553c3aafdab4beac98 2026-04-01 05:55:35,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:55:35,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca10553c3aafdab4beac98 HTTP/1.1" 200 None 2026-04-01 05:55:35,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:55:35,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-01 05:55:35,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:55:35,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca10553c3aafdab4beac98 HTTP/1.1" 200 None 2026-04-01 05:55:35,750 - lava_callback - ERROR - Error processing callback for node 69ca10553c3aafdab4beac98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:56:40,892 - lava_callback - INFO - Received callback for node 69ca108a3c3aafdab4beaca7 INFO: 172.20.0.1:33172 - "POST /node/69ca108a3c3aafdab4beaca7 HTTP/1.0" 202 Accepted 2026-04-01 05:56:40,898 - lava_callback - INFO - Starting async job submit for node 69ca108a3c3aafdab4beaca7 2026-04-01 05:56:40,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:56:40,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca108a3c3aafdab4beaca7 HTTP/1.1" 200 None 2026-04-01 05:56:40,915 - lava_callback - DEBUG - Processing node 69ca108a3c3aafdab4beaca7: baseline-arm64-clabbe 2026-04-01 05:56:40,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca108a3c3aafdab4beaca7 2026-04-01 05:56:40,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:56:40,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca108a3c3aafdab4beaca7 HTTP/1.1" 200 None 2026-04-01 05:56:40,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:56:40,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:56:40,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:56:40,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca108a3c3aafdab4beaca7 HTTP/1.1" 200 None 2026-04-01 05:56:40,995 - lava_callback - ERROR - Error processing callback for node 69ca108a3c3aafdab4beaca7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60786 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:58:00,979 - lava_callback - INFO - Received callback for node 69ca10d63c3aafdab4beacc1 INFO: 172.20.0.1:59146 - "POST /node/69ca10d63c3aafdab4beacc1 HTTP/1.0" 202 Accepted 2026-04-01 05:58:00,985 - lava_callback - INFO - Starting async job submit for node 69ca10d63c3aafdab4beacc1 2026-04-01 05:58:00,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:58:01,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca10d63c3aafdab4beacc1 HTTP/1.1" 200 None 2026-04-01 05:58:01,004 - lava_callback - DEBUG - Processing node 69ca10d63c3aafdab4beacc1: baseline-arm64-clabbe 2026-04-01 05:58:01,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca10d63c3aafdab4beacc1 2026-04-01 05:58:01,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:58:01,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca10d63c3aafdab4beacc1 HTTP/1.1" 200 None 2026-04-01 05:58:01,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:58:01,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 05:58:01,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:58:01,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca10d63c3aafdab4beacc1 HTTP/1.1" 200 None 2026-04-01 05:58:01,057 - lava_callback - ERROR - Error processing callback for node 69ca10d63c3aafdab4beacc1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52966 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:59:01,023 - lava_callback - INFO - Received callback for node 69ca11143c3aafdab4beaccb INFO: 172.20.0.1:43968 - "POST /node/69ca11143c3aafdab4beaccb HTTP/1.0" 202 Accepted 2026-04-01 05:59:01,029 - lava_callback - INFO - Starting async job submit for node 69ca11143c3aafdab4beaccb 2026-04-01 05:59:01,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:01,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11143c3aafdab4beaccb HTTP/1.1" 200 None 2026-04-01 05:59:01,049 - lava_callback - DEBUG - Processing node 69ca11143c3aafdab4beaccb: baseline-arm-clabbe 2026-04-01 05:59:01,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca11143c3aafdab4beaccb 2026-04-01 05:59:01,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:01,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11143c3aafdab4beaccb HTTP/1.1" 200 None 2026-04-01 05:59:01,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:01,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-01 05:59:01,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:01,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca11143c3aafdab4beaccb HTTP/1.1" 200 None 2026-04-01 05:59:01,108 - lava_callback - ERROR - Error processing callback for node 69ca11143c3aafdab4beaccb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK 2026-04-01 05:59:40,976 - lava_callback - INFO - Received callback for node 69ca11393c3aafdab4beacd6 INFO: 172.20.0.1:42434 - "POST /node/69ca11393c3aafdab4beacd6 HTTP/1.0" 202 Accepted 2026-04-01 05:59:40,981 - lava_callback - INFO - Starting async job submit for node 69ca11393c3aafdab4beacd6 2026-04-01 05:59:40,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:40,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11393c3aafdab4beacd6 HTTP/1.1" 200 None 2026-04-01 05:59:40,997 - lava_callback - DEBUG - Processing node 69ca11393c3aafdab4beacd6: baseline-arm-clabbe 2026-04-01 05:59:40,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca11393c3aafdab4beacd6 2026-04-01 05:59:40,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:41,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11393c3aafdab4beacd6 HTTP/1.1" 200 None 2026-04-01 05:59:41,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:41,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-01 05:59:41,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 05:59:41,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca11393c3aafdab4beacd6 HTTP/1.1" 200 None 2026-04-01 05:59:41,049 - lava_callback - ERROR - Error processing callback for node 69ca11393c3aafdab4beacd6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32902 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:00:21,001 - lava_callback - INFO - Received callback for node 69ca11713c3aafdab4beace2 INFO: 172.20.0.1:33006 - "POST /node/69ca11713c3aafdab4beace2 HTTP/1.0" 202 Accepted 2026-04-01 06:00:21,008 - lava_callback - INFO - Starting async job submit for node 69ca11713c3aafdab4beace2 2026-04-01 06:00:21,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:00:21,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11713c3aafdab4beace2 HTTP/1.1" 200 None 2026-04-01 06:00:21,031 - lava_callback - DEBUG - Processing node 69ca11713c3aafdab4beace2: baseline-arm64-clabbe 2026-04-01 06:00:21,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca11713c3aafdab4beace2 2026-04-01 06:00:21,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:00:21,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11713c3aafdab4beace2 HTTP/1.1" 200 None 2026-04-01 06:00:21,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:00:21,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 06:00:21,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:00:21,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca11713c3aafdab4beace2 HTTP/1.1" 200 None 2026-04-01 06:00:21,114 - lava_callback - ERROR - Error processing callback for node 69ca11713c3aafdab4beace2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48870 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:01:41,124 - lava_callback - INFO - Received callback for node 69ca11af3c3aafdab4beacef INFO: 172.20.0.1:55788 - "POST /node/69ca11af3c3aafdab4beacef HTTP/1.0" 202 Accepted 2026-04-01 06:01:41,131 - lava_callback - INFO - Starting async job submit for node 69ca11af3c3aafdab4beacef 2026-04-01 06:01:41,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:01:41,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11af3c3aafdab4beacef HTTP/1.1" 200 None 2026-04-01 06:01:41,152 - lava_callback - DEBUG - Processing node 69ca11af3c3aafdab4beacef: baseline-arm-clabbe 2026-04-01 06:01:41,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca11af3c3aafdab4beacef 2026-04-01 06:01:41,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:01:41,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11af3c3aafdab4beacef HTTP/1.1" 200 None 2026-04-01 06:01:41,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:01:41,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-01 06:01:41,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:01:41,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca11af3c3aafdab4beacef HTTP/1.1" 200 None 2026-04-01 06:01:41,221 - lava_callback - ERROR - Error processing callback for node 69ca11af3c3aafdab4beacef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51048 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:02:21,127 - lava_callback - INFO - Received callback for node 69ca11e23c3aafdab4bead02 INFO: 172.20.0.1:55500 - "POST /node/69ca11e23c3aafdab4bead02 HTTP/1.0" 202 Accepted 2026-04-01 06:02:21,133 - lava_callback - INFO - Starting async job submit for node 69ca11e23c3aafdab4bead02 2026-04-01 06:02:21,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:02:21,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11e23c3aafdab4bead02 HTTP/1.1" 200 None 2026-04-01 06:02:21,156 - lava_callback - DEBUG - Processing node 69ca11e23c3aafdab4bead02: baseline-arm64-clabbe 2026-04-01 06:02:21,156 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca11e23c3aafdab4bead02 2026-04-01 06:02:21,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:02:21,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca11e23c3aafdab4bead02 HTTP/1.1" 200 None 2026-04-01 06:02:21,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:02:21,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 06:02:21,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:02:21,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca11e23c3aafdab4bead02 HTTP/1.1" 200 None 2026-04-01 06:02:21,229 - lava_callback - ERROR - Error processing callback for node 69ca11e23c3aafdab4bead02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47012 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:03:21,200 - lava_callback - INFO - Received callback for node 69ca121f3c3aafdab4bead09 INFO: 172.20.0.1:48122 - "POST /node/69ca121f3c3aafdab4bead09 HTTP/1.0" 202 Accepted 2026-04-01 06:03:21,209 - lava_callback - INFO - Starting async job submit for node 69ca121f3c3aafdab4bead09 2026-04-01 06:03:21,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:03:21,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca121f3c3aafdab4bead09 HTTP/1.1" 200 None 2026-04-01 06:03:21,224 - lava_callback - DEBUG - Processing node 69ca121f3c3aafdab4bead09: baseline-arm-clabbe 2026-04-01 06:03:21,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca121f3c3aafdab4bead09 2026-04-01 06:03:21,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:03:21,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca121f3c3aafdab4bead09 HTTP/1.1" 200 None 2026-04-01 06:03:21,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:03:21,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-01 06:03:21,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:03:21,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca121f3c3aafdab4bead09 HTTP/1.1" 200 None 2026-04-01 06:03:21,282 - lava_callback - ERROR - Error processing callback for node 69ca121f3c3aafdab4bead09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:04:01,213 - lava_callback - INFO - Received callback for node 69ca12443c3aafdab4bead10 INFO: 172.20.0.1:58078 - "POST /node/69ca12443c3aafdab4bead10 HTTP/1.0" 202 Accepted 2026-04-01 06:04:01,223 - lava_callback - INFO - Starting async job submit for node 69ca12443c3aafdab4bead10 2026-04-01 06:04:01,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:01,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca12443c3aafdab4bead10 HTTP/1.1" 200 None 2026-04-01 06:04:01,246 - lava_callback - DEBUG - Processing node 69ca12443c3aafdab4bead10: baseline-arm-clabbe 2026-04-01 06:04:01,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca12443c3aafdab4bead10 2026-04-01 06:04:01,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:01,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca12443c3aafdab4bead10 HTTP/1.1" 200 None 2026-04-01 06:04:01,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:01,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-01 06:04:01,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:01,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca12443c3aafdab4bead10 HTTP/1.1" 200 None 2026-04-01 06:04:01,314 - lava_callback - ERROR - Error processing callback for node 69ca12443c3aafdab4bead10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47042 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:04:23,815 - lava_callback - INFO - Received callback for node 69cbfcd68df9359ff4898445 INFO: 172.20.0.1:35584 - "POST /node/69cbfcd68df9359ff4898445 HTTP/1.0" 202 Accepted 2026-04-01 06:04:23,824 - lava_callback - INFO - Starting async job submit for node 69cbfcd68df9359ff4898445 2026-04-01 06:04:23,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:23,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcd68df9359ff4898445 HTTP/1.1" 200 None 2026-04-01 06:04:23,868 - lava_callback - DEBUG - Processing node 69cbfcd68df9359ff4898445: baseline-arm64-broonie 2026-04-01 06:04:24,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:04:24,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:04:24,318 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd68df9359ff4898445/log.txt.gz 2026-04-01 06:04:24,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:04:24,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:04:24,654 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd68df9359ff4898445/lava-logs.yaml 2026-04-01 06:04:24,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:04:24,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:04:24,833 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd68df9359ff4898445/lava_callback.json.gz 2026-04-01 06:04:24,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:24,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcd68df9359ff4898445 HTTP/1.1" 200 None 2026-04-01 06:04:24,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:24,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 06:04:24,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:24,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcd68df9359ff4898445 HTTP/1.1" 200 None 2026-04-01 06:04:24,916 - lava_callback - INFO - Completed processing callback for node 69cbfcd68df9359ff4898445 INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51232 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:04:41,308 - lava_callback - INFO - Received callback for node 69ca12763c3aafdab4bead21 INFO: 172.20.0.1:44378 - "POST /node/69ca12763c3aafdab4bead21 HTTP/1.0" 202 Accepted 2026-04-01 06:04:41,314 - lava_callback - INFO - Starting async job submit for node 69ca12763c3aafdab4bead21 2026-04-01 06:04:41,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:41,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca12763c3aafdab4bead21 HTTP/1.1" 200 None 2026-04-01 06:04:41,334 - lava_callback - DEBUG - Processing node 69ca12763c3aafdab4bead21: baseline-arm64-clabbe 2026-04-01 06:04:41,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca12763c3aafdab4bead21 2026-04-01 06:04:41,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:41,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca12763c3aafdab4bead21 HTTP/1.1" 200 None 2026-04-01 06:04:41,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:41,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-04-01 06:04:41,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:41,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca12763c3aafdab4bead21 HTTP/1.1" 200 None 2026-04-01 06:04:41,385 - lava_callback - ERROR - Error processing callback for node 69ca12763c3aafdab4bead21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-01 06:04:45,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:04:45,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43878 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:05:43,477 - lava_callback - INFO - Received callback for node 69cc59838df9359ff489a16a INFO: 172.20.0.1:51796 - "POST /node/69cc59838df9359ff489a16a HTTP/1.0" 202 Accepted 2026-04-01 06:05:43,484 - lava_callback - INFO - Starting async job submit for node 69cc59838df9359ff489a16a 2026-04-01 06:05:43,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:05:43,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc59838df9359ff489a16a HTTP/1.1" 200 None 2026-04-01 06:05:43,521 - lava_callback - DEBUG - Processing node 69cc59838df9359ff489a16a: baseline-arm64-broonie 2026-04-01 06:05:43,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:05:44,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:05:44,068 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc59838df9359ff489a16a/log.txt.gz 2026-04-01 06:05:44,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:05:44,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:05:44,412 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc59838df9359ff489a16a/lava-logs.yaml 2026-04-01 06:05:44,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:05:44,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:05:44,617 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc59838df9359ff489a16a/lava_callback.json.gz 2026-04-01 06:05:44,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:05:44,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc59838df9359ff489a16a HTTP/1.1" 200 None 2026-04-01 06:05:44,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:05:44,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 06:05:44,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:05:44,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc59838df9359ff489a16a HTTP/1.1" 200 None 2026-04-01 06:05:44,703 - lava_callback - INFO - Completed processing callback for node 69cc59838df9359ff489a16a 2026-04-01 06:05:45,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:05:46,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60070 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:07:03,587 - lava_callback - INFO - Received callback for node 69ccb0778cd9cbcdc945bdd7 INFO: 172.20.0.1:39860 - "POST /node/69ccb0778cd9cbcdc945bdd7 HTTP/1.0" 202 Accepted 2026-04-01 06:07:03,595 - lava_callback - INFO - Starting async job submit for node 69ccb0778cd9cbcdc945bdd7 2026-04-01 06:07:03,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:07:03,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb0778cd9cbcdc945bdd7 HTTP/1.1" 200 None 2026-04-01 06:07:03,631 - lava_callback - DEBUG - Processing node 69ccb0778cd9cbcdc945bdd7: baseline-arm64-broonie 2026-04-01 06:07:03,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:07:04,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:07:04,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ccb0778cd9cbcdc945bdd7/log.txt.gz 2026-04-01 06:07:04,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:07:04,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:07:04,433 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ccb0778cd9cbcdc945bdd7/lava-logs.yaml 2026-04-01 06:07:04,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:07:04,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:07:04,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ccb0778cd9cbcdc945bdd7/lava_callback.json.gz 2026-04-01 06:07:04,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:07:04,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb0778cd9cbcdc945bdd7 HTTP/1.1" 200 None 2026-04-01 06:07:04,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:07:04,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 06:07:04,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:07:04,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb0778cd9cbcdc945bdd7 HTTP/1.1" 200 None 2026-04-01 06:07:04,690 - lava_callback - INFO - Completed processing callback for node 69ccb0778cd9cbcdc945bdd7 INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39302 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 06:07:16,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:07:16,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57042 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:42:58,382 - lava_callback - INFO - Received callback for node 69ca1b6f3c3aafdab4bead9d INFO: 172.20.0.1:57046 - "POST /node/69ca1b6f3c3aafdab4bead9d HTTP/1.0" 202 Accepted 2026-04-01 06:42:58,387 - lava_callback - INFO - Starting async job submit for node 69ca1b6f3c3aafdab4bead9d 2026-04-01 06:42:58,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:42:58,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1b6f3c3aafdab4bead9d HTTP/1.1" 200 None 2026-04-01 06:42:58,403 - lava_callback - DEBUG - Processing node 69ca1b6f3c3aafdab4bead9d: baseline-arm-baylibre 2026-04-01 06:42:58,403 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1b6f3c3aafdab4bead9d 2026-04-01 06:42:58,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:42:58,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1b6f3c3aafdab4bead9d HTTP/1.1" 200 None 2026-04-01 06:42:58,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:42:58,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:42:58,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:42:58,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1b6f3c3aafdab4bead9d HTTP/1.1" 200 None 2026-04-01 06:42:58,461 - lava_callback - ERROR - Error processing callback for node 69ca1b6f3c3aafdab4bead9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33216 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:43:38,285 - lava_callback - INFO - Received callback for node 69ca1b943c3aafdab4beada6 INFO: 172.20.0.1:33222 - "POST /node/69ca1b943c3aafdab4beada6 HTTP/1.0" 202 Accepted 2026-04-01 06:43:38,292 - lava_callback - INFO - Starting async job submit for node 69ca1b943c3aafdab4beada6 2026-04-01 06:43:38,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:43:38,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1b943c3aafdab4beada6 HTTP/1.1" 200 None 2026-04-01 06:43:38,309 - lava_callback - DEBUG - Processing node 69ca1b943c3aafdab4beada6: baseline-arm-baylibre 2026-04-01 06:43:38,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1b943c3aafdab4beada6 2026-04-01 06:43:38,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:43:38,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1b943c3aafdab4beada6 HTTP/1.1" 200 None 2026-04-01 06:43:38,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:43:38,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:43:38,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:43:38,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1b943c3aafdab4beada6 HTTP/1.1" 200 None 2026-04-01 06:43:38,362 - lava_callback - ERROR - Error processing callback for node 69ca1b943c3aafdab4beada6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43994 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:44:18,489 - lava_callback - INFO - Received callback for node 69ca1bb93c3aafdab4beadb0 INFO: 172.20.0.1:44010 - "POST /node/69ca1bb93c3aafdab4beadb0 HTTP/1.0" 202 Accepted 2026-04-01 06:44:18,495 - lava_callback - INFO - Starting async job submit for node 69ca1bb93c3aafdab4beadb0 2026-04-01 06:44:18,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:18,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1bb93c3aafdab4beadb0 HTTP/1.1" 200 None 2026-04-01 06:44:18,511 - lava_callback - DEBUG - Processing node 69ca1bb93c3aafdab4beadb0: baseline-arm-baylibre 2026-04-01 06:44:18,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1bb93c3aafdab4beadb0 2026-04-01 06:44:18,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:18,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1bb93c3aafdab4beadb0 HTTP/1.1" 200 None 2026-04-01 06:44:18,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:18,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:44:18,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:18,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1bb93c3aafdab4beadb0 HTTP/1.1" 200 None 2026-04-01 06:44:18,566 - lava_callback - ERROR - Error processing callback for node 69ca1bb93c3aafdab4beadb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57706 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:44:58,392 - lava_callback - INFO - Received callback for node 69ca1bdd3c3aafdab4beadb8 INFO: 172.20.0.1:57720 - "POST /node/69ca1bdd3c3aafdab4beadb8 HTTP/1.0" 202 Accepted 2026-04-01 06:44:58,397 - lava_callback - INFO - Starting async job submit for node 69ca1bdd3c3aafdab4beadb8 2026-04-01 06:44:58,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:58,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1bdd3c3aafdab4beadb8 HTTP/1.1" 200 None 2026-04-01 06:44:58,413 - lava_callback - DEBUG - Processing node 69ca1bdd3c3aafdab4beadb8: baseline-arm-baylibre 2026-04-01 06:44:58,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1bdd3c3aafdab4beadb8 2026-04-01 06:44:58,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:58,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1bdd3c3aafdab4beadb8 HTTP/1.1" 200 None 2026-04-01 06:44:58,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:58,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:44:58,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:44:58,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1bdd3c3aafdab4beadb8 HTTP/1.1" 200 None 2026-04-01 06:44:58,465 - lava_callback - ERROR - Error processing callback for node 69ca1bdd3c3aafdab4beadb8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:45:38,473 - lava_callback - INFO - Received callback for node 69ca1c023c3aafdab4beadc1 INFO: 172.20.0.1:37760 - "POST /node/69ca1c023c3aafdab4beadc1 HTTP/1.0" 202 Accepted 2026-04-01 06:45:38,478 - lava_callback - INFO - Starting async job submit for node 69ca1c023c3aafdab4beadc1 2026-04-01 06:45:38,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:45:38,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1c023c3aafdab4beadc1 HTTP/1.1" 200 None 2026-04-01 06:45:38,515 - lava_callback - DEBUG - Processing node 69ca1c023c3aafdab4beadc1: baseline-arm-baylibre 2026-04-01 06:45:38,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1c023c3aafdab4beadc1 2026-04-01 06:45:38,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:45:38,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1c023c3aafdab4beadc1 HTTP/1.1" 200 None 2026-04-01 06:45:38,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:45:38,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:45:38,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:45:38,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1c023c3aafdab4beadc1 HTTP/1.1" 200 None 2026-04-01 06:45:38,589 - lava_callback - ERROR - Error processing callback for node 69ca1c023c3aafdab4beadc1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:47:23,197 - lava_callback - INFO - Received callback for node 69ca1c6f3c3aafdab4beade2 INFO: 172.20.0.1:53052 - "POST /node/69ca1c6f3c3aafdab4beade2 HTTP/1.0" 202 Accepted 2026-04-01 06:47:23,203 - lava_callback - INFO - Starting async job submit for node 69ca1c6f3c3aafdab4beade2 2026-04-01 06:47:23,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:47:23,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1c6f3c3aafdab4beade2 HTTP/1.1" 200 None 2026-04-01 06:47:23,223 - lava_callback - DEBUG - Processing node 69ca1c6f3c3aafdab4beade2: baseline-arm-clabbe 2026-04-01 06:47:23,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1c6f3c3aafdab4beade2 2026-04-01 06:47:23,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:47:23,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1c6f3c3aafdab4beade2 HTTP/1.1" 200 None 2026-04-01 06:47:23,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:47:23,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:47:23,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:47:23,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1c6f3c3aafdab4beade2 HTTP/1.1" 200 None 2026-04-01 06:47:23,288 - lava_callback - ERROR - Error processing callback for node 69ca1c6f3c3aafdab4beade2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57546 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:48:03,142 - lava_callback - INFO - Received callback for node 69ca1c943c3aafdab4beade9 INFO: 172.20.0.1:54116 - "POST /node/69ca1c943c3aafdab4beade9 HTTP/1.0" 202 Accepted 2026-04-01 06:48:03,147 - lava_callback - INFO - Starting async job submit for node 69ca1c943c3aafdab4beade9 2026-04-01 06:48:03,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:03,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1c943c3aafdab4beade9 HTTP/1.1" 200 None 2026-04-01 06:48:03,164 - lava_callback - DEBUG - Processing node 69ca1c943c3aafdab4beade9: baseline-arm-clabbe 2026-04-01 06:48:03,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1c943c3aafdab4beade9 2026-04-01 06:48:03,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:03,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1c943c3aafdab4beade9 HTTP/1.1" 200 None 2026-04-01 06:48:03,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:03,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:48:03,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:03,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1c943c3aafdab4beade9 HTTP/1.1" 200 None 2026-04-01 06:48:03,223 - lava_callback - ERROR - Error processing callback for node 69ca1c943c3aafdab4beade9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:48:43,165 - lava_callback - INFO - Received callback for node 69ca1cb83c3aafdab4beadf2 INFO: 172.20.0.1:53532 - "POST /node/69ca1cb83c3aafdab4beadf2 HTTP/1.0" 202 Accepted 2026-04-01 06:48:43,171 - lava_callback - INFO - Starting async job submit for node 69ca1cb83c3aafdab4beadf2 2026-04-01 06:48:43,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:43,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1cb83c3aafdab4beadf2 HTTP/1.1" 200 None 2026-04-01 06:48:43,195 - lava_callback - DEBUG - Processing node 69ca1cb83c3aafdab4beadf2: baseline-arm-clabbe 2026-04-01 06:48:43,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1cb83c3aafdab4beadf2 2026-04-01 06:48:43,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:43,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1cb83c3aafdab4beadf2 HTTP/1.1" 200 None 2026-04-01 06:48:43,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:43,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:48:43,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:48:43,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1cb83c3aafdab4beadf2 HTTP/1.1" 200 None 2026-04-01 06:48:43,252 - lava_callback - ERROR - Error processing callback for node 69ca1cb83c3aafdab4beadf2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50754 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:49:23,179 - lava_callback - INFO - Received callback for node 69ca1cde3c3aafdab4beadfd INFO: 172.20.0.1:59054 - "POST /node/69ca1cde3c3aafdab4beadfd HTTP/1.0" 202 Accepted 2026-04-01 06:49:23,186 - lava_callback - INFO - Starting async job submit for node 69ca1cde3c3aafdab4beadfd 2026-04-01 06:49:23,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:23,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1cde3c3aafdab4beadfd HTTP/1.1" 200 None 2026-04-01 06:49:23,202 - lava_callback - DEBUG - Processing node 69ca1cde3c3aafdab4beadfd: baseline-arm-clabbe 2026-04-01 06:49:23,202 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1cde3c3aafdab4beadfd 2026-04-01 06:49:23,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:23,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1cde3c3aafdab4beadfd HTTP/1.1" 200 None 2026-04-01 06:49:23,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:23,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:49:23,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:23,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1cde3c3aafdab4beadfd HTTP/1.1" 200 None 2026-04-01 06:49:23,283 - lava_callback - ERROR - Error processing callback for node 69ca1cde3c3aafdab4beadfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41852 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:49:43,218 - lava_callback - INFO - Received callback for node 69ca1d023c3aafdab4beae08 INFO: 172.20.0.1:34136 - "POST /node/69ca1d023c3aafdab4beae08 HTTP/1.0" 202 Accepted 2026-04-01 06:49:43,224 - lava_callback - INFO - Starting async job submit for node 69ca1d023c3aafdab4beae08 2026-04-01 06:49:43,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:43,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d023c3aafdab4beae08 HTTP/1.1" 200 None 2026-04-01 06:49:43,240 - lava_callback - DEBUG - Processing node 69ca1d023c3aafdab4beae08: baseline-arm-clabbe 2026-04-01 06:49:43,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1d023c3aafdab4beae08 2026-04-01 06:49:43,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:43,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d023c3aafdab4beae08 HTTP/1.1" 200 None 2026-04-01 06:49:43,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:43,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-01 06:49:43,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:49:43,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1d023c3aafdab4beae08 HTTP/1.1" 200 None 2026-04-01 06:49:43,296 - lava_callback - ERROR - Error processing callback for node 69ca1d023c3aafdab4beae08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48018 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:50:18,792 - lava_callback - INFO - Received callback for node 69ca1d233c3aafdab4beae0f INFO: 172.20.0.1:48020 - "POST /node/69ca1d233c3aafdab4beae0f HTTP/1.0" 202 Accepted 2026-04-01 06:50:18,798 - lava_callback - INFO - Starting async job submit for node 69ca1d233c3aafdab4beae0f 2026-04-01 06:50:18,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:50:18,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d233c3aafdab4beae0f HTTP/1.1" 200 None 2026-04-01 06:50:18,825 - lava_callback - DEBUG - Processing node 69ca1d233c3aafdab4beae0f: baseline-arm-baylibre 2026-04-01 06:50:18,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1d233c3aafdab4beae0f 2026-04-01 06:50:18,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:50:18,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d233c3aafdab4beae0f HTTP/1.1" 200 None 2026-04-01 06:50:18,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:50:18,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 06:50:18,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:50:18,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1d233c3aafdab4beae0f HTTP/1.1" 200 None 2026-04-01 06:50:18,881 - lava_callback - ERROR - Error processing callback for node 69ca1d233c3aafdab4beae0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44882 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:51:09,115 - lava_callback - INFO - Received callback for node 69cbfcda8df9359ff489844b INFO: 172.20.0.1:44894 - "POST /node/69cbfcda8df9359ff489844b HTTP/1.0" 202 Accepted 2026-04-01 06:51:09,121 - lava_callback - INFO - Starting async job submit for node 69cbfcda8df9359ff489844b 2026-04-01 06:51:09,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:09,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcda8df9359ff489844b HTTP/1.1" 200 None 2026-04-01 06:51:09,162 - lava_callback - DEBUG - Processing node 69cbfcda8df9359ff489844b: baseline-arm64-broonie 2026-04-01 06:51:09,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:51:09,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:51:09,621 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcda8df9359ff489844b/log.txt.gz 2026-04-01 06:51:09,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:51:10,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:51:10,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcda8df9359ff489844b/lava-logs.yaml 2026-04-01 06:51:10,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:51:10,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:51:10,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcda8df9359ff489844b/lava_callback.json.gz 2026-04-01 06:51:10,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:10,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcda8df9359ff489844b HTTP/1.1" 200 None 2026-04-01 06:51:10,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:10,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 06:51:10,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:10,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcda8df9359ff489844b HTTP/1.1" 200 None 2026-04-01 06:51:10,268 - lava_callback - INFO - Completed processing callback for node 69cbfcda8df9359ff489844b INFO: 172.20.0.1:44414 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 06:51:16,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:16,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53578 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:51:38,784 - lava_callback - INFO - Received callback for node 69ca1d6e3c3aafdab4beae21 INFO: 172.20.0.1:53582 - "POST /node/69ca1d6e3c3aafdab4beae21 HTTP/1.0" 202 Accepted 2026-04-01 06:51:38,790 - lava_callback - INFO - Starting async job submit for node 69ca1d6e3c3aafdab4beae21 2026-04-01 06:51:38,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:38,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d6e3c3aafdab4beae21 HTTP/1.1" 200 None 2026-04-01 06:51:38,806 - lava_callback - DEBUG - Processing node 69ca1d6e3c3aafdab4beae21: baseline-arm-baylibre 2026-04-01 06:51:38,806 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1d6e3c3aafdab4beae21 2026-04-01 06:51:38,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:38,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d6e3c3aafdab4beae21 HTTP/1.1" 200 None 2026-04-01 06:51:38,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:38,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 06:51:38,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:51:38,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1d6e3c3aafdab4beae21 HTTP/1.1" 200 None 2026-04-01 06:51:38,864 - lava_callback - ERROR - Error processing callback for node 69ca1d6e3c3aafdab4beae21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47422 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:52:18,803 - lava_callback - INFO - Received callback for node 69ca1d963c3aafdab4beae2e INFO: 172.20.0.1:47432 - "POST /node/69ca1d963c3aafdab4beae2e HTTP/1.0" 202 Accepted 2026-04-01 06:52:18,809 - lava_callback - INFO - Starting async job submit for node 69ca1d963c3aafdab4beae2e 2026-04-01 06:52:18,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:18,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d963c3aafdab4beae2e HTTP/1.1" 200 None 2026-04-01 06:52:18,825 - lava_callback - DEBUG - Processing node 69ca1d963c3aafdab4beae2e: baseline-arm-baylibre 2026-04-01 06:52:18,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1d963c3aafdab4beae2e 2026-04-01 06:52:18,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:18,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1d963c3aafdab4beae2e HTTP/1.1" 200 None 2026-04-01 06:52:18,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:18,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 06:52:18,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:18,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1d963c3aafdab4beae2e HTTP/1.1" 200 None 2026-04-01 06:52:18,890 - lava_callback - ERROR - Error processing callback for node 69ca1d963c3aafdab4beae2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:52:49,187 - lava_callback - INFO - Received callback for node 69cc5a2c8df9359ff489a1c3 INFO: 172.20.0.1:56284 - "POST /node/69cc5a2c8df9359ff489a1c3 HTTP/1.0" 202 Accepted 2026-04-01 06:52:49,193 - lava_callback - INFO - Starting async job submit for node 69cc5a2c8df9359ff489a1c3 2026-04-01 06:52:49,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:49,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc5a2c8df9359ff489a1c3 HTTP/1.1" 200 None 2026-04-01 06:52:49,242 - lava_callback - DEBUG - Processing node 69cc5a2c8df9359ff489a1c3: baseline-arm64-broonie 2026-04-01 06:52:49,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:52:49,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:52:49,706 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc5a2c8df9359ff489a1c3/log.txt.gz 2026-04-01 06:52:49,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:52:50,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:52:50,212 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc5a2c8df9359ff489a1c3/lava-logs.yaml 2026-04-01 06:52:50,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:52:50,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:52:50,380 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cc5a2c8df9359ff489a1c3/lava_callback.json.gz 2026-04-01 06:52:50,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:50,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc5a2c8df9359ff489a1c3 HTTP/1.1" 200 None 2026-04-01 06:52:50,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:50,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 06:52:50,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:52:50,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc5a2c8df9359ff489a1c3 HTTP/1.1" 200 None 2026-04-01 06:52:50,468 - lava_callback - INFO - Completed processing callback for node 69cc5a2c8df9359ff489a1c3 INFO: 172.20.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34130 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-01 06:53:16,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:53:16,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42720 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:54:29,268 - lava_callback - INFO - Received callback for node 69ccb0d48cd9cbcdc945bdfc INFO: 172.20.0.1:42724 - "POST /node/69ccb0d48cd9cbcdc945bdfc HTTP/1.0" 202 Accepted 2026-04-01 06:54:29,274 - lava_callback - INFO - Starting async job submit for node 69ccb0d48cd9cbcdc945bdfc 2026-04-01 06:54:29,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:54:29,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb0d48cd9cbcdc945bdfc HTTP/1.1" 200 None 2026-04-01 06:54:29,312 - lava_callback - DEBUG - Processing node 69ccb0d48cd9cbcdc945bdfc: baseline-arm64-broonie 2026-04-01 06:54:29,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:54:29,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:54:29,689 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ccb0d48cd9cbcdc945bdfc/log.txt.gz 2026-04-01 06:54:29,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:54:30,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:54:30,053 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ccb0d48cd9cbcdc945bdfc/lava-logs.yaml 2026-04-01 06:54:30,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-01 06:54:30,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-01 06:54:30,210 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ccb0d48cd9cbcdc945bdfc/lava_callback.json.gz 2026-04-01 06:54:30,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:54:30,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb0d48cd9cbcdc945bdfc HTTP/1.1" 200 None 2026-04-01 06:54:30,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:54:30,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-01 06:54:30,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:54:30,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb0d48cd9cbcdc945bdfc HTTP/1.1" 200 None 2026-04-01 06:54:30,293 - lava_callback - INFO - Completed processing callback for node 69ccb0d48cd9cbcdc945bdfc INFO: 172.20.0.1:42598 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:54:46,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:54:46,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:56:03,478 - lava_callback - INFO - Received callback for node 69ca1e753c3aafdab4beae3a INFO: 172.20.0.1:52832 - "POST /node/69ca1e753c3aafdab4beae3a HTTP/1.0" 202 Accepted 2026-04-01 06:56:03,484 - lava_callback - INFO - Starting async job submit for node 69ca1e753c3aafdab4beae3a 2026-04-01 06:56:03,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:56:03,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1e753c3aafdab4beae3a HTTP/1.1" 200 None 2026-04-01 06:56:03,503 - lava_callback - DEBUG - Processing node 69ca1e753c3aafdab4beae3a: baseline-arm-clabbe 2026-04-01 06:56:03,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1e753c3aafdab4beae3a 2026-04-01 06:56:03,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:56:03,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1e753c3aafdab4beae3a HTTP/1.1" 200 None 2026-04-01 06:56:03,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:56:03,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 06:56:03,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:56:03,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1e753c3aafdab4beae3a HTTP/1.1" 200 None 2026-04-01 06:56:03,561 - lava_callback - ERROR - Error processing callback for node 69ca1e753c3aafdab4beae3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34248 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:57:23,547 - lava_callback - INFO - Received callback for node 69ca1ec53c3aafdab4beae48 INFO: 172.20.0.1:50218 - "POST /node/69ca1ec53c3aafdab4beae48 HTTP/1.0" 202 Accepted 2026-04-01 06:57:23,552 - lava_callback - INFO - Starting async job submit for node 69ca1ec53c3aafdab4beae48 2026-04-01 06:57:23,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:23,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1ec53c3aafdab4beae48 HTTP/1.1" 200 None 2026-04-01 06:57:23,576 - lava_callback - DEBUG - Processing node 69ca1ec53c3aafdab4beae48: baseline-arm-clabbe 2026-04-01 06:57:23,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1ec53c3aafdab4beae48 2026-04-01 06:57:23,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:23,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1ec53c3aafdab4beae48 HTTP/1.1" 200 None 2026-04-01 06:57:23,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:23,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 06:57:23,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:23,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1ec53c3aafdab4beae48 HTTP/1.1" 200 None 2026-04-01 06:57:23,634 - lava_callback - ERROR - Error processing callback for node 69ca1ec53c3aafdab4beae48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44720 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:57:59,142 - lava_callback - INFO - Received callback for node 69ca1ee63c3aafdab4beae53 INFO: 172.20.0.1:44726 - "POST /node/69ca1ee63c3aafdab4beae53 HTTP/1.0" 202 Accepted 2026-04-01 06:57:59,148 - lava_callback - INFO - Starting async job submit for node 69ca1ee63c3aafdab4beae53 2026-04-01 06:57:59,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:59,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1ee63c3aafdab4beae53 HTTP/1.1" 200 None 2026-04-01 06:57:59,165 - lava_callback - DEBUG - Processing node 69ca1ee63c3aafdab4beae53: baseline-arm-baylibre 2026-04-01 06:57:59,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1ee63c3aafdab4beae53 2026-04-01 06:57:59,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:59,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1ee63c3aafdab4beae53 HTTP/1.1" 200 None 2026-04-01 06:57:59,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:59,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 06:57:59,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:57:59,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1ee63c3aafdab4beae53 HTTP/1.1" 200 None 2026-04-01 06:57:59,223 - lava_callback - ERROR - Error processing callback for node 69ca1ee63c3aafdab4beae53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:58:43,537 - lava_callback - INFO - Received callback for node 69ca1f0f3c3aafdab4beae5f INFO: 172.20.0.1:42346 - "POST /node/69ca1f0f3c3aafdab4beae5f HTTP/1.0" 202 Accepted 2026-04-01 06:58:43,543 - lava_callback - INFO - Starting async job submit for node 69ca1f0f3c3aafdab4beae5f 2026-04-01 06:58:43,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:58:43,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f0f3c3aafdab4beae5f HTTP/1.1" 200 None 2026-04-01 06:58:43,562 - lava_callback - DEBUG - Processing node 69ca1f0f3c3aafdab4beae5f: baseline-arm-clabbe 2026-04-01 06:58:43,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1f0f3c3aafdab4beae5f 2026-04-01 06:58:43,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:58:43,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f0f3c3aafdab4beae5f HTTP/1.1" 200 None 2026-04-01 06:58:43,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:58:43,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 06:58:43,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:58:43,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1f0f3c3aafdab4beae5f HTTP/1.1" 200 None 2026-04-01 06:58:43,613 - lava_callback - ERROR - Error processing callback for node 69ca1f0f3c3aafdab4beae5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54488 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:59:03,518 - lava_callback - INFO - Received callback for node 69ca1f343c3aafdab4beae6e INFO: 172.20.0.1:53516 - "POST /node/69ca1f343c3aafdab4beae6e HTTP/1.0" 202 Accepted 2026-04-01 06:59:03,524 - lava_callback - INFO - Starting async job submit for node 69ca1f343c3aafdab4beae6e 2026-04-01 06:59:03,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:03,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f343c3aafdab4beae6e HTTP/1.1" 200 None 2026-04-01 06:59:03,540 - lava_callback - DEBUG - Processing node 69ca1f343c3aafdab4beae6e: baseline-arm-clabbe 2026-04-01 06:59:03,541 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1f343c3aafdab4beae6e 2026-04-01 06:59:03,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:03,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f343c3aafdab4beae6e HTTP/1.1" 200 None 2026-04-01 06:59:03,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:03,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 06:59:03,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:03,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1f343c3aafdab4beae6e HTTP/1.1" 200 None 2026-04-01 06:59:03,592 - lava_callback - ERROR - Error processing callback for node 69ca1f343c3aafdab4beae6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57768 - "GET / HTTP/1.0" 200 OK 2026-04-01 06:59:39,196 - lava_callback - INFO - Received callback for node 69ca1f533c3aafdab4beae7b INFO: 172.20.0.1:57772 - "POST /node/69ca1f533c3aafdab4beae7b HTTP/1.0" 202 Accepted 2026-04-01 06:59:39,202 - lava_callback - INFO - Starting async job submit for node 69ca1f533c3aafdab4beae7b 2026-04-01 06:59:39,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:39,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f533c3aafdab4beae7b HTTP/1.1" 200 None 2026-04-01 06:59:39,218 - lava_callback - DEBUG - Processing node 69ca1f533c3aafdab4beae7b: baseline-arm-baylibre 2026-04-01 06:59:39,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1f533c3aafdab4beae7b 2026-04-01 06:59:39,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:39,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f533c3aafdab4beae7b HTTP/1.1" 200 None 2026-04-01 06:59:39,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:39,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 06:59:39,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 06:59:39,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1f533c3aafdab4beae7b HTTP/1.1" 200 None 2026-04-01 06:59:39,276 - lava_callback - ERROR - Error processing callback for node 69ca1f533c3aafdab4beae7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50520 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:00:19,208 - lava_callback - INFO - Received callback for node 69ca1f783c3aafdab4beae88 INFO: 172.20.0.1:50532 - "POST /node/69ca1f783c3aafdab4beae88 HTTP/1.0" 202 Accepted 2026-04-01 07:00:19,214 - lava_callback - INFO - Starting async job submit for node 69ca1f783c3aafdab4beae88 2026-04-01 07:00:19,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:00:19,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f783c3aafdab4beae88 HTTP/1.1" 200 None 2026-04-01 07:00:19,230 - lava_callback - DEBUG - Processing node 69ca1f783c3aafdab4beae88: baseline-arm-baylibre 2026-04-01 07:00:19,230 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1f783c3aafdab4beae88 2026-04-01 07:00:19,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:00:19,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1f783c3aafdab4beae88 HTTP/1.1" 200 None 2026-04-01 07:00:19,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:00:19,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:00:19,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:00:19,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1f783c3aafdab4beae88 HTTP/1.1" 200 None 2026-04-01 07:00:19,299 - lava_callback - ERROR - Error processing callback for node 69ca1f783c3aafdab4beae88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43538 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:01:03,675 - lava_callback - INFO - Received callback for node 69ca1fa23c3aafdab4beae96 INFO: 172.20.0.1:48834 - "POST /node/69ca1fa23c3aafdab4beae96 HTTP/1.0" 202 Accepted 2026-04-01 07:01:03,681 - lava_callback - INFO - Starting async job submit for node 69ca1fa23c3aafdab4beae96 2026-04-01 07:01:03,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:03,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1fa23c3aafdab4beae96 HTTP/1.1" 200 None 2026-04-01 07:01:03,697 - lava_callback - DEBUG - Processing node 69ca1fa23c3aafdab4beae96: baseline-arm-clabbe 2026-04-01 07:01:03,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1fa23c3aafdab4beae96 2026-04-01 07:01:03,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:03,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1fa23c3aafdab4beae96 HTTP/1.1" 200 None 2026-04-01 07:01:03,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:03,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:01:03,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:03,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1fa23c3aafdab4beae96 HTTP/1.1" 200 None 2026-04-01 07:01:03,753 - lava_callback - ERROR - Error processing callback for node 69ca1fa23c3aafdab4beae96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43800 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:01:43,692 - lava_callback - INFO - Received callback for node 69ca1fc73c3aafdab4beaea1 INFO: 172.20.0.1:48458 - "POST /node/69ca1fc73c3aafdab4beaea1 HTTP/1.0" 202 Accepted 2026-04-01 07:01:43,698 - lava_callback - INFO - Starting async job submit for node 69ca1fc73c3aafdab4beaea1 2026-04-01 07:01:43,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:43,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1fc73c3aafdab4beaea1 HTTP/1.1" 200 None 2026-04-01 07:01:43,714 - lava_callback - DEBUG - Processing node 69ca1fc73c3aafdab4beaea1: baseline-arm-clabbe 2026-04-01 07:01:43,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1fc73c3aafdab4beaea1 2026-04-01 07:01:43,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:43,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1fc73c3aafdab4beaea1 HTTP/1.1" 200 None 2026-04-01 07:01:43,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:43,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:01:43,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:01:43,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1fc73c3aafdab4beaea1 HTTP/1.1" 200 None 2026-04-01 07:01:43,782 - lava_callback - ERROR - Error processing callback for node 69ca1fc73c3aafdab4beaea1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48350 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:02:19,373 - lava_callback - INFO - Received callback for node 69ca1fe83c3aafdab4beaead INFO: 172.20.0.1:48358 - "POST /node/69ca1fe83c3aafdab4beaead HTTP/1.0" 202 Accepted 2026-04-01 07:02:19,378 - lava_callback - INFO - Starting async job submit for node 69ca1fe83c3aafdab4beaead 2026-04-01 07:02:19,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:19,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1fe83c3aafdab4beaead HTTP/1.1" 200 None 2026-04-01 07:02:19,401 - lava_callback - DEBUG - Processing node 69ca1fe83c3aafdab4beaead: baseline-arm-baylibre 2026-04-01 07:02:19,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca1fe83c3aafdab4beaead 2026-04-01 07:02:19,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:19,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca1fe83c3aafdab4beaead HTTP/1.1" 200 None 2026-04-01 07:02:19,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:19,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:02:19,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:19,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca1fe83c3aafdab4beaead HTTP/1.1" 200 None 2026-04-01 07:02:19,457 - lava_callback - ERROR - Error processing callback for node 69ca1fe83c3aafdab4beaead: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56500 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:02:39,341 - lava_callback - INFO - Received callback for node 69ca200c3c3aafdab4beaebb INFO: 172.20.0.1:56504 - "POST /node/69ca200c3c3aafdab4beaebb HTTP/1.0" 202 Accepted 2026-04-01 07:02:39,349 - lava_callback - INFO - Starting async job submit for node 69ca200c3c3aafdab4beaebb 2026-04-01 07:02:39,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:39,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca200c3c3aafdab4beaebb HTTP/1.1" 200 None 2026-04-01 07:02:39,367 - lava_callback - DEBUG - Processing node 69ca200c3c3aafdab4beaebb: baseline-arm-baylibre 2026-04-01 07:02:39,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca200c3c3aafdab4beaebb 2026-04-01 07:02:39,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:39,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca200c3c3aafdab4beaebb HTTP/1.1" 200 None 2026-04-01 07:02:39,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:39,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:02:39,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:02:39,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca200c3c3aafdab4beaebb HTTP/1.1" 200 None 2026-04-01 07:02:39,454 - lava_callback - ERROR - Error processing callback for node 69ca200c3c3aafdab4beaebb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45464 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:03:23,842 - lava_callback - INFO - Received callback for node 69ca20353c3aafdab4beaeca INFO: 172.20.0.1:46744 - "POST /node/69ca20353c3aafdab4beaeca HTTP/1.0" 202 Accepted 2026-04-01 07:03:23,848 - lava_callback - INFO - Starting async job submit for node 69ca20353c3aafdab4beaeca 2026-04-01 07:03:23,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:03:23,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20353c3aafdab4beaeca HTTP/1.1" 200 None 2026-04-01 07:03:23,870 - lava_callback - DEBUG - Processing node 69ca20353c3aafdab4beaeca: baseline-arm-clabbe 2026-04-01 07:03:23,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca20353c3aafdab4beaeca 2026-04-01 07:03:23,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:03:23,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20353c3aafdab4beaeca HTTP/1.1" 200 None 2026-04-01 07:03:23,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:03:23,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:03:23,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:03:23,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca20353c3aafdab4beaeca HTTP/1.1" 200 None 2026-04-01 07:03:23,935 - lava_callback - ERROR - Error processing callback for node 69ca20353c3aafdab4beaeca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38522 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:04:03,771 - lava_callback - INFO - Received callback for node 69ca20593c3aafdab4beaed9 INFO: 172.20.0.1:56126 - "POST /node/69ca20593c3aafdab4beaed9 HTTP/1.0" 202 Accepted 2026-04-01 07:04:03,777 - lava_callback - INFO - Starting async job submit for node 69ca20593c3aafdab4beaed9 2026-04-01 07:04:03,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:03,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20593c3aafdab4beaed9 HTTP/1.1" 200 None 2026-04-01 07:04:03,792 - lava_callback - DEBUG - Processing node 69ca20593c3aafdab4beaed9: baseline-arm-clabbe 2026-04-01 07:04:03,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca20593c3aafdab4beaed9 2026-04-01 07:04:03,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:03,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20593c3aafdab4beaed9 HTTP/1.1" 200 None 2026-04-01 07:04:03,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:03,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:04:03,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:03,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca20593c3aafdab4beaed9 HTTP/1.1" 200 None 2026-04-01 07:04:03,850 - lava_callback - ERROR - Error processing callback for node 69ca20593c3aafdab4beaed9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58098 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:04:39,497 - lava_callback - INFO - Received callback for node 69ca207b3c3aafdab4beaee7 INFO: 172.20.0.1:58104 - "POST /node/69ca207b3c3aafdab4beaee7 HTTP/1.0" 202 Accepted 2026-04-01 07:04:39,503 - lava_callback - INFO - Starting async job submit for node 69ca207b3c3aafdab4beaee7 2026-04-01 07:04:39,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:39,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca207b3c3aafdab4beaee7 HTTP/1.1" 200 None 2026-04-01 07:04:39,519 - lava_callback - DEBUG - Processing node 69ca207b3c3aafdab4beaee7: baseline-arm-baylibre 2026-04-01 07:04:39,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca207b3c3aafdab4beaee7 2026-04-01 07:04:39,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:39,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca207b3c3aafdab4beaee7 HTTP/1.1" 200 None 2026-04-01 07:04:39,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:39,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:04:39,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:04:39,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca207b3c3aafdab4beaee7 HTTP/1.1" 200 None 2026-04-01 07:04:39,574 - lava_callback - ERROR - Error processing callback for node 69ca207b3c3aafdab4beaee7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54346 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:05:23,864 - lava_callback - INFO - Received callback for node 69ca20a13c3aafdab4beaef6 INFO: 172.20.0.1:45852 - "POST /node/69ca20a13c3aafdab4beaef6 HTTP/1.0" 202 Accepted 2026-04-01 07:05:23,869 - lava_callback - INFO - Starting async job submit for node 69ca20a13c3aafdab4beaef6 2026-04-01 07:05:23,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:23,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20a13c3aafdab4beaef6 HTTP/1.1" 200 None 2026-04-01 07:05:23,889 - lava_callback - DEBUG - Processing node 69ca20a13c3aafdab4beaef6: baseline-arm-clabbe 2026-04-01 07:05:23,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca20a13c3aafdab4beaef6 2026-04-01 07:05:23,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:23,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20a13c3aafdab4beaef6 HTTP/1.1" 200 None 2026-04-01 07:05:23,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:23,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:05:23,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:23,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca20a13c3aafdab4beaef6 HTTP/1.1" 200 None 2026-04-01 07:05:23,947 - lava_callback - ERROR - Error processing callback for node 69ca20a13c3aafdab4beaef6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51242 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:05:59,581 - lava_callback - INFO - Received callback for node 69ca20c53c3aafdab4beaf07 INFO: 172.20.0.1:51254 - "POST /node/69ca20c53c3aafdab4beaf07 HTTP/1.0" 202 Accepted 2026-04-01 07:05:59,587 - lava_callback - INFO - Starting async job submit for node 69ca20c53c3aafdab4beaf07 2026-04-01 07:05:59,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:59,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20c53c3aafdab4beaf07 HTTP/1.1" 200 None 2026-04-01 07:05:59,603 - lava_callback - DEBUG - Processing node 69ca20c53c3aafdab4beaf07: baseline-arm-baylibre 2026-04-01 07:05:59,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca20c53c3aafdab4beaf07 2026-04-01 07:05:59,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:59,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20c53c3aafdab4beaf07 HTTP/1.1" 200 None 2026-04-01 07:05:59,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:59,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:05:59,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:05:59,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca20c53c3aafdab4beaf07 HTTP/1.1" 200 None 2026-04-01 07:05:59,656 - lava_callback - ERROR - Error processing callback for node 69ca20c53c3aafdab4beaf07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47286 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:06:19,536 - lava_callback - INFO - Received callback for node 69ca20e63c3aafdab4beaf12 INFO: 172.20.0.1:47290 - "POST /node/69ca20e63c3aafdab4beaf12 HTTP/1.0" 202 Accepted 2026-04-01 07:06:19,542 - lava_callback - INFO - Starting async job submit for node 69ca20e63c3aafdab4beaf12 2026-04-01 07:06:19,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:06:19,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20e63c3aafdab4beaf12 HTTP/1.1" 200 None 2026-04-01 07:06:19,558 - lava_callback - DEBUG - Processing node 69ca20e63c3aafdab4beaf12: baseline-arm-baylibre 2026-04-01 07:06:19,558 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca20e63c3aafdab4beaf12 2026-04-01 07:06:19,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:06:19,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca20e63c3aafdab4beaf12 HTTP/1.1" 200 None 2026-04-01 07:06:19,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:06:19,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:06:19,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:06:19,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca20e63c3aafdab4beaf12 HTTP/1.1" 200 None 2026-04-01 07:06:19,620 - lava_callback - ERROR - Error processing callback for node 69ca20e63c3aafdab4beaf12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37588 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:07:04,140 - lava_callback - INFO - Received callback for node 69ca210e3c3aafdab4beaf22 INFO: 172.20.0.1:51114 - "POST /node/69ca210e3c3aafdab4beaf22 HTTP/1.0" 202 Accepted 2026-04-01 07:07:04,145 - lava_callback - INFO - Starting async job submit for node 69ca210e3c3aafdab4beaf22 2026-04-01 07:07:04,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:04,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca210e3c3aafdab4beaf22 HTTP/1.1" 200 None 2026-04-01 07:07:04,168 - lava_callback - DEBUG - Processing node 69ca210e3c3aafdab4beaf22: baseline-arm-clabbe 2026-04-01 07:07:04,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca210e3c3aafdab4beaf22 2026-04-01 07:07:04,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:04,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca210e3c3aafdab4beaf22 HTTP/1.1" 200 None 2026-04-01 07:07:04,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:04,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:07:04,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:04,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca210e3c3aafdab4beaf22 HTTP/1.1" 200 None 2026-04-01 07:07:04,229 - lava_callback - ERROR - Error processing callback for node 69ca210e3c3aafdab4beaf22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56096 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:07:43,913 - lava_callback - INFO - Received callback for node 69ca21333c3aafdab4beaf30 INFO: 172.20.0.1:52398 - "POST /node/69ca21333c3aafdab4beaf30 HTTP/1.0" 202 Accepted 2026-04-01 07:07:43,919 - lava_callback - INFO - Starting async job submit for node 69ca21333c3aafdab4beaf30 2026-04-01 07:07:43,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:43,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21333c3aafdab4beaf30 HTTP/1.1" 200 None 2026-04-01 07:07:43,950 - lava_callback - DEBUG - Processing node 69ca21333c3aafdab4beaf30: baseline-arm-clabbe 2026-04-01 07:07:43,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca21333c3aafdab4beaf30 2026-04-01 07:07:43,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:43,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21333c3aafdab4beaf30 HTTP/1.1" 200 None 2026-04-01 07:07:43,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:43,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:07:43,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:07:44,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca21333c3aafdab4beaf30 HTTP/1.1" 200 None 2026-04-01 07:07:44,001 - lava_callback - ERROR - Error processing callback for node 69ca21333c3aafdab4beaf30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60500 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:08:23,997 - lava_callback - INFO - Received callback for node 69ca21583c3aafdab4beaf44 INFO: 172.20.0.1:57692 - "POST /node/69ca21583c3aafdab4beaf44 HTTP/1.0" 202 Accepted 2026-04-01 07:08:24,003 - lava_callback - INFO - Starting async job submit for node 69ca21583c3aafdab4beaf44 2026-04-01 07:08:24,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:08:24,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21583c3aafdab4beaf44 HTTP/1.1" 200 None 2026-04-01 07:08:24,021 - lava_callback - DEBUG - Processing node 69ca21583c3aafdab4beaf44: baseline-arm-clabbe 2026-04-01 07:08:24,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca21583c3aafdab4beaf44 2026-04-01 07:08:24,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:08:24,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21583c3aafdab4beaf44 HTTP/1.1" 200 None 2026-04-01 07:08:24,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:08:24,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:08:24,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:08:24,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca21583c3aafdab4beaf44 HTTP/1.1" 200 None 2026-04-01 07:08:24,079 - lava_callback - ERROR - Error processing callback for node 69ca21583c3aafdab4beaf44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60502 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:09:03,997 - lava_callback - INFO - Received callback for node 69ca217d3c3aafdab4beaf4c INFO: 172.20.0.1:44352 - "POST /node/69ca217d3c3aafdab4beaf4c HTTP/1.0" 202 Accepted 2026-04-01 07:09:04,003 - lava_callback - INFO - Starting async job submit for node 69ca217d3c3aafdab4beaf4c 2026-04-01 07:09:04,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:04,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca217d3c3aafdab4beaf4c HTTP/1.1" 200 None 2026-04-01 07:09:04,018 - lava_callback - DEBUG - Processing node 69ca217d3c3aafdab4beaf4c: baseline-arm-clabbe 2026-04-01 07:09:04,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca217d3c3aafdab4beaf4c 2026-04-01 07:09:04,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:04,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca217d3c3aafdab4beaf4c HTTP/1.1" 200 None 2026-04-01 07:09:04,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:04,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:09:04,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:04,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca217d3c3aafdab4beaf4c HTTP/1.1" 200 None 2026-04-01 07:09:04,077 - lava_callback - ERROR - Error processing callback for node 69ca217d3c3aafdab4beaf4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:09:44,018 - lava_callback - INFO - Received callback for node 69ca21a23c3aafdab4beaf58 INFO: 172.20.0.1:41198 - "POST /node/69ca21a23c3aafdab4beaf58 HTTP/1.0" 202 Accepted 2026-04-01 07:09:44,023 - lava_callback - INFO - Starting async job submit for node 69ca21a23c3aafdab4beaf58 2026-04-01 07:09:44,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:44,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21a23c3aafdab4beaf58 HTTP/1.1" 200 None 2026-04-01 07:09:44,039 - lava_callback - DEBUG - Processing node 69ca21a23c3aafdab4beaf58: baseline-arm-clabbe 2026-04-01 07:09:44,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca21a23c3aafdab4beaf58 2026-04-01 07:09:44,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:44,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21a23c3aafdab4beaf58 HTTP/1.1" 200 None 2026-04-01 07:09:44,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:44,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:09:44,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:09:44,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca21a23c3aafdab4beaf58 HTTP/1.1" 200 None 2026-04-01 07:09:44,114 - lava_callback - ERROR - Error processing callback for node 69ca21a23c3aafdab4beaf58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41938 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:10:04,281 - lava_callback - INFO - Received callback for node 69ca21c63c3aafdab4beaf64 INFO: 172.20.0.1:36630 - "POST /node/69ca21c63c3aafdab4beaf64 HTTP/1.0" 202 Accepted 2026-04-01 07:10:04,287 - lava_callback - INFO - Starting async job submit for node 69ca21c63c3aafdab4beaf64 2026-04-01 07:10:04,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:04,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21c63c3aafdab4beaf64 HTTP/1.1" 200 None 2026-04-01 07:10:04,307 - lava_callback - DEBUG - Processing node 69ca21c63c3aafdab4beaf64: baseline-arm-clabbe 2026-04-01 07:10:04,307 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca21c63c3aafdab4beaf64 2026-04-01 07:10:04,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:04,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21c63c3aafdab4beaf64 HTTP/1.1" 200 None 2026-04-01 07:10:04,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:04,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:10:04,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:04,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca21c63c3aafdab4beaf64 HTTP/1.1" 200 None 2026-04-01 07:10:04,365 - lava_callback - ERROR - Error processing callback for node 69ca21c63c3aafdab4beaf64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53638 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:10:44,103 - lava_callback - INFO - Received callback for node 69ca21eb3c3aafdab4beaf71 INFO: 172.20.0.1:52252 - "POST /node/69ca21eb3c3aafdab4beaf71 HTTP/1.0" 202 Accepted 2026-04-01 07:10:44,109 - lava_callback - INFO - Starting async job submit for node 69ca21eb3c3aafdab4beaf71 2026-04-01 07:10:44,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:44,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21eb3c3aafdab4beaf71 HTTP/1.1" 200 None 2026-04-01 07:10:44,135 - lava_callback - DEBUG - Processing node 69ca21eb3c3aafdab4beaf71: baseline-arm-clabbe 2026-04-01 07:10:44,135 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca21eb3c3aafdab4beaf71 2026-04-01 07:10:44,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:44,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca21eb3c3aafdab4beaf71 HTTP/1.1" 200 None 2026-04-01 07:10:44,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:44,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-01 07:10:44,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:10:44,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca21eb3c3aafdab4beaf71 HTTP/1.1" 200 None 2026-04-01 07:10:44,208 - lava_callback - ERROR - Error processing callback for node 69ca21eb3c3aafdab4beaf71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:11:24,080 - lava_callback - INFO - Received callback for node 69ca22103c3aafdab4beaf83 INFO: 172.20.0.1:38788 - "POST /node/69ca22103c3aafdab4beaf83 HTTP/1.0" 202 Accepted 2026-04-01 07:11:24,089 - lava_callback - INFO - Starting async job submit for node 69ca22103c3aafdab4beaf83 2026-04-01 07:11:24,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:11:24,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22103c3aafdab4beaf83 HTTP/1.1" 200 None 2026-04-01 07:11:24,108 - lava_callback - DEBUG - Processing node 69ca22103c3aafdab4beaf83: baseline-arm-clabbe 2026-04-01 07:11:24,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca22103c3aafdab4beaf83 2026-04-01 07:11:24,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:11:24,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22103c3aafdab4beaf83 HTTP/1.1" 200 None 2026-04-01 07:11:24,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:11:24,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:11:24,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:11:24,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca22103c3aafdab4beaf83 HTTP/1.1" 200 None 2026-04-01 07:11:24,172 - lava_callback - ERROR - Error processing callback for node 69ca22103c3aafdab4beaf83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36598 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:12:04,113 - lava_callback - INFO - Received callback for node 69ca22353c3aafdab4beaf91 INFO: 172.20.0.1:49716 - "POST /node/69ca22353c3aafdab4beaf91 HTTP/1.0" 202 Accepted 2026-04-01 07:12:04,118 - lava_callback - INFO - Starting async job submit for node 69ca22353c3aafdab4beaf91 2026-04-01 07:12:04,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:04,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22353c3aafdab4beaf91 HTTP/1.1" 200 None 2026-04-01 07:12:04,135 - lava_callback - DEBUG - Processing node 69ca22353c3aafdab4beaf91: baseline-arm-clabbe 2026-04-01 07:12:04,135 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca22353c3aafdab4beaf91 2026-04-01 07:12:04,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:04,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22353c3aafdab4beaf91 HTTP/1.1" 200 None 2026-04-01 07:12:04,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:04,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:12:04,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:04,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca22353c3aafdab4beaf91 HTTP/1.1" 200 None 2026-04-01 07:12:04,196 - lava_callback - ERROR - Error processing callback for node 69ca22353c3aafdab4beaf91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59388 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:12:44,131 - lava_callback - INFO - Received callback for node 69ca22593c3aafdab4beaf9f INFO: 172.20.0.1:54736 - "POST /node/69ca22593c3aafdab4beaf9f HTTP/1.0" 202 Accepted 2026-04-01 07:12:44,137 - lava_callback - INFO - Starting async job submit for node 69ca22593c3aafdab4beaf9f 2026-04-01 07:12:44,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:44,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22593c3aafdab4beaf9f HTTP/1.1" 200 None 2026-04-01 07:12:44,153 - lava_callback - DEBUG - Processing node 69ca22593c3aafdab4beaf9f: baseline-arm-clabbe 2026-04-01 07:12:44,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca22593c3aafdab4beaf9f 2026-04-01 07:12:44,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:44,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22593c3aafdab4beaf9f HTTP/1.1" 200 None 2026-04-01 07:12:44,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:44,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:12:44,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:12:44,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca22593c3aafdab4beaf9f HTTP/1.1" 200 None 2026-04-01 07:12:44,216 - lava_callback - ERROR - Error processing callback for node 69ca22593c3aafdab4beaf9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55782 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:12:59,982 - lava_callback - INFO - Received callback for node 69ca227a3c3aafdab4beafaa INFO: 172.20.0.1:41004 - "POST /node/69ca227a3c3aafdab4beafaa HTTP/1.0" 202 Accepted 2026-04-01 07:12:59,988 - lava_callback - INFO - Starting async job submit for node 69ca227a3c3aafdab4beafaa 2026-04-01 07:12:59,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:13:00,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca227a3c3aafdab4beafaa HTTP/1.1" 200 None 2026-04-01 07:13:00,011 - lava_callback - DEBUG - Processing node 69ca227a3c3aafdab4beafaa: baseline-arm-baylibre 2026-04-01 07:13:00,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca227a3c3aafdab4beafaa 2026-04-01 07:13:00,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:13:00,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca227a3c3aafdab4beafaa HTTP/1.1" 200 None 2026-04-01 07:13:00,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:13:00,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:13:00,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:13:00,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca227a3c3aafdab4beafaa HTTP/1.1" 200 None 2026-04-01 07:13:00,069 - lava_callback - ERROR - Error processing callback for node 69ca227a3c3aafdab4beafaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:14:04,232 - lava_callback - INFO - Received callback for node 69ca22a63c3aafdab4beafbb INFO: 172.20.0.1:38472 - "POST /node/69ca22a63c3aafdab4beafbb HTTP/1.0" 202 Accepted 2026-04-01 07:14:04,237 - lava_callback - INFO - Starting async job submit for node 69ca22a63c3aafdab4beafbb 2026-04-01 07:14:04,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:04,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22a63c3aafdab4beafbb HTTP/1.1" 200 None 2026-04-01 07:14:04,253 - lava_callback - DEBUG - Processing node 69ca22a63c3aafdab4beafbb: baseline-arm-clabbe 2026-04-01 07:14:04,254 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca22a63c3aafdab4beafbb 2026-04-01 07:14:04,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:04,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22a63c3aafdab4beafbb HTTP/1.1" 200 None 2026-04-01 07:14:04,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:04,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:14:04,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:04,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca22a63c3aafdab4beafbb HTTP/1.1" 200 None 2026-04-01 07:14:04,305 - lava_callback - ERROR - Error processing callback for node 69ca22a63c3aafdab4beafbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:14:24,187 - lava_callback - INFO - Received callback for node 69ca22cb3c3aafdab4beafc6 INFO: 172.20.0.1:59332 - "POST /node/69ca22cb3c3aafdab4beafc6 HTTP/1.0" 202 Accepted 2026-04-01 07:14:24,193 - lava_callback - INFO - Starting async job submit for node 69ca22cb3c3aafdab4beafc6 2026-04-01 07:14:24,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:24,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22cb3c3aafdab4beafc6 HTTP/1.1" 200 None 2026-04-01 07:14:24,209 - lava_callback - DEBUG - Processing node 69ca22cb3c3aafdab4beafc6: baseline-arm-clabbe 2026-04-01 07:14:24,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca22cb3c3aafdab4beafc6 2026-04-01 07:14:24,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:24,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca22cb3c3aafdab4beafc6 HTTP/1.1" 200 None 2026-04-01 07:14:24,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:24,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:14:24,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:14:24,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca22cb3c3aafdab4beafc6 HTTP/1.1" 200 None 2026-04-01 07:14:24,266 - lava_callback - ERROR - Error processing callback for node 69ca22cb3c3aafdab4beafc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47402 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:15:44,301 - lava_callback - INFO - Received callback for node 69ca23123c3aafdab4beafe9 INFO: 172.20.0.1:37242 - "POST /node/69ca23123c3aafdab4beafe9 HTTP/1.0" 202 Accepted 2026-04-01 07:15:44,306 - lava_callback - INFO - Starting async job submit for node 69ca23123c3aafdab4beafe9 2026-04-01 07:15:44,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:15:44,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca23123c3aafdab4beafe9 HTTP/1.1" 200 None 2026-04-01 07:15:44,323 - lava_callback - DEBUG - Processing node 69ca23123c3aafdab4beafe9: baseline-arm-clabbe 2026-04-01 07:15:44,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca23123c3aafdab4beafe9 2026-04-01 07:15:44,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:15:44,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca23123c3aafdab4beafe9 HTTP/1.1" 200 None 2026-04-01 07:15:44,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:15:44,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:15:44,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:15:44,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca23123c3aafdab4beafe9 HTTP/1.1" 200 None 2026-04-01 07:15:44,393 - lava_callback - ERROR - Error processing callback for node 69ca23123c3aafdab4beafe9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47564 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:16:24,259 - lava_callback - INFO - Received callback for node 69ca23373c3aafdab4beb01d INFO: 172.20.0.1:43040 - "POST /node/69ca23373c3aafdab4beb01d HTTP/1.0" 202 Accepted 2026-04-01 07:16:24,264 - lava_callback - INFO - Starting async job submit for node 69ca23373c3aafdab4beb01d 2026-04-01 07:16:24,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:16:24,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca23373c3aafdab4beb01d HTTP/1.1" 200 None 2026-04-01 07:16:24,280 - lava_callback - DEBUG - Processing node 69ca23373c3aafdab4beb01d: baseline-arm-clabbe 2026-04-01 07:16:24,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca23373c3aafdab4beb01d 2026-04-01 07:16:24,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:16:24,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca23373c3aafdab4beb01d HTTP/1.1" 200 None 2026-04-01 07:16:24,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:16:24,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:16:24,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:16:24,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca23373c3aafdab4beb01d HTTP/1.1" 200 None 2026-04-01 07:16:24,331 - lava_callback - ERROR - Error processing callback for node 69ca23373c3aafdab4beb01d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48814 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:17:04,467 - lava_callback - INFO - Received callback for node 69ca235c3c3aafdab4beb028 INFO: 172.20.0.1:40432 - "POST /node/69ca235c3c3aafdab4beb028 HTTP/1.0" 202 Accepted 2026-04-01 07:17:04,472 - lava_callback - INFO - Starting async job submit for node 69ca235c3c3aafdab4beb028 2026-04-01 07:17:04,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:04,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca235c3c3aafdab4beb028 HTTP/1.1" 200 None 2026-04-01 07:17:04,489 - lava_callback - DEBUG - Processing node 69ca235c3c3aafdab4beb028: baseline-arm-clabbe 2026-04-01 07:17:04,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca235c3c3aafdab4beb028 2026-04-01 07:17:04,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:04,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca235c3c3aafdab4beb028 HTTP/1.1" 200 None 2026-04-01 07:17:04,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:04,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:17:04,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:04,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca235c3c3aafdab4beb028 HTTP/1.1" 200 None 2026-04-01 07:17:04,562 - lava_callback - ERROR - Error processing callback for node 69ca235c3c3aafdab4beb028: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38600 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:17:24,310 - lava_callback - INFO - Received callback for node 69ca23813c3aafdab4beb037 INFO: 172.20.0.1:54426 - "POST /node/69ca23813c3aafdab4beb037 HTTP/1.0" 202 Accepted 2026-04-01 07:17:24,315 - lava_callback - INFO - Starting async job submit for node 69ca23813c3aafdab4beb037 2026-04-01 07:17:24,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:24,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca23813c3aafdab4beb037 HTTP/1.1" 200 None 2026-04-01 07:17:24,331 - lava_callback - DEBUG - Processing node 69ca23813c3aafdab4beb037: baseline-arm-clabbe 2026-04-01 07:17:24,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca23813c3aafdab4beb037 2026-04-01 07:17:24,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:24,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca23813c3aafdab4beb037 HTTP/1.1" 200 None 2026-04-01 07:17:24,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:24,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-01 07:17:24,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:17:24,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca23813c3aafdab4beb037 HTTP/1.1" 200 None 2026-04-01 07:17:24,386 - lava_callback - ERROR - Error processing callback for node 69ca23813c3aafdab4beb037: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34962 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:23:20,738 - lava_callback - INFO - Received callback for node 69ca24df3c3aafdab4beb074 INFO: 172.20.0.1:58108 - "POST /node/69ca24df3c3aafdab4beb074 HTTP/1.0" 202 Accepted 2026-04-01 07:23:20,744 - lava_callback - INFO - Starting async job submit for node 69ca24df3c3aafdab4beb074 2026-04-01 07:23:20,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:23:20,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca24df3c3aafdab4beb074 HTTP/1.1" 200 None 2026-04-01 07:23:20,760 - lava_callback - DEBUG - Processing node 69ca24df3c3aafdab4beb074: baseline-arm64-baylibre 2026-04-01 07:23:20,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca24df3c3aafdab4beb074 2026-04-01 07:23:20,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:23:20,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca24df3c3aafdab4beb074 HTTP/1.1" 200 None 2026-04-01 07:23:20,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:23:20,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:23:20,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:23:20,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca24df3c3aafdab4beb074 HTTP/1.1" 200 None 2026-04-01 07:23:20,811 - lava_callback - ERROR - Error processing callback for node 69ca24df3c3aafdab4beb074: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35134 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:24:40,593 - lava_callback - INFO - Received callback for node 69ca252d3c3aafdab4beb07a INFO: 172.20.0.1:46774 - "POST /node/69ca252d3c3aafdab4beb07a HTTP/1.0" 202 Accepted 2026-04-01 07:24:40,600 - lava_callback - INFO - Starting async job submit for node 69ca252d3c3aafdab4beb07a 2026-04-01 07:24:40,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:24:40,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca252d3c3aafdab4beb07a HTTP/1.1" 200 None 2026-04-01 07:24:40,622 - lava_callback - DEBUG - Processing node 69ca252d3c3aafdab4beb07a: baseline-arm64-baylibre 2026-04-01 07:24:40,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca252d3c3aafdab4beb07a 2026-04-01 07:24:40,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:24:40,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca252d3c3aafdab4beb07a HTTP/1.1" 200 None 2026-04-01 07:24:40,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:24:40,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:24:40,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:24:40,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca252d3c3aafdab4beb07a HTTP/1.1" 200 None 2026-04-01 07:24:40,687 - lava_callback - ERROR - Error processing callback for node 69ca252d3c3aafdab4beb07a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49588 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:26:00,689 - lava_callback - INFO - Received callback for node 69ca257c3c3aafdab4beb085 INFO: 172.20.0.1:47030 - "POST /node/69ca257c3c3aafdab4beb085 HTTP/1.0" 202 Accepted 2026-04-01 07:26:00,695 - lava_callback - INFO - Starting async job submit for node 69ca257c3c3aafdab4beb085 2026-04-01 07:26:00,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:26:00,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca257c3c3aafdab4beb085 HTTP/1.1" 200 None 2026-04-01 07:26:00,712 - lava_callback - DEBUG - Processing node 69ca257c3c3aafdab4beb085: baseline-arm64-baylibre 2026-04-01 07:26:00,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca257c3c3aafdab4beb085 2026-04-01 07:26:00,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:26:00,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca257c3c3aafdab4beb085 HTTP/1.1" 200 None 2026-04-01 07:26:00,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:26:00,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:26:00,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:26:00,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca257c3c3aafdab4beb085 HTTP/1.1" 200 None 2026-04-01 07:26:00,774 - lava_callback - ERROR - Error processing callback for node 69ca257c3c3aafdab4beb085: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47536 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:27:20,705 - lava_callback - INFO - Received callback for node 69ca25cb3c3aafdab4beb08c INFO: 172.20.0.1:33302 - "POST /node/69ca25cb3c3aafdab4beb08c HTTP/1.0" 202 Accepted 2026-04-01 07:27:20,710 - lava_callback - INFO - Starting async job submit for node 69ca25cb3c3aafdab4beb08c 2026-04-01 07:27:20,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:27:20,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca25cb3c3aafdab4beb08c HTTP/1.1" 200 None 2026-04-01 07:27:20,745 - lava_callback - DEBUG - Processing node 69ca25cb3c3aafdab4beb08c: baseline-arm64-baylibre 2026-04-01 07:27:20,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca25cb3c3aafdab4beb08c 2026-04-01 07:27:20,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:27:20,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca25cb3c3aafdab4beb08c HTTP/1.1" 200 None 2026-04-01 07:27:20,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:27:20,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:27:20,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:27:20,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca25cb3c3aafdab4beb08c HTTP/1.1" 200 None 2026-04-01 07:27:20,812 - lava_callback - ERROR - Error processing callback for node 69ca25cb3c3aafdab4beb08c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45800 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:28:41,119 - lava_callback - INFO - Received callback for node 69ca26193c3aafdab4beb098 INFO: 172.20.0.1:43856 - "POST /node/69ca26193c3aafdab4beb098 HTTP/1.0" 202 Accepted 2026-04-01 07:28:41,125 - lava_callback - INFO - Starting async job submit for node 69ca26193c3aafdab4beb098 2026-04-01 07:28:41,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:28:41,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca26193c3aafdab4beb098 HTTP/1.1" 200 None 2026-04-01 07:28:41,147 - lava_callback - DEBUG - Processing node 69ca26193c3aafdab4beb098: baseline-arm64-baylibre 2026-04-01 07:28:41,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca26193c3aafdab4beb098 2026-04-01 07:28:41,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:28:41,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca26193c3aafdab4beb098 HTTP/1.1" 200 None 2026-04-01 07:28:41,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:28:41,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:28:41,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:28:41,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca26193c3aafdab4beb098 HTTP/1.1" 200 None 2026-04-01 07:28:41,221 - lava_callback - ERROR - Error processing callback for node 69ca26193c3aafdab4beb098: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46582 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:30:01,049 - lava_callback - INFO - Received callback for node 69ca26693c3aafdab4beb0a7 INFO: 172.20.0.1:56756 - "POST /node/69ca26693c3aafdab4beb0a7 HTTP/1.0" 202 Accepted 2026-04-01 07:30:01,054 - lava_callback - INFO - Starting async job submit for node 69ca26693c3aafdab4beb0a7 2026-04-01 07:30:01,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:30:01,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca26693c3aafdab4beb0a7 HTTP/1.1" 200 None 2026-04-01 07:30:01,071 - lava_callback - DEBUG - Processing node 69ca26693c3aafdab4beb0a7: baseline-arm64-baylibre 2026-04-01 07:30:01,071 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca26693c3aafdab4beb0a7 2026-04-01 07:30:01,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:30:01,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca26693c3aafdab4beb0a7 HTTP/1.1" 200 None 2026-04-01 07:30:01,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:30:01,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:30:01,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:30:01,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca26693c3aafdab4beb0a7 HTTP/1.1" 200 None 2026-04-01 07:30:01,171 - lava_callback - ERROR - Error processing callback for node 69ca26693c3aafdab4beb0a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:31:25,036 - lava_callback - INFO - Received callback for node 69ca26c03c3aafdab4beb0b8 INFO: 172.20.0.1:51156 - "POST /node/69ca26c03c3aafdab4beb0b8 HTTP/1.0" 202 Accepted 2026-04-01 07:31:25,042 - lava_callback - INFO - Starting async job submit for node 69ca26c03c3aafdab4beb0b8 2026-04-01 07:31:25,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:31:25,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca26c03c3aafdab4beb0b8 HTTP/1.1" 200 None 2026-04-01 07:31:25,059 - lava_callback - DEBUG - Processing node 69ca26c03c3aafdab4beb0b8: baseline-arm64-clabbe 2026-04-01 07:31:25,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca26c03c3aafdab4beb0b8 2026-04-01 07:31:25,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:31:25,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca26c03c3aafdab4beb0b8 HTTP/1.1" 200 None 2026-04-01 07:31:25,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:31:25,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:31:25,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:31:25,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca26c03c3aafdab4beb0b8 HTTP/1.1" 200 None 2026-04-01 07:31:25,116 - lava_callback - ERROR - Error processing callback for node 69ca26c03c3aafdab4beb0b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:32:45,102 - lava_callback - INFO - Received callback for node 69ca27113c3aafdab4beb0c2 INFO: 172.20.0.1:43368 - "POST /node/69ca27113c3aafdab4beb0c2 HTTP/1.0" 202 Accepted 2026-04-01 07:32:45,108 - lava_callback - INFO - Starting async job submit for node 69ca27113c3aafdab4beb0c2 2026-04-01 07:32:45,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:32:45,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca27113c3aafdab4beb0c2 HTTP/1.1" 200 None 2026-04-01 07:32:45,124 - lava_callback - DEBUG - Processing node 69ca27113c3aafdab4beb0c2: baseline-arm64-clabbe 2026-04-01 07:32:45,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca27113c3aafdab4beb0c2 2026-04-01 07:32:45,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:32:45,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca27113c3aafdab4beb0c2 HTTP/1.1" 200 None 2026-04-01 07:32:45,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:32:45,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:32:45,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:32:45,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca27113c3aafdab4beb0c2 HTTP/1.1" 200 None 2026-04-01 07:32:45,182 - lava_callback - ERROR - Error processing callback for node 69ca27113c3aafdab4beb0c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51582 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:34:05,133 - lava_callback - INFO - Received callback for node 69ca275f3c3aafdab4beb0d4 INFO: 172.20.0.1:54354 - "POST /node/69ca275f3c3aafdab4beb0d4 HTTP/1.0" 202 Accepted 2026-04-01 07:34:05,138 - lava_callback - INFO - Starting async job submit for node 69ca275f3c3aafdab4beb0d4 2026-04-01 07:34:05,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:34:05,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca275f3c3aafdab4beb0d4 HTTP/1.1" 200 None 2026-04-01 07:34:05,163 - lava_callback - DEBUG - Processing node 69ca275f3c3aafdab4beb0d4: baseline-arm64-clabbe 2026-04-01 07:34:05,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca275f3c3aafdab4beb0d4 2026-04-01 07:34:05,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:34:05,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca275f3c3aafdab4beb0d4 HTTP/1.1" 200 None 2026-04-01 07:34:05,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:34:05,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:34:05,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:34:05,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca275f3c3aafdab4beb0d4 HTTP/1.1" 200 None 2026-04-01 07:34:05,245 - lava_callback - ERROR - Error processing callback for node 69ca275f3c3aafdab4beb0d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34708 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:35:25,228 - lava_callback - INFO - Received callback for node 69ca27ae3c3aafdab4beb0e0 INFO: 172.20.0.1:33586 - "POST /node/69ca27ae3c3aafdab4beb0e0 HTTP/1.0" 202 Accepted 2026-04-01 07:35:25,234 - lava_callback - INFO - Starting async job submit for node 69ca27ae3c3aafdab4beb0e0 2026-04-01 07:35:25,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:35:25,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca27ae3c3aafdab4beb0e0 HTTP/1.1" 200 None 2026-04-01 07:35:25,250 - lava_callback - DEBUG - Processing node 69ca27ae3c3aafdab4beb0e0: baseline-arm64-clabbe 2026-04-01 07:35:25,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca27ae3c3aafdab4beb0e0 2026-04-01 07:35:25,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:35:25,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca27ae3c3aafdab4beb0e0 HTTP/1.1" 200 None 2026-04-01 07:35:25,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:35:25,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:35:25,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:35:25,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca27ae3c3aafdab4beb0e0 HTTP/1.1" 200 None 2026-04-01 07:35:25,304 - lava_callback - ERROR - Error processing callback for node 69ca27ae3c3aafdab4beb0e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41634 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:36:45,300 - lava_callback - INFO - Received callback for node 69ca27fd3c3aafdab4beb0eb INFO: 172.20.0.1:36974 - "POST /node/69ca27fd3c3aafdab4beb0eb HTTP/1.0" 202 Accepted 2026-04-01 07:36:45,306 - lava_callback - INFO - Starting async job submit for node 69ca27fd3c3aafdab4beb0eb 2026-04-01 07:36:45,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:36:45,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca27fd3c3aafdab4beb0eb HTTP/1.1" 200 None 2026-04-01 07:36:45,323 - lava_callback - DEBUG - Processing node 69ca27fd3c3aafdab4beb0eb: baseline-arm64-clabbe 2026-04-01 07:36:45,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca27fd3c3aafdab4beb0eb 2026-04-01 07:36:45,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:36:45,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca27fd3c3aafdab4beb0eb HTTP/1.1" 200 None 2026-04-01 07:36:45,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:36:45,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:36:45,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:36:45,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca27fd3c3aafdab4beb0eb HTTP/1.1" 200 None 2026-04-01 07:36:45,389 - lava_callback - ERROR - Error processing callback for node 69ca27fd3c3aafdab4beb0eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:38:05,344 - lava_callback - INFO - Received callback for node 69ca284b3c3aafdab4beb0fc INFO: 172.20.0.1:43740 - "POST /node/69ca284b3c3aafdab4beb0fc HTTP/1.0" 202 Accepted 2026-04-01 07:38:05,353 - lava_callback - INFO - Starting async job submit for node 69ca284b3c3aafdab4beb0fc 2026-04-01 07:38:05,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:38:05,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca284b3c3aafdab4beb0fc HTTP/1.1" 200 None 2026-04-01 07:38:05,369 - lava_callback - DEBUG - Processing node 69ca284b3c3aafdab4beb0fc: baseline-arm64-clabbe 2026-04-01 07:38:05,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca284b3c3aafdab4beb0fc 2026-04-01 07:38:05,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:38:05,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca284b3c3aafdab4beb0fc HTTP/1.1" 200 None 2026-04-01 07:38:05,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:38:05,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-01 07:38:05,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:38:05,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca284b3c3aafdab4beb0fc HTTP/1.1" 200 None 2026-04-01 07:38:05,427 - lava_callback - ERROR - Error processing callback for node 69ca284b3c3aafdab4beb0fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55488 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:39:01,428 - lava_callback - INFO - Received callback for node 69ca28903c3aafdab4beb102 INFO: 172.20.0.1:53418 - "POST /node/69ca28903c3aafdab4beb102 HTTP/1.0" 202 Accepted 2026-04-01 07:39:01,434 - lava_callback - INFO - Starting async job submit for node 69ca28903c3aafdab4beb102 2026-04-01 07:39:01,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:39:01,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca28903c3aafdab4beb102 HTTP/1.1" 200 None 2026-04-01 07:39:01,455 - lava_callback - DEBUG - Processing node 69ca28903c3aafdab4beb102: baseline-arm64-baylibre 2026-04-01 07:39:01,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca28903c3aafdab4beb102 2026-04-01 07:39:01,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:39:01,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca28903c3aafdab4beb102 HTTP/1.1" 200 None 2026-04-01 07:39:01,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:39:01,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:39:01,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:39:01,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca28903c3aafdab4beb102 HTTP/1.1" 200 None 2026-04-01 07:39:01,524 - lava_callback - ERROR - Error processing callback for node 69ca28903c3aafdab4beb102: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43502 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:40:21,518 - lava_callback - INFO - Received callback for node 69ca28e03c3aafdab4beb10f INFO: 172.20.0.1:37342 - "POST /node/69ca28e03c3aafdab4beb10f HTTP/1.0" 202 Accepted 2026-04-01 07:40:21,524 - lava_callback - INFO - Starting async job submit for node 69ca28e03c3aafdab4beb10f 2026-04-01 07:40:21,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:40:21,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca28e03c3aafdab4beb10f HTTP/1.1" 200 None 2026-04-01 07:40:21,540 - lava_callback - DEBUG - Processing node 69ca28e03c3aafdab4beb10f: baseline-arm64-baylibre 2026-04-01 07:40:21,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca28e03c3aafdab4beb10f 2026-04-01 07:40:21,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:40:21,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca28e03c3aafdab4beb10f HTTP/1.1" 200 None 2026-04-01 07:40:21,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:40:21,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:40:21,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:40:21,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca28e03c3aafdab4beb10f HTTP/1.1" 200 None 2026-04-01 07:40:21,605 - lava_callback - ERROR - Error processing callback for node 69ca28e03c3aafdab4beb10f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:43:01,793 - lava_callback - INFO - Received callback for node 69ca29793c3aafdab4beb129 INFO: 172.20.0.1:50354 - "POST /node/69ca29793c3aafdab4beb129 HTTP/1.0" 202 Accepted 2026-04-01 07:43:01,799 - lava_callback - INFO - Starting async job submit for node 69ca29793c3aafdab4beb129 2026-04-01 07:43:01,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:43:01,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca29793c3aafdab4beb129 HTTP/1.1" 200 None 2026-04-01 07:43:01,822 - lava_callback - DEBUG - Processing node 69ca29793c3aafdab4beb129: baseline-arm64-baylibre 2026-04-01 07:43:01,822 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca29793c3aafdab4beb129 2026-04-01 07:43:01,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:43:01,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca29793c3aafdab4beb129 HTTP/1.1" 200 None 2026-04-01 07:43:01,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:43:01,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:43:01,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:43:01,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca29793c3aafdab4beb129 HTTP/1.1" 200 None 2026-04-01 07:43:01,890 - lava_callback - ERROR - Error processing callback for node 69ca29793c3aafdab4beb129: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49384 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:46:41,740 - lava_callback - INFO - Received callback for node 69ca2a5f3c3aafdab4beb152 INFO: 172.20.0.1:58510 - "POST /node/69ca2a5f3c3aafdab4beb152 HTTP/1.0" 202 Accepted 2026-04-01 07:46:41,746 - lava_callback - INFO - Starting async job submit for node 69ca2a5f3c3aafdab4beb152 2026-04-01 07:46:41,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:46:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2a5f3c3aafdab4beb152 HTTP/1.1" 200 None 2026-04-01 07:46:41,776 - lava_callback - DEBUG - Processing node 69ca2a5f3c3aafdab4beb152: baseline-arm64-baylibre 2026-04-01 07:46:41,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2a5f3c3aafdab4beb152 2026-04-01 07:46:41,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:46:41,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2a5f3c3aafdab4beb152 HTTP/1.1" 200 None 2026-04-01 07:46:41,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:46:41,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:46:41,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:46:41,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2a5f3c3aafdab4beb152 HTTP/1.1" 200 None 2026-04-01 07:46:41,839 - lava_callback - ERROR - Error processing callback for node 69ca2a5f3c3aafdab4beb152: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44892 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:48:01,859 - lava_callback - INFO - Received callback for node 69ca2aab3c3aafdab4beb15c INFO: 172.20.0.1:40612 - "POST /node/69ca2aab3c3aafdab4beb15c HTTP/1.0" 202 Accepted 2026-04-01 07:48:01,864 - lava_callback - INFO - Starting async job submit for node 69ca2aab3c3aafdab4beb15c 2026-04-01 07:48:01,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:48:01,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2aab3c3aafdab4beb15c HTTP/1.1" 200 None 2026-04-01 07:48:01,880 - lava_callback - DEBUG - Processing node 69ca2aab3c3aafdab4beb15c: baseline-arm64-baylibre 2026-04-01 07:48:01,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2aab3c3aafdab4beb15c 2026-04-01 07:48:01,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:48:01,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2aab3c3aafdab4beb15c HTTP/1.1" 200 None 2026-04-01 07:48:01,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:48:01,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:48:01,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:48:01,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2aab3c3aafdab4beb15c HTTP/1.1" 200 None 2026-04-01 07:48:01,936 - lava_callback - ERROR - Error processing callback for node 69ca2aab3c3aafdab4beb15c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37784 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:49:21,897 - lava_callback - INFO - Received callback for node 69ca2af53c3aafdab4beb16b INFO: 172.20.0.1:59406 - "POST /node/69ca2af53c3aafdab4beb16b HTTP/1.0" 202 Accepted 2026-04-01 07:49:21,906 - lava_callback - INFO - Starting async job submit for node 69ca2af53c3aafdab4beb16b 2026-04-01 07:49:21,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:49:21,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2af53c3aafdab4beb16b HTTP/1.1" 200 None 2026-04-01 07:49:21,923 - lava_callback - DEBUG - Processing node 69ca2af53c3aafdab4beb16b: baseline-arm64-baylibre 2026-04-01 07:49:21,923 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2af53c3aafdab4beb16b 2026-04-01 07:49:21,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:49:21,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2af53c3aafdab4beb16b HTTP/1.1" 200 None 2026-04-01 07:49:21,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:49:21,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:49:21,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:49:21,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2af53c3aafdab4beb16b HTTP/1.1" 200 None 2026-04-01 07:49:21,998 - lava_callback - ERROR - Error processing callback for node 69ca2af53c3aafdab4beb16b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47938 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:52:02,255 - lava_callback - INFO - Received callback for node 69ca2b8e3c3aafdab4beb180 INFO: 172.20.0.1:41832 - "POST /node/69ca2b8e3c3aafdab4beb180 HTTP/1.0" 202 Accepted 2026-04-01 07:52:02,261 - lava_callback - INFO - Starting async job submit for node 69ca2b8e3c3aafdab4beb180 2026-04-01 07:52:02,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:52:02,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2b8e3c3aafdab4beb180 HTTP/1.1" 200 None 2026-04-01 07:52:02,296 - lava_callback - DEBUG - Processing node 69ca2b8e3c3aafdab4beb180: baseline-arm64-baylibre 2026-04-01 07:52:02,296 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2b8e3c3aafdab4beb180 2026-04-01 07:52:02,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:52:02,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2b8e3c3aafdab4beb180 HTTP/1.1" 200 None 2026-04-01 07:52:02,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:52:02,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:52:02,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:52:02,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2b8e3c3aafdab4beb180 HTTP/1.1" 200 None 2026-04-01 07:52:02,354 - lava_callback - ERROR - Error processing callback for node 69ca2b8e3c3aafdab4beb180: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40312 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:53:02,060 - lava_callback - INFO - Received callback for node 69ca2bda3c3aafdab4beb18b INFO: 172.20.0.1:47456 - "POST /node/69ca2bda3c3aafdab4beb18b HTTP/1.0" 202 Accepted 2026-04-01 07:53:02,066 - lava_callback - INFO - Starting async job submit for node 69ca2bda3c3aafdab4beb18b 2026-04-01 07:53:02,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:53:02,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2bda3c3aafdab4beb18b HTTP/1.1" 200 None 2026-04-01 07:53:02,083 - lava_callback - DEBUG - Processing node 69ca2bda3c3aafdab4beb18b: baseline-arm64-baylibre 2026-04-01 07:53:02,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2bda3c3aafdab4beb18b 2026-04-01 07:53:02,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:53:02,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2bda3c3aafdab4beb18b HTTP/1.1" 200 None 2026-04-01 07:53:02,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:53:02,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:53:02,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:53:02,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2bda3c3aafdab4beb18b HTTP/1.1" 200 None 2026-04-01 07:53:02,145 - lava_callback - ERROR - Error processing callback for node 69ca2bda3c3aafdab4beb18b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36434 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:54:26,064 - lava_callback - INFO - Received callback for node 69ca2c2c3c3aafdab4beb19b INFO: 172.20.0.1:45930 - "POST /node/69ca2c2c3c3aafdab4beb19b HTTP/1.0" 202 Accepted 2026-04-01 07:54:26,070 - lava_callback - INFO - Starting async job submit for node 69ca2c2c3c3aafdab4beb19b 2026-04-01 07:54:26,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:54:26,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2c2c3c3aafdab4beb19b HTTP/1.1" 200 None 2026-04-01 07:54:26,086 - lava_callback - DEBUG - Processing node 69ca2c2c3c3aafdab4beb19b: baseline-arm64-clabbe 2026-04-01 07:54:26,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2c2c3c3aafdab4beb19b 2026-04-01 07:54:26,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:54:26,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2c2c3c3aafdab4beb19b HTTP/1.1" 200 None 2026-04-01 07:54:26,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:54:26,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:54:26,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:54:26,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2c2c3c3aafdab4beb19b HTTP/1.1" 200 None 2026-04-01 07:54:26,139 - lava_callback - ERROR - Error processing callback for node 69ca2c2c3c3aafdab4beb19b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44272 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:55:46,098 - lava_callback - INFO - Received callback for node 69ca2c783c3aafdab4beb1a3 INFO: 172.20.0.1:42746 - "POST /node/69ca2c783c3aafdab4beb1a3 HTTP/1.0" 202 Accepted 2026-04-01 07:55:46,104 - lava_callback - INFO - Starting async job submit for node 69ca2c783c3aafdab4beb1a3 2026-04-01 07:55:46,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:55:46,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2c783c3aafdab4beb1a3 HTTP/1.1" 200 None 2026-04-01 07:55:46,121 - lava_callback - DEBUG - Processing node 69ca2c783c3aafdab4beb1a3: baseline-arm64-clabbe 2026-04-01 07:55:46,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2c783c3aafdab4beb1a3 2026-04-01 07:55:46,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:55:46,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2c783c3aafdab4beb1a3 HTTP/1.1" 200 None 2026-04-01 07:55:46,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:55:46,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:55:46,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:55:46,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2c783c3aafdab4beb1a3 HTTP/1.1" 200 None 2026-04-01 07:55:46,178 - lava_callback - ERROR - Error processing callback for node 69ca2c783c3aafdab4beb1a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47718 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:57:06,123 - lava_callback - INFO - Received callback for node 69ca2cc93c3aafdab4beb1b2 INFO: 172.20.0.1:40590 - "POST /node/69ca2cc93c3aafdab4beb1b2 HTTP/1.0" 202 Accepted 2026-04-01 07:57:06,129 - lava_callback - INFO - Starting async job submit for node 69ca2cc93c3aafdab4beb1b2 2026-04-01 07:57:06,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:57:06,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2cc93c3aafdab4beb1b2 HTTP/1.1" 200 None 2026-04-01 07:57:06,154 - lava_callback - DEBUG - Processing node 69ca2cc93c3aafdab4beb1b2: baseline-arm64-clabbe 2026-04-01 07:57:06,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2cc93c3aafdab4beb1b2 2026-04-01 07:57:06,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:57:06,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2cc93c3aafdab4beb1b2 HTTP/1.1" 200 None 2026-04-01 07:57:06,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:57:06,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:57:06,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:57:06,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2cc93c3aafdab4beb1b2 HTTP/1.1" 200 None 2026-04-01 07:57:06,217 - lava_callback - ERROR - Error processing callback for node 69ca2cc93c3aafdab4beb1b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43760 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:58:26,165 - lava_callback - INFO - Received callback for node 69ca2d173c3aafdab4beb1c7 INFO: 172.20.0.1:37750 - "POST /node/69ca2d173c3aafdab4beb1c7 HTTP/1.0" 202 Accepted 2026-04-01 07:58:26,171 - lava_callback - INFO - Starting async job submit for node 69ca2d173c3aafdab4beb1c7 2026-04-01 07:58:26,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:58:26,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2d173c3aafdab4beb1c7 HTTP/1.1" 200 None 2026-04-01 07:58:26,193 - lava_callback - DEBUG - Processing node 69ca2d173c3aafdab4beb1c7: baseline-arm64-clabbe 2026-04-01 07:58:26,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2d173c3aafdab4beb1c7 2026-04-01 07:58:26,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:58:26,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2d173c3aafdab4beb1c7 HTTP/1.1" 200 None 2026-04-01 07:58:26,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:58:26,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:58:26,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:58:26,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2d173c3aafdab4beb1c7 HTTP/1.1" 200 None 2026-04-01 07:58:26,252 - lava_callback - ERROR - Error processing callback for node 69ca2d173c3aafdab4beb1c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47474 - "GET / HTTP/1.0" 200 OK 2026-04-01 07:59:46,278 - lava_callback - INFO - Received callback for node 69ca2d653c3aafdab4beb1ce INFO: 172.20.0.1:45414 - "POST /node/69ca2d653c3aafdab4beb1ce HTTP/1.0" 202 Accepted 2026-04-01 07:59:46,284 - lava_callback - INFO - Starting async job submit for node 69ca2d653c3aafdab4beb1ce 2026-04-01 07:59:46,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:59:46,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2d653c3aafdab4beb1ce HTTP/1.1" 200 None 2026-04-01 07:59:46,300 - lava_callback - DEBUG - Processing node 69ca2d653c3aafdab4beb1ce: baseline-arm64-clabbe 2026-04-01 07:59:46,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2d653c3aafdab4beb1ce 2026-04-01 07:59:46,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:59:46,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2d653c3aafdab4beb1ce HTTP/1.1" 200 None 2026-04-01 07:59:46,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:59:46,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 07:59:46,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 07:59:46,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2d653c3aafdab4beb1ce HTTP/1.1" 200 None 2026-04-01 07:59:46,359 - lava_callback - ERROR - Error processing callback for node 69ca2d653c3aafdab4beb1ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:01:06,349 - lava_callback - INFO - Received callback for node 69ca2db23c3aafdab4beb1e0 INFO: 172.20.0.1:39786 - "POST /node/69ca2db23c3aafdab4beb1e0 HTTP/1.0" 202 Accepted 2026-04-01 08:01:06,354 - lava_callback - INFO - Starting async job submit for node 69ca2db23c3aafdab4beb1e0 2026-04-01 08:01:06,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:01:06,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2db23c3aafdab4beb1e0 HTTP/1.1" 200 None 2026-04-01 08:01:06,371 - lava_callback - DEBUG - Processing node 69ca2db23c3aafdab4beb1e0: baseline-arm64-clabbe 2026-04-01 08:01:06,371 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2db23c3aafdab4beb1e0 2026-04-01 08:01:06,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:01:06,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2db23c3aafdab4beb1e0 HTTP/1.1" 200 None 2026-04-01 08:01:06,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:01:06,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 08:01:06,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:01:06,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2db23c3aafdab4beb1e0 HTTP/1.1" 200 None 2026-04-01 08:01:06,426 - lava_callback - ERROR - Error processing callback for node 69ca2db23c3aafdab4beb1e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:02:26,407 - lava_callback - INFO - Received callback for node 69ca2dff3c3aafdab4beb1ea INFO: 172.20.0.1:41130 - "POST /node/69ca2dff3c3aafdab4beb1ea HTTP/1.0" 202 Accepted 2026-04-01 08:02:26,413 - lava_callback - INFO - Starting async job submit for node 69ca2dff3c3aafdab4beb1ea 2026-04-01 08:02:26,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:02:26,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2dff3c3aafdab4beb1ea HTTP/1.1" 200 None 2026-04-01 08:02:26,433 - lava_callback - DEBUG - Processing node 69ca2dff3c3aafdab4beb1ea: baseline-arm64-clabbe 2026-04-01 08:02:26,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2dff3c3aafdab4beb1ea 2026-04-01 08:02:26,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:02:26,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2dff3c3aafdab4beb1ea HTTP/1.1" 200 None 2026-04-01 08:02:26,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:02:26,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 08:02:26,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:02:26,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2dff3c3aafdab4beb1ea HTTP/1.1" 200 None 2026-04-01 08:02:26,490 - lava_callback - ERROR - Error processing callback for node 69ca2dff3c3aafdab4beb1ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:03:46,452 - lava_callback - INFO - Received callback for node 69ca2e4c3c3aafdab4beb1f6 INFO: 172.20.0.1:52148 - "POST /node/69ca2e4c3c3aafdab4beb1f6 HTTP/1.0" 202 Accepted 2026-04-01 08:03:46,458 - lava_callback - INFO - Starting async job submit for node 69ca2e4c3c3aafdab4beb1f6 2026-04-01 08:03:46,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:03:46,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2e4c3c3aafdab4beb1f6 HTTP/1.1" 200 None 2026-04-01 08:03:46,479 - lava_callback - DEBUG - Processing node 69ca2e4c3c3aafdab4beb1f6: baseline-arm64-clabbe 2026-04-01 08:03:46,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2e4c3c3aafdab4beb1f6 2026-04-01 08:03:46,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:03:46,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2e4c3c3aafdab4beb1f6 HTTP/1.1" 200 None 2026-04-01 08:03:46,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:03:46,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 08:03:46,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:03:46,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2e4c3c3aafdab4beb1f6 HTTP/1.1" 200 None 2026-04-01 08:03:46,533 - lava_callback - ERROR - Error processing callback for node 69ca2e4c3c3aafdab4beb1f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40370 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:04:46,495 - lava_callback - INFO - Received callback for node 69ca2e983c3aafdab4beb205 INFO: 172.20.0.1:45322 - "POST /node/69ca2e983c3aafdab4beb205 HTTP/1.0" 202 Accepted 2026-04-01 08:04:46,501 - lava_callback - INFO - Starting async job submit for node 69ca2e983c3aafdab4beb205 2026-04-01 08:04:46,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:04:46,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2e983c3aafdab4beb205 HTTP/1.1" 200 None 2026-04-01 08:04:46,517 - lava_callback - DEBUG - Processing node 69ca2e983c3aafdab4beb205: baseline-arm64-clabbe 2026-04-01 08:04:46,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2e983c3aafdab4beb205 2026-04-01 08:04:46,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:04:46,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2e983c3aafdab4beb205 HTTP/1.1" 200 None 2026-04-01 08:04:46,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:04:46,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 08:04:46,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:45324 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:04:46,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2e983c3aafdab4beb205 HTTP/1.1" 200 None 2026-04-01 08:04:46,572 - lava_callback - ERROR - Error processing callback for node 69ca2e983c3aafdab4beb205: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55390 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:06:06,556 - lava_callback - INFO - Received callback for node 69ca2ee53c3aafdab4beb211 INFO: 172.20.0.1:55400 - "POST /node/69ca2ee53c3aafdab4beb211 HTTP/1.0" 202 Accepted 2026-04-01 08:06:06,561 - lava_callback - INFO - Starting async job submit for node 69ca2ee53c3aafdab4beb211 2026-04-01 08:06:06,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:06:06,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2ee53c3aafdab4beb211 HTTP/1.1" 200 None 2026-04-01 08:06:06,581 - lava_callback - DEBUG - Processing node 69ca2ee53c3aafdab4beb211: baseline-arm64-clabbe 2026-04-01 08:06:06,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2ee53c3aafdab4beb211 2026-04-01 08:06:06,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:06:06,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2ee53c3aafdab4beb211 HTTP/1.1" 200 None 2026-04-01 08:06:06,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:06:06,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 08:06:06,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:06:06,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2ee53c3aafdab4beb211 HTTP/1.1" 200 None 2026-04-01 08:06:06,640 - lava_callback - ERROR - Error processing callback for node 69ca2ee53c3aafdab4beb211: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:07:26,639 - lava_callback - INFO - Received callback for node 69ca2f323c3aafdab4beb224 INFO: 172.20.0.1:51244 - "POST /node/69ca2f323c3aafdab4beb224 HTTP/1.0" 202 Accepted 2026-04-01 08:07:26,645 - lava_callback - INFO - Starting async job submit for node 69ca2f323c3aafdab4beb224 2026-04-01 08:07:26,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:07:26,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2f323c3aafdab4beb224 HTTP/1.1" 200 None 2026-04-01 08:07:26,664 - lava_callback - DEBUG - Processing node 69ca2f323c3aafdab4beb224: baseline-arm64-clabbe 2026-04-01 08:07:26,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2f323c3aafdab4beb224 2026-04-01 08:07:26,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:07:26,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2f323c3aafdab4beb224 HTTP/1.1" 200 None 2026-04-01 08:07:26,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:07:26,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 08:07:26,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:07:26,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2f323c3aafdab4beb224 HTTP/1.1" 200 None 2026-04-01 08:07:26,723 - lava_callback - ERROR - Error processing callback for node 69ca2f323c3aafdab4beb224: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33150 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:08:46,829 - lava_callback - INFO - Received callback for node 69ca2f7f3c3aafdab4beb22b INFO: 172.20.0.1:33164 - "POST /node/69ca2f7f3c3aafdab4beb22b HTTP/1.0" 202 Accepted 2026-04-01 08:08:46,835 - lava_callback - INFO - Starting async job submit for node 69ca2f7f3c3aafdab4beb22b 2026-04-01 08:08:46,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:08:46,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2f7f3c3aafdab4beb22b HTTP/1.1" 200 None 2026-04-01 08:08:46,851 - lava_callback - DEBUG - Processing node 69ca2f7f3c3aafdab4beb22b: baseline-arm64-clabbe 2026-04-01 08:08:46,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2f7f3c3aafdab4beb22b 2026-04-01 08:08:46,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:08:46,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2f7f3c3aafdab4beb22b HTTP/1.1" 200 None 2026-04-01 08:08:46,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:08:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-01 08:08:46,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:08:46,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2f7f3c3aafdab4beb22b HTTP/1.1" 200 None 2026-04-01 08:08:46,901 - lava_callback - ERROR - Error processing callback for node 69ca2f7f3c3aafdab4beb22b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36324 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:10:03,550 - lava_callback - INFO - Received callback for node 69ca2fc63c3aafdab4beb23d INFO: 172.20.0.1:54080 - "POST /node/69ca2fc63c3aafdab4beb23d HTTP/1.0" 202 Accepted 2026-04-01 08:10:03,556 - lava_callback - INFO - Starting async job submit for node 69ca2fc63c3aafdab4beb23d 2026-04-01 08:10:03,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:10:03,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2fc63c3aafdab4beb23d HTTP/1.1" 200 None 2026-04-01 08:10:03,572 - lava_callback - DEBUG - Processing node 69ca2fc63c3aafdab4beb23d: baseline-arm64-baylibre 2026-04-01 08:10:03,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca2fc63c3aafdab4beb23d 2026-04-01 08:10:03,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:10:03,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca2fc63c3aafdab4beb23d HTTP/1.1" 200 None 2026-04-01 08:10:03,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:10:03,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 08:10:03,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:10:03,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca2fc63c3aafdab4beb23d HTTP/1.1" 200 None 2026-04-01 08:10:03,633 - lava_callback - ERROR - Error processing callback for node 69ca2fc63c3aafdab4beb23d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:11:06,789 - lava_callback - INFO - Received callback for node 69ca30183c3aafdab4beb24c INFO: 172.20.0.1:48724 - "POST /node/69ca30183c3aafdab4beb24c HTTP/1.0" 202 Accepted 2026-04-01 08:11:06,794 - lava_callback - INFO - Starting async job submit for node 69ca30183c3aafdab4beb24c 2026-04-01 08:11:06,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:11:06,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca30183c3aafdab4beb24c HTTP/1.1" 200 None 2026-04-01 08:11:06,810 - lava_callback - DEBUG - Processing node 69ca30183c3aafdab4beb24c: baseline-arm64-clabbe 2026-04-01 08:11:06,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca30183c3aafdab4beb24c 2026-04-01 08:11:06,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:11:06,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca30183c3aafdab4beb24c HTTP/1.1" 200 None 2026-04-01 08:11:06,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:11:06,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 08:11:06,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:11:06,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca30183c3aafdab4beb24c HTTP/1.1" 200 None 2026-04-01 08:11:06,873 - lava_callback - ERROR - Error processing callback for node 69ca30183c3aafdab4beb24c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37978 - "GET / HTTP/1.0" 200 OK 2026-04-01 08:12:26,822 - lava_callback - INFO - Received callback for node 69ca30663c3aafdab4beb259 INFO: 172.20.0.1:37994 - "POST /node/69ca30663c3aafdab4beb259 HTTP/1.0" 202 Accepted 2026-04-01 08:12:26,830 - lava_callback - INFO - Starting async job submit for node 69ca30663c3aafdab4beb259 2026-04-01 08:12:26,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:12:26,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca30663c3aafdab4beb259 HTTP/1.1" 200 None 2026-04-01 08:12:26,853 - lava_callback - DEBUG - Processing node 69ca30663c3aafdab4beb259: baseline-arm64-clabbe 2026-04-01 08:12:26,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ca30663c3aafdab4beb259 2026-04-01 08:12:26,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:12:26,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca30663c3aafdab4beb259 HTTP/1.1" 200 None 2026-04-01 08:12:26,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:12:26,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-01 08:12:26,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 08:12:26,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca30663c3aafdab4beb259 HTTP/1.1" 200 None 2026-04-01 08:12:26,930 - lava_callback - ERROR - Error processing callback for node 69ca30663c3aafdab4beb259: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]