INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:45892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:28:34,541 - lava_callback - INFO - Received callback for node 69ca33b10e33d030878a8837 2026-03-30 08:28:34,548 - lava_callback - INFO - Starting async job submit for node 69ca33b10e33d030878a8837 INFO: 172.21.0.1:41596 - "POST /node/69ca33b10e33d030878a8837 HTTP/1.0" 202 Accepted 2026-03-30 08:28:34,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:28:34,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca33b10e33d030878a8837 HTTP/1.1" 200 None 2026-03-30 08:28:34,770 - lava_callback - DEBUG - Processing node 69ca33b10e33d030878a8837: baseline-arm 2026-03-30 08:28:34,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:28:35,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:28:35,205 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca33b10e33d030878a8837/log.txt.gz 2026-03-30 08:28:35,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:28:35,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:28:35,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca33b10e33d030878a8837/lava-logs.yaml 2026-03-30 08:28:35,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:28:35,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:28:35,729 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca33b10e33d030878a8837/lava_callback.json.gz 2026-03-30 08:28:35,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:28:35,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca33b10e33d030878a8837 HTTP/1.1" 200 None 2026-03-30 08:28:35,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:28:35,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31f70e33d030878a87f5 HTTP/1.1" 200 None 2026-03-30 08:28:35,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:28:35,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca33b10e33d030878a8837 HTTP/1.1" 200 None 2026-03-30 08:28:35,844 - lava_callback - INFO - Completed processing callback for node 69ca33b10e33d030878a8837 INFO: 172.21.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56220 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:29:05,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:29:05,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49988 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:39:06,322 - lava_callback - INFO - Received callback for node 69ca36930e33d030878a8858 INFO: 172.21.0.1:52092 - "POST /node/69ca36930e33d030878a8858 HTTP/1.0" 202 Accepted 2026-03-30 08:39:06,328 - lava_callback - INFO - Starting async job submit for node 69ca36930e33d030878a8858 2026-03-30 08:39:06,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:06,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36930e33d030878a8858 HTTP/1.1" 200 None 2026-03-30 08:39:06,361 - lava_callback - DEBUG - Processing node 69ca36930e33d030878a8858: baseline-x86 2026-03-30 08:39:06,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:52096 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:39:06,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:06,678 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36930e33d030878a8858/log.txt.gz 2026-03-30 08:39:06,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:06,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:06,971 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36930e33d030878a8858/lava-logs.yaml 2026-03-30 08:39:06,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:07,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:07,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36930e33d030878a8858/lava_callback.json.gz 2026-03-30 08:39:07,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:07,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36930e33d030878a8858 HTTP/1.1" 200 None 2026-03-30 08:39:07,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:07,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca32150e33d030878a8819 HTTP/1.1" 200 None 2026-03-30 08:39:07,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:07,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca36930e33d030878a8858 HTTP/1.1" 200 None 2026-03-30 08:39:07,215 - lava_callback - INFO - Completed processing callback for node 69ca36930e33d030878a8858 2026-03-30 08:39:10,590 - lava_callback - INFO - Received callback for node 69ca36950e33d030878a8862 INFO: 172.21.0.1:54562 - "POST /node/69ca36950e33d030878a8862 HTTP/1.0" 202 Accepted 2026-03-30 08:39:10,596 - lava_callback - INFO - Starting async job submit for node 69ca36950e33d030878a8862 2026-03-30 08:39:10,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:10,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36950e33d030878a8862 HTTP/1.1" 200 None 2026-03-30 08:39:10,634 - lava_callback - DEBUG - Processing node 69ca36950e33d030878a8862: baseline-x86 2026-03-30 08:39:10,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:10,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:10,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36950e33d030878a8862/log.txt.gz 2026-03-30 08:39:11,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:11,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:11,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36950e33d030878a8862/lava-logs.yaml 2026-03-30 08:39:11,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:11,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:11,515 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36950e33d030878a8862/lava_callback.json.gz 2026-03-30 08:39:11,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36950e33d030878a8862 HTTP/1.1" 200 None 2026-03-30 08:39:11,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:11,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca32150e33d030878a8819 HTTP/1.1" 200 None 2026-03-30 08:39:11,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:11,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca36950e33d030878a8862 HTTP/1.1" 200 None 2026-03-30 08:39:11,602 - lava_callback - INFO - Completed processing callback for node 69ca36950e33d030878a8862 INFO: 172.21.0.1:54572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:39:25,098 - lava_callback - INFO - Received callback for node 69c793b3be430e18c5d1603e INFO: 172.21.0.1:33300 - "POST /node/69c793b3be430e18c5d1603e HTTP/1.0" 202 Accepted 2026-03-30 08:39:25,104 - lava_callback - INFO - Starting async job submit for node 69c793b3be430e18c5d1603e 2026-03-30 08:39:25,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b3be430e18c5d1603e HTTP/1.1" 200 None 2026-03-30 08:39:25,120 - lava_callback - DEBUG - Processing node 69c793b3be430e18c5d1603e: baseline-arm-clabbe 2026-03-30 08:39:25,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c793b3be430e18c5d1603e 2026-03-30 08:39:25,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b3be430e18c5d1603e HTTP/1.1" 200 None 2026-03-30 08:39:25,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 08:39:25,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b3be430e18c5d1603e HTTP/1.1" 200 None 2026-03-30 08:39:25,183 - lava_callback - ERROR - Error processing callback for node 69c793b3be430e18c5d1603e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:39:25,765 - lava_callback - INFO - Received callback for node 69c793b7be430e18c5d1604b INFO: 172.21.0.1:33312 - "POST /node/69c793b7be430e18c5d1604b HTTP/1.0" 202 Accepted 2026-03-30 08:39:25,770 - lava_callback - INFO - Starting async job submit for node 69c793b7be430e18c5d1604b 2026-03-30 08:39:25,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b7be430e18c5d1604b HTTP/1.1" 200 None 2026-03-30 08:39:25,793 - lava_callback - DEBUG - Processing node 69c793b7be430e18c5d1604b: baseline-arm-clabbe 2026-03-30 08:39:25,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c793b7be430e18c5d1604b 2026-03-30 08:39:25,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b7be430e18c5d1604b HTTP/1.1" 200 None 2026-03-30 08:39:25,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 08:39:25,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:25,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b7be430e18c5d1604b HTTP/1.1" 200 None 2026-03-30 08:39:25,845 - lava_callback - ERROR - Error processing callback for node 69c793b7be430e18c5d1604b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:39:26,442 - lava_callback - INFO - Received callback for node 69c793b8be430e18c5d1604e INFO: 172.21.0.1:33316 - "POST /node/69c793b8be430e18c5d1604e HTTP/1.0" 202 Accepted 2026-03-30 08:39:26,449 - lava_callback - INFO - Starting async job submit for node 69c793b8be430e18c5d1604e 2026-03-30 08:39:26,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:26,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b8be430e18c5d1604e HTTP/1.1" 200 None 2026-03-30 08:39:26,467 - lava_callback - DEBUG - Processing node 69c793b8be430e18c5d1604e: baseline-arm-clabbe 2026-03-30 08:39:26,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c793b8be430e18c5d1604e 2026-03-30 08:39:26,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:26,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b8be430e18c5d1604e HTTP/1.1" 200 None 2026-03-30 08:39:26,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:26,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 08:39:26,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:39:26,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b8be430e18c5d1604e HTTP/1.1" 200 None 2026-03-30 08:39:26,536 - lava_callback - ERROR - Error processing callback for node 69c793b8be430e18c5d1604e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:39:27,122 - lava_callback - INFO - Received callback for node 69c793b9be430e18c5d16051 INFO: 172.21.0.1:33342 - "POST /node/69c793b9be430e18c5d16051 HTTP/1.0" 202 Accepted 2026-03-30 08:39:27,128 - lava_callback - INFO - Starting async job submit for node 69c793b9be430e18c5d16051 2026-03-30 08:39:27,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:27,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b9be430e18c5d16051 HTTP/1.1" 200 None 2026-03-30 08:39:27,144 - lava_callback - DEBUG - Processing node 69c793b9be430e18c5d16051: baseline-arm-clabbe 2026-03-30 08:39:27,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c793b9be430e18c5d16051 2026-03-30 08:39:27,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:27,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b9be430e18c5d16051 HTTP/1.1" 200 None 2026-03-30 08:39:27,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:27,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 08:39:27,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:27,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b9be430e18c5d16051 HTTP/1.1" 200 None 2026-03-30 08:39:27,221 - lava_callback - ERROR - Error processing callback for node 69c793b9be430e18c5d16051: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:39:35,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:35,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53040 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:39:45,067 - lava_callback - INFO - Received callback for node 69c793bcbe430e18c5d1605c INFO: 172.21.0.1:59966 - "POST /node/69c793bcbe430e18c5d1605c HTTP/1.0" 202 Accepted 2026-03-30 08:39:45,073 - lava_callback - INFO - Starting async job submit for node 69c793bcbe430e18c5d1605c 2026-03-30 08:39:45,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:45,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793bcbe430e18c5d1605c HTTP/1.1" 200 None 2026-03-30 08:39:45,092 - lava_callback - DEBUG - Processing node 69c793bcbe430e18c5d1605c: baseline-arm-clabbe 2026-03-30 08:39:45,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c793bcbe430e18c5d1605c 2026-03-30 08:39:45,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:45,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793bcbe430e18c5d1605c HTTP/1.1" 200 None 2026-03-30 08:39:45,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:45,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 08:39:45,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:45,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793bcbe430e18c5d1605c HTTP/1.1" 200 None 2026-03-30 08:39:45,160 - lava_callback - ERROR - Error processing callback for node 69c793bcbe430e18c5d1605c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53460 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:39:57,512 - lava_callback - INFO - Received callback for node 69ca36940e33d030878a885c INFO: 172.21.0.1:53476 - "POST /node/69ca36940e33d030878a885c HTTP/1.0" 202 Accepted 2026-03-30 08:39:57,518 - lava_callback - INFO - Starting async job submit for node 69ca36940e33d030878a885c 2026-03-30 08:39:57,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:57,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36940e33d030878a885c HTTP/1.1" 200 None 2026-03-30 08:39:57,551 - lava_callback - DEBUG - Processing node 69ca36940e33d030878a885c: baseline-x86 2026-03-30 08:39:57,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:57,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:57,978 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36940e33d030878a885c/log.txt.gz 2026-03-30 08:39:58,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:58,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:58,337 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36940e33d030878a885c/lava-logs.yaml 2026-03-30 08:39:58,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:39:58,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:39:58,470 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36940e33d030878a885c/lava_callback.json.gz 2026-03-30 08:39:58,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:58,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36940e33d030878a885c HTTP/1.1" 200 None 2026-03-30 08:39:58,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:58,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca32150e33d030878a8819 HTTP/1.1" 200 None 2026-03-30 08:39:58,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:39:58,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca36940e33d030878a885c HTTP/1.1" 200 None 2026-03-30 08:39:58,555 - lava_callback - INFO - Completed processing callback for node 69ca36940e33d030878a885c 2026-03-30 08:40:05,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:40:05,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 08:40:15,244 - lava_callback - INFO - Received callback for node 69ca36950e33d030878a885f INFO: 172.21.0.1:52818 - "POST /node/69ca36950e33d030878a885f HTTP/1.0" 202 Accepted 2026-03-30 08:40:15,298 - lava_callback - INFO - Starting async job submit for node 69ca36950e33d030878a885f 2026-03-30 08:40:15,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:40:15,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36950e33d030878a885f HTTP/1.1" 200 None 2026-03-30 08:40:15,347 - lava_callback - DEBUG - Processing node 69ca36950e33d030878a885f: baseline-x86 2026-03-30 08:40:15,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:40:15,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:40:15,890 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36950e33d030878a885f/log.txt.gz 2026-03-30 08:40:16,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:40:16,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:40:16,264 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36950e33d030878a885f/lava-logs.yaml 2026-03-30 08:40:16,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:40:16,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:40:16,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ca36950e33d030878a885f/lava_callback.json.gz 2026-03-30 08:40:16,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:40:16,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca36950e33d030878a885f HTTP/1.1" 200 None 2026-03-30 08:40:16,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:40:16,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca32150e33d030878a8819 HTTP/1.1" 200 None 2026-03-30 08:40:16,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:40:16,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca36950e33d030878a885f HTTP/1.1" 200 None 2026-03-30 08:40:16,490 - lava_callback - INFO - Completed processing callback for node 69ca36950e33d030878a885f INFO: 172.21.0.1:52828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:40:35,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:40:35,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57878 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:42:35,333 - lava_callback - INFO - Received callback for node 69ca37680e33d030878a88ea INFO: 172.21.0.1:52938 - "POST /node/69ca37680e33d030878a88ea HTTP/1.0" 202 Accepted 2026-03-30 08:42:35,342 - lava_callback - INFO - Starting async job submit for node 69ca37680e33d030878a88ea 2026-03-30 08:42:35,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:35,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37680e33d030878a88ea HTTP/1.1" 200 None 2026-03-30 08:42:35,371 - lava_callback - DEBUG - Processing node 69ca37680e33d030878a88ea: baseline-arm 2026-03-30 08:42:35,379 - lava_callback - WARNING - Failed to upload log 2026-03-30 08:42:35,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:42:35,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:42:35,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37680e33d030878a88ea/lava-logs.yaml 2026-03-30 08:42:35,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:42:35,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:42:35,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37680e33d030878a88ea/lava_callback.json.gz 2026-03-30 08:42:35,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:35,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37680e33d030878a88ea HTTP/1.1" 200 None 2026-03-30 08:42:35,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:35,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:42:35,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:35,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37680e33d030878a88ea HTTP/1.1" 200 None 2026-03-30 08:42:35,764 - lava_callback - INFO - Completed processing callback for node 69ca37680e33d030878a88ea 2026-03-30 08:42:35,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:35,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32910 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:42:53,391 - lava_callback - INFO - Received callback for node 69ca376b0e33d030878a88f6 INFO: 172.21.0.1:34120 - "POST /node/69ca376b0e33d030878a88f6 HTTP/1.0" 202 Accepted 2026-03-30 08:42:53,437 - lava_callback - INFO - Starting async job submit for node 69ca376b0e33d030878a88f6 2026-03-30 08:42:53,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:53,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca376b0e33d030878a88f6 HTTP/1.1" 200 None 2026-03-30 08:42:53,477 - lava_callback - DEBUG - Processing node 69ca376b0e33d030878a88f6: baseline-arm 2026-03-30 08:42:53,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:42:53,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:42:53,764 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca376b0e33d030878a88f6/log.txt.gz 2026-03-30 08:42:53,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:42:53,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:42:53,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca376b0e33d030878a88f6/lava-logs.yaml 2026-03-30 08:42:53,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:42:54,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:42:54,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca376b0e33d030878a88f6/lava_callback.json.gz 2026-03-30 08:42:54,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:54,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca376b0e33d030878a88f6 HTTP/1.1" 200 None 2026-03-30 08:42:54,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:54,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:42:54,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:42:54,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca376b0e33d030878a88f6 HTTP/1.1" 200 None 2026-03-30 08:42:54,194 - lava_callback - INFO - Completed processing callback for node 69ca376b0e33d030878a88f6 INFO: 172.21.0.1:34130 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:43:05,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:05,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53912 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:43:10,590 - lava_callback - INFO - Received callback for node 69ca37690e33d030878a88ed INFO: 172.21.0.1:53710 - "POST /node/69ca37690e33d030878a88ed HTTP/1.0" 202 Accepted 2026-03-30 08:43:10,638 - lava_callback - INFO - Starting async job submit for node 69ca37690e33d030878a88ed 2026-03-30 08:43:10,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:10,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37690e33d030878a88ed HTTP/1.1" 200 None 2026-03-30 08:43:10,698 - lava_callback - DEBUG - Processing node 69ca37690e33d030878a88ed: baseline-arm 2026-03-30 08:43:10,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:11,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:11,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37690e33d030878a88ed/log.txt.gz 2026-03-30 08:43:11,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:11,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:11,299 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37690e33d030878a88ed/lava-logs.yaml 2026-03-30 08:43:11,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:11,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:11,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37690e33d030878a88ed/lava_callback.json.gz 2026-03-30 08:43:11,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:11,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37690e33d030878a88ed HTTP/1.1" 200 None 2026-03-30 08:43:11,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:11,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:43:11,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:11,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37690e33d030878a88ed HTTP/1.1" 200 None 2026-03-30 08:43:11,503 - lava_callback - INFO - Completed processing callback for node 69ca37690e33d030878a88ed INFO: 172.21.0.1:53718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58758 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:43:35,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:35,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42834 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:43:54,524 - lava_callback - INFO - Received callback for node 69ca37670e33d030878a88e7 INFO: 172.21.0.1:51120 - "POST /node/69ca37670e33d030878a88e7 HTTP/1.0" 202 Accepted 2026-03-30 08:43:54,529 - lava_callback - INFO - Starting async job submit for node 69ca37670e33d030878a88e7 2026-03-30 08:43:54,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:54,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37670e33d030878a88e7 HTTP/1.1" 200 None 2026-03-30 08:43:54,565 - lava_callback - DEBUG - Processing node 69ca37670e33d030878a88e7: baseline-arm 2026-03-30 08:43:54,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:54,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:54,896 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37670e33d030878a88e7/log.txt.gz 2026-03-30 08:43:55,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:55,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:55,235 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37670e33d030878a88e7/lava-logs.yaml 2026-03-30 08:43:55,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:55,475 - lava_callback - INFO - Received callback for node 69ca37760e33d030878a890d 2026-03-30 08:43:55,483 - lava_callback - INFO - Starting async job submit for node 69ca37760e33d030878a890d INFO: 172.21.0.1:51122 - "POST /node/69ca37760e33d030878a890d HTTP/1.0" 202 Accepted 2026-03-30 08:43:55,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:55,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37760e33d030878a890d HTTP/1.1" 200 None 2026-03-30 08:43:55,529 - lava_callback - DEBUG - Processing node 69ca37760e33d030878a890d: baseline-arm-broonie 2026-03-30 08:43:55,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:55,637 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37670e33d030878a88e7/lava_callback.json.gz 2026-03-30 08:43:55,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:55,755 - lava_callback - INFO - Received callback for node 69ca378b0e33d030878a8978 2026-03-30 08:43:55,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:55,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37670e33d030878a88e7 HTTP/1.1" 200 None 2026-03-30 08:43:55,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:55,796 - lava_callback - INFO - Starting async job submit for node 69ca378b0e33d030878a8978 INFO: 172.21.0.1:51132 - "POST /node/69ca378b0e33d030878a8978 HTTP/1.0" 202 Accepted 2026-03-30 08:43:55,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:43:55,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:55,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:55,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca378b0e33d030878a8978 HTTP/1.1" 200 None 2026-03-30 08:43:55,855 - lava_callback - DEBUG - Processing node 69ca378b0e33d030878a8978: baseline-arm-pengutronix 2026-03-30 08:43:55,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37670e33d030878a88e7 HTTP/1.1" 200 None 2026-03-30 08:43:55,882 - lava_callback - INFO - Completed processing callback for node 69ca37670e33d030878a88e7 2026-03-30 08:43:55,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:56,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37760e33d030878a890d/log.txt.gz 2026-03-30 08:43:56,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:56,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:56,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:56,322 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca378b0e33d030878a8978/log.txt.gz 2026-03-30 08:43:56,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:56,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:56,452 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37760e33d030878a890d/lava-logs.yaml 2026-03-30 08:43:56,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:51144 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:43:56,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:56,563 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca378b0e33d030878a8978/lava-logs.yaml 2026-03-30 08:43:56,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:56,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:56,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37760e33d030878a890d/lava_callback.json.gz 2026-03-30 08:43:56,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:56,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:56,708 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca378b0e33d030878a8978/lava_callback.json.gz 2026-03-30 08:43:56,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:56,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37760e33d030878a890d HTTP/1.1" 200 None 2026-03-30 08:43:56,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca378b0e33d030878a8978 HTTP/1.1" 200 None 2026-03-30 08:43:56,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:56,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:56,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:43:56,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:43:56,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:56,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:56,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37760e33d030878a890d HTTP/1.1" 200 None 2026-03-30 08:43:56,804 - lava_callback - INFO - Completed processing callback for node 69ca37760e33d030878a890d 2026-03-30 08:43:56,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca378b0e33d030878a8978 HTTP/1.1" 200 None 2026-03-30 08:43:56,807 - lava_callback - INFO - Completed processing callback for node 69ca378b0e33d030878a8978 2026-03-30 08:43:59,632 - lava_callback - INFO - Received callback for node 69ca37880e33d030878a896f INFO: 172.21.0.1:51156 - "POST /node/69ca37880e33d030878a896f HTTP/1.0" 202 Accepted 2026-03-30 08:43:59,638 - lava_callback - INFO - Starting async job submit for node 69ca37880e33d030878a896f 2026-03-30 08:43:59,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:43:59,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37880e33d030878a896f HTTP/1.1" 200 None 2026-03-30 08:43:59,677 - lava_callback - DEBUG - Processing node 69ca37880e33d030878a896f: baseline-arm-pengutronix 2026-03-30 08:43:59,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:43:59,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:43:59,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37880e33d030878a896f/log.txt.gz 2026-03-30 08:44:00,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:00,238 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:00,240 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37880e33d030878a896f/lava-logs.yaml 2026-03-30 08:44:00,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:00,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:00,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37880e33d030878a896f/lava_callback.json.gz 2026-03-30 08:44:00,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:00,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37880e33d030878a896f HTTP/1.1" 200 None 2026-03-30 08:44:00,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:00,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:44:00,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:00,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37880e33d030878a896f HTTP/1.1" 200 None 2026-03-30 08:44:00,460 - lava_callback - INFO - Completed processing callback for node 69ca37880e33d030878a896f 2026-03-30 08:44:00,999 - lava_callback - INFO - Received callback for node 69ca37870e33d030878a896c INFO: 172.21.0.1:50276 - "POST /node/69ca37870e33d030878a896c HTTP/1.0" 202 Accepted 2026-03-30 08:44:01,005 - lava_callback - INFO - Starting async job submit for node 69ca37870e33d030878a896c 2026-03-30 08:44:01,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:01,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37870e33d030878a896c HTTP/1.1" 200 None 2026-03-30 08:44:01,044 - lava_callback - DEBUG - Processing node 69ca37870e33d030878a896c: baseline-arm-pengutronix 2026-03-30 08:44:01,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:01,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:01,342 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37870e33d030878a896c/log.txt.gz 2026-03-30 08:44:01,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:01,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:01,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37870e33d030878a896c/lava-logs.yaml 2026-03-30 08:44:01,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:01,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:01,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37870e33d030878a896c/lava_callback.json.gz 2026-03-30 08:44:01,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:01,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37870e33d030878a896c HTTP/1.1" 200 None 2026-03-30 08:44:01,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:01,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:44:01,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:01,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37870e33d030878a896c HTTP/1.1" 200 None 2026-03-30 08:44:01,850 - lava_callback - INFO - Completed processing callback for node 69ca37870e33d030878a896c 2026-03-30 08:44:03,024 - lava_callback - INFO - Received callback for node 69ca37790e33d030878a8913 INFO: 172.21.0.1:50282 - "POST /node/69ca37790e33d030878a8913 HTTP/1.0" 202 Accepted 2026-03-30 08:44:03,030 - lava_callback - INFO - Starting async job submit for node 69ca37790e33d030878a8913 2026-03-30 08:44:03,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:03,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37790e33d030878a8913 HTTP/1.1" 200 None 2026-03-30 08:44:03,068 - lava_callback - DEBUG - Processing node 69ca37790e33d030878a8913: baseline-arm-broonie 2026-03-30 08:44:03,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:03,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:03,428 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37790e33d030878a8913/log.txt.gz 2026-03-30 08:44:03,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:03,669 - lava_callback - INFO - Received callback for node 69ca377d0e33d030878a8925 INFO: 172.21.0.1:50286 - "POST /node/69ca377d0e33d030878a8925 HTTP/1.0" 202 Accepted 2026-03-30 08:44:03,676 - lava_callback - INFO - Starting async job submit for node 69ca377d0e33d030878a8925 2026-03-30 08:44:03,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:03,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca377d0e33d030878a8925 HTTP/1.1" 200 None 2026-03-30 08:44:03,721 - lava_callback - DEBUG - Processing node 69ca377d0e33d030878a8925: baseline-arm-clabbe 2026-03-30 08:44:03,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:03,831 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37790e33d030878a8913/lava-logs.yaml 2026-03-30 08:44:03,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:03,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:04,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:04,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca377d0e33d030878a8925/log.txt.gz 2026-03-30 08:44:04,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:04,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37790e33d030878a8913/lava_callback.json.gz 2026-03-30 08:44:04,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:04,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37790e33d030878a8913 HTTP/1.1" 200 None 2026-03-30 08:44:04,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:04,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:04,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:44:04,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:04,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37790e33d030878a8913 HTTP/1.1" 200 None 2026-03-30 08:44:04,303 - lava_callback - INFO - Completed processing callback for node 69ca37790e33d030878a8913 2026-03-30 08:44:04,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:04,511 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca377d0e33d030878a8925/lava-logs.yaml 2026-03-30 08:44:04,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:04,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:04,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca377d0e33d030878a8925/lava_callback.json.gz 2026-03-30 08:44:04,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:04,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca377d0e33d030878a8925 HTTP/1.1" 200 None 2026-03-30 08:44:04,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:04,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:44:04,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:04,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca377d0e33d030878a8925 HTTP/1.1" 200 None 2026-03-30 08:44:04,764 - lava_callback - INFO - Completed processing callback for node 69ca377d0e33d030878a8925 2026-03-30 08:44:05,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:06,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:50296 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:44:06,625 - lava_callback - INFO - Received callback for node 69ca376a0e33d030878a88f3 INFO: 172.21.0.1:50312 - "POST /node/69ca376a0e33d030878a88f3 HTTP/1.0" 202 Accepted 2026-03-30 08:44:06,631 - lava_callback - INFO - Starting async job submit for node 69ca376a0e33d030878a88f3 2026-03-30 08:44:06,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:06,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca376a0e33d030878a88f3 HTTP/1.1" 200 None 2026-03-30 08:44:06,666 - lava_callback - DEBUG - Processing node 69ca376a0e33d030878a88f3: baseline-arm 2026-03-30 08:44:06,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:07,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:07,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca376a0e33d030878a88f3/log.txt.gz 2026-03-30 08:44:07,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:07,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:07,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca376a0e33d030878a88f3/lava-logs.yaml 2026-03-30 08:44:07,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:07,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:07,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca376a0e33d030878a88f3/lava_callback.json.gz 2026-03-30 08:44:07,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:07,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca376a0e33d030878a88f3 HTTP/1.1" 200 None 2026-03-30 08:44:07,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:07,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:44:07,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:07,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca376a0e33d030878a88f3 HTTP/1.1" 200 None 2026-03-30 08:44:07,546 - lava_callback - INFO - Completed processing callback for node 69ca376a0e33d030878a88f3 2026-03-30 08:44:14,843 - lava_callback - INFO - Received callback for node 69ca37780e33d030878a8910 INFO: 172.21.0.1:49164 - "POST /node/69ca37780e33d030878a8910 HTTP/1.0" 202 Accepted 2026-03-30 08:44:14,849 - lava_callback - INFO - Starting async job submit for node 69ca37780e33d030878a8910 2026-03-30 08:44:14,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:14,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37780e33d030878a8910 HTTP/1.1" 200 None 2026-03-30 08:44:14,887 - lava_callback - DEBUG - Processing node 69ca37780e33d030878a8910: baseline-arm-broonie 2026-03-30 08:44:15,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:49176 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 08:44:15,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:15,261 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37780e33d030878a8910/log.txt.gz 2026-03-30 08:44:15,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:15,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:15,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37780e33d030878a8910/lava-logs.yaml 2026-03-30 08:44:15,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:44:15,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:44:15,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37780e33d030878a8910/lava_callback.json.gz 2026-03-30 08:44:15,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:15,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37780e33d030878a8910 HTTP/1.1" 200 None 2026-03-30 08:44:15,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:15,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:44:15,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:15,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37780e33d030878a8910 HTTP/1.1" 200 None 2026-03-30 08:44:15,875 - lava_callback - INFO - Completed processing callback for node 69ca37780e33d030878a8910 INFO: 172.21.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54140 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:44:36,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:44:36,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46134 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:45:07,619 - lava_callback - INFO - Received callback for node 69ca37a20e33d030878a89af INFO: 172.21.0.1:46146 - "POST /node/69ca37a20e33d030878a89af HTTP/1.0" 202 Accepted 2026-03-30 08:45:07,629 - lava_callback - INFO - Starting async job submit for node 69ca37a20e33d030878a89af 2026-03-30 08:45:07,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:07,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a20e33d030878a89af HTTP/1.1" 200 None 2026-03-30 08:45:07,690 - lava_callback - DEBUG - Processing node 69ca37a20e33d030878a89af: ltp-dio 2026-03-30 08:45:08,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:08,545 - lava_callback - INFO - Received callback for node 69c79500be430e18c5d161fd INFO: 172.21.0.1:46152 - "POST /node/69c79500be430e18c5d161fd HTTP/1.0" 202 Accepted 2026-03-30 08:45:08,552 - lava_callback - INFO - Starting async job submit for node 69c79500be430e18c5d161fd 2026-03-30 08:45:08,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:08,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79500be430e18c5d161fd HTTP/1.1" 200 None 2026-03-30 08:45:08,571 - lava_callback - DEBUG - Processing node 69c79500be430e18c5d161fd: baseline-arm-baylibre 2026-03-30 08:45:08,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79500be430e18c5d161fd 2026-03-30 08:45:08,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:08,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79500be430e18c5d161fd HTTP/1.1" 200 None 2026-03-30 08:45:08,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:08,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 08:45:08,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:08,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79500be430e18c5d161fd HTTP/1.1" 200 None 2026-03-30 08:45:08,649 - lava_callback - ERROR - Error processing callback for node 69c79500be430e18c5d161fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:45:09,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:09,424 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69ca37a20e33d030878a89af/log.txt.gz 2026-03-30 08:45:09,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:10,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:10,820 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69ca37a20e33d030878a89af/lava-logs.yaml 2026-03-30 08:45:10,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:11,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:11,779 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69ca37a20e33d030878a89af/lava_callback.json.gz 2026-03-30 08:45:11,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:11,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a20e33d030878a89af HTTP/1.1" 200 None 2026-03-30 08:45:11,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:11,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:45:11,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:11,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37a20e33d030878a89af HTTP/1.1" 200 None 2026-03-30 08:45:11,898 - lava_callback - INFO - Completed processing callback for node 69ca37a20e33d030878a89af INFO: 172.21.0.1:51076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59706 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:45:30,753 - lava_callback - INFO - Received callback for node 69ca377e0e33d030878a8929 INFO: 172.21.0.1:51964 - "POST /node/69ca377e0e33d030878a8929 HTTP/1.0" 202 Accepted 2026-03-30 08:45:30,761 - lava_callback - INFO - Starting async job submit for node 69ca377e0e33d030878a8929 2026-03-30 08:45:30,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:30,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca377e0e33d030878a8929 HTTP/1.1" 200 None 2026-03-30 08:45:30,835 - lava_callback - DEBUG - Processing node 69ca377e0e33d030878a8929: baseline-arm-clabbe 2026-03-30 08:45:31,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:32,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:32,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca377e0e33d030878a8929/log.txt.gz 2026-03-30 08:45:32,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:33,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:33,368 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca377e0e33d030878a8929/lava-logs.yaml 2026-03-30 08:45:33,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:34,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:34,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca377e0e33d030878a8929/lava_callback.json.gz 2026-03-30 08:45:34,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:34,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca377e0e33d030878a8929 HTTP/1.1" 200 None 2026-03-30 08:45:34,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:34,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:45:34,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:34,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca377e0e33d030878a8929 HTTP/1.1" 200 None 2026-03-30 08:45:34,370 - lava_callback - INFO - Completed processing callback for node 69ca377e0e33d030878a8929 2026-03-30 08:45:36,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:36,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51978 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:45:45,433 - lava_callback - INFO - Received callback for node 69c7952bbe430e18c5d1620c INFO: 172.21.0.1:58912 - "POST /node/69c7952bbe430e18c5d1620c HTTP/1.0" 202 Accepted 2026-03-30 08:45:45,442 - lava_callback - INFO - Starting async job submit for node 69c7952bbe430e18c5d1620c 2026-03-30 08:45:45,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:45,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7952bbe430e18c5d1620c HTTP/1.1" 200 None 2026-03-30 08:45:45,467 - lava_callback - DEBUG - Processing node 69c7952bbe430e18c5d1620c: baseline-arm-clabbe 2026-03-30 08:45:45,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7952bbe430e18c5d1620c 2026-03-30 08:45:45,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:45,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7952bbe430e18c5d1620c HTTP/1.1" 200 None 2026-03-30 08:45:45,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:45,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 08:45:45,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:45,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7952bbe430e18c5d1620c HTTP/1.1" 200 None 2026-03-30 08:45:45,546 - lava_callback - ERROR - Error processing callback for node 69c7952bbe430e18c5d1620c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:45:56,755 - lava_callback - INFO - Received callback for node 69ca383b0e33d030878a8a5f INFO: 172.21.0.1:47576 - "POST /node/69ca383b0e33d030878a8a5f HTTP/1.0" 202 Accepted 2026-03-30 08:45:56,763 - lava_callback - INFO - Starting async job submit for node 69ca383b0e33d030878a8a5f 2026-03-30 08:45:56,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:56,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca383b0e33d030878a8a5f HTTP/1.1" 200 None 2026-03-30 08:45:56,793 - lava_callback - DEBUG - Processing node 69ca383b0e33d030878a8a5f: baseline-arm 2026-03-30 08:45:56,801 - lava_callback - WARNING - Failed to upload log 2026-03-30 08:45:56,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:57,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:57,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca383b0e33d030878a8a5f/lava-logs.yaml 2026-03-30 08:45:57,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:45:58,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:45:58,867 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca383b0e33d030878a8a5f/lava_callback.json.gz 2026-03-30 08:45:58,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:58,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca383b0e33d030878a8a5f HTTP/1.1" 200 None 2026-03-30 08:45:58,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:58,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:45:58,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:45:58,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca383b0e33d030878a8a5f HTTP/1.1" 200 None 2026-03-30 08:45:59,000 - lava_callback - INFO - Completed processing callback for node 69ca383b0e33d030878a8a5f 2026-03-30 08:46:06,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:06,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:46:15,084 - lava_callback - INFO - Received callback for node 69ca33b20e33d030878a883a INFO: 172.21.0.1:53680 - "POST /node/69ca33b20e33d030878a883a HTTP/1.0" 202 Accepted 2026-03-30 08:46:15,094 - lava_callback - INFO - Starting async job submit for node 69ca33b20e33d030878a883a 2026-03-30 08:46:15,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:53696 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 08:46:15,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca33b20e33d030878a883a HTTP/1.1" 200 None 2026-03-30 08:46:15,192 - lava_callback - DEBUG - Processing node 69ca33b20e33d030878a883a: ltp-smoke 2026-03-30 08:46:15,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:46:16,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:46:16,455 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca33b20e33d030878a883a/log.txt.gz INFO: 172.21.0.1:53708 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:46:16,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:46:17,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:46:17,574 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca33b20e33d030878a883a/lava-logs.yaml 2026-03-30 08:46:17,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:46:18,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:46:18,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca33b20e33d030878a883a/lava_callback.json.gz 2026-03-30 08:46:18,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:18,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca33b20e33d030878a883a HTTP/1.1" 200 None 2026-03-30 08:46:18,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:18,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31f70e33d030878a87f5 HTTP/1.1" 200 None 2026-03-30 08:46:18,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:18,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca33b20e33d030878a883a HTTP/1.1" 200 None 2026-03-30 08:46:18,529 - lava_callback - INFO - Completed processing callback for node 69ca33b20e33d030878a883a INFO: 172.21.0.1:58866 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:46:25,273 - lava_callback - INFO - Received callback for node 69c79552be430e18c5d16229 INFO: 172.21.0.1:58878 - "POST /node/69c79552be430e18c5d16229 HTTP/1.0" 202 Accepted 2026-03-30 08:46:25,280 - lava_callback - INFO - Starting async job submit for node 69c79552be430e18c5d16229 2026-03-30 08:46:25,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:25,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79552be430e18c5d16229 HTTP/1.1" 200 None 2026-03-30 08:46:25,307 - lava_callback - DEBUG - Processing node 69c79552be430e18c5d16229: baseline-arm-clabbe 2026-03-30 08:46:25,307 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79552be430e18c5d16229 2026-03-30 08:46:25,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:25,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79552be430e18c5d16229 HTTP/1.1" 200 None 2026-03-30 08:46:25,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:25,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 08:46:25,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:25,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79552be430e18c5d16229 HTTP/1.1" 200 None 2026-03-30 08:46:25,377 - lava_callback - ERROR - Error processing callback for node 69c79552be430e18c5d16229: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58888 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:46:36,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:46:36,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:47:05,367 - lava_callback - INFO - Received callback for node 69c7957abe430e18c5d1623e INFO: 172.21.0.1:36642 - "POST /node/69c7957abe430e18c5d1623e HTTP/1.0" 202 Accepted 2026-03-30 08:47:05,373 - lava_callback - INFO - Starting async job submit for node 69c7957abe430e18c5d1623e 2026-03-30 08:47:05,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:05,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7957abe430e18c5d1623e HTTP/1.1" 200 None 2026-03-30 08:47:05,390 - lava_callback - DEBUG - Processing node 69c7957abe430e18c5d1623e: baseline-arm-clabbe 2026-03-30 08:47:05,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7957abe430e18c5d1623e 2026-03-30 08:47:05,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:05,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7957abe430e18c5d1623e HTTP/1.1" 200 None 2026-03-30 08:47:05,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:05,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 08:47:05,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:05,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7957abe430e18c5d1623e HTTP/1.1" 200 None 2026-03-30 08:47:05,460 - lava_callback - ERROR - Error processing callback for node 69c7957abe430e18c5d1623e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36654 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:47:13,722 - lava_callback - INFO - Received callback for node 69ca37810e33d030878a8935 INFO: 172.21.0.1:53116 - "POST /node/69ca37810e33d030878a8935 HTTP/1.0" 202 Accepted 2026-03-30 08:47:13,728 - lava_callback - INFO - Starting async job submit for node 69ca37810e33d030878a8935 2026-03-30 08:47:13,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:13,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37810e33d030878a8935 HTTP/1.1" 200 None 2026-03-30 08:47:13,765 - lava_callback - DEBUG - Processing node 69ca37810e33d030878a8935: baseline-arm-clabbe 2026-03-30 08:47:14,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:47:14,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:47:14,210 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca37810e33d030878a8935/log.txt.gz 2026-03-30 08:47:14,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:47:14,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:47:14,538 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca37810e33d030878a8935/lava-logs.yaml 2026-03-30 08:47:14,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:47:14,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:47:14,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca37810e33d030878a8935/lava_callback.json.gz 2026-03-30 08:47:14,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:14,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37810e33d030878a8935 HTTP/1.1" 200 None 2026-03-30 08:47:14,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:14,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:47:14,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:14,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37810e33d030878a8935 HTTP/1.1" 200 None 2026-03-30 08:47:14,761 - lava_callback - INFO - Completed processing callback for node 69ca37810e33d030878a8935 INFO: 172.21.0.1:53128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:47:25,016 - lava_callback - INFO - Received callback for node 69ca37ae0e33d030878a89c7 INFO: 172.21.0.1:60584 - "POST /node/69ca37ae0e33d030878a89c7 HTTP/1.0" 202 Accepted 2026-03-30 08:47:25,023 - lava_callback - INFO - Starting async job submit for node 69ca37ae0e33d030878a89c7 2026-03-30 08:47:25,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:25,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37ae0e33d030878a89c7 HTTP/1.1" 200 None 2026-03-30 08:47:25,065 - lava_callback - DEBUG - Processing node 69ca37ae0e33d030878a89c7: ltp-syscalls INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:47:25,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:47:25,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:47:25,966 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69ca37ae0e33d030878a89c7/log.txt.gz 2026-03-30 08:47:26,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:60604 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:47:26,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:47:26,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69ca37ae0e33d030878a89c7/lava-logs.yaml 2026-03-30 08:47:26,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:47:26,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:47:26,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69ca37ae0e33d030878a89c7/lava_callback.json.gz 2026-03-30 08:47:26,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:26,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37ae0e33d030878a89c7 HTTP/1.1" 200 None 2026-03-30 08:47:26,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:26,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:47:26,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:26,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37ae0e33d030878a89c7 HTTP/1.1" 200 None 2026-03-30 08:47:26,904 - lava_callback - INFO - Completed processing callback for node 69ca37ae0e33d030878a89c7 2026-03-30 08:47:36,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:36,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45682 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:47:45,527 - lava_callback - INFO - Received callback for node 69c795a1be430e18c5d1624e INFO: 172.21.0.1:37910 - "POST /node/69c795a1be430e18c5d1624e HTTP/1.0" 202 Accepted 2026-03-30 08:47:45,533 - lava_callback - INFO - Starting async job submit for node 69c795a1be430e18c5d1624e 2026-03-30 08:47:45,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:45,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c795a1be430e18c5d1624e HTTP/1.1" 200 None 2026-03-30 08:47:45,563 - lava_callback - DEBUG - Processing node 69c795a1be430e18c5d1624e: baseline-arm-clabbe 2026-03-30 08:47:45,564 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c795a1be430e18c5d1624e 2026-03-30 08:47:45,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:45,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c795a1be430e18c5d1624e HTTP/1.1" 200 None 2026-03-30 08:47:45,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:45,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 08:47:45,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:47:45,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c795a1be430e18c5d1624e HTTP/1.1" 200 None 2026-03-30 08:47:45,616 - lava_callback - ERROR - Error processing callback for node 69c795a1be430e18c5d1624e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:48:48,907 - lava_callback - INFO - Received callback for node 69ca37820e33d030878a895c INFO: 172.21.0.1:56382 - "POST /node/69ca37820e33d030878a895c HTTP/1.0" 202 Accepted 2026-03-30 08:48:48,914 - lava_callback - INFO - Starting async job submit for node 69ca37820e33d030878a895c 2026-03-30 08:48:48,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:48:48,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37820e33d030878a895c HTTP/1.1" 200 None 2026-03-30 08:48:48,971 - lava_callback - DEBUG - Processing node 69ca37820e33d030878a895c: baseline-arm-clabbe 2026-03-30 08:48:49,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:48:49,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:48:49,549 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca37820e33d030878a895c/log.txt.gz 2026-03-30 08:48:49,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:48:50,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:48:50,088 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca37820e33d030878a895c/lava-logs.yaml 2026-03-30 08:48:50,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:48:50,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:48:50,301 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ca37820e33d030878a895c/lava_callback.json.gz 2026-03-30 08:48:50,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:48:50,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37820e33d030878a895c HTTP/1.1" 200 None 2026-03-30 08:48:50,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:48:50,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:48:50,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:48:50,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37820e33d030878a895c HTTP/1.1" 200 None 2026-03-30 08:48:50,382 - lava_callback - INFO - Completed processing callback for node 69ca37820e33d030878a895c INFO: 172.21.0.1:59658 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:49:05,543 - lava_callback - INFO - Received callback for node 69c795ecbe430e18c5d16290 INFO: 172.21.0.1:60420 - "POST /node/69c795ecbe430e18c5d16290 HTTP/1.0" 202 Accepted 2026-03-30 08:49:05,551 - lava_callback - INFO - Starting async job submit for node 69c795ecbe430e18c5d16290 2026-03-30 08:49:05,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:05,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c795ecbe430e18c5d16290 HTTP/1.1" 200 None 2026-03-30 08:49:05,578 - lava_callback - DEBUG - Processing node 69c795ecbe430e18c5d16290: baseline-arm-clabbe 2026-03-30 08:49:05,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c795ecbe430e18c5d16290 2026-03-30 08:49:05,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:05,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c795ecbe430e18c5d16290 HTTP/1.1" 200 None 2026-03-30 08:49:05,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:05,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 08:49:05,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:05,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c795ecbe430e18c5d16290 HTTP/1.1" 200 None 2026-03-30 08:49:05,677 - lava_callback - ERROR - Error processing callback for node 69c795ecbe430e18c5d16290: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:49:06,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:06,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58092 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:49:23,223 - lava_callback - INFO - Received callback for node 69ca37750e33d030878a890a INFO: 172.21.0.1:41252 - "POST /node/69ca37750e33d030878a890a HTTP/1.0" 202 Accepted 2026-03-30 08:49:23,232 - lava_callback - INFO - Starting async job submit for node 69ca37750e33d030878a890a 2026-03-30 08:49:23,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:23,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37750e33d030878a890a HTTP/1.1" 200 None 2026-03-30 08:49:23,310 - lava_callback - DEBUG - Processing node 69ca37750e33d030878a890a: baseline-arm-broonie 2026-03-30 08:49:23,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:49:23,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:49:23,776 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37750e33d030878a890a/log.txt.gz 2026-03-30 08:49:23,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:49:24,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:49:24,113 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37750e33d030878a890a/lava-logs.yaml 2026-03-30 08:49:24,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:49:24,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:49:24,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ca37750e33d030878a890a/lava_callback.json.gz 2026-03-30 08:49:24,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:24,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37750e33d030878a890a HTTP/1.1" 200 None 2026-03-30 08:49:24,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:24,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:49:24,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:24,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37750e33d030878a890a HTTP/1.1" 200 None 2026-03-30 08:49:24,443 - lava_callback - INFO - Completed processing callback for node 69ca37750e33d030878a890a INFO: 172.21.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41276 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:49:36,028 - lava_callback - INFO - Received callback for node 69ca39140e33d030878a8aea INFO: 172.21.0.1:53594 - "POST /node/69ca39140e33d030878a8aea HTTP/1.0" 202 Accepted 2026-03-30 08:49:36,033 - lava_callback - INFO - Starting async job submit for node 69ca39140e33d030878a8aea 2026-03-30 08:49:36,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:36,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39140e33d030878a8aea HTTP/1.1" 200 None 2026-03-30 08:49:36,051 - lava_callback - DEBUG - Processing node 69ca39140e33d030878a8aea: baseline-arm64 2026-03-30 08:49:36,058 - lava_callback - WARNING - Failed to upload log 2026-03-30 08:49:36,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:49:36,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:49:36,178 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39140e33d030878a8aea/lava-logs.yaml 2026-03-30 08:49:36,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:49:36,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:36,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 08:49:36,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:49:36,371 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39140e33d030878a8aea/lava_callback.json.gz 2026-03-30 08:49:36,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:36,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39140e33d030878a8aea HTTP/1.1" 200 None 2026-03-30 08:49:36,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:36,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:49:36,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:49:36,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39140e33d030878a8aea HTTP/1.1" 200 None 2026-03-30 08:49:36,448 - lava_callback - INFO - Completed processing callback for node 69ca39140e33d030878a8aea INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50232 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:50:06,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:06,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34332 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:50:09,596 - lava_callback - INFO - Received callback for node 69ca39220e33d030878a8b14 INFO: 172.21.0.1:34340 - "POST /node/69ca39220e33d030878a8b14 HTTP/1.0" 202 Accepted 2026-03-30 08:50:09,606 - lava_callback - INFO - Starting async job submit for node 69ca39220e33d030878a8b14 2026-03-30 08:50:09,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:09,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39220e33d030878a8b14 HTTP/1.1" 200 None 2026-03-30 08:50:09,641 - lava_callback - DEBUG - Processing node 69ca39220e33d030878a8b14: baseline-arm64 2026-03-30 08:50:09,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:09,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:09,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39220e33d030878a8b14/log.txt.gz 2026-03-30 08:50:10,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:10,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:10,217 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39220e33d030878a8b14/lava-logs.yaml 2026-03-30 08:50:10,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:10,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:10,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39220e33d030878a8b14/lava_callback.json.gz 2026-03-30 08:50:10,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:10,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39220e33d030878a8b14 HTTP/1.1" 200 None 2026-03-30 08:50:10,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:10,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:10,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:10,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39220e33d030878a8b14 HTTP/1.1" 200 None 2026-03-30 08:50:10,548 - lava_callback - INFO - Completed processing callback for node 69ca39220e33d030878a8b14 2026-03-30 08:50:12,545 - lava_callback - INFO - Received callback for node 69ca39330e33d030878a8b51 INFO: 172.21.0.1:38706 - "POST /node/69ca39330e33d030878a8b51 HTTP/1.0" 202 Accepted 2026-03-30 08:50:12,551 - lava_callback - INFO - Starting async job submit for node 69ca39330e33d030878a8b51 2026-03-30 08:50:12,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:12,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39330e33d030878a8b51 HTTP/1.1" 200 None 2026-03-30 08:50:12,568 - lava_callback - DEBUG - Processing node 69ca39330e33d030878a8b51: baseline-nfs-arm64 2026-03-30 08:50:12,570 - lava_callback - WARNING - Failed to upload log 2026-03-30 08:50:12,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:12,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:12,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39330e33d030878a8b51/lava-logs.yaml 2026-03-30 08:50:12,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:12,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:12,812 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39330e33d030878a8b51/lava_callback.json.gz 2026-03-30 08:50:12,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:12,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39330e33d030878a8b51 HTTP/1.1" 200 None 2026-03-30 08:50:12,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:12,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:12,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:12,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39330e33d030878a8b51 HTTP/1.1" 200 None 2026-03-30 08:50:12,877 - lava_callback - INFO - Completed processing callback for node 69ca39330e33d030878a8b51 INFO: 172.21.0.1:38708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:50:30,250 - lava_callback - INFO - Received callback for node 69ca39150e33d030878a8af4 INFO: 172.21.0.1:49308 - "POST /node/69ca39150e33d030878a8af4 HTTP/1.0" 202 Accepted 2026-03-30 08:50:30,256 - lava_callback - INFO - Starting async job submit for node 69ca39150e33d030878a8af4 2026-03-30 08:50:30,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:30,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39150e33d030878a8af4 HTTP/1.1" 200 None 2026-03-30 08:50:30,320 - lava_callback - DEBUG - Processing node 69ca39150e33d030878a8af4: baseline-arm64 2026-03-30 08:50:30,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:30,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:30,882 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39150e33d030878a8af4/log.txt.gz 2026-03-30 08:50:31,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:31,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:31,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39150e33d030878a8af4/lava-logs.yaml 2026-03-30 08:50:31,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:31,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:31,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39150e33d030878a8af4/lava_callback.json.gz 2026-03-30 08:50:31,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:31,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39150e33d030878a8af4 HTTP/1.1" 200 None 2026-03-30 08:50:31,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:31,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:31,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:31,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39150e33d030878a8af4 HTTP/1.1" 200 None 2026-03-30 08:50:31,498 - lava_callback - INFO - Completed processing callback for node 69ca39150e33d030878a8af4 2026-03-30 08:50:36,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:36,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49314 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:50:37,788 - lava_callback - INFO - Received callback for node 69ca39170e33d030878a8afa INFO: 172.21.0.1:49318 - "POST /node/69ca39170e33d030878a8afa HTTP/1.0" 202 Accepted 2026-03-30 08:50:37,795 - lava_callback - INFO - Starting async job submit for node 69ca39170e33d030878a8afa 2026-03-30 08:50:37,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:37,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39170e33d030878a8afa HTTP/1.1" 200 None 2026-03-30 08:50:37,842 - lava_callback - DEBUG - Processing node 69ca39170e33d030878a8afa: baseline-arm64 2026-03-30 08:50:38,054 - lava_callback - WARNING - Failed to upload log 2026-03-30 08:50:38,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:38,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:38,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39170e33d030878a8afa/lava-logs.yaml 2026-03-30 08:50:38,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:38,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:38,439 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39170e33d030878a8afa/lava_callback.json.gz 2026-03-30 08:50:38,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:38,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39170e33d030878a8afa HTTP/1.1" 200 None 2026-03-30 08:50:38,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:38,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:38,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:38,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39170e33d030878a8afa HTTP/1.1" 200 None 2026-03-30 08:50:38,581 - lava_callback - INFO - Completed processing callback for node 69ca39170e33d030878a8afa INFO: 172.21.0.1:39146 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:50:47,537 - lava_callback - INFO - Received callback for node 69ca391a0e33d030878a8afd INFO: 172.21.0.1:39154 - "POST /node/69ca391a0e33d030878a8afd HTTP/1.0" 202 Accepted 2026-03-30 08:50:47,544 - lava_callback - INFO - Starting async job submit for node 69ca391a0e33d030878a8afd 2026-03-30 08:50:47,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:47,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391a0e33d030878a8afd HTTP/1.1" 200 None 2026-03-30 08:50:47,590 - lava_callback - DEBUG - Processing node 69ca391a0e33d030878a8afd: baseline-arm64 2026-03-30 08:50:47,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:47,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:47,843 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391a0e33d030878a8afd/log.txt.gz 2026-03-30 08:50:47,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:48,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:48,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391a0e33d030878a8afd/lava-logs.yaml 2026-03-30 08:50:48,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:48,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:48,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391a0e33d030878a8afd/lava_callback.json.gz 2026-03-30 08:50:48,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:48,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391a0e33d030878a8afd HTTP/1.1" 200 None 2026-03-30 08:50:48,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:48,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:48,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:48,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca391a0e33d030878a8afd HTTP/1.1" 200 None 2026-03-30 08:50:48,395 - lava_callback - INFO - Completed processing callback for node 69ca391a0e33d030878a8afd 2026-03-30 08:50:53,680 - lava_callback - INFO - Received callback for node 69ca39140e33d030878a8af1 INFO: 172.21.0.1:40386 - "POST /node/69ca39140e33d030878a8af1 HTTP/1.0" 202 Accepted 2026-03-30 08:50:53,687 - lava_callback - INFO - Starting async job submit for node 69ca39140e33d030878a8af1 2026-03-30 08:50:53,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:53,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39140e33d030878a8af1 HTTP/1.1" 200 None 2026-03-30 08:50:53,723 - lava_callback - DEBUG - Processing node 69ca39140e33d030878a8af1: baseline-arm64 2026-03-30 08:50:54,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:54,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:54,288 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39140e33d030878a8af1/log.txt.gz 2026-03-30 08:50:54,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:54,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:54,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39140e33d030878a8af1/lava-logs.yaml 2026-03-30 08:50:54,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:54,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:54,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39140e33d030878a8af1/lava_callback.json.gz 2026-03-30 08:50:54,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:54,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39140e33d030878a8af1 HTTP/1.1" 200 None 2026-03-30 08:50:54,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:54,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:54,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:54,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39140e33d030878a8af1 HTTP/1.1" 200 None 2026-03-30 08:50:54,911 - lava_callback - INFO - Completed processing callback for node 69ca39140e33d030878a8af1 2026-03-30 08:50:55,378 - lava_callback - INFO - Received callback for node 69ca39280e33d030878a8b38 INFO: 172.21.0.1:40388 - "POST /node/69ca39280e33d030878a8b38 HTTP/1.0" 202 Accepted 2026-03-30 08:50:55,386 - lava_callback - INFO - Starting async job submit for node 69ca39280e33d030878a8b38 2026-03-30 08:50:55,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:55,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39280e33d030878a8b38 HTTP/1.1" 200 None 2026-03-30 08:50:55,437 - lava_callback - DEBUG - Processing node 69ca39280e33d030878a8b38: baseline-arm64 2026-03-30 08:50:55,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:55,832 - lava_callback - INFO - Received callback for node 69ca391e0e33d030878a8b06 INFO: 172.21.0.1:40400 - "POST /node/69ca391e0e33d030878a8b06 HTTP/1.0" 202 Accepted 2026-03-30 08:50:55,840 - lava_callback - INFO - Starting async job submit for node 69ca391e0e33d030878a8b06 2026-03-30 08:50:55,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:55,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:55,879 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39280e33d030878a8b38/log.txt.gz 2026-03-30 08:50:55,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391e0e33d030878a8b06 HTTP/1.1" 200 None 2026-03-30 08:50:55,903 - lava_callback - DEBUG - Processing node 69ca391e0e33d030878a8b06: baseline-arm64 2026-03-30 08:50:56,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40414 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:50:56,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:56,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:56,646 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391e0e33d030878a8b06/log.txt.gz 2026-03-30 08:50:56,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:56,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39280e33d030878a8b38/lava-logs.yaml 2026-03-30 08:50:56,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:56,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:57,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:57,024 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39280e33d030878a8b38/lava_callback.json.gz 2026-03-30 08:50:57,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:57,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39280e33d030878a8b38 HTTP/1.1" 200 None 2026-03-30 08:50:57,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:57,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:57,081 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391e0e33d030878a8b06/lava-logs.yaml 2026-03-30 08:50:57,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:57,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:57,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:57,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39280e33d030878a8b38 HTTP/1.1" 200 None 2026-03-30 08:50:57,114 - lava_callback - INFO - Completed processing callback for node 69ca39280e33d030878a8b38 2026-03-30 08:50:57,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:57,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391e0e33d030878a8b06/lava_callback.json.gz 2026-03-30 08:50:57,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:57,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391e0e33d030878a8b06 HTTP/1.1" 200 None 2026-03-30 08:50:57,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:57,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:57,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:57,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca391e0e33d030878a8b06 HTTP/1.1" 200 None 2026-03-30 08:50:57,393 - lava_callback - INFO - Completed processing callback for node 69ca391e0e33d030878a8b06 2026-03-30 08:50:57,660 - lava_callback - INFO - Received callback for node 69ca39160e33d030878a8af7 INFO: 172.21.0.1:40426 - "POST /node/69ca39160e33d030878a8af7 HTTP/1.0" 202 Accepted 2026-03-30 08:50:57,666 - lava_callback - INFO - Starting async job submit for node 69ca39160e33d030878a8af7 2026-03-30 08:50:57,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:57,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39160e33d030878a8af7 HTTP/1.1" 200 None 2026-03-30 08:50:57,705 - lava_callback - DEBUG - Processing node 69ca39160e33d030878a8af7: baseline-arm64 2026-03-30 08:50:57,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:58,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:58,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39160e33d030878a8af7/log.txt.gz 2026-03-30 08:50:58,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:58,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:58,421 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39160e33d030878a8af7/lava-logs.yaml 2026-03-30 08:50:58,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:58,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:58,562 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39160e33d030878a8af7/lava_callback.json.gz 2026-03-30 08:50:58,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:58,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39160e33d030878a8af7 HTTP/1.1" 200 None 2026-03-30 08:50:58,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:58,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:50:58,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:58,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39160e33d030878a8af7 HTTP/1.1" 200 None 2026-03-30 08:50:58,646 - lava_callback - INFO - Completed processing callback for node 69ca39160e33d030878a8af7 2026-03-30 08:50:59,026 - lava_callback - INFO - Received callback for node 69ca39260e33d030878a8b1d INFO: 172.21.0.1:40440 - "POST /node/69ca39260e33d030878a8b1d HTTP/1.0" 202 Accepted 2026-03-30 08:50:59,033 - lava_callback - INFO - Starting async job submit for node 69ca39260e33d030878a8b1d 2026-03-30 08:50:59,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:50:59,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39260e33d030878a8b1d HTTP/1.1" 200 None 2026-03-30 08:50:59,100 - lava_callback - DEBUG - Processing node 69ca39260e33d030878a8b1d: baseline-arm64 2026-03-30 08:50:59,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:59,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:59,595 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39260e33d030878a8b1d/log.txt.gz 2026-03-30 08:50:59,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:59,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:50:59,967 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39260e33d030878a8b1d/lava-logs.yaml 2026-03-30 08:50:59,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:50:59,993 - lava_callback - INFO - Received callback for node 69ca39230e33d030878a8b17 INFO: 172.21.0.1:46588 - "POST /node/69ca39230e33d030878a8b17 HTTP/1.0" 202 Accepted 2026-03-30 08:51:00,000 - lava_callback - INFO - Starting async job submit for node 69ca39230e33d030878a8b17 2026-03-30 08:51:00,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:00,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39230e33d030878a8b17 HTTP/1.1" 200 None 2026-03-30 08:51:00,052 - lava_callback - DEBUG - Processing node 69ca39230e33d030878a8b17: baseline-arm64 2026-03-30 08:51:00,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:00,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39260e33d030878a8b1d/lava_callback.json.gz 2026-03-30 08:51:00,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:00,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39260e33d030878a8b1d HTTP/1.1" 200 None 2026-03-30 08:51:00,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:00,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:00,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:00,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:00,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39260e33d030878a8b1d HTTP/1.1" 200 None 2026-03-30 08:51:00,406 - lava_callback - INFO - Completed processing callback for node 69ca39260e33d030878a8b1d 2026-03-30 08:51:00,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:00,540 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39230e33d030878a8b17/log.txt.gz 2026-03-30 08:51:00,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:00,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:00,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39230e33d030878a8b17/lava-logs.yaml 2026-03-30 08:51:00,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:01,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:01,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39230e33d030878a8b17/lava_callback.json.gz 2026-03-30 08:51:01,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:01,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39230e33d030878a8b17 HTTP/1.1" 200 None 2026-03-30 08:51:01,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:01,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:01,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:01,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39230e33d030878a8b17 HTTP/1.1" 200 None 2026-03-30 08:51:01,212 - lava_callback - INFO - Completed processing callback for node 69ca39230e33d030878a8b17 2026-03-30 08:51:03,443 - lava_callback - INFO - Received callback for node 69ca379b0e33d030878a8986 INFO: 172.21.0.1:46598 - "POST /node/69ca379b0e33d030878a8986 HTTP/1.0" 202 Accepted 2026-03-30 08:51:03,453 - lava_callback - INFO - Starting async job submit for node 69ca379b0e33d030878a8986 2026-03-30 08:51:03,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:03,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca379b0e33d030878a8986 HTTP/1.1" 200 None 2026-03-30 08:51:03,501 - lava_callback - DEBUG - Processing node 69ca379b0e33d030878a8986: ltp-cap_bounds 2026-03-30 08:51:03,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:04,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:04,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ca379b0e33d030878a8986/log.txt.gz 2026-03-30 08:51:04,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:04,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:04,501 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ca379b0e33d030878a8986/lava-logs.yaml 2026-03-30 08:51:04,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:04,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:04,661 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ca379b0e33d030878a8986/lava_callback.json.gz 2026-03-30 08:51:04,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:04,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca379b0e33d030878a8986 HTTP/1.1" 200 None 2026-03-30 08:51:04,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:04,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:51:04,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:04,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca379b0e33d030878a8986 HTTP/1.1" 200 None 2026-03-30 08:51:04,754 - lava_callback - INFO - Completed processing callback for node 69ca379b0e33d030878a8986 2026-03-30 08:51:06,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:06,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49974 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:51:16,805 - lava_callback - INFO - Received callback for node 69ca391d0e33d030878a8b03 INFO: 172.21.0.1:49982 - "POST /node/69ca391d0e33d030878a8b03 HTTP/1.0" 202 Accepted 2026-03-30 08:51:16,815 - lava_callback - INFO - Starting async job submit for node 69ca391d0e33d030878a8b03 2026-03-30 08:51:16,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:16,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391d0e33d030878a8b03 HTTP/1.1" 200 None 2026-03-30 08:51:16,860 - lava_callback - DEBUG - Processing node 69ca391d0e33d030878a8b03: baseline-arm64 2026-03-30 08:51:17,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:17,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:17,360 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391d0e33d030878a8b03/log.txt.gz 2026-03-30 08:51:17,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:17,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:17,776 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391d0e33d030878a8b03/lava-logs.yaml 2026-03-30 08:51:17,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:17,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:17,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391d0e33d030878a8b03/lava_callback.json.gz 2026-03-30 08:51:17,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:18,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391d0e33d030878a8b03 HTTP/1.1" 200 None 2026-03-30 08:51:18,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:18,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:18,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:18,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca391d0e33d030878a8b03 HTTP/1.1" 200 None 2026-03-30 08:51:18,055 - lava_callback - INFO - Completed processing callback for node 69ca391d0e33d030878a8b03 2026-03-30 08:51:23,692 - lava_callback - INFO - Received callback for node 69ca379e0e33d030878a898c INFO: 172.21.0.1:37516 - "POST /node/69ca379e0e33d030878a898c HTTP/1.0" 202 Accepted 2026-03-30 08:51:23,701 - lava_callback - INFO - Starting async job submit for node 69ca379e0e33d030878a898c 2026-03-30 08:51:23,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:23,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca379e0e33d030878a898c HTTP/1.1" 200 None 2026-03-30 08:51:23,761 - lava_callback - DEBUG - Processing node 69ca379e0e33d030878a898c: ltp-controllers 2026-03-30 08:51:24,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:24,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:24,514 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69ca379e0e33d030878a898c/log.txt.gz 2026-03-30 08:51:24,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:25,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:25,075 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69ca379e0e33d030878a898c/lava-logs.yaml 2026-03-30 08:51:25,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:25,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:25,258 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69ca379e0e33d030878a898c/lava_callback.json.gz INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:51:25,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:25,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca379e0e33d030878a898c HTTP/1.1" 200 None 2026-03-30 08:51:25,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:25,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:51:25,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:25,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca379e0e33d030878a898c HTTP/1.1" 200 None 2026-03-30 08:51:25,361 - lava_callback - INFO - Completed processing callback for node 69ca379e0e33d030878a898c INFO: 172.21.0.1:37540 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:51:28,918 - lava_callback - INFO - Received callback for node 69ca39390e33d030878a8b61 INFO: 172.21.0.1:37542 - "POST /node/69ca39390e33d030878a8b61 HTTP/1.0" 202 Accepted 2026-03-30 08:51:28,928 - lava_callback - INFO - Starting async job submit for node 69ca39390e33d030878a8b61 2026-03-30 08:51:28,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:28,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39390e33d030878a8b61 HTTP/1.1" 200 None 2026-03-30 08:51:28,969 - lava_callback - DEBUG - Processing node 69ca39390e33d030878a8b61: baseline-nfs-arm64 2026-03-30 08:51:29,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:29,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:29,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39390e33d030878a8b61/log.txt.gz 2026-03-30 08:51:29,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:30,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:30,139 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39390e33d030878a8b61/lava-logs.yaml 2026-03-30 08:51:30,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:30,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:30,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39390e33d030878a8b61/lava_callback.json.gz 2026-03-30 08:51:30,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:30,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39390e33d030878a8b61 HTTP/1.1" 200 None 2026-03-30 08:51:30,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:30,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:30,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:30,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39390e33d030878a8b61 HTTP/1.1" 200 None 2026-03-30 08:51:30,433 - lava_callback - INFO - Completed processing callback for node 69ca39390e33d030878a8b61 2026-03-30 08:51:35,287 - lava_callback - INFO - Received callback for node 69ca379d0e33d030878a8989 INFO: 172.21.0.1:49860 - "POST /node/69ca379d0e33d030878a8989 HTTP/1.0" 202 Accepted 2026-03-30 08:51:35,295 - lava_callback - INFO - Starting async job submit for node 69ca379d0e33d030878a8989 2026-03-30 08:51:35,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:35,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca379d0e33d030878a8989 HTTP/1.1" 200 None 2026-03-30 08:51:35,348 - lava_callback - DEBUG - Processing node 69ca379d0e33d030878a8989: ltp-containers 2026-03-30 08:51:35,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:35,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:35,948 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69ca379d0e33d030878a8989/log.txt.gz 2026-03-30 08:51:36,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:36,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:36,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 08:51:36,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:36,506 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69ca379d0e33d030878a8989/lava-logs.yaml INFO: 172.21.0.1:49864 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:51:36,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:36,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:36,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69ca379d0e33d030878a8989/lava_callback.json.gz 2026-03-30 08:51:36,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:36,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca379d0e33d030878a8989 HTTP/1.1" 200 None 2026-03-30 08:51:36,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:36,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:51:36,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:36,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca379d0e33d030878a8989 HTTP/1.1" 200 None 2026-03-30 08:51:36,787 - lava_callback - INFO - Completed processing callback for node 69ca379d0e33d030878a8989 2026-03-30 08:51:37,146 - lava_callback - INFO - Received callback for node 69ca39440e33d030878a8b85 INFO: 172.21.0.1:49872 - "POST /node/69ca39440e33d030878a8b85 HTTP/1.0" 202 Accepted 2026-03-30 08:51:37,154 - lava_callback - INFO - Starting async job submit for node 69ca39440e33d030878a8b85 2026-03-30 08:51:37,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:37,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39440e33d030878a8b85 HTTP/1.1" 200 None 2026-03-30 08:51:37,196 - lava_callback - DEBUG - Processing node 69ca39440e33d030878a8b85: baseline-arm64-broonie 2026-03-30 08:51:37,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:37,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:37,592 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39440e33d030878a8b85/log.txt.gz 2026-03-30 08:51:37,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:38,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:38,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39440e33d030878a8b85/lava-logs.yaml 2026-03-30 08:51:38,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:38,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:38,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39440e33d030878a8b85/lava_callback.json.gz 2026-03-30 08:51:38,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:38,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39440e33d030878a8b85 HTTP/1.1" 200 None 2026-03-30 08:51:38,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:38,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:38,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:38,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39440e33d030878a8b85 HTTP/1.1" 200 None 2026-03-30 08:51:38,381 - lava_callback - INFO - Completed processing callback for node 69ca39440e33d030878a8b85 2026-03-30 08:51:38,461 - lava_callback - INFO - Received callback for node 69ca392e0e33d030878a8b45 INFO: 172.21.0.1:49882 - "POST /node/69ca392e0e33d030878a8b45 HTTP/1.0" 202 Accepted 2026-03-30 08:51:38,473 - lava_callback - INFO - Starting async job submit for node 69ca392e0e33d030878a8b45 2026-03-30 08:51:38,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:38,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392e0e33d030878a8b45 HTTP/1.1" 200 None 2026-03-30 08:51:38,524 - lava_callback - DEBUG - Processing node 69ca392e0e33d030878a8b45: baseline-nfs-arm64 2026-03-30 08:51:38,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:39,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:39,080 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392e0e33d030878a8b45/log.txt.gz 2026-03-30 08:51:39,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:39,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:39,539 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392e0e33d030878a8b45/lava-logs.yaml 2026-03-30 08:51:39,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:39,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:39,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392e0e33d030878a8b45/lava_callback.json.gz 2026-03-30 08:51:39,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:39,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392e0e33d030878a8b45 HTTP/1.1" 200 None 2026-03-30 08:51:39,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:39,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:39,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:39,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca392e0e33d030878a8b45 HTTP/1.1" 200 None 2026-03-30 08:51:39,807 - lava_callback - INFO - Completed processing callback for node 69ca392e0e33d030878a8b45 2026-03-30 08:51:41,217 - lava_callback - INFO - Received callback for node 69ca395f0e33d030878a8bf0 INFO: 172.21.0.1:46888 - "POST /node/69ca395f0e33d030878a8bf0 HTTP/1.0" 202 Accepted 2026-03-30 08:51:41,224 - lava_callback - INFO - Starting async job submit for node 69ca395f0e33d030878a8bf0 2026-03-30 08:51:41,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:41,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395f0e33d030878a8bf0 HTTP/1.1" 200 None 2026-03-30 08:51:41,282 - lava_callback - DEBUG - Processing node 69ca395f0e33d030878a8bf0: baseline-arm64-foundriesio 2026-03-30 08:51:41,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:41,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:41,718 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69ca395f0e33d030878a8bf0/log.txt.gz 2026-03-30 08:51:41,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:42,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:42,054 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69ca395f0e33d030878a8bf0/lava-logs.yaml 2026-03-30 08:51:42,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:42,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:42,264 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69ca395f0e33d030878a8bf0/lava_callback.json.gz 2026-03-30 08:51:42,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:42,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395f0e33d030878a8bf0 HTTP/1.1" 200 None 2026-03-30 08:51:42,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:42,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:42,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:42,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca395f0e33d030878a8bf0 HTTP/1.1" 200 None 2026-03-30 08:51:42,357 - lava_callback - INFO - Completed processing callback for node 69ca395f0e33d030878a8bf0 INFO: 172.21.0.1:46904 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:51:49,519 - lava_callback - INFO - Received callback for node 69ca392b0e33d030878a8b3f INFO: 172.21.0.1:46916 - "POST /node/69ca392b0e33d030878a8b3f HTTP/1.0" 202 Accepted 2026-03-30 08:51:49,527 - lava_callback - INFO - Starting async job submit for node 69ca392b0e33d030878a8b3f 2026-03-30 08:51:49,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392b0e33d030878a8b3f HTTP/1.1" 200 None 2026-03-30 08:51:49,565 - lava_callback - DEBUG - Processing node 69ca392b0e33d030878a8b3f: baseline-nfs-arm64 2026-03-30 08:51:49,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:50,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:50,030 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392b0e33d030878a8b3f/log.txt.gz 2026-03-30 08:51:50,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:50,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:50,432 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392b0e33d030878a8b3f/lava-logs.yaml 2026-03-30 08:51:50,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:50,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:50,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392b0e33d030878a8b3f/lava_callback.json.gz 2026-03-30 08:51:50,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:50,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392b0e33d030878a8b3f HTTP/1.1" 200 None 2026-03-30 08:51:50,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:50,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:50,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:50,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca392b0e33d030878a8b3f HTTP/1.1" 200 None 2026-03-30 08:51:50,673 - lava_callback - INFO - Completed processing callback for node 69ca392b0e33d030878a8b3f 2026-03-30 08:51:51,259 - lava_callback - INFO - Received callback for node 69ca395d0e33d030878a8beb INFO: 172.21.0.1:47978 - "POST /node/69ca395d0e33d030878a8beb HTTP/1.0" 202 Accepted 2026-03-30 08:51:51,265 - lava_callback - INFO - Starting async job submit for node 69ca395d0e33d030878a8beb 2026-03-30 08:51:51,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:51,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395d0e33d030878a8beb HTTP/1.1" 200 None 2026-03-30 08:51:51,301 - lava_callback - DEBUG - Processing node 69ca395d0e33d030878a8beb: baseline-arm64-pengutronix 2026-03-30 08:51:51,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:51,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:51,661 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395d0e33d030878a8beb/log.txt.gz 2026-03-30 08:51:51,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:52,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:52,004 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395d0e33d030878a8beb/lava-logs.yaml 2026-03-30 08:51:52,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:52,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:52,153 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395d0e33d030878a8beb/lava_callback.json.gz 2026-03-30 08:51:52,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:52,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395d0e33d030878a8beb HTTP/1.1" 200 None 2026-03-30 08:51:52,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:52,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:52,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:52,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca395d0e33d030878a8beb HTTP/1.1" 200 None 2026-03-30 08:51:52,237 - lava_callback - INFO - Completed processing callback for node 69ca395d0e33d030878a8beb 2026-03-30 08:51:55,310 - lava_callback - INFO - Received callback for node 69ca394c0e33d030878a8bb9 INFO: 172.21.0.1:47986 - "POST /node/69ca394c0e33d030878a8bb9 HTTP/1.0" 202 Accepted 2026-03-30 08:51:55,316 - lava_callback - INFO - Starting async job submit for node 69ca394c0e33d030878a8bb9 2026-03-30 08:51:55,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:55,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394c0e33d030878a8bb9 HTTP/1.1" 200 None 2026-03-30 08:51:55,353 - lava_callback - DEBUG - Processing node 69ca394c0e33d030878a8bb9: baseline-arm64-broonie 2026-03-30 08:51:55,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:55,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:55,788 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394c0e33d030878a8bb9/log.txt.gz 2026-03-30 08:51:55,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:56,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:56,133 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394c0e33d030878a8bb9/lava-logs.yaml 2026-03-30 08:51:56,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:51:56,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:51:56,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394c0e33d030878a8bb9/lava_callback.json.gz 2026-03-30 08:51:56,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:56,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394c0e33d030878a8bb9 HTTP/1.1" 200 None 2026-03-30 08:51:56,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:56,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:51:56,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:51:56,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca394c0e33d030878a8bb9 HTTP/1.1" 200 None 2026-03-30 08:51:56,369 - lava_callback - INFO - Completed processing callback for node 69ca394c0e33d030878a8bb9 INFO: 172.21.0.1:48000 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:52:01,691 - lava_callback - INFO - Received callback for node 69ca39370e33d030878a8b5a INFO: 172.21.0.1:51764 - "POST /node/69ca39370e33d030878a8b5a HTTP/1.0" 202 Accepted 2026-03-30 08:52:01,698 - lava_callback - INFO - Starting async job submit for node 69ca39370e33d030878a8b5a 2026-03-30 08:52:01,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:01,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39370e33d030878a8b5a HTTP/1.1" 200 None 2026-03-30 08:52:01,736 - lava_callback - DEBUG - Processing node 69ca39370e33d030878a8b5a: baseline-nfs-arm64 2026-03-30 08:52:02,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:02,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:02,225 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39370e33d030878a8b5a/log.txt.gz 2026-03-30 08:52:02,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:02,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:02,634 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39370e33d030878a8b5a/lava-logs.yaml 2026-03-30 08:52:02,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:02,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:02,787 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39370e33d030878a8b5a/lava_callback.json.gz 2026-03-30 08:52:02,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:02,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39370e33d030878a8b5a HTTP/1.1" 200 None 2026-03-30 08:52:02,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:02,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:02,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:02,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39370e33d030878a8b5a HTTP/1.1" 200 None 2026-03-30 08:52:02,884 - lava_callback - INFO - Completed processing callback for node 69ca39370e33d030878a8b5a 2026-03-30 08:52:03,221 - lava_callback - INFO - Received callback for node 69ca39480e33d030878a8bb0 INFO: 172.21.0.1:51766 - "POST /node/69ca39480e33d030878a8bb0 HTTP/1.0" 202 Accepted 2026-03-30 08:52:03,228 - lava_callback - INFO - Starting async job submit for node 69ca39480e33d030878a8bb0 2026-03-30 08:52:03,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:03,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39480e33d030878a8bb0 HTTP/1.1" 200 None 2026-03-30 08:52:03,268 - lava_callback - DEBUG - Processing node 69ca39480e33d030878a8bb0: baseline-arm64-broonie 2026-03-30 08:52:03,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:03,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:03,676 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39480e33d030878a8bb0/log.txt.gz 2026-03-30 08:52:03,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:04,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:04,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39480e33d030878a8bb0/lava-logs.yaml 2026-03-30 08:52:04,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:04,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:04,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39480e33d030878a8bb0/lava_callback.json.gz 2026-03-30 08:52:04,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:04,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39480e33d030878a8bb0 HTTP/1.1" 200 None 2026-03-30 08:52:04,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:04,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:04,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:04,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39480e33d030878a8bb0 HTTP/1.1" 200 None 2026-03-30 08:52:04,267 - lava_callback - INFO - Completed processing callback for node 69ca39480e33d030878a8bb0 2026-03-30 08:52:04,670 - lava_callback - INFO - Received callback for node 69ca394a0e33d030878a8bb3 INFO: 172.21.0.1:51768 - "POST /node/69ca394a0e33d030878a8bb3 HTTP/1.0" 202 Accepted 2026-03-30 08:52:04,677 - lava_callback - INFO - Starting async job submit for node 69ca394a0e33d030878a8bb3 2026-03-30 08:52:04,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:04,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394a0e33d030878a8bb3 HTTP/1.1" 200 None 2026-03-30 08:52:04,723 - lava_callback - DEBUG - Processing node 69ca394a0e33d030878a8bb3: baseline-arm64-broonie 2026-03-30 08:52:04,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:05,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:05,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394a0e33d030878a8bb3/log.txt.gz 2026-03-30 08:52:05,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:05,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:05,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394a0e33d030878a8bb3/lava-logs.yaml 2026-03-30 08:52:05,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:05,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:05,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394a0e33d030878a8bb3/lava_callback.json.gz 2026-03-30 08:52:05,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:05,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394a0e33d030878a8bb3 HTTP/1.1" 200 None 2026-03-30 08:52:05,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:05,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:05,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:05,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca394a0e33d030878a8bb3 HTTP/1.1" 200 None 2026-03-30 08:52:05,678 - lava_callback - INFO - Completed processing callback for node 69ca394a0e33d030878a8bb3 2026-03-30 08:52:06,059 - lava_callback - INFO - Received callback for node 69ca39500e33d030878a8bbf INFO: 172.21.0.1:51776 - "POST /node/69ca39500e33d030878a8bbf HTTP/1.0" 202 Accepted 2026-03-30 08:52:06,067 - lava_callback - INFO - Starting async job submit for node 69ca39500e33d030878a8bbf 2026-03-30 08:52:06,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:06,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39500e33d030878a8bbf HTTP/1.1" 200 None 2026-03-30 08:52:06,109 - lava_callback - DEBUG - Processing node 69ca39500e33d030878a8bbf: baseline-arm64-broonie 2026-03-30 08:52:06,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:06,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:06,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:52:06,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:06,518 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39500e33d030878a8bbf/log.txt.gz 2026-03-30 08:52:06,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:06,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:06,830 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39500e33d030878a8bbf/lava-logs.yaml 2026-03-30 08:52:06,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:06,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:06,968 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39500e33d030878a8bbf/lava_callback.json.gz 2026-03-30 08:52:06,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:07,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39500e33d030878a8bbf HTTP/1.1" 200 None 2026-03-30 08:52:07,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:07,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:07,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:07,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39500e33d030878a8bbf HTTP/1.1" 200 None 2026-03-30 08:52:07,060 - lava_callback - INFO - Completed processing callback for node 69ca39500e33d030878a8bbf 2026-03-30 08:52:07,527 - lava_callback - INFO - Received callback for node 69ca39340e33d030878a8b54 INFO: 172.21.0.1:51786 - "POST /node/69ca39340e33d030878a8b54 HTTP/1.0" 202 Accepted 2026-03-30 08:52:07,534 - lava_callback - INFO - Starting async job submit for node 69ca39340e33d030878a8b54 2026-03-30 08:52:07,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:07,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39340e33d030878a8b54 HTTP/1.1" 200 None 2026-03-30 08:52:07,585 - lava_callback - DEBUG - Processing node 69ca39340e33d030878a8b54: baseline-nfs-arm64 2026-03-30 08:52:07,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:08,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:08,107 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39340e33d030878a8b54/log.txt.gz 2026-03-30 08:52:08,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:08,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:08,490 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39340e33d030878a8b54/lava-logs.yaml 2026-03-30 08:52:08,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:08,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:08,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39340e33d030878a8b54/lava_callback.json.gz 2026-03-30 08:52:08,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:08,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39340e33d030878a8b54 HTTP/1.1" 200 None 2026-03-30 08:52:08,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:08,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:08,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:08,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39340e33d030878a8b54 HTTP/1.1" 200 None 2026-03-30 08:52:08,714 - lava_callback - INFO - Completed processing callback for node 69ca39340e33d030878a8b54 INFO: 172.21.0.1:42890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42904 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:52:19,942 - lava_callback - INFO - Received callback for node 69ca395a0e33d030878a8bde INFO: 172.21.0.1:50952 - "POST /node/69ca395a0e33d030878a8bde HTTP/1.0" 202 Accepted 2026-03-30 08:52:19,948 - lava_callback - INFO - Starting async job submit for node 69ca395a0e33d030878a8bde 2026-03-30 08:52:19,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:19,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395a0e33d030878a8bde HTTP/1.1" 200 None 2026-03-30 08:52:19,990 - lava_callback - DEBUG - Processing node 69ca395a0e33d030878a8bde: baseline-arm64-pengutronix 2026-03-30 08:52:20,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:20,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:20,291 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395a0e33d030878a8bde/log.txt.gz 2026-03-30 08:52:20,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:20,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:20,610 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395a0e33d030878a8bde/lava-logs.yaml 2026-03-30 08:52:20,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:20,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:20,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395a0e33d030878a8bde/lava_callback.json.gz 2026-03-30 08:52:20,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:20,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395a0e33d030878a8bde HTTP/1.1" 200 None 2026-03-30 08:52:20,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:20,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:20,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:20,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca395a0e33d030878a8bde HTTP/1.1" 200 None 2026-03-30 08:52:20,858 - lava_callback - INFO - Completed processing callback for node 69ca395a0e33d030878a8bde 2026-03-30 08:52:23,262 - lava_callback - INFO - Received callback for node 69ca394e0e33d030878a8bbc INFO: 172.21.0.1:50964 - "POST /node/69ca394e0e33d030878a8bbc HTTP/1.0" 202 Accepted 2026-03-30 08:52:23,268 - lava_callback - INFO - Starting async job submit for node 69ca394e0e33d030878a8bbc 2026-03-30 08:52:23,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:23,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394e0e33d030878a8bbc HTTP/1.1" 200 None 2026-03-30 08:52:23,306 - lava_callback - DEBUG - Processing node 69ca394e0e33d030878a8bbc: baseline-arm64-broonie 2026-03-30 08:52:23,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:23,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:23,729 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394e0e33d030878a8bbc/log.txt.gz 2026-03-30 08:52:23,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:24,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:24,062 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394e0e33d030878a8bbc/lava-logs.yaml 2026-03-30 08:52:24,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:24,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:24,229 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394e0e33d030878a8bbc/lava_callback.json.gz 2026-03-30 08:52:24,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:24,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394e0e33d030878a8bbc HTTP/1.1" 200 None 2026-03-30 08:52:24,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:24,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:24,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:24,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca394e0e33d030878a8bbc HTTP/1.1" 200 None 2026-03-30 08:52:24,313 - lava_callback - INFO - Completed processing callback for node 69ca394e0e33d030878a8bbc 2026-03-30 08:52:24,749 - lava_callback - INFO - Received callback for node 69ca39510e33d030878a8bc2 INFO: 172.21.0.1:50972 - "POST /node/69ca39510e33d030878a8bc2 HTTP/1.0" 202 Accepted 2026-03-30 08:52:24,755 - lava_callback - INFO - Starting async job submit for node 69ca39510e33d030878a8bc2 2026-03-30 08:52:24,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:24,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39510e33d030878a8bc2 HTTP/1.1" 200 None 2026-03-30 08:52:24,793 - lava_callback - DEBUG - Processing node 69ca39510e33d030878a8bc2: baseline-arm64-broonie 2026-03-30 08:52:25,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:25,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:25,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39510e33d030878a8bc2/log.txt.gz INFO: 172.21.0.1:50986 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:52:25,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:25,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:25,510 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39510e33d030878a8bc2/lava-logs.yaml 2026-03-30 08:52:25,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:25,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:25,645 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39510e33d030878a8bc2/lava_callback.json.gz 2026-03-30 08:52:25,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:25,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39510e33d030878a8bc2 HTTP/1.1" 200 None 2026-03-30 08:52:25,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:25,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:25,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:25,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39510e33d030878a8bc2 HTTP/1.1" 200 None 2026-03-30 08:52:25,752 - lava_callback - INFO - Completed processing callback for node 69ca39510e33d030878a8bc2 INFO: 172.21.0.1:50994 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:52:27,118 - lava_callback - INFO - Received callback for node 69ca395c0e33d030878a8be1 INFO: 172.21.0.1:51008 - "POST /node/69ca395c0e33d030878a8be1 HTTP/1.0" 202 Accepted 2026-03-30 08:52:27,124 - lava_callback - INFO - Starting async job submit for node 69ca395c0e33d030878a8be1 2026-03-30 08:52:27,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:27,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395c0e33d030878a8be1 HTTP/1.1" 200 None 2026-03-30 08:52:27,162 - lava_callback - DEBUG - Processing node 69ca395c0e33d030878a8be1: baseline-arm64-pengutronix 2026-03-30 08:52:27,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:27,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:27,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395c0e33d030878a8be1/log.txt.gz 2026-03-30 08:52:27,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:27,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:27,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395c0e33d030878a8be1/lava-logs.yaml 2026-03-30 08:52:27,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:27,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:27,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ca395c0e33d030878a8be1/lava_callback.json.gz 2026-03-30 08:52:27,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:27,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca395c0e33d030878a8be1 HTTP/1.1" 200 None 2026-03-30 08:52:27,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:28,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:28,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:28,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca395c0e33d030878a8be1 HTTP/1.1" 200 None 2026-03-30 08:52:28,029 - lava_callback - INFO - Completed processing callback for node 69ca395c0e33d030878a8be1 2026-03-30 08:52:36,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:36,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:47344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40068 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:52:55,171 - lava_callback - INFO - Received callback for node 69ca39460e33d030878a8bad INFO: 172.21.0.1:35734 - "POST /node/69ca39460e33d030878a8bad HTTP/1.0" 202 Accepted 2026-03-30 08:52:55,177 - lava_callback - INFO - Starting async job submit for node 69ca39460e33d030878a8bad 2026-03-30 08:52:55,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:55,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39460e33d030878a8bad HTTP/1.1" 200 None 2026-03-30 08:52:55,219 - lava_callback - DEBUG - Processing node 69ca39460e33d030878a8bad: baseline-arm64-broonie 2026-03-30 08:52:55,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:55,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:55,590 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39460e33d030878a8bad/log.txt.gz 2026-03-30 08:52:55,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:55,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:55,907 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39460e33d030878a8bad/lava-logs.yaml 2026-03-30 08:52:55,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:56,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:56,061 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca39460e33d030878a8bad/lava_callback.json.gz 2026-03-30 08:52:56,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:56,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39460e33d030878a8bad HTTP/1.1" 200 None 2026-03-30 08:52:56,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:56,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:56,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:56,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39460e33d030878a8bad HTTP/1.1" 200 None 2026-03-30 08:52:56,146 - lava_callback - INFO - Completed processing callback for node 69ca39460e33d030878a8bad INFO: 172.21.0.1:35746 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:52:58,377 - lava_callback - INFO - Received callback for node 69ca391b0e33d030878a8b00 INFO: 172.21.0.1:35760 - "POST /node/69ca391b0e33d030878a8b00 HTTP/1.0" 202 Accepted 2026-03-30 08:52:58,383 - lava_callback - INFO - Starting async job submit for node 69ca391b0e33d030878a8b00 2026-03-30 08:52:58,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:58,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391b0e33d030878a8b00 HTTP/1.1" 200 None 2026-03-30 08:52:58,422 - lava_callback - DEBUG - Processing node 69ca391b0e33d030878a8b00: baseline-arm64 2026-03-30 08:52:58,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:58,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:58,948 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391b0e33d030878a8b00/log.txt.gz 2026-03-30 08:52:59,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:59,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:59,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391b0e33d030878a8b00/lava-logs.yaml 2026-03-30 08:52:59,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:52:59,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:52:59,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca391b0e33d030878a8b00/lava_callback.json.gz 2026-03-30 08:52:59,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:59,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca391b0e33d030878a8b00 HTTP/1.1" 200 None 2026-03-30 08:52:59,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:59,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:52:59,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:52:59,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca391b0e33d030878a8b00 HTTP/1.1" 200 None 2026-03-30 08:52:59,595 - lava_callback - INFO - Completed processing callback for node 69ca391b0e33d030878a8b00 2026-03-30 08:53:01,062 - lava_callback - INFO - Received callback for node 69ca39d50e33d030878a92a9 INFO: 172.21.0.1:46632 - "POST /node/69ca39d50e33d030878a92a9 HTTP/1.0" 202 Accepted 2026-03-30 08:53:01,068 - lava_callback - INFO - Starting async job submit for node 69ca39d50e33d030878a92a9 2026-03-30 08:53:01,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:01,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39d50e33d030878a92a9 HTTP/1.1" 200 None 2026-03-30 08:53:01,086 - lava_callback - DEBUG - Processing node 69ca39d50e33d030878a92a9: baseline-arm 2026-03-30 08:53:01,091 - lava_callback - WARNING - Failed to upload log 2026-03-30 08:53:01,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:01,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:01,217 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca39d50e33d030878a92a9/lava-logs.yaml 2026-03-30 08:53:01,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:01,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:01,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca39d50e33d030878a92a9/lava_callback.json.gz 2026-03-30 08:53:01,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:01,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39d50e33d030878a92a9 HTTP/1.1" 200 None 2026-03-30 08:53:01,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:01,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:53:01,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:01,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39d50e33d030878a92a9 HTTP/1.1" 200 None 2026-03-30 08:53:01,404 - lava_callback - INFO - Completed processing callback for node 69ca39d50e33d030878a92a9 2026-03-30 08:53:06,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:06,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38738 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:53:43,472 - lava_callback - INFO - Received callback for node 69ca37a10e33d030878a89ac INFO: 172.21.0.1:46296 - "POST /node/69ca37a10e33d030878a89ac HTTP/1.0" 202 Accepted 2026-03-30 08:53:43,478 - lava_callback - INFO - Starting async job submit for node 69ca37a10e33d030878a89ac 2026-03-30 08:53:43,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:43,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a10e33d030878a89ac HTTP/1.1" 200 None 2026-03-30 08:53:43,526 - lava_callback - DEBUG - Processing node 69ca37a10e33d030878a89ac: ltp-cve 2026-03-30 08:53:43,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:44,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:44,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69ca37a10e33d030878a89ac/log.txt.gz 2026-03-30 08:53:44,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:44,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:44,554 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69ca37a10e33d030878a89ac/lava-logs.yaml 2026-03-30 08:53:44,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:44,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:44,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69ca37a10e33d030878a89ac/lava_callback.json.gz 2026-03-30 08:53:44,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:44,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a10e33d030878a89ac HTTP/1.1" 200 None 2026-03-30 08:53:44,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:44,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:53:44,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:44,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37a10e33d030878a89ac HTTP/1.1" 200 None 2026-03-30 08:53:44,853 - lava_callback - INFO - Completed processing callback for node 69ca37a10e33d030878a89ac INFO: 172.21.0.1:46298 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:53:49,007 - lava_callback - INFO - Received callback for node 69c79710be430e18c5d16373 INFO: 172.21.0.1:46308 - "POST /node/69c79710be430e18c5d16373 HTTP/1.0" 202 Accepted 2026-03-30 08:53:49,013 - lava_callback - INFO - Starting async job submit for node 69c79710be430e18c5d16373 2026-03-30 08:53:49,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:49,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79710be430e18c5d16373 HTTP/1.1" 200 None 2026-03-30 08:53:49,029 - lava_callback - DEBUG - Processing node 69c79710be430e18c5d16373: baseline-arm-baylibre 2026-03-30 08:53:49,029 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79710be430e18c5d16373 2026-03-30 08:53:49,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:49,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79710be430e18c5d16373 HTTP/1.1" 200 None 2026-03-30 08:53:49,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:49,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 08:53:49,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:49,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79710be430e18c5d16373 HTTP/1.1" 200 None 2026-03-30 08:53:49,089 - lava_callback - ERROR - Error processing callback for node 69c79710be430e18c5d16373: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:53:55,210 - lava_callback - INFO - Received callback for node 69ca394b0e33d030878a8bb6 INFO: 172.21.0.1:38548 - "POST /node/69ca394b0e33d030878a8bb6 HTTP/1.0" 202 Accepted 2026-03-30 08:53:55,216 - lava_callback - INFO - Starting async job submit for node 69ca394b0e33d030878a8bb6 2026-03-30 08:53:55,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:55,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394b0e33d030878a8bb6 HTTP/1.1" 200 None 2026-03-30 08:53:55,253 - lava_callback - DEBUG - Processing node 69ca394b0e33d030878a8bb6: baseline-arm64-broonie 2026-03-30 08:53:55,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:55,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:55,678 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394b0e33d030878a8bb6/log.txt.gz 2026-03-30 08:53:55,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:56,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:56,006 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394b0e33d030878a8bb6/lava-logs.yaml 2026-03-30 08:53:56,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:53:56,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:53:56,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ca394b0e33d030878a8bb6/lava_callback.json.gz 2026-03-30 08:53:56,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:56,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca394b0e33d030878a8bb6 HTTP/1.1" 200 None 2026-03-30 08:53:56,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:56,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:53:56,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:53:56,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca394b0e33d030878a8bb6 HTTP/1.1" 200 None 2026-03-30 08:53:56,247 - lava_callback - INFO - Completed processing callback for node 69ca394b0e33d030878a8bb6 INFO: 172.21.0.1:38558 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:54:06,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:06,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50002 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:54:14,210 - lava_callback - INFO - Received callback for node 69ca39200e33d030878a8b0c INFO: 172.21.0.1:47300 - "POST /node/69ca39200e33d030878a8b0c HTTP/1.0" 202 Accepted 2026-03-30 08:54:14,216 - lava_callback - INFO - Starting async job submit for node 69ca39200e33d030878a8b0c 2026-03-30 08:54:14,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:14,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39200e33d030878a8b0c HTTP/1.1" 200 None 2026-03-30 08:54:14,253 - lava_callback - DEBUG - Processing node 69ca39200e33d030878a8b0c: baseline-arm64 2026-03-30 08:54:14,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:14,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:14,688 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39200e33d030878a8b0c/log.txt.gz 2026-03-30 08:54:14,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:15,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:15,084 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39200e33d030878a8b0c/lava-logs.yaml 2026-03-30 08:54:15,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:47302 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 08:54:15,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:15,216 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39200e33d030878a8b0c/lava_callback.json.gz 2026-03-30 08:54:15,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:15,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39200e33d030878a8b0c HTTP/1.1" 200 None 2026-03-30 08:54:15,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:15,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:54:15,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:15,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39200e33d030878a8b0c HTTP/1.1" 200 None 2026-03-30 08:54:15,371 - lava_callback - INFO - Completed processing callback for node 69ca39200e33d030878a8b0c INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51072 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:54:28,927 - lava_callback - INFO - Received callback for node 69c7973bbe430e18c5d16411 INFO: 172.21.0.1:51080 - "POST /node/69c7973bbe430e18c5d16411 HTTP/1.0" 202 Accepted 2026-03-30 08:54:28,932 - lava_callback - INFO - Starting async job submit for node 69c7973bbe430e18c5d16411 2026-03-30 08:54:28,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:28,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7973bbe430e18c5d16411 HTTP/1.1" 200 None 2026-03-30 08:54:28,948 - lava_callback - DEBUG - Processing node 69c7973bbe430e18c5d16411: baseline-arm64-baylibre 2026-03-30 08:54:28,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7973bbe430e18c5d16411 2026-03-30 08:54:28,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:28,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7973bbe430e18c5d16411 HTTP/1.1" 200 None 2026-03-30 08:54:28,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:29,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:29,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:29,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7973bbe430e18c5d16411 HTTP/1.1" 200 None 2026-03-30 08:54:29,027 - lava_callback - ERROR - Error processing callback for node 69c7973bbe430e18c5d16411: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:29,610 - lava_callback - INFO - Received callback for node 69c7973cbe430e18c5d16414 INFO: 172.21.0.1:51086 - "POST /node/69c7973cbe430e18c5d16414 HTTP/1.0" 202 Accepted 2026-03-30 08:54:29,616 - lava_callback - INFO - Starting async job submit for node 69c7973cbe430e18c5d16414 2026-03-30 08:54:29,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:29,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7973cbe430e18c5d16414 HTTP/1.1" 200 None 2026-03-30 08:54:29,632 - lava_callback - DEBUG - Processing node 69c7973cbe430e18c5d16414: baseline-arm64-baylibre 2026-03-30 08:54:29,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7973cbe430e18c5d16414 2026-03-30 08:54:29,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:29,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7973cbe430e18c5d16414 HTTP/1.1" 200 None 2026-03-30 08:54:29,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:29,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:29,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:29,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7973cbe430e18c5d16414 HTTP/1.1" 200 None 2026-03-30 08:54:29,688 - lava_callback - ERROR - Error processing callback for node 69c7973cbe430e18c5d16414: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:30,270 - lava_callback - INFO - Received callback for node 69ca39300e33d030878a8b4b INFO: 172.21.0.1:46150 - "POST /node/69ca39300e33d030878a8b4b HTTP/1.0" 202 Accepted 2026-03-30 08:54:30,277 - lava_callback - INFO - Starting async job submit for node 69ca39300e33d030878a8b4b 2026-03-30 08:54:30,300 - lava_callback - INFO - Received callback for node 69c79741be430e18c5d16421 INFO: 172.21.0.1:46162 - "POST /node/69c79741be430e18c5d16421 HTTP/1.0" 202 Accepted 2026-03-30 08:54:30,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:30,307 - lava_callback - INFO - Starting async job submit for node 69c79741be430e18c5d16421 2026-03-30 08:54:30,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:30,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79741be430e18c5d16421 HTTP/1.1" 200 None 2026-03-30 08:54:30,323 - lava_callback - DEBUG - Processing node 69c79741be430e18c5d16421: baseline-arm64-baylibre 2026-03-30 08:54:30,324 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79741be430e18c5d16421 2026-03-30 08:54:30,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39300e33d030878a8b4b HTTP/1.1" 200 None 2026-03-30 08:54:30,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:30,326 - lava_callback - DEBUG - Processing node 69ca39300e33d030878a8b4b: baseline-nfs-arm64 2026-03-30 08:54:30,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79741be430e18c5d16421 HTTP/1.1" 200 None 2026-03-30 08:54:30,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:30,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:30,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:30,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79741be430e18c5d16421 HTTP/1.1" 200 None 2026-03-30 08:54:30,719 - lava_callback - ERROR - Error processing callback for node 69c79741be430e18c5d16421: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:30,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:30,988 - lava_callback - INFO - Received callback for node 69c79734be430e18c5d163fa INFO: 172.21.0.1:46176 - "POST /node/69c79734be430e18c5d163fa HTTP/1.0" 202 Accepted 2026-03-30 08:54:30,994 - lava_callback - INFO - Starting async job submit for node 69c79734be430e18c5d163fa 2026-03-30 08:54:30,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79734be430e18c5d163fa HTTP/1.1" 200 None 2026-03-30 08:54:31,012 - lava_callback - DEBUG - Processing node 69c79734be430e18c5d163fa: baseline-arm-baylibre 2026-03-30 08:54:31,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79734be430e18c5d163fa 2026-03-30 08:54:31,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79734be430e18c5d163fa HTTP/1.1" 200 None 2026-03-30 08:54:31,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 08:54:31,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79734be430e18c5d163fa HTTP/1.1" 200 None 2026-03-30 08:54:31,078 - lava_callback - ERROR - Error processing callback for node 69c79734be430e18c5d163fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:31,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:31,082 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39300e33d030878a8b4b/log.txt.gz 2026-03-30 08:54:31,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:31,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:31,586 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39300e33d030878a8b4b/lava-logs.yaml 2026-03-30 08:54:31,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:31,646 - lava_callback - INFO - Received callback for node 69c79739be430e18c5d1640c INFO: 172.21.0.1:46182 - "POST /node/69c79739be430e18c5d1640c HTTP/1.0" 202 Accepted 2026-03-30 08:54:31,655 - lava_callback - INFO - Starting async job submit for node 69c79739be430e18c5d1640c 2026-03-30 08:54:31,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79739be430e18c5d1640c HTTP/1.1" 200 None 2026-03-30 08:54:31,674 - lava_callback - DEBUG - Processing node 69c79739be430e18c5d1640c: baseline-arm64-baylibre 2026-03-30 08:54:31,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79739be430e18c5d1640c 2026-03-30 08:54:31,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79739be430e18c5d1640c HTTP/1.1" 200 None 2026-03-30 08:54:31,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:31,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79739be430e18c5d1640c HTTP/1.1" 200 None 2026-03-30 08:54:31,739 - lava_callback - ERROR - Error processing callback for node 69c79739be430e18c5d1640c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:31,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:31,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39300e33d030878a8b4b/lava_callback.json.gz 2026-03-30 08:54:31,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39300e33d030878a8b4b HTTP/1.1" 200 None 2026-03-30 08:54:31,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:54:31,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:31,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39300e33d030878a8b4b HTTP/1.1" 200 None 2026-03-30 08:54:31,921 - lava_callback - INFO - Completed processing callback for node 69ca39300e33d030878a8b4b 2026-03-30 08:54:32,312 - lava_callback - INFO - Received callback for node 69c7973ebe430e18c5d1641b INFO: 172.21.0.1:46184 - "POST /node/69c7973ebe430e18c5d1641b HTTP/1.0" 202 Accepted 2026-03-30 08:54:32,317 - lava_callback - INFO - Starting async job submit for node 69c7973ebe430e18c5d1641b 2026-03-30 08:54:32,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:32,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7973ebe430e18c5d1641b HTTP/1.1" 200 None 2026-03-30 08:54:32,334 - lava_callback - DEBUG - Processing node 69c7973ebe430e18c5d1641b: baseline-arm64-baylibre 2026-03-30 08:54:32,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7973ebe430e18c5d1641b 2026-03-30 08:54:32,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:32,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7973ebe430e18c5d1641b HTTP/1.1" 200 None 2026-03-30 08:54:32,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:32,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:32,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:32,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7973ebe430e18c5d1641b HTTP/1.1" 200 None 2026-03-30 08:54:32,414 - lava_callback - ERROR - Error processing callback for node 69c7973ebe430e18c5d1641b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:32,982 - lava_callback - INFO - Received callback for node 69c79740be430e18c5d1641e INFO: 172.21.0.1:46188 - "POST /node/69c79740be430e18c5d1641e HTTP/1.0" 202 Accepted 2026-03-30 08:54:32,988 - lava_callback - INFO - Starting async job submit for node 69c79740be430e18c5d1641e 2026-03-30 08:54:32,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:33,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79740be430e18c5d1641e HTTP/1.1" 200 None 2026-03-30 08:54:33,005 - lava_callback - DEBUG - Processing node 69c79740be430e18c5d1641e: baseline-arm64-baylibre 2026-03-30 08:54:33,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79740be430e18c5d1641e 2026-03-30 08:54:33,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:33,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79740be430e18c5d1641e HTTP/1.1" 200 None 2026-03-30 08:54:33,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:33,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:33,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:33,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79740be430e18c5d1641e HTTP/1.1" 200 None 2026-03-30 08:54:33,065 - lava_callback - ERROR - Error processing callback for node 69c79740be430e18c5d1641e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:36,262 - lava_callback - INFO - Received callback for node 69ca392c0e33d030878a8b42 INFO: 172.21.0.1:46194 - "POST /node/69ca392c0e33d030878a8b42 HTTP/1.0" 202 Accepted 2026-03-30 08:54:36,270 - lava_callback - INFO - Starting async job submit for node 69ca392c0e33d030878a8b42 2026-03-30 08:54:36,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:36,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392c0e33d030878a8b42 HTTP/1.1" 200 None 2026-03-30 08:54:36,314 - lava_callback - DEBUG - Processing node 69ca392c0e33d030878a8b42: baseline-nfs-arm64 2026-03-30 08:54:36,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:46202 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:54:36,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 08:54:36,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:37,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:37,097 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392c0e33d030878a8b42/log.txt.gz 2026-03-30 08:54:37,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:37,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:37,677 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392c0e33d030878a8b42/lava-logs.yaml 2026-03-30 08:54:37,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:54:37,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:54:37,861 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392c0e33d030878a8b42/lava_callback.json.gz 2026-03-30 08:54:37,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:37,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392c0e33d030878a8b42 HTTP/1.1" 200 None 2026-03-30 08:54:37,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:37,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:54:37,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:37,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca392c0e33d030878a8b42 HTTP/1.1" 200 None 2026-03-30 08:54:37,982 - lava_callback - INFO - Completed processing callback for node 69ca392c0e33d030878a8b42 2026-03-30 08:54:45,729 - lava_callback - INFO - Received callback for node 69c79752be430e18c5d1646f INFO: 172.21.0.1:54456 - "POST /node/69c79752be430e18c5d1646f HTTP/1.0" 202 Accepted 2026-03-30 08:54:45,735 - lava_callback - INFO - Starting async job submit for node 69c79752be430e18c5d1646f 2026-03-30 08:54:45,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:45,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79752be430e18c5d1646f HTTP/1.1" 200 None 2026-03-30 08:54:45,751 - lava_callback - DEBUG - Processing node 69c79752be430e18c5d1646f: baseline-arm64-clabbe 2026-03-30 08:54:45,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79752be430e18c5d1646f 2026-03-30 08:54:45,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:45,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79752be430e18c5d1646f HTTP/1.1" 200 None 2026-03-30 08:54:45,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:45,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:45,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:45,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79752be430e18c5d1646f HTTP/1.1" 200 None 2026-03-30 08:54:45,818 - lava_callback - ERROR - Error processing callback for node 69c79752be430e18c5d1646f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:54:46,415 - lava_callback - INFO - Received callback for node 69c79753be430e18c5d16474 INFO: 172.21.0.1:54468 - "POST /node/69c79753be430e18c5d16474 HTTP/1.0" 202 Accepted 2026-03-30 08:54:46,420 - lava_callback - INFO - Starting async job submit for node 69c79753be430e18c5d16474 2026-03-30 08:54:46,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:46,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79753be430e18c5d16474 HTTP/1.1" 200 None 2026-03-30 08:54:46,437 - lava_callback - DEBUG - Processing node 69c79753be430e18c5d16474: baseline-arm64-clabbe 2026-03-30 08:54:46,437 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79753be430e18c5d16474 2026-03-30 08:54:46,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:46,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79753be430e18c5d16474 HTTP/1.1" 200 None 2026-03-30 08:54:46,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:46,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:54:46,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:54:46,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79753be430e18c5d16474 HTTP/1.1" 200 None 2026-03-30 08:54:46,502 - lava_callback - ERROR - Error processing callback for node 69c79753be430e18c5d16474: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:55:03,873 - lava_callback - INFO - Received callback for node 69ca37a40e33d030878a89b2 INFO: 172.21.0.1:34504 - "POST /node/69ca37a40e33d030878a89b2 HTTP/1.0" 202 Accepted 2026-03-30 08:55:03,882 - lava_callback - INFO - Starting async job submit for node 69ca37a40e33d030878a89b2 2026-03-30 08:55:03,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:03,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a40e33d030878a89b2 HTTP/1.1" 200 None 2026-03-30 08:55:03,949 - lava_callback - DEBUG - Processing node 69ca37a40e33d030878a89b2: ltp-fs_bind 2026-03-30 08:55:04,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:55:04,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:04,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ca37a40e33d030878a89b2/log.txt.gz 2026-03-30 08:55:04,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:55:05,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:05,263 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ca37a40e33d030878a89b2/lava-logs.yaml 2026-03-30 08:55:05,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:55:05,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:05,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ca37a40e33d030878a89b2/lava_callback.json.gz 2026-03-30 08:55:05,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:05,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a40e33d030878a89b2 HTTP/1.1" 200 None 2026-03-30 08:55:05,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:05,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:55:05,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37a40e33d030878a89b2 HTTP/1.1" 200 None 2026-03-30 08:55:05,547 - lava_callback - INFO - Completed processing callback for node 69ca37a40e33d030878a89b2 2026-03-30 08:55:05,901 - lava_callback - INFO - Received callback for node 69c79754be430e18c5d16477 INFO: 172.21.0.1:34512 - "POST /node/69c79754be430e18c5d16477 HTTP/1.0" 202 Accepted 2026-03-30 08:55:05,909 - lava_callback - INFO - Starting async job submit for node 69c79754be430e18c5d16477 2026-03-30 08:55:05,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:05,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79754be430e18c5d16477 HTTP/1.1" 200 None 2026-03-30 08:55:05,940 - lava_callback - DEBUG - Processing node 69c79754be430e18c5d16477: baseline-arm64-clabbe 2026-03-30 08:55:05,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79754be430e18c5d16477 2026-03-30 08:55:05,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:05,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79754be430e18c5d16477 HTTP/1.1" 200 None 2026-03-30 08:55:05,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:05,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:55:05,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:06,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79754be430e18c5d16477 HTTP/1.1" 200 None 2026-03-30 08:55:06,039 - lava_callback - ERROR - Error processing callback for node 69c79754be430e18c5d16477: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34526 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:55:06,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:06,597 - lava_callback - INFO - Received callback for node 69c79755be430e18c5d1647a 2026-03-30 08:55:06,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34532 - "POST /node/69c79755be430e18c5d1647a HTTP/1.0" 202 Accepted 2026-03-30 08:55:06,608 - lava_callback - INFO - Starting async job submit for node 69c79755be430e18c5d1647a 2026-03-30 08:55:06,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:06,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79755be430e18c5d1647a HTTP/1.1" 200 None 2026-03-30 08:55:06,639 - lava_callback - DEBUG - Processing node 69c79755be430e18c5d1647a: baseline-arm64-clabbe 2026-03-30 08:55:06,639 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79755be430e18c5d1647a 2026-03-30 08:55:06,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:06,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79755be430e18c5d1647a HTTP/1.1" 200 None 2026-03-30 08:55:06,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:06,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:55:06,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:06,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79755be430e18c5d1647a HTTP/1.1" 200 None 2026-03-30 08:55:06,725 - lava_callback - ERROR - Error processing callback for node 69c79755be430e18c5d1647a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:55:07,268 - lava_callback - INFO - Received callback for node 69c79757be430e18c5d16480 INFO: 172.21.0.1:34544 - "POST /node/69c79757be430e18c5d16480 HTTP/1.0" 202 Accepted 2026-03-30 08:55:07,279 - lava_callback - INFO - Starting async job submit for node 69c79757be430e18c5d16480 2026-03-30 08:55:07,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:07,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79757be430e18c5d16480 HTTP/1.1" 200 None 2026-03-30 08:55:07,309 - lava_callback - DEBUG - Processing node 69c79757be430e18c5d16480: baseline-arm64-clabbe 2026-03-30 08:55:07,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79757be430e18c5d16480 2026-03-30 08:55:07,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:07,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79757be430e18c5d16480 HTTP/1.1" 200 None 2026-03-30 08:55:07,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:07,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:55:07,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:07,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79757be430e18c5d16480 HTTP/1.1" 200 None 2026-03-30 08:55:07,405 - lava_callback - ERROR - Error processing callback for node 69c79757be430e18c5d16480: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:55:08,040 - lava_callback - INFO - Received callback for node 69c79758be430e18c5d16483 INFO: 172.21.0.1:34546 - "POST /node/69c79758be430e18c5d16483 HTTP/1.0" 202 Accepted 2026-03-30 08:55:08,049 - lava_callback - INFO - Starting async job submit for node 69c79758be430e18c5d16483 2026-03-30 08:55:08,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:08,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79758be430e18c5d16483 HTTP/1.1" 200 None 2026-03-30 08:55:08,081 - lava_callback - DEBUG - Processing node 69c79758be430e18c5d16483: baseline-arm64-clabbe 2026-03-30 08:55:08,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79758be430e18c5d16483 2026-03-30 08:55:08,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:08,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79758be430e18c5d16483 HTTP/1.1" 200 None 2026-03-30 08:55:08,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:08,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 08:55:08,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:08,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79758be430e18c5d16483 HTTP/1.1" 200 None 2026-03-30 08:55:08,184 - lava_callback - ERROR - Error processing callback for node 69c79758be430e18c5d16483: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:55:23,781 - lava_callback - INFO - Received callback for node 69ca37a50e33d030878a89b5 INFO: 172.21.0.1:42820 - "POST /node/69ca37a50e33d030878a89b5 HTTP/1.0" 202 Accepted 2026-03-30 08:55:23,792 - lava_callback - INFO - Starting async job submit for node 69ca37a50e33d030878a89b5 2026-03-30 08:55:23,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:23,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a50e33d030878a89b5 HTTP/1.1" 200 None 2026-03-30 08:55:23,844 - lava_callback - DEBUG - Processing node 69ca37a50e33d030878a89b5: ltp-fsx 2026-03-30 08:55:24,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:55:24,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:24,834 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69ca37a50e33d030878a89b5/log.txt.gz 2026-03-30 08:55:25,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42822 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:55:25,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:25,816 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69ca37a50e33d030878a89b5/lava-logs.yaml 2026-03-30 08:55:25,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:55:26,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:26,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69ca37a50e33d030878a89b5/lava_callback.json.gz 2026-03-30 08:55:26,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:26,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a50e33d030878a89b5 HTTP/1.1" 200 None 2026-03-30 08:55:26,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:55:26,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:26,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37a50e33d030878a89b5 HTTP/1.1" 200 None 2026-03-30 08:55:26,396 - lava_callback - INFO - Completed processing callback for node 69ca37a50e33d030878a89b5 INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:55:28,977 - lava_callback - INFO - Received callback for node 69c7977dbe430e18c5d164d6 INFO: 172.21.0.1:42836 - "POST /node/69c7977dbe430e18c5d164d6 HTTP/1.0" 202 Accepted 2026-03-30 08:55:28,984 - lava_callback - INFO - Starting async job submit for node 69c7977dbe430e18c5d164d6 2026-03-30 08:55:28,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:29,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7977dbe430e18c5d164d6 HTTP/1.1" 200 None 2026-03-30 08:55:29,008 - lava_callback - DEBUG - Processing node 69c7977dbe430e18c5d164d6: baseline-arm-baylibre 2026-03-30 08:55:29,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7977dbe430e18c5d164d6 2026-03-30 08:55:29,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:29,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7977dbe430e18c5d164d6 HTTP/1.1" 200 None 2026-03-30 08:55:29,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:29,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 08:55:29,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:29,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7977dbe430e18c5d164d6 HTTP/1.1" 200 None 2026-03-30 08:55:29,083 - lava_callback - ERROR - Error processing callback for node 69c7977dbe430e18c5d164d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 08:55:35,536 - lava_callback - INFO - Received callback for node 69ca37a70e33d030878a89b8 INFO: 172.21.0.1:37948 - "POST /node/69ca37a70e33d030878a89b8 HTTP/1.0" 202 Accepted 2026-03-30 08:55:35,546 - lava_callback - INFO - Starting async job submit for node 69ca37a70e33d030878a89b8 2026-03-30 08:55:35,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:35,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a70e33d030878a89b8 HTTP/1.1" 200 None 2026-03-30 08:55:35,597 - lava_callback - DEBUG - Processing node 69ca37a70e33d030878a89b8: ltp-input 2026-03-30 08:55:36,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:37950 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:55:36,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:36,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 08:55:36,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:36,781 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69ca37a70e33d030878a89b8/log.txt.gz 2026-03-30 08:55:37,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:55:37,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:37,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69ca37a70e33d030878a89b8/lava-logs.yaml 2026-03-30 08:55:37,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:55:38,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:55:38,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69ca37a70e33d030878a89b8/lava_callback.json.gz 2026-03-30 08:55:38,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:38,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a70e33d030878a89b8 HTTP/1.1" 200 None 2026-03-30 08:55:38,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:38,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:55:38,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:55:38,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37a70e33d030878a89b8 HTTP/1.1" 200 None 2026-03-30 08:55:38,493 - lava_callback - INFO - Completed processing callback for node 69ca37a70e33d030878a89b8 INFO: 172.21.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37336 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:56:06,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:56:06,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 08:56:08,984 - lava_callback - INFO - Received callback for node 69c797a2be430e18c5d16530 INFO: 172.21.0.1:37342 - "POST /node/69c797a2be430e18c5d16530 HTTP/1.0" 202 Accepted 2026-03-30 08:56:08,991 - lava_callback - INFO - Starting async job submit for node 69c797a2be430e18c5d16530 2026-03-30 08:56:08,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:56:09,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c797a2be430e18c5d16530 HTTP/1.1" 200 None 2026-03-30 08:56:09,010 - lava_callback - DEBUG - Processing node 69c797a2be430e18c5d16530: baseline-arm-baylibre 2026-03-30 08:56:09,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c797a2be430e18c5d16530 2026-03-30 08:56:09,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:56:09,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c797a2be430e18c5d16530 HTTP/1.1" 200 None 2026-03-30 08:56:09,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:56:09,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 08:56:09,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:56:09,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c797a2be430e18c5d16530 HTTP/1.1" 200 None 2026-03-30 08:56:09,090 - lava_callback - ERROR - Error processing callback for node 69c797a2be430e18c5d16530: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57454 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:57:41,859 - lava_callback - INFO - Received callback for node 69ca392a0e33d030878a8b3c INFO: 172.21.0.1:51842 - "POST /node/69ca392a0e33d030878a8b3c HTTP/1.0" 202 Accepted 2026-03-30 08:57:41,866 - lava_callback - INFO - Starting async job submit for node 69ca392a0e33d030878a8b3c 2026-03-30 08:57:41,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:57:41,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392a0e33d030878a8b3c HTTP/1.1" 200 None 2026-03-30 08:57:41,924 - lava_callback - DEBUG - Processing node 69ca392a0e33d030878a8b3c: baseline-arm64 2026-03-30 08:57:42,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:57:42,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:57:42,474 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca392a0e33d030878a8b3c/log.txt.gz 2026-03-30 08:57:42,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:57:42,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:57:42,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca392a0e33d030878a8b3c/lava-logs.yaml 2026-03-30 08:57:43,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:57:43,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:57:43,159 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca392a0e33d030878a8b3c/lava_callback.json.gz DEBUG: Setup failed for 69ca392a0e33d030878a8b3c. Transitting job node result: incomplete -> fail 2026-03-30 08:57:43,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:57:43,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392a0e33d030878a8b3c HTTP/1.1" 200 None 2026-03-30 08:57:43,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:57:43,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:57:43,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:57:43,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca392a0e33d030878a8b3c HTTP/1.1" 200 None 2026-03-30 08:57:43,404 - lava_callback - INFO - Completed processing callback for node 69ca392a0e33d030878a8b3c INFO: 172.21.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48270 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:58:03,764 - lava_callback - INFO - Received callback for node 69ca37a80e33d030878a89bb INFO: 172.21.0.1:38340 - "POST /node/69ca37a80e33d030878a89bb HTTP/1.0" 202 Accepted 2026-03-30 08:58:03,771 - lava_callback - INFO - Starting async job submit for node 69ca37a80e33d030878a89bb 2026-03-30 08:58:03,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:03,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a80e33d030878a89bb HTTP/1.1" 200 None 2026-03-30 08:58:03,810 - lava_callback - DEBUG - Processing node 69ca37a80e33d030878a89bb: ltp-ipc 2026-03-30 08:58:04,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:04,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:04,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69ca37a80e33d030878a89bb/log.txt.gz 2026-03-30 08:58:04,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:04,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:04,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69ca37a80e33d030878a89bb/lava-logs.yaml 2026-03-30 08:58:04,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:05,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:05,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69ca37a80e33d030878a89bb/lava_callback.json.gz 2026-03-30 08:58:05,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:05,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37a80e33d030878a89bb HTTP/1.1" 200 None 2026-03-30 08:58:05,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:05,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:58:05,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:05,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37a80e33d030878a89bb HTTP/1.1" 200 None 2026-03-30 08:58:05,165 - lava_callback - INFO - Completed processing callback for node 69ca37a80e33d030878a89bb INFO: 172.21.0.1:38342 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:58:06,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:06,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44192 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:58:29,134 - lava_callback - INFO - Received callback for node 69c7982fbe430e18c5d165ea INFO: 172.21.0.1:44204 - "POST /node/69c7982fbe430e18c5d165ea HTTP/1.0" 202 Accepted 2026-03-30 08:58:29,143 - lava_callback - INFO - Starting async job submit for node 69c7982fbe430e18c5d165ea 2026-03-30 08:58:29,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:29,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7982fbe430e18c5d165ea HTTP/1.1" 200 None 2026-03-30 08:58:29,181 - lava_callback - DEBUG - Processing node 69c7982fbe430e18c5d165ea: baseline-arm-baylibre 2026-03-30 08:58:29,181 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7982fbe430e18c5d165ea 2026-03-30 08:58:29,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:29,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7982fbe430e18c5d165ea HTTP/1.1" 200 None 2026-03-30 08:58:29,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:29,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 08:58:29,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:29,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7982fbe430e18c5d165ea HTTP/1.1" 200 None 2026-03-30 08:58:29,244 - lava_callback - ERROR - Error processing callback for node 69c7982fbe430e18c5d165ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52782 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:58:56,532 - lava_callback - INFO - Received callback for node 69ca3b2c0e33d030878a93a7 INFO: 172.21.0.1:52794 - "POST /node/69ca3b2c0e33d030878a93a7 HTTP/1.0" 202 Accepted 2026-03-30 08:58:56,539 - lava_callback - INFO - Starting async job submit for node 69ca3b2c0e33d030878a93a7 2026-03-30 08:58:56,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:56,556 - lava_callback - INFO - Received callback for node 69ca3b280e33d030878a93a4 INFO: 172.21.0.1:52806 - "POST /node/69ca3b280e33d030878a93a4 HTTP/1.0" 202 Accepted 2026-03-30 08:58:56,562 - lava_callback - INFO - Starting async job submit for node 69ca3b280e33d030878a93a4 2026-03-30 08:58:56,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:56,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3b2c0e33d030878a93a7 HTTP/1.1" 200 None 2026-03-30 08:58:56,595 - lava_callback - DEBUG - Processing node 69ca3b2c0e33d030878a93a7: baseline-arm64 2026-03-30 08:58:56,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3b280e33d030878a93a4 HTTP/1.1" 200 None 2026-03-30 08:58:56,621 - lava_callback - DEBUG - Processing node 69ca3b280e33d030878a93a4: baseline-arm64 2026-03-30 08:58:56,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:56,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:57,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:57,026 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3b2c0e33d030878a93a7/log.txt.gz 2026-03-30 08:58:57,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:57,119 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3b280e33d030878a93a4/log.txt.gz 2026-03-30 08:58:57,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:57,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:57,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:57,396 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3b2c0e33d030878a93a7/lava-logs.yaml 2026-03-30 08:58:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:57,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:57,491 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3b280e33d030878a93a4/lava-logs.yaml 2026-03-30 08:58:57,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:58:57,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:57,550 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3b2c0e33d030878a93a7/lava_callback.json.gz 2026-03-30 08:58:57,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:57,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3b2c0e33d030878a93a7 HTTP/1.1" 200 None 2026-03-30 08:58:57,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:57,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:58:57,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:57,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca3b2c0e33d030878a93a7 HTTP/1.1" 200 None 2026-03-30 08:58:57,640 - lava_callback - INFO - Completed processing callback for node 69ca3b2c0e33d030878a93a7 2026-03-30 08:58:57,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:58:57,662 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3b280e33d030878a93a4/lava_callback.json.gz 2026-03-30 08:58:57,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:57,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3b280e33d030878a93a4 HTTP/1.1" 200 None 2026-03-30 08:58:57,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:57,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:58:57,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:58:57,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca3b280e33d030878a93a4 HTTP/1.1" 200 None 2026-03-30 08:58:57,798 - lava_callback - INFO - Completed processing callback for node 69ca3b280e33d030878a93a4 2026-03-30 08:59:03,711 - lava_callback - INFO - Received callback for node 69ca37aa0e33d030878a89be INFO: 172.21.0.1:38126 - "POST /node/69ca37aa0e33d030878a89be HTTP/1.0" 202 Accepted 2026-03-30 08:59:03,717 - lava_callback - INFO - Starting async job submit for node 69ca37aa0e33d030878a89be 2026-03-30 08:59:03,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:03,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37aa0e33d030878a89be HTTP/1.1" 200 None 2026-03-30 08:59:03,759 - lava_callback - DEBUG - Processing node 69ca37aa0e33d030878a89be: ltp-pty 2026-03-30 08:59:04,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:04,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:04,271 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69ca37aa0e33d030878a89be/log.txt.gz 2026-03-30 08:59:04,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:04,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:04,688 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69ca37aa0e33d030878a89be/lava-logs.yaml 2026-03-30 08:59:04,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:04,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:04,843 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69ca37aa0e33d030878a89be/lava_callback.json.gz 2026-03-30 08:59:04,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:04,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37aa0e33d030878a89be HTTP/1.1" 200 None 2026-03-30 08:59:04,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:04,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:59:04,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:04,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37aa0e33d030878a89be HTTP/1.1" 200 None 2026-03-30 08:59:04,933 - lava_callback - INFO - Completed processing callback for node 69ca37aa0e33d030878a89be INFO: 172.21.0.1:38134 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:59:06,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:06,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 08:59:10,307 - lava_callback - INFO - Received callback for node 69ca37690e33d030878a88f0 INFO: 172.21.0.1:55410 - "POST /node/69ca37690e33d030878a88f0 HTTP/1.0" 202 Accepted 2026-03-30 08:59:10,314 - lava_callback - INFO - Starting async job submit for node 69ca37690e33d030878a88f0 2026-03-30 08:59:10,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:10,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37690e33d030878a88f0 HTTP/1.1" 200 None 2026-03-30 08:59:10,382 - lava_callback - DEBUG - Processing node 69ca37690e33d030878a88f0: baseline-arm 2026-03-30 08:59:10,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:11,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:11,146 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37690e33d030878a88f0/log.txt.gz 2026-03-30 08:59:11,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:11,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:11,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37690e33d030878a88f0/lava-logs.yaml 2026-03-30 08:59:11,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:11,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:11,895 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca37690e33d030878a88f0/lava_callback.json.gz DEBUG: Setup failed for 69ca37690e33d030878a88f0. Transitting job node result: incomplete -> fail 2026-03-30 08:59:12,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:12,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37690e33d030878a88f0 HTTP/1.1" 200 None 2026-03-30 08:59:12,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:12,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:59:12,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:12,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37690e33d030878a88f0 HTTP/1.1" 200 None 2026-03-30 08:59:12,201 - lava_callback - INFO - Completed processing callback for node 69ca37690e33d030878a88f0 INFO: 172.21.0.1:55418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55430 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:59:23,952 - lava_callback - INFO - Received callback for node 69ca37ab0e33d030878a89c1 INFO: 172.21.0.1:59676 - "POST /node/69ca37ab0e33d030878a89c1 HTTP/1.0" 202 Accepted 2026-03-30 08:59:23,958 - lava_callback - INFO - Starting async job submit for node 69ca37ab0e33d030878a89c1 2026-03-30 08:59:23,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:23,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37ab0e33d030878a89c1 HTTP/1.1" 200 None 2026-03-30 08:59:24,000 - lava_callback - DEBUG - Processing node 69ca37ab0e33d030878a89c1: ltp-sched 2026-03-30 08:59:24,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:24,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:24,660 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69ca37ab0e33d030878a89c1/log.txt.gz 2026-03-30 08:59:24,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:25,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:25,113 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69ca37ab0e33d030878a89c1/lava-logs.yaml 2026-03-30 08:59:25,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:25,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:25,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69ca37ab0e33d030878a89c1/lava_callback.json.gz 2026-03-30 08:59:25,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:25,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37ab0e33d030878a89c1 HTTP/1.1" 200 None 2026-03-30 08:59:25,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:25,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:59:25,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:25,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37ab0e33d030878a89c1 HTTP/1.1" 200 None 2026-03-30 08:59:25,379 - lava_callback - INFO - Completed processing callback for node 69ca37ab0e33d030878a89c1 INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59702 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:59:35,883 - lava_callback - INFO - Received callback for node 69ca37ad0e33d030878a89c4 INFO: 172.21.0.1:58528 - "POST /node/69ca37ad0e33d030878a89c4 HTTP/1.0" 202 Accepted 2026-03-30 08:59:35,890 - lava_callback - INFO - Starting async job submit for node 69ca37ad0e33d030878a89c4 2026-03-30 08:59:35,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:35,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37ad0e33d030878a89c4 HTTP/1.1" 200 None 2026-03-30 08:59:35,930 - lava_callback - DEBUG - Processing node 69ca37ad0e33d030878a89c4: ltp-smoke 2026-03-30 08:59:36,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:36,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:36,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca37ad0e33d030878a89c4/log.txt.gz INFO: 172.21.0.1:58532 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:59:36,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:36,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:36,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 08:59:36,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:36,900 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca37ad0e33d030878a89c4/lava-logs.yaml 2026-03-30 08:59:36,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:37,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:37,081 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca37ad0e33d030878a89c4/lava_callback.json.gz 2026-03-30 08:59:37,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:37,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37ad0e33d030878a89c4 HTTP/1.1" 200 None 2026-03-30 08:59:37,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:37,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 08:59:37,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:37,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37ad0e33d030878a89c4 HTTP/1.1" 200 None 2026-03-30 08:59:37,189 - lava_callback - INFO - Completed processing callback for node 69ca37ad0e33d030878a89c4 INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:59:49,259 - lava_callback - INFO - Received callback for node 69c79877be430e18c5d16620 INFO: 172.21.0.1:36962 - "POST /node/69c79877be430e18c5d16620 HTTP/1.0" 202 Accepted 2026-03-30 08:59:49,265 - lava_callback - INFO - Starting async job submit for node 69c79877be430e18c5d16620 2026-03-30 08:59:49,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:49,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79877be430e18c5d16620 HTTP/1.1" 200 None 2026-03-30 08:59:49,281 - lava_callback - DEBUG - Processing node 69c79877be430e18c5d16620: baseline-arm-baylibre 2026-03-30 08:59:49,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79877be430e18c5d16620 2026-03-30 08:59:49,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:49,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79877be430e18c5d16620 HTTP/1.1" 200 None 2026-03-30 08:59:49,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:49,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 08:59:49,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:49,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79877be430e18c5d16620 HTTP/1.1" 200 None 2026-03-30 08:59:49,348 - lava_callback - ERROR - Error processing callback for node 69c79877be430e18c5d16620: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46492 - "GET / HTTP/1.0" 200 OK 2026-03-30 08:59:57,317 - lava_callback - INFO - Received callback for node 69ca39210e33d030878a8b11 INFO: 172.21.0.1:46508 - "POST /node/69ca39210e33d030878a8b11 HTTP/1.0" 202 Accepted 2026-03-30 08:59:57,325 - lava_callback - INFO - Starting async job submit for node 69ca39210e33d030878a8b11 2026-03-30 08:59:57,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:57,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39210e33d030878a8b11 HTTP/1.1" 200 None 2026-03-30 08:59:57,388 - lava_callback - DEBUG - Processing node 69ca39210e33d030878a8b11: baseline-arm64 2026-03-30 08:59:58,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:58,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:58,302 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39210e33d030878a8b11/log.txt.gz 2026-03-30 08:59:58,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:59,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:59,006 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39210e33d030878a8b11/lava-logs.yaml 2026-03-30 08:59:59,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 08:59:59,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 08:59:59,182 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39210e33d030878a8b11/lava_callback.json.gz 2026-03-30 08:59:59,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:59,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39210e33d030878a8b11 HTTP/1.1" 200 None 2026-03-30 08:59:59,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:59,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 08:59:59,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 08:59:59,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39210e33d030878a8b11 HTTP/1.1" 200 None 2026-03-30 08:59:59,278 - lava_callback - INFO - Completed processing callback for node 69ca39210e33d030878a8b11 INFO: 172.21.0.1:37354 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:00:06,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:00:06,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:01:05,935 - lava_callback - INFO - Received callback for node 69c798c0be430e18c5d16642 INFO: 172.21.0.1:57100 - "POST /node/69c798c0be430e18c5d16642 HTTP/1.0" 202 Accepted 2026-03-30 09:01:05,941 - lava_callback - INFO - Starting async job submit for node 69c798c0be430e18c5d16642 2026-03-30 09:01:05,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:01:05,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c798c0be430e18c5d16642 HTTP/1.1" 200 None 2026-03-30 09:01:05,956 - lava_callback - DEBUG - Processing node 69c798c0be430e18c5d16642: baseline-arm-clabbe 2026-03-30 09:01:05,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c798c0be430e18c5d16642 2026-03-30 09:01:05,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:01:05,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c798c0be430e18c5d16642 HTTP/1.1" 200 None 2026-03-30 09:01:05,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:01:05,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 09:01:05,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:01:06,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c798c0be430e18c5d16642 HTTP/1.1" 200 None 2026-03-30 09:01:06,015 - lava_callback - ERROR - Error processing callback for node 69c798c0be430e18c5d16642: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57116 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:02:08,940 - lava_callback - INFO - Received callback for node 69ca39560e33d030878a8bd1 INFO: 172.21.0.1:57120 - "POST /node/69ca39560e33d030878a8bd1 HTTP/1.0" 202 Accepted 2026-03-30 09:02:08,947 - lava_callback - INFO - Starting async job submit for node 69ca39560e33d030878a8bd1 2026-03-30 09:02:08,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:08,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39560e33d030878a8bd1 HTTP/1.1" 200 None 2026-03-30 09:02:09,000 - lava_callback - DEBUG - Processing node 69ca39560e33d030878a8bd1: baseline-arm64-clabbe 2026-03-30 09:02:09,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:02:10,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:02:10,030 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69ca39560e33d030878a8bd1/log.txt.gz 2026-03-30 09:02:10,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:02:10,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:02:10,583 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69ca39560e33d030878a8bd1/lava-logs.yaml 2026-03-30 09:02:10,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:02:10,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:02:10,784 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69ca39560e33d030878a8bd1/lava_callback.json.gz 2026-03-30 09:02:10,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:10,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39560e33d030878a8bd1 HTTP/1.1" 200 None 2026-03-30 09:02:10,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:02:10,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:10,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39560e33d030878a8bd1 HTTP/1.1" 200 None 2026-03-30 09:02:10,921 - lava_callback - INFO - Completed processing callback for node 69ca39560e33d030878a8bd1 INFO: 172.21.0.1:41406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41416 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:02:23,776 - lava_callback - INFO - Received callback for node 69ca37b00e33d030878a89ca INFO: 172.21.0.1:33512 - "POST /node/69ca37b00e33d030878a89ca HTTP/1.0" 202 Accepted 2026-03-30 09:02:23,784 - lava_callback - INFO - Starting async job submit for node 69ca37b00e33d030878a89ca 2026-03-30 09:02:23,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:23,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37b00e33d030878a89ca HTTP/1.1" 200 None 2026-03-30 09:02:23,825 - lava_callback - DEBUG - Processing node 69ca37b00e33d030878a89ca: ltp-syscalls-ipc 2026-03-30 09:02:24,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:02:24,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:02:24,779 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69ca37b00e33d030878a89ca/log.txt.gz 2026-03-30 09:02:25,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:02:25,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:02:25,210 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69ca37b00e33d030878a89ca/lava-logs.yaml 2026-03-30 09:02:25,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33514 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:02:25,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:02:25,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69ca37b00e33d030878a89ca/lava_callback.json.gz 2026-03-30 09:02:25,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:25,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37b00e33d030878a89ca HTTP/1.1" 200 None 2026-03-30 09:02:25,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:25,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 09:02:25,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:25,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37b00e33d030878a89ca HTTP/1.1" 200 None 2026-03-30 09:02:25,890 - lava_callback - INFO - Completed processing callback for node 69ca37b00e33d030878a89ca INFO: 172.21.0.1:33528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49890 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:02:36,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:36,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 09:02:46,117 - lava_callback - INFO - Received callback for node 69c7992cbe430e18c5d1667f INFO: 172.21.0.1:43922 - "POST /node/69c7992cbe430e18c5d1667f HTTP/1.0" 202 Accepted 2026-03-30 09:02:46,123 - lava_callback - INFO - Starting async job submit for node 69c7992cbe430e18c5d1667f 2026-03-30 09:02:46,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:46,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7992cbe430e18c5d1667f HTTP/1.1" 200 None 2026-03-30 09:02:46,153 - lava_callback - DEBUG - Processing node 69c7992cbe430e18c5d1667f: baseline-arm-clabbe 2026-03-30 09:02:46,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7992cbe430e18c5d1667f 2026-03-30 09:02:46,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:46,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7992cbe430e18c5d1667f HTTP/1.1" 200 None 2026-03-30 09:02:46,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:46,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 09:02:46,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:02:46,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7992cbe430e18c5d1667f HTTP/1.1" 200 None 2026-03-30 09:02:46,255 - lava_callback - ERROR - Error processing callback for node 69c7992cbe430e18c5d1667f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38098 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:03:03,789 - lava_callback - INFO - Received callback for node 69ca37b10e33d030878a89cd INFO: 172.21.0.1:48452 - "POST /node/69ca37b10e33d030878a89cd HTTP/1.0" 202 Accepted 2026-03-30 09:03:03,799 - lava_callback - INFO - Starting async job submit for node 69ca37b10e33d030878a89cd 2026-03-30 09:03:03,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:03,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37b10e33d030878a89cd HTTP/1.1" 200 None 2026-03-30 09:03:03,871 - lava_callback - DEBUG - Processing node 69ca37b10e33d030878a89cd: ltp-timers 2026-03-30 09:03:04,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:03:04,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:03:04,653 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69ca37b10e33d030878a89cd/log.txt.gz 2026-03-30 09:03:04,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:03:05,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:03:05,043 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69ca37b10e33d030878a89cd/lava-logs.yaml 2026-03-30 09:03:05,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:03:05,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:03:05,932 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69ca37b10e33d030878a89cd/lava_callback.json.gz 2026-03-30 09:03:05,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:05,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37b10e33d030878a89cd HTTP/1.1" 200 None 2026-03-30 09:03:05,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:06,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 09:03:06,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:06,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37b10e33d030878a89cd HTTP/1.1" 200 None 2026-03-30 09:03:06,029 - lava_callback - INFO - Completed processing callback for node 69ca37b10e33d030878a89cd INFO: 172.21.0.1:48456 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:03:06,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:06,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:03:22,940 - lava_callback - INFO - Received callback for node 69ca39310e33d030878a8b4e INFO: 172.21.0.1:33782 - "POST /node/69ca39310e33d030878a8b4e HTTP/1.0" 202 Accepted 2026-03-30 09:03:22,951 - lava_callback - INFO - Starting async job submit for node 69ca39310e33d030878a8b4e 2026-03-30 09:03:22,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:23,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39310e33d030878a8b4e HTTP/1.1" 200 None 2026-03-30 09:03:23,003 - lava_callback - DEBUG - Processing node 69ca39310e33d030878a8b4e: baseline-nfs-arm64 2026-03-30 09:03:24,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:03:24,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:03:24,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39310e33d030878a8b4e/log.txt.gz 2026-03-30 09:03:25,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33798 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:03:25,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:03:25,872 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39310e33d030878a8b4e/lava-logs.yaml 2026-03-30 09:03:25,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:03:26,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:03:26,087 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39310e33d030878a8b4e/lava_callback.json.gz 2026-03-30 09:03:26,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:26,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39310e33d030878a8b4e HTTP/1.1" 200 None 2026-03-30 09:03:26,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:26,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:03:26,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:26,191 - lava_callback - INFO - Received callback for node 69c79952be430e18c5d1668b 2026-03-30 09:03:26,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39310e33d030878a8b4e HTTP/1.1" 200 None 2026-03-30 09:03:26,221 - lava_callback - INFO - Completed processing callback for node 69ca39310e33d030878a8b4e INFO: 172.21.0.1:33802 - "POST /node/69c79952be430e18c5d1668b HTTP/1.0" 202 Accepted 2026-03-30 09:03:26,230 - lava_callback - INFO - Starting async job submit for node 69c79952be430e18c5d1668b 2026-03-30 09:03:26,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:26,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79952be430e18c5d1668b HTTP/1.1" 200 None 2026-03-30 09:03:26,280 - lava_callback - DEBUG - Processing node 69c79952be430e18c5d1668b: baseline-arm-clabbe 2026-03-30 09:03:26,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79952be430e18c5d1668b 2026-03-30 09:03:26,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:26,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79952be430e18c5d1668b HTTP/1.1" 200 None 2026-03-30 09:03:26,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:26,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 09:03:26,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:26,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79952be430e18c5d1668b HTTP/1.1" 200 None 2026-03-30 09:03:26,361 - lava_callback - ERROR - Error processing callback for node 69c79952be430e18c5d1668b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39908 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:03:36,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:03:36,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47792 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:04:12,922 - lava_callback - INFO - Received callback for node 69ca3c800e33d030878a942d INFO: 172.21.0.1:38020 - "POST /node/69ca3c800e33d030878a942d HTTP/1.0" 202 Accepted 2026-03-30 09:04:12,933 - lava_callback - INFO - Starting async job submit for node 69ca3c800e33d030878a942d 2026-03-30 09:04:12,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:04:12,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3c800e33d030878a942d HTTP/1.1" 200 None 2026-03-30 09:04:12,965 - lava_callback - DEBUG - Processing node 69ca3c800e33d030878a942d: baseline-nfs-arm64 2026-03-30 09:04:12,967 - lava_callback - WARNING - Failed to upload log 2026-03-30 09:04:12,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:04:13,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:04:13,209 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca3c800e33d030878a942d/lava-logs.yaml 2026-03-30 09:04:13,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:04:13,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:04:13,353 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca3c800e33d030878a942d/lava_callback.json.gz 2026-03-30 09:04:13,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:04:13,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3c800e33d030878a942d HTTP/1.1" 200 None 2026-03-30 09:04:13,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:04:13,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:04:13,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:04:13,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca3c800e33d030878a942d HTTP/1.1" 200 None 2026-03-30 09:04:13,419 - lava_callback - INFO - Completed processing callback for node 69ca3c800e33d030878a942d INFO: 172.21.0.1:38024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42600 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:04:36,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:04:36,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44428 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:05:26,290 - lava_callback - INFO - Received callback for node 69c799c4be430e18c5d166ae INFO: 172.21.0.1:44442 - "POST /node/69c799c4be430e18c5d166ae HTTP/1.0" 202 Accepted 2026-03-30 09:05:26,297 - lava_callback - INFO - Starting async job submit for node 69c799c4be430e18c5d166ae 2026-03-30 09:05:26,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:26,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c799c4be430e18c5d166ae HTTP/1.1" 200 None 2026-03-30 09:05:26,327 - lava_callback - DEBUG - Processing node 69c799c4be430e18c5d166ae: baseline-arm-clabbe 2026-03-30 09:05:26,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c799c4be430e18c5d166ae 2026-03-30 09:05:26,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:26,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c799c4be430e18c5d166ae HTTP/1.1" 200 None 2026-03-30 09:05:26,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:26,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 09:05:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:26,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c799c4be430e18c5d166ae HTTP/1.1" 200 None 2026-03-30 09:05:26,395 - lava_callback - ERROR - Error processing callback for node 69c799c4be430e18c5d166ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33188 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:05:57,221 - lava_callback - INFO - Received callback for node 69ca3cc30e33d030878a943b INFO: 172.21.0.1:33194 - "POST /node/69ca3cc30e33d030878a943b HTTP/1.0" 202 Accepted 2026-03-30 09:05:57,228 - lava_callback - INFO - Starting async job submit for node 69ca3cc30e33d030878a943b 2026-03-30 09:05:57,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:57,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3cc30e33d030878a943b HTTP/1.1" 200 None 2026-03-30 09:05:57,314 - lava_callback - DEBUG - Processing node 69ca3cc30e33d030878a943b: baseline-arm64 2026-03-30 09:05:57,487 - lava_callback - WARNING - Failed to upload log 2026-03-30 09:05:57,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:05:57,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:05:57,921 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3cc30e33d030878a943b/lava-logs.yaml 2026-03-30 09:05:57,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:05:58,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:05:58,229 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3cc30e33d030878a943b/lava_callback.json.gz 2026-03-30 09:05:58,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:58,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3cc30e33d030878a943b HTTP/1.1" 200 None 2026-03-30 09:05:58,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:58,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:05:58,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:05:58,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca3cc30e33d030878a943b HTTP/1.1" 200 None 2026-03-30 09:05:58,446 - lava_callback - INFO - Completed processing callback for node 69ca3cc30e33d030878a943b INFO: 172.21.0.1:45628 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:06:06,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:06:06,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:06:56,073 - lava_callback - INFO - Received callback for node 69ca37b30e33d030878a89d0 INFO: 172.21.0.1:50810 - "POST /node/69ca37b30e33d030878a89d0 HTTP/1.0" 202 Accepted 2026-03-30 09:06:56,079 - lava_callback - INFO - Starting async job submit for node 69ca37b30e33d030878a89d0 2026-03-30 09:06:56,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:06:56,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37b30e33d030878a89d0 HTTP/1.1" 200 None 2026-03-30 09:06:56,131 - lava_callback - DEBUG - Processing node 69ca37b30e33d030878a89d0: ltp-watchqueue 2026-03-30 09:06:56,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50824 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:06:56,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:06:56,678 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69ca37b30e33d030878a89d0/log.txt.gz 2026-03-30 09:06:56,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:06:57,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:06:57,138 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69ca37b30e33d030878a89d0/lava-logs.yaml 2026-03-30 09:06:57,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:06:57,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:06:57,312 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69ca37b30e33d030878a89d0/lava_callback.json.gz 2026-03-30 09:06:57,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:06:57,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37b30e33d030878a89d0 HTTP/1.1" 200 None 2026-03-30 09:06:57,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:06:57,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 09:06:57,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:06:57,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37b30e33d030878a89d0 HTTP/1.1" 200 None 2026-03-30 09:06:57,461 - lava_callback - INFO - Completed processing callback for node 69ca37b30e33d030878a89d0 INFO: 172.21.0.1:45286 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:07:06,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:07:06,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57070 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:07:16,803 - lava_callback - INFO - Received callback for node 69ca3d0f0e33d030878a944b INFO: 172.21.0.1:57080 - "POST /node/69ca3d0f0e33d030878a944b HTTP/1.0" 202 Accepted 2026-03-30 09:07:16,810 - lava_callback - INFO - Starting async job submit for node 69ca3d0f0e33d030878a944b 2026-03-30 09:07:16,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:07:16,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3d0f0e33d030878a944b HTTP/1.1" 200 None 2026-03-30 09:07:16,857 - lava_callback - DEBUG - Processing node 69ca3d0f0e33d030878a944b: baseline-arm64 2026-03-30 09:07:17,008 - lava_callback - WARNING - Failed to upload log 2026-03-30 09:07:17,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:07:17,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:07:17,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3d0f0e33d030878a944b/lava-logs.yaml 2026-03-30 09:07:17,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:07:17,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:07:17,558 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3d0f0e33d030878a944b/lava_callback.json.gz 2026-03-30 09:07:17,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:07:17,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3d0f0e33d030878a944b HTTP/1.1" 200 None 2026-03-30 09:07:17,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:07:17,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:07:17,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:07:17,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca3d0f0e33d030878a944b HTTP/1.1" 200 None 2026-03-30 09:07:17,722 - lava_callback - INFO - Completed processing callback for node 69ca3d0f0e33d030878a944b INFO: 172.21.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60070 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:07:36,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:07:36,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48664 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:08:31,618 - lava_callback - INFO - Received callback for node 69ca3d7b0e33d030878a9468 INFO: 172.21.0.1:38590 - "POST /node/69ca3d7b0e33d030878a9468 HTTP/1.0" 202 Accepted 2026-03-30 09:08:31,624 - lava_callback - INFO - Starting async job submit for node 69ca3d7b0e33d030878a9468 2026-03-30 09:08:31,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:08:31,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3d7b0e33d030878a9468 HTTP/1.1" 200 None 2026-03-30 09:08:31,643 - lava_callback - DEBUG - Processing node 69ca3d7b0e33d030878a9468: baseline-arm 2026-03-30 09:08:31,648 - lava_callback - WARNING - Failed to upload log 2026-03-30 09:08:31,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:08:31,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:08:31,948 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca3d7b0e33d030878a9468/lava-logs.yaml 2026-03-30 09:08:31,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:08:32,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:08:32,142 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca3d7b0e33d030878a9468/lava_callback.json.gz 2026-03-30 09:08:32,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:08:32,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3d7b0e33d030878a9468 HTTP/1.1" 200 None 2026-03-30 09:08:32,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:08:32,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 09:08:32,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:08:32,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca3d7b0e33d030878a9468 HTTP/1.1" 200 None 2026-03-30 09:08:32,252 - lava_callback - INFO - Completed processing callback for node 69ca3d7b0e33d030878a9468 INFO: 172.21.0.1:38598 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:08:36,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:08:36,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50152 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:09:46,479 - lava_callback - INFO - Received callback for node 69c79ad4be430e18c5d166e6 INFO: 172.21.0.1:56620 - "POST /node/69c79ad4be430e18c5d166e6 HTTP/1.0" 202 Accepted 2026-03-30 09:09:46,485 - lava_callback - INFO - Starting async job submit for node 69c79ad4be430e18c5d166e6 2026-03-30 09:09:46,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:09:46,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79ad4be430e18c5d166e6 HTTP/1.1" 200 None 2026-03-30 09:09:46,501 - lava_callback - DEBUG - Processing node 69c79ad4be430e18c5d166e6: baseline-arm-clabbe 2026-03-30 09:09:46,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79ad4be430e18c5d166e6 2026-03-30 09:09:46,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:09:46,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79ad4be430e18c5d166e6 HTTP/1.1" 200 None 2026-03-30 09:09:46,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:09:46,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 09:09:46,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:09:46,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79ad4be430e18c5d166e6 HTTP/1.1" 200 None 2026-03-30 09:09:46,565 - lava_callback - ERROR - Error processing callback for node 69c79ad4be430e18c5d166e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:10:36,085 - lava_callback - INFO - Received callback for node 69ca39fb0e33d030878a92cc INFO: 172.21.0.1:55270 - "POST /node/69ca39fb0e33d030878a92cc HTTP/1.0" 202 Accepted 2026-03-30 09:10:36,091 - lava_callback - INFO - Starting async job submit for node 69ca39fb0e33d030878a92cc 2026-03-30 09:10:36,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:10:36,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39fb0e33d030878a92cc HTTP/1.1" 200 None 2026-03-30 09:10:36,170 - lava_callback - DEBUG - Processing node 69ca39fb0e33d030878a92cc: ltp-smoke INFO: 172.21.0.1:55284 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:10:36,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:10:36,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:10:36,830 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca39fb0e33d030878a92cc/log.txt.gz 2026-03-30 09:10:37,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:10:37,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:10:37,339 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca39fb0e33d030878a92cc/lava-logs.yaml 2026-03-30 09:10:37,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:10:37,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:10:37,524 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca39fb0e33d030878a92cc/lava_callback.json.gz 2026-03-30 09:10:37,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:10:37,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39fb0e33d030878a92cc HTTP/1.1" 200 None 2026-03-30 09:10:37,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:10:37,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31f70e33d030878a87f5 HTTP/1.1" 200 None 2026-03-30 09:10:37,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:10:37,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39fb0e33d030878a92cc HTTP/1.1" 200 None 2026-03-30 09:10:37,866 - lava_callback - INFO - Completed processing callback for node 69ca39fb0e33d030878a92cc INFO: 172.21.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56242 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:11:03,168 - lava_callback - INFO - Received callback for node 69ca392f0e33d030878a8b48 INFO: 172.21.0.1:60068 - "POST /node/69ca392f0e33d030878a8b48 HTTP/1.0" 202 Accepted 2026-03-30 09:11:03,176 - lava_callback - INFO - Starting async job submit for node 69ca392f0e33d030878a8b48 2026-03-30 09:11:03,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:03,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392f0e33d030878a8b48 HTTP/1.1" 200 None 2026-03-30 09:11:03,235 - lava_callback - DEBUG - Processing node 69ca392f0e33d030878a8b48: baseline-nfs-arm64 2026-03-30 09:11:04,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:11:04,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:11:04,426 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392f0e33d030878a8b48/log.txt.gz 2026-03-30 09:11:05,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:11:05,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:11:05,268 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392f0e33d030878a8b48/lava-logs.yaml 2026-03-30 09:11:05,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:11:05,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:11:05,485 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca392f0e33d030878a8b48/lava_callback.json.gz DEBUG: Setup failed for 69ca392f0e33d030878a8b48. Transitting job node result: incomplete -> fail 2026-03-30 09:11:05,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:05,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca392f0e33d030878a8b48 HTTP/1.1" 200 None 2026-03-30 09:11:05,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:11:05,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:05,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca392f0e33d030878a8b48 HTTP/1.1" 200 None 2026-03-30 09:11:05,747 - lava_callback - INFO - Completed processing callback for node 69ca392f0e33d030878a8b48 INFO: 172.21.0.1:60076 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:11:06,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:07,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59448 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 09:11:16,083 - lava_callback - INFO - Received callback for node 69ca3de70e33d030878a948d INFO: 172.21.0.1:59454 - "POST /node/69ca3de70e33d030878a948d HTTP/1.0" 202 Accepted 2026-03-30 09:11:16,090 - lava_callback - INFO - Starting async job submit for node 69ca3de70e33d030878a948d 2026-03-30 09:11:16,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:16,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3de70e33d030878a948d HTTP/1.1" 200 None 2026-03-30 09:11:16,131 - lava_callback - DEBUG - Processing node 69ca3de70e33d030878a948d: baseline-arm64 2026-03-30 09:11:16,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:11:16,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:11:16,605 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3de70e33d030878a948d/log.txt.gz 2026-03-30 09:11:16,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:11:16,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:11:16,998 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3de70e33d030878a948d/lava-logs.yaml 2026-03-30 09:11:17,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:11:17,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:11:17,179 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca3de70e33d030878a948d/lava_callback.json.gz 2026-03-30 09:11:17,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:17,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca3de70e33d030878a948d HTTP/1.1" 200 None 2026-03-30 09:11:17,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:17,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:11:17,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:17,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca3de70e33d030878a948d HTTP/1.1" 200 None 2026-03-30 09:11:17,341 - lava_callback - INFO - Completed processing callback for node 69ca3de70e33d030878a948d INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:11:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:11:37,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55116 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 09:17:15,478 - lava_callback - INFO - Received callback for node 69ca37890e33d030878a8974 INFO: 172.21.0.1:55118 - "POST /node/69ca37890e33d030878a8974 HTTP/1.0" 202 Accepted 2026-03-30 09:17:15,525 - lava_callback - INFO - Starting async job submit for node 69ca37890e33d030878a8974 2026-03-30 09:17:15,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:15,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37890e33d030878a8974 HTTP/1.1" 200 None 2026-03-30 09:17:15,567 - lava_callback - DEBUG - Processing node 69ca37890e33d030878a8974: baseline-arm-pengutronix 2026-03-30 09:17:15,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:17:15,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:17:15,828 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37890e33d030878a8974/log.txt.gz 2026-03-30 09:17:15,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:17:16,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:17:16,119 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37890e33d030878a8974/lava-logs.yaml 2026-03-30 09:17:16,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:17:16,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:17:16,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca37890e33d030878a8974/lava_callback.json.gz DEBUG: Setup failed for 69ca37890e33d030878a8974. Transitting job node result: incomplete -> fail 2026-03-30 09:17:16,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:16,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca37890e33d030878a8974 HTTP/1.1" 200 None 2026-03-30 09:17:16,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:16,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 09:17:16,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:16,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca37890e33d030878a8974 HTTP/1.1" 200 None 2026-03-30 09:17:16,433 - lava_callback - INFO - Completed processing callback for node 69ca37890e33d030878a8974 INFO: 172.21.0.1:55126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:17:37,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:37,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36594 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:17:50,321 - lava_callback - INFO - Received callback for node 69c79cb1be430e18c5d16768 INFO: 172.21.0.1:36876 - "POST /node/69c79cb1be430e18c5d16768 HTTP/1.0" 202 Accepted 2026-03-30 09:17:50,326 - lava_callback - INFO - Starting async job submit for node 69c79cb1be430e18c5d16768 2026-03-30 09:17:50,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:50,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79cb1be430e18c5d16768 HTTP/1.1" 200 None 2026-03-30 09:17:50,343 - lava_callback - DEBUG - Processing node 69c79cb1be430e18c5d16768: baseline-arm64-baylibre 2026-03-30 09:17:50,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79cb1be430e18c5d16768 2026-03-30 09:17:50,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:50,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79cb1be430e18c5d16768 HTTP/1.1" 200 None 2026-03-30 09:17:50,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:50,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:17:50,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:17:50,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79cb1be430e18c5d16768 HTTP/1.1" 200 None 2026-03-30 09:17:50,401 - lava_callback - ERROR - Error processing callback for node 69c79cb1be430e18c5d16768: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59004 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:19:10,319 - lava_callback - INFO - Received callback for node 69c79cfebe430e18c5d16777 INFO: 172.21.0.1:49588 - "POST /node/69c79cfebe430e18c5d16777 HTTP/1.0" 202 Accepted 2026-03-30 09:19:10,325 - lava_callback - INFO - Starting async job submit for node 69c79cfebe430e18c5d16777 2026-03-30 09:19:10,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:19:10,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79cfebe430e18c5d16777 HTTP/1.1" 200 None 2026-03-30 09:19:10,340 - lava_callback - DEBUG - Processing node 69c79cfebe430e18c5d16777: baseline-arm64-baylibre 2026-03-30 09:19:10,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79cfebe430e18c5d16777 2026-03-30 09:19:10,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:19:10,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79cfebe430e18c5d16777 HTTP/1.1" 200 None 2026-03-30 09:19:10,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:19:10,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:19:10,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:19:10,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79cfebe430e18c5d16777 HTTP/1.1" 200 None 2026-03-30 09:19:10,401 - lava_callback - ERROR - Error processing callback for node 69c79cfebe430e18c5d16777: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:20:30,392 - lava_callback - INFO - Received callback for node 69c79d50be430e18c5d16781 INFO: 172.21.0.1:51702 - "POST /node/69c79d50be430e18c5d16781 HTTP/1.0" 202 Accepted 2026-03-30 09:20:30,401 - lava_callback - INFO - Starting async job submit for node 69c79d50be430e18c5d16781 2026-03-30 09:20:30,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:20:30,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79d50be430e18c5d16781 HTTP/1.1" 200 None 2026-03-30 09:20:30,420 - lava_callback - DEBUG - Processing node 69c79d50be430e18c5d16781: baseline-arm64-baylibre 2026-03-30 09:20:30,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79d50be430e18c5d16781 2026-03-30 09:20:30,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:20:30,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79d50be430e18c5d16781 HTTP/1.1" 200 None 2026-03-30 09:20:30,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:20:30,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:20:30,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:20:30,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79d50be430e18c5d16781 HTTP/1.1" 200 None 2026-03-30 09:20:30,536 - lava_callback - ERROR - Error processing callback for node 69c79d50be430e18c5d16781: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:21:50,518 - lava_callback - INFO - Received callback for node 69c79da1be430e18c5d1678b INFO: 172.21.0.1:44528 - "POST /node/69c79da1be430e18c5d1678b HTTP/1.0" 202 Accepted 2026-03-30 09:21:50,523 - lava_callback - INFO - Starting async job submit for node 69c79da1be430e18c5d1678b 2026-03-30 09:21:50,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:21:50,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79da1be430e18c5d1678b HTTP/1.1" 200 None 2026-03-30 09:21:50,539 - lava_callback - DEBUG - Processing node 69c79da1be430e18c5d1678b: baseline-arm64-baylibre 2026-03-30 09:21:50,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79da1be430e18c5d1678b 2026-03-30 09:21:50,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:21:50,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79da1be430e18c5d1678b HTTP/1.1" 200 None 2026-03-30 09:21:50,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:21:50,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:21:50,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:21:50,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79da1be430e18c5d1678b HTTP/1.1" 200 None 2026-03-30 09:21:50,603 - lava_callback - ERROR - Error processing callback for node 69c79da1be430e18c5d1678b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:23:10,543 - lava_callback - INFO - Received callback for node 69c79defbe430e18c5d16791 INFO: 172.21.0.1:51256 - "POST /node/69c79defbe430e18c5d16791 HTTP/1.0" 202 Accepted 2026-03-30 09:23:10,548 - lava_callback - INFO - Starting async job submit for node 69c79defbe430e18c5d16791 2026-03-30 09:23:10,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:23:10,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79defbe430e18c5d16791 HTTP/1.1" 200 None 2026-03-30 09:23:10,575 - lava_callback - DEBUG - Processing node 69c79defbe430e18c5d16791: baseline-arm64-baylibre 2026-03-30 09:23:10,575 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79defbe430e18c5d16791 2026-03-30 09:23:10,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:23:10,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79defbe430e18c5d16791 HTTP/1.1" 200 None 2026-03-30 09:23:10,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:23:10,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:23:10,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:23:10,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79defbe430e18c5d16791 HTTP/1.1" 200 None 2026-03-30 09:23:10,641 - lava_callback - ERROR - Error processing callback for node 69c79defbe430e18c5d16791: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44712 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:24:30,609 - lava_callback - INFO - Received callback for node 69c79e3fbe430e18c5d1679b INFO: 172.21.0.1:40832 - "POST /node/69c79e3fbe430e18c5d1679b HTTP/1.0" 202 Accepted 2026-03-30 09:24:30,615 - lava_callback - INFO - Starting async job submit for node 69c79e3fbe430e18c5d1679b 2026-03-30 09:24:30,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:24:30,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79e3fbe430e18c5d1679b HTTP/1.1" 200 None 2026-03-30 09:24:30,630 - lava_callback - DEBUG - Processing node 69c79e3fbe430e18c5d1679b: baseline-arm64-baylibre 2026-03-30 09:24:30,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79e3fbe430e18c5d1679b 2026-03-30 09:24:30,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:24:30,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79e3fbe430e18c5d1679b HTTP/1.1" 200 None 2026-03-30 09:24:30,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:24:30,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:24:30,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:24:30,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79e3fbe430e18c5d1679b HTTP/1.1" 200 None 2026-03-30 09:24:30,692 - lava_callback - ERROR - Error processing callback for node 69c79e3fbe430e18c5d1679b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44736 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:25:35,031 - lava_callback - INFO - Received callback for node 69c96c5ff2866272ea51e91e INFO: 172.21.0.1:42218 - "POST /node/69c96c5ff2866272ea51e91e HTTP/1.0" 202 Accepted 2026-03-30 09:25:35,038 - lava_callback - INFO - Starting async job submit for node 69c96c5ff2866272ea51e91e 2026-03-30 09:25:35,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:25:35,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96c5ff2866272ea51e91e HTTP/1.1" 200 None 2026-03-30 09:25:35,080 - lava_callback - DEBUG - Processing node 69c96c5ff2866272ea51e91e: baseline-nfs-arm64 2026-03-30 09:25:35,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:25:35,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:25:35,773 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96c5ff2866272ea51e91e/log.txt.gz 2026-03-30 09:25:36,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:25:36,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:25:36,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96c5ff2866272ea51e91e/lava-logs.yaml 2026-03-30 09:25:36,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:25:36,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:25:36,428 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96c5ff2866272ea51e91e/lava_callback.json.gz 2026-03-30 09:25:36,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:25:36,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96c5ff2866272ea51e91e HTTP/1.1" 200 None 2026-03-30 09:25:36,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:25:36,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-30 09:25:36,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42228 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:25:36,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96c5ff2866272ea51e91e HTTP/1.1" 200 None 2026-03-30 09:25:36,567 - lava_callback - INFO - Completed processing callback for node 69c96c5ff2866272ea51e91e 2026-03-30 09:25:37,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:25:37,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50260 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:27:07,270 - lava_callback - INFO - Received callback for node 69c79ed7be430e18c5d167af INFO: 172.21.0.1:50274 - "POST /node/69c79ed7be430e18c5d167af HTTP/1.0" 202 Accepted 2026-03-30 09:27:07,275 - lava_callback - INFO - Starting async job submit for node 69c79ed7be430e18c5d167af 2026-03-30 09:27:07,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:27:07,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79ed7be430e18c5d167af HTTP/1.1" 200 None 2026-03-30 09:27:07,293 - lava_callback - DEBUG - Processing node 69c79ed7be430e18c5d167af: baseline-arm64-clabbe 2026-03-30 09:27:07,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79ed7be430e18c5d167af 2026-03-30 09:27:07,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:27:07,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79ed7be430e18c5d167af HTTP/1.1" 200 None 2026-03-30 09:27:07,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:27:07,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:27:07,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:27:07,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79ed7be430e18c5d167af HTTP/1.1" 200 None 2026-03-30 09:27:07,381 - lava_callback - ERROR - Error processing callback for node 69c79ed7be430e18c5d167af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:28:27,368 - lava_callback - INFO - Received callback for node 69c79f26be430e18c5d167b6 INFO: 172.21.0.1:57506 - "POST /node/69c79f26be430e18c5d167b6 HTTP/1.0" 202 Accepted 2026-03-30 09:28:27,374 - lava_callback - INFO - Starting async job submit for node 69c79f26be430e18c5d167b6 2026-03-30 09:28:27,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:28:27,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79f26be430e18c5d167b6 HTTP/1.1" 200 None 2026-03-30 09:28:27,389 - lava_callback - DEBUG - Processing node 69c79f26be430e18c5d167b6: baseline-arm64-clabbe 2026-03-30 09:28:27,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79f26be430e18c5d167b6 2026-03-30 09:28:27,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:28:27,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79f26be430e18c5d167b6 HTTP/1.1" 200 None 2026-03-30 09:28:27,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:28:27,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:28:27,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:28:27,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79f26be430e18c5d167b6 HTTP/1.1" 200 None 2026-03-30 09:28:27,459 - lava_callback - ERROR - Error processing callback for node 69c79f26be430e18c5d167b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45504 - "GET /node/69aff4b5ae557f2600f1028b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37076 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:29:39,250 - lava_callback - INFO - Received callback for node 69c96ae7f2866272ea51e8ea INFO: 172.21.0.1:37092 - "POST /node/69c96ae7f2866272ea51e8ea HTTP/1.0" 202 Accepted 2026-03-30 09:29:39,263 - lava_callback - INFO - Starting async job submit for node 69c96ae7f2866272ea51e8ea 2026-03-30 09:29:39,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:39,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96ae7f2866272ea51e8ea HTTP/1.1" 200 None 2026-03-30 09:29:39,307 - lava_callback - DEBUG - Processing node 69c96ae7f2866272ea51e8ea: baseline-arm64 2026-03-30 09:29:39,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:29:39,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:29:39,861 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96ae7f2866272ea51e8ea/log.txt.gz 2026-03-30 09:29:40,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:29:40,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:29:40,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96ae7f2866272ea51e8ea/lava-logs.yaml 2026-03-30 09:29:40,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:29:40,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:29:40,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96ae7f2866272ea51e8ea/lava_callback.json.gz 2026-03-30 09:29:40,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:40,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96ae7f2866272ea51e8ea HTTP/1.1" 200 None 2026-03-30 09:29:40,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:40,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-30 09:29:40,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:40,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96ae7f2866272ea51e8ea HTTP/1.1" 200 None 2026-03-30 09:29:40,612 - lava_callback - INFO - Completed processing callback for node 69c96ae7f2866272ea51e8ea INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:29:47,405 - lava_callback - INFO - Received callback for node 69c79f75be430e18c5d167bf INFO: 172.21.0.1:34784 - "POST /node/69c79f75be430e18c5d167bf HTTP/1.0" 202 Accepted 2026-03-30 09:29:47,411 - lava_callback - INFO - Starting async job submit for node 69c79f75be430e18c5d167bf 2026-03-30 09:29:47,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:47,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79f75be430e18c5d167bf HTTP/1.1" 200 None 2026-03-30 09:29:47,434 - lava_callback - DEBUG - Processing node 69c79f75be430e18c5d167bf: baseline-arm64-clabbe 2026-03-30 09:29:47,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79f75be430e18c5d167bf 2026-03-30 09:29:47,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:47,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79f75be430e18c5d167bf HTTP/1.1" 200 None 2026-03-30 09:29:47,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:47,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:29:47,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:29:47,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79f75be430e18c5d167bf HTTP/1.1" 200 None 2026-03-30 09:29:47,498 - lava_callback - ERROR - Error processing callback for node 69c79f75be430e18c5d167bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58528 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:30:07,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:30:07,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38574 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:31:07,457 - lava_callback - INFO - Received callback for node 69c79fc6be430e18c5d167cb INFO: 172.21.0.1:38584 - "POST /node/69c79fc6be430e18c5d167cb HTTP/1.0" 202 Accepted 2026-03-30 09:31:07,462 - lava_callback - INFO - Starting async job submit for node 69c79fc6be430e18c5d167cb 2026-03-30 09:31:07,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:31:07,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79fc6be430e18c5d167cb HTTP/1.1" 200 None 2026-03-30 09:31:07,477 - lava_callback - DEBUG - Processing node 69c79fc6be430e18c5d167cb: baseline-arm64-clabbe 2026-03-30 09:31:07,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c79fc6be430e18c5d167cb 2026-03-30 09:31:07,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:31:07,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79fc6be430e18c5d167cb HTTP/1.1" 200 None 2026-03-30 09:31:07,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:31:07,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:31:07,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:31:07,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79fc6be430e18c5d167cb HTTP/1.1" 200 None 2026-03-30 09:31:07,531 - lava_callback - ERROR - Error processing callback for node 69c79fc6be430e18c5d167cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37404 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:32:27,464 - lava_callback - INFO - Received callback for node 69c7a016be430e18c5d167d0 INFO: 172.21.0.1:37416 - "POST /node/69c7a016be430e18c5d167d0 HTTP/1.0" 202 Accepted 2026-03-30 09:32:27,470 - lava_callback - INFO - Starting async job submit for node 69c7a016be430e18c5d167d0 2026-03-30 09:32:27,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:32:27,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a016be430e18c5d167d0 HTTP/1.1" 200 None 2026-03-30 09:32:27,501 - lava_callback - DEBUG - Processing node 69c7a016be430e18c5d167d0: baseline-arm64-clabbe 2026-03-30 09:32:27,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a016be430e18c5d167d0 2026-03-30 09:32:27,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:32:27,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a016be430e18c5d167d0 HTTP/1.1" 200 None 2026-03-30 09:32:27,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:32:27,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:32:27,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:32:27,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a016be430e18c5d167d0 HTTP/1.1" 200 None 2026-03-30 09:32:27,558 - lava_callback - ERROR - Error processing callback for node 69c7a016be430e18c5d167d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34468 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:33:47,514 - lava_callback - INFO - Received callback for node 69c7a066be430e18c5d167e2 INFO: 172.21.0.1:34480 - "POST /node/69c7a066be430e18c5d167e2 HTTP/1.0" 202 Accepted 2026-03-30 09:33:47,519 - lava_callback - INFO - Starting async job submit for node 69c7a066be430e18c5d167e2 2026-03-30 09:33:47,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:33:47,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a066be430e18c5d167e2 HTTP/1.1" 200 None 2026-03-30 09:33:47,542 - lava_callback - DEBUG - Processing node 69c7a066be430e18c5d167e2: baseline-arm64-clabbe 2026-03-30 09:33:47,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a066be430e18c5d167e2 2026-03-30 09:33:47,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:33:47,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a066be430e18c5d167e2 HTTP/1.1" 200 None 2026-03-30 09:33:47,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:33:47,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 09:33:47,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:33:47,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a066be430e18c5d167e2 HTTP/1.1" 200 None 2026-03-30 09:33:47,601 - lava_callback - ERROR - Error processing callback for node 69c7a066be430e18c5d167e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57438 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:36:51,554 - lava_callback - INFO - Received callback for node 69c7a121be430e18c5d16803 INFO: 172.21.0.1:39378 - "POST /node/69c7a121be430e18c5d16803 HTTP/1.0" 202 Accepted 2026-03-30 09:36:51,561 - lava_callback - INFO - Starting async job submit for node 69c7a121be430e18c5d16803 2026-03-30 09:36:51,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:36:51,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a121be430e18c5d16803 HTTP/1.1" 200 None 2026-03-30 09:36:51,584 - lava_callback - DEBUG - Processing node 69c7a121be430e18c5d16803: baseline-arm64-baylibre 2026-03-30 09:36:51,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a121be430e18c5d16803 2026-03-30 09:36:51,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:36:51,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a121be430e18c5d16803 HTTP/1.1" 200 None 2026-03-30 09:36:51,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:36:51,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:36:51,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:36:51,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a121be430e18c5d16803 HTTP/1.1" 200 None 2026-03-30 09:36:51,651 - lava_callback - ERROR - Error processing callback for node 69c7a121be430e18c5d16803: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44300 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:37:35,465 - lava_callback - INFO - Received callback for node 69ca42a20e33d030878a95a1 INFO: 172.21.0.1:59208 - "POST /node/69ca42a20e33d030878a95a1 HTTP/1.0" 202 Accepted 2026-03-30 09:37:35,472 - lava_callback - INFO - Starting async job submit for node 69ca42a20e33d030878a95a1 2026-03-30 09:37:35,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:37:35,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca42a20e33d030878a95a1 HTTP/1.1" 200 None 2026-03-30 09:37:35,528 - lava_callback - DEBUG - Processing node 69ca42a20e33d030878a95a1: baseline-arm64 2026-03-30 09:37:35,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:37:36,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:37:36,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca42a20e33d030878a95a1/log.txt.gz 2026-03-30 09:37:36,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59220 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:37:36,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:37:36,645 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca42a20e33d030878a95a1/lava-logs.yaml 2026-03-30 09:37:36,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:37:36,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:37:36,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca42a20e33d030878a95a1/lava_callback.json.gz DEBUG: Setup failed for 69ca42a20e33d030878a95a1. Transitting job node result: incomplete -> fail 2026-03-30 09:37:36,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:37:36,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca42a20e33d030878a95a1 HTTP/1.1" 200 None 2026-03-30 09:37:36,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:37:36,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:37:36,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:37:36,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca42a20e33d030878a95a1 HTTP/1.1" 200 None 2026-03-30 09:37:37,039 - lava_callback - INFO - Completed processing callback for node 69ca42a20e33d030878a95a1 2026-03-30 09:37:37,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:37:37,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41402 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:38:27,742 - lava_callback - INFO - Received callback for node 69ca39250e33d030878a8b1a INFO: 172.21.0.1:41414 - "POST /node/69ca39250e33d030878a8b1a HTTP/1.0" 202 Accepted 2026-03-30 09:38:27,749 - lava_callback - INFO - Starting async job submit for node 69ca39250e33d030878a8b1a 2026-03-30 09:38:27,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:27,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39250e33d030878a8b1a HTTP/1.1" 200 None 2026-03-30 09:38:27,793 - lava_callback - DEBUG - Processing node 69ca39250e33d030878a8b1a: baseline-arm64 2026-03-30 09:38:28,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:38:28,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:38:28,431 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39250e33d030878a8b1a/log.txt.gz 2026-03-30 09:38:28,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:38:28,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:38:28,932 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39250e33d030878a8b1a/lava-logs.yaml 2026-03-30 09:38:28,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:38:29,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:38:29,120 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca39250e33d030878a8b1a/lava_callback.json.gz 2026-03-30 09:38:29,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:29,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39250e33d030878a8b1a HTTP/1.1" 200 None 2026-03-30 09:38:29,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:29,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:38:29,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:29,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39250e33d030878a8b1a HTTP/1.1" 200 None 2026-03-30 09:38:29,261 - lava_callback - INFO - Completed processing callback for node 69ca39250e33d030878a8b1a INFO: 172.21.0.1:36884 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:38:37,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:37,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53464 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:38:54,292 - lava_callback - INFO - Received callback for node 69ca44a30e33d030878a9629 INFO: 172.21.0.1:52994 - "POST /node/69ca44a30e33d030878a9629 HTTP/1.0" 202 Accepted 2026-03-30 09:38:54,298 - lava_callback - INFO - Starting async job submit for node 69ca44a30e33d030878a9629 2026-03-30 09:38:54,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:54,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca44a30e33d030878a9629 HTTP/1.1" 200 None 2026-03-30 09:38:54,315 - lava_callback - DEBUG - Processing node 69ca44a30e33d030878a9629: baseline-nfs-arm64 2026-03-30 09:38:54,317 - lava_callback - WARNING - Failed to upload log 2026-03-30 09:38:54,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:38:54,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:38:54,459 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca44a30e33d030878a9629/lava-logs.yaml 2026-03-30 09:38:54,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:38:54,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:38:54,600 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca44a30e33d030878a9629/lava_callback.json.gz 2026-03-30 09:38:54,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:54,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca44a30e33d030878a9629 HTTP/1.1" 200 None 2026-03-30 09:38:54,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:54,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:38:54,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:38:54,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca44a30e33d030878a9629 HTTP/1.1" 200 None 2026-03-30 09:38:54,662 - lava_callback - INFO - Completed processing callback for node 69ca44a30e33d030878a9629 INFO: 172.21.0.1:52996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:39:07,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:39:07,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:39:31,428 - lava_callback - INFO - Received callback for node 69c7a1bfbe430e18c5d1681f INFO: 172.21.0.1:54634 - "POST /node/69c7a1bfbe430e18c5d1681f HTTP/1.0" 202 Accepted 2026-03-30 09:39:31,434 - lava_callback - INFO - Starting async job submit for node 69c7a1bfbe430e18c5d1681f 2026-03-30 09:39:31,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:39:31,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a1bfbe430e18c5d1681f HTTP/1.1" 200 None 2026-03-30 09:39:31,466 - lava_callback - DEBUG - Processing node 69c7a1bfbe430e18c5d1681f: baseline-arm64-baylibre 2026-03-30 09:39:31,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a1bfbe430e18c5d1681f 2026-03-30 09:39:31,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:39:31,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a1bfbe430e18c5d1681f HTTP/1.1" 200 None 2026-03-30 09:39:31,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:39:31,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:39:31,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:39:31,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a1bfbe430e18c5d1681f HTTP/1.1" 200 None 2026-03-30 09:39:31,527 - lava_callback - ERROR - Error processing callback for node 69c7a1bfbe430e18c5d1681f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46800 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:41:18,159 - lava_callback - INFO - Received callback for node 69ca45090e33d030878a9644 INFO: 172.21.0.1:46812 - "POST /node/69ca45090e33d030878a9644 HTTP/1.0" 202 Accepted 2026-03-30 09:41:18,165 - lava_callback - INFO - Starting async job submit for node 69ca45090e33d030878a9644 2026-03-30 09:41:18,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:41:18,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45090e33d030878a9644 HTTP/1.1" 200 None 2026-03-30 09:41:18,218 - lava_callback - DEBUG - Processing node 69ca45090e33d030878a9644: baseline-arm64 2026-03-30 09:41:18,364 - lava_callback - WARNING - Failed to upload log 2026-03-30 09:41:18,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:41:18,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:41:18,632 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca45090e33d030878a9644/lava-logs.yaml 2026-03-30 09:41:18,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:41:18,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:41:18,810 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca45090e33d030878a9644/lava_callback.json.gz 2026-03-30 09:41:18,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:41:18,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45090e33d030878a9644 HTTP/1.1" 200 None 2026-03-30 09:41:18,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:41:18,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:41:18,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:41:18,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca45090e33d030878a9644 HTTP/1.1" 200 None 2026-03-30 09:41:18,962 - lava_callback - INFO - Completed processing callback for node 69ca45090e33d030878a9644 INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:41:37,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:41:37,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33238 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:42:56,637 - lava_callback - INFO - Received callback for node 69ca39350e33d030878a8b57 INFO: 172.21.0.1:33250 - "POST /node/69ca39350e33d030878a8b57 HTTP/1.0" 202 Accepted 2026-03-30 09:42:56,644 - lava_callback - INFO - Starting async job submit for node 69ca39350e33d030878a8b57 2026-03-30 09:42:56,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:42:56,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39350e33d030878a8b57 HTTP/1.1" 200 None 2026-03-30 09:42:56,700 - lava_callback - DEBUG - Processing node 69ca39350e33d030878a8b57: baseline-nfs-arm64 2026-03-30 09:42:57,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:42:57,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:42:57,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39350e33d030878a8b57/log.txt.gz 2026-03-30 09:42:57,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:42:57,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:42:57,942 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39350e33d030878a8b57/lava-logs.yaml 2026-03-30 09:42:57,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:42:58,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:42:58,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca39350e33d030878a8b57/lava_callback.json.gz 2026-03-30 09:42:58,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:42:58,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca39350e33d030878a8b57 HTTP/1.1" 200 None 2026-03-30 09:42:58,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:42:58,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:42:58,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:42:58,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca39350e33d030878a8b57 HTTP/1.1" 200 None 2026-03-30 09:42:58,281 - lava_callback - INFO - Completed processing callback for node 69ca39350e33d030878a8b57 INFO: 172.21.0.1:33662 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:43:07,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:07,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:43:23,594 - lava_callback - INFO - Received callback for node 69ca45530e33d030878a964e INFO: 172.21.0.1:59090 - "POST /node/69ca45530e33d030878a964e HTTP/1.0" 202 Accepted 2026-03-30 09:43:23,601 - lava_callback - INFO - Starting async job submit for node 69ca45530e33d030878a964e 2026-03-30 09:43:23,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:23,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45530e33d030878a964e HTTP/1.1" 200 None 2026-03-30 09:43:23,646 - lava_callback - DEBUG - Processing node 69ca45530e33d030878a964e: baseline-arm64 2026-03-30 09:43:23,851 - lava_callback - WARNING - Failed to upload log 2026-03-30 09:43:23,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:43:24,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:43:24,297 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca45530e33d030878a964e/lava-logs.yaml 2026-03-30 09:43:24,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:43:24,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:43:24,448 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca45530e33d030878a964e/lava_callback.json.gz 2026-03-30 09:43:24,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:24,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45530e33d030878a964e HTTP/1.1" 200 None 2026-03-30 09:43:24,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:24,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:43:24,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:24,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca45530e33d030878a964e HTTP/1.1" 200 None 2026-03-30 09:43:24,638 - lava_callback - INFO - Completed processing callback for node 69ca45530e33d030878a964e INFO: 172.21.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59116 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:43:31,657 - lava_callback - INFO - Received callback for node 69c7a2aebe430e18c5d16849 INFO: 172.21.0.1:43758 - "POST /node/69c7a2aebe430e18c5d16849 HTTP/1.0" 202 Accepted 2026-03-30 09:43:31,663 - lava_callback - INFO - Starting async job submit for node 69c7a2aebe430e18c5d16849 2026-03-30 09:43:31,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:31,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a2aebe430e18c5d16849 HTTP/1.1" 200 None 2026-03-30 09:43:31,682 - lava_callback - DEBUG - Processing node 69c7a2aebe430e18c5d16849: baseline-arm64-baylibre 2026-03-30 09:43:31,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a2aebe430e18c5d16849 2026-03-30 09:43:31,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:31,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a2aebe430e18c5d16849 HTTP/1.1" 200 None 2026-03-30 09:43:31,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:31,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:43:31,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:31,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a2aebe430e18c5d16849 HTTP/1.1" 200 None 2026-03-30 09:43:31,748 - lava_callback - ERROR - Error processing callback for node 69c7a2aebe430e18c5d16849: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:43:37,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:43:37,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:45:51,879 - lava_callback - INFO - Received callback for node 69c7a340be430e18c5d16859 INFO: 172.21.0.1:56794 - "POST /node/69c7a340be430e18c5d16859 HTTP/1.0" 202 Accepted 2026-03-30 09:45:51,885 - lava_callback - INFO - Starting async job submit for node 69c7a340be430e18c5d16859 2026-03-30 09:45:51,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:45:51,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a340be430e18c5d16859 HTTP/1.1" 200 None 2026-03-30 09:45:51,900 - lava_callback - DEBUG - Processing node 69c7a340be430e18c5d16859: baseline-arm64-baylibre 2026-03-30 09:45:51,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a340be430e18c5d16859 2026-03-30 09:45:51,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:45:51,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a340be430e18c5d16859 HTTP/1.1" 200 None 2026-03-30 09:45:51,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:45:51,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:45:51,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:45:51,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a340be430e18c5d16859 HTTP/1.1" 200 None 2026-03-30 09:45:51,957 - lava_callback - ERROR - Error processing callback for node 69c7a340be430e18c5d16859: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56806 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:46:00,873 - lava_callback - INFO - Received callback for node 69ca46110e33d030878a9679 INFO: 172.21.0.1:55034 - "POST /node/69ca46110e33d030878a9679 HTTP/1.0" 202 Accepted 2026-03-30 09:46:00,879 - lava_callback - INFO - Starting async job submit for node 69ca46110e33d030878a9679 2026-03-30 09:46:00,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:46:00,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca46110e33d030878a9679 HTTP/1.1" 200 None 2026-03-30 09:46:00,922 - lava_callback - DEBUG - Processing node 69ca46110e33d030878a9679: baseline-arm64 2026-03-30 09:46:01,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:46:01,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:46:01,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca46110e33d030878a9679/log.txt.gz 2026-03-30 09:46:01,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:46:02,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:46:02,029 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca46110e33d030878a9679/lava-logs.yaml 2026-03-30 09:46:02,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:46:02,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:46:02,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca46110e33d030878a9679/lava_callback.json.gz 2026-03-30 09:46:02,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:46:02,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca46110e33d030878a9679 HTTP/1.1" 200 None 2026-03-30 09:46:02,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:46:02,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 09:46:02,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:46:02,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca46110e33d030878a9679 HTTP/1.1" 200 None 2026-03-30 09:46:02,399 - lava_callback - INFO - Completed processing callback for node 69ca46110e33d030878a9679 INFO: 172.21.0.1:55046 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:46:07,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:46:07,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53416 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:49:52,080 - lava_callback - INFO - Received callback for node 69c7a432be430e18c5d16884 INFO: 172.21.0.1:60954 - "POST /node/69c7a432be430e18c5d16884 HTTP/1.0" 202 Accepted 2026-03-30 09:49:52,086 - lava_callback - INFO - Starting async job submit for node 69c7a432be430e18c5d16884 2026-03-30 09:49:52,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:49:52,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a432be430e18c5d16884 HTTP/1.1" 200 None 2026-03-30 09:49:52,110 - lava_callback - DEBUG - Processing node 69c7a432be430e18c5d16884: baseline-arm64-baylibre 2026-03-30 09:49:52,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a432be430e18c5d16884 2026-03-30 09:49:52,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:49:52,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a432be430e18c5d16884 HTTP/1.1" 200 None 2026-03-30 09:49:52,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:49:52,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:49:52,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:49:52,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a432be430e18c5d16884 HTTP/1.1" 200 None 2026-03-30 09:49:52,171 - lava_callback - ERROR - Error processing callback for node 69c7a432be430e18c5d16884: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60964 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:50:03,725 - lava_callback - INFO - Received callback for node 69ca430b0e33d030878a95ba INFO: 172.21.0.1:49132 - "POST /node/69ca430b0e33d030878a95ba HTTP/1.0" 202 Accepted 2026-03-30 09:50:03,732 - lava_callback - INFO - Starting async job submit for node 69ca430b0e33d030878a95ba 2026-03-30 09:50:03,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:50:03,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca430b0e33d030878a95ba HTTP/1.1" 200 None 2026-03-30 09:50:03,802 - lava_callback - DEBUG - Processing node 69ca430b0e33d030878a95ba: baseline-arm 2026-03-30 09:50:04,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:50:04,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:50:04,591 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca430b0e33d030878a95ba/log.txt.gz 2026-03-30 09:50:04,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:50:05,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:50:05,204 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca430b0e33d030878a95ba/lava-logs.yaml 2026-03-30 09:50:05,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 09:50:05,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 09:50:05,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca430b0e33d030878a95ba/lava_callback.json.gz DEBUG: Setup failed for 69ca430b0e33d030878a95ba. Transitting job node result: incomplete -> fail 2026-03-30 09:50:05,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:50:05,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca430b0e33d030878a95ba HTTP/1.1" 200 None 2026-03-30 09:50:05,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:50:05,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 09:50:05,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:50:05,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca430b0e33d030878a95ba HTTP/1.1" 200 None 2026-03-30 09:50:05,664 - lava_callback - INFO - Completed processing callback for node 69ca430b0e33d030878a95ba INFO: 172.21.0.1:49148 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:50:07,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:50:07,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58800 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:52:28,328 - lava_callback - INFO - Received callback for node 69c7a4d7be430e18c5d168a2 INFO: 172.21.0.1:58804 - "POST /node/69c7a4d7be430e18c5d168a2 HTTP/1.0" 202 Accepted 2026-03-30 09:52:28,334 - lava_callback - INFO - Starting async job submit for node 69c7a4d7be430e18c5d168a2 2026-03-30 09:52:28,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:52:28,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a4d7be430e18c5d168a2 HTTP/1.1" 200 None 2026-03-30 09:52:28,350 - lava_callback - DEBUG - Processing node 69c7a4d7be430e18c5d168a2: baseline-arm64-clabbe 2026-03-30 09:52:28,351 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a4d7be430e18c5d168a2 2026-03-30 09:52:28,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:52:28,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a4d7be430e18c5d168a2 HTTP/1.1" 200 None 2026-03-30 09:52:28,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:52:28,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:52:28,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:52:28,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a4d7be430e18c5d168a2 HTTP/1.1" 200 None 2026-03-30 09:52:28,409 - lava_callback - ERROR - Error processing callback for node 69c7a4d7be430e18c5d168a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44332 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:55:08,253 - lava_callback - INFO - Received callback for node 69c7a576be430e18c5d168be INFO: 172.21.0.1:44340 - "POST /node/69c7a576be430e18c5d168be HTTP/1.0" 202 Accepted 2026-03-30 09:55:08,259 - lava_callback - INFO - Starting async job submit for node 69c7a576be430e18c5d168be 2026-03-30 09:55:08,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:55:08,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a576be430e18c5d168be HTTP/1.1" 200 None 2026-03-30 09:55:08,275 - lava_callback - DEBUG - Processing node 69c7a576be430e18c5d168be: baseline-arm64-clabbe 2026-03-30 09:55:08,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a576be430e18c5d168be 2026-03-30 09:55:08,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:55:08,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a576be430e18c5d168be HTTP/1.1" 200 None 2026-03-30 09:55:08,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:55:08,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:55:08,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:55:08,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a576be430e18c5d168be HTTP/1.1" 200 None 2026-03-30 09:55:08,355 - lava_callback - ERROR - Error processing callback for node 69c7a576be430e18c5d168be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46058 - "GET / HTTP/1.0" 200 OK 2026-03-30 09:58:28,540 - lava_callback - INFO - Received callback for node 69c7a639be430e18c5d168e4 INFO: 172.21.0.1:46060 - "POST /node/69c7a639be430e18c5d168e4 HTTP/1.0" 202 Accepted 2026-03-30 09:58:28,546 - lava_callback - INFO - Starting async job submit for node 69c7a639be430e18c5d168e4 2026-03-30 09:58:28,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:58:28,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a639be430e18c5d168e4 HTTP/1.1" 200 None 2026-03-30 09:58:28,562 - lava_callback - DEBUG - Processing node 69c7a639be430e18c5d168e4: baseline-arm64-clabbe 2026-03-30 09:58:28,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a639be430e18c5d168e4 2026-03-30 09:58:28,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:58:28,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a639be430e18c5d168e4 HTTP/1.1" 200 None 2026-03-30 09:58:28,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:58:28,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 09:58:28,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 09:58:28,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a639be430e18c5d168e4 HTTP/1.1" 200 None 2026-03-30 09:58:28,621 - lava_callback - ERROR - Error processing callback for node 69c7a639be430e18c5d168e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59492 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:01:08,664 - lava_callback - INFO - Received callback for node 69c7a6d9be430e18c5d16901 INFO: 172.21.0.1:59494 - "POST /node/69c7a6d9be430e18c5d16901 HTTP/1.0" 202 Accepted 2026-03-30 10:01:08,670 - lava_callback - INFO - Starting async job submit for node 69c7a6d9be430e18c5d16901 2026-03-30 10:01:08,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:08,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a6d9be430e18c5d16901 HTTP/1.1" 200 None 2026-03-30 10:01:08,699 - lava_callback - DEBUG - Processing node 69c7a6d9be430e18c5d16901: baseline-arm64-clabbe 2026-03-30 10:01:08,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a6d9be430e18c5d16901 2026-03-30 10:01:08,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:08,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a6d9be430e18c5d16901 HTTP/1.1" 200 None 2026-03-30 10:01:08,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:08,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 10:01:08,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:08,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a6d9be430e18c5d16901 HTTP/1.1" 200 None 2026-03-30 10:01:08,767 - lava_callback - ERROR - Error processing callback for node 69c7a6d9be430e18c5d16901: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45186 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:01:18,017 - lava_callback - INFO - Received callback for node 69ca45c20e33d030878a966a INFO: 172.21.0.1:45188 - "POST /node/69ca45c20e33d030878a966a HTTP/1.0" 202 Accepted 2026-03-30 10:01:18,024 - lava_callback - INFO - Starting async job submit for node 69ca45c20e33d030878a966a 2026-03-30 10:01:18,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:18,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45c20e33d030878a966a HTTP/1.1" 200 None 2026-03-30 10:01:18,097 - lava_callback - DEBUG - Processing node 69ca45c20e33d030878a966a: ltp-smoke 2026-03-30 10:01:18,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:01:18,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:01:18,750 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca45c20e33d030878a966a/log.txt.gz 2026-03-30 10:01:19,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:01:19,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:01:19,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca45c20e33d030878a966a/lava-logs.yaml 2026-03-30 10:01:19,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:01:19,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:01:19,595 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca45c20e33d030878a966a/lava_callback.json.gz 2026-03-30 10:01:19,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:19,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45c20e33d030878a966a HTTP/1.1" 200 None 2026-03-30 10:01:19,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:19,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31f70e33d030878a87f5 HTTP/1.1" 200 None 2026-03-30 10:01:19,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:19,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca45c20e33d030878a966a HTTP/1.1" 200 None 2026-03-30 10:01:19,912 - lava_callback - INFO - Completed processing callback for node 69ca45c20e33d030878a966a INFO: 172.21.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:01:37,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:01:37,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39108 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:02:28,700 - lava_callback - INFO - Received callback for node 69c7a72abe430e18c5d16919 INFO: 172.21.0.1:39122 - "POST /node/69c7a72abe430e18c5d16919 HTTP/1.0" 202 Accepted 2026-03-30 10:02:28,706 - lava_callback - INFO - Starting async job submit for node 69c7a72abe430e18c5d16919 2026-03-30 10:02:28,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:02:28,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a72abe430e18c5d16919 HTTP/1.1" 200 None 2026-03-30 10:02:28,730 - lava_callback - DEBUG - Processing node 69c7a72abe430e18c5d16919: baseline-arm64-clabbe 2026-03-30 10:02:28,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a72abe430e18c5d16919 2026-03-30 10:02:28,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:02:28,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a72abe430e18c5d16919 HTTP/1.1" 200 None 2026-03-30 10:02:28,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:02:28,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 10:02:28,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:02:28,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a72abe430e18c5d16919 HTTP/1.1" 200 None 2026-03-30 10:02:28,791 - lava_callback - ERROR - Error processing callback for node 69c7a72abe430e18c5d16919: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56230 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:03:42,580 - lava_callback - INFO - Received callback for node 69ca45ca0e33d030878a9670 INFO: 172.21.0.1:35016 - "POST /node/69ca45ca0e33d030878a9670 HTTP/1.0" 202 Accepted 2026-03-30 10:03:42,588 - lava_callback - INFO - Starting async job submit for node 69ca45ca0e33d030878a9670 2026-03-30 10:03:42,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:03:42,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45ca0e33d030878a9670 HTTP/1.1" 200 None 2026-03-30 10:03:42,646 - lava_callback - DEBUG - Processing node 69ca45ca0e33d030878a9670: baseline-nfs-arm64 2026-03-30 10:03:43,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:03:43,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:03:43,883 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca45ca0e33d030878a9670/log.txt.gz 2026-03-30 10:03:44,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:03:44,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:03:44,745 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca45ca0e33d030878a9670/lava-logs.yaml 2026-03-30 10:03:44,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:03:44,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:03:44,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca45ca0e33d030878a9670/lava_callback.json.gz DEBUG: Setup failed for 69ca45ca0e33d030878a9670. Transitting job node result: incomplete -> fail 2026-03-30 10:03:45,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:03:45,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca45ca0e33d030878a9670 HTTP/1.1" 200 None 2026-03-30 10:03:45,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:03:45,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:03:45,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:03:45,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca45ca0e33d030878a9670 HTTP/1.1" 200 None 2026-03-30 10:03:45,226 - lava_callback - INFO - Completed processing callback for node 69ca45ca0e33d030878a9670 INFO: 172.21.0.1:35028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46606 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:04:07,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:04:07,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36162 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:05:08,750 - lava_callback - INFO - Received callback for node 69c7a7c8be430e18c5d1692c INFO: 172.21.0.1:36172 - "POST /node/69c7a7c8be430e18c5d1692c HTTP/1.0" 202 Accepted 2026-03-30 10:05:08,755 - lava_callback - INFO - Starting async job submit for node 69c7a7c8be430e18c5d1692c 2026-03-30 10:05:08,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:08,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a7c8be430e18c5d1692c HTTP/1.1" 200 None 2026-03-30 10:05:08,772 - lava_callback - DEBUG - Processing node 69c7a7c8be430e18c5d1692c: baseline-arm64-clabbe 2026-03-30 10:05:08,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a7c8be430e18c5d1692c 2026-03-30 10:05:08,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:08,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a7c8be430e18c5d1692c HTTP/1.1" 200 None 2026-03-30 10:05:08,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:08,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 10:05:08,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:08,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a7c8be430e18c5d1692c HTTP/1.1" 200 None 2026-03-30 10:05:08,829 - lava_callback - ERROR - Error processing callback for node 69c7a7c8be430e18c5d1692c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-30 10:05:14,061 - lava_callback - INFO - Received callback for node 69ca48460e33d030878a96d4 INFO: 172.21.0.1:49908 - "POST /node/69ca48460e33d030878a96d4 HTTP/1.0" 202 Accepted 2026-03-30 10:05:14,068 - lava_callback - INFO - Starting async job submit for node 69ca48460e33d030878a96d4 2026-03-30 10:05:14,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:14,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca48460e33d030878a96d4 HTTP/1.1" 200 None 2026-03-30 10:05:14,109 - lava_callback - DEBUG - Processing node 69ca48460e33d030878a96d4: baseline-nfs-arm64 2026-03-30 10:05:14,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:05:14,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:05:14,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca48460e33d030878a96d4/log.txt.gz 2026-03-30 10:05:15,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:49922 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 10:05:15,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:05:15,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca48460e33d030878a96d4/lava-logs.yaml 2026-03-30 10:05:15,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:05:15,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:05:15,622 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca48460e33d030878a96d4/lava_callback.json.gz 2026-03-30 10:05:15,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:15,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca48460e33d030878a96d4 HTTP/1.1" 200 None 2026-03-30 10:05:15,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:15,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-30 10:05:15,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:15,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca48460e33d030878a96d4 HTTP/1.1" 200 None 2026-03-30 10:05:15,776 - lava_callback - INFO - Completed processing callback for node 69ca48460e33d030878a96d4 INFO: 172.21.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56004 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:05:37,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:05:37,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57472 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:06:32,163 - lava_callback - INFO - Received callback for node 69ca46640e33d030878a968a INFO: 172.21.0.1:60056 - "POST /node/69ca46640e33d030878a968a HTTP/1.0" 202 Accepted 2026-03-30 10:06:32,169 - lava_callback - INFO - Starting async job submit for node 69ca46640e33d030878a968a 2026-03-30 10:06:32,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:06:32,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca46640e33d030878a968a HTTP/1.1" 200 None 2026-03-30 10:06:32,208 - lava_callback - DEBUG - Processing node 69ca46640e33d030878a968a: baseline-arm-pengutronix 2026-03-30 10:06:32,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:06:32,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:06:32,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca46640e33d030878a968a/log.txt.gz 2026-03-30 10:06:32,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:06:32,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:06:32,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca46640e33d030878a968a/lava-logs.yaml 2026-03-30 10:06:32,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:06:32,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:06:32,897 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca46640e33d030878a968a/lava_callback.json.gz DEBUG: Setup failed for 69ca46640e33d030878a968a. Transitting job node result: incomplete -> fail 2026-03-30 10:06:32,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:06:32,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca46640e33d030878a968a HTTP/1.1" 200 None 2026-03-30 10:06:32,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:06:32,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 10:06:32,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:06:33,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca46640e33d030878a968a HTTP/1.1" 200 None 2026-03-30 10:06:33,062 - lava_callback - INFO - Completed processing callback for node 69ca46640e33d030878a968a INFO: 172.21.0.1:60058 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:06:37,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:06:37,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:08:12,982 - lava_callback - INFO - Received callback for node 69c7a87abe430e18c5d16949 INFO: 172.21.0.1:55960 - "POST /node/69c7a87abe430e18c5d16949 HTTP/1.0" 202 Accepted 2026-03-30 10:08:12,987 - lava_callback - INFO - Starting async job submit for node 69c7a87abe430e18c5d16949 2026-03-30 10:08:12,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:13,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a87abe430e18c5d16949 HTTP/1.1" 200 None 2026-03-30 10:08:13,003 - lava_callback - DEBUG - Processing node 69c7a87abe430e18c5d16949: baseline-arm64-baylibre 2026-03-30 10:08:13,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a87abe430e18c5d16949 2026-03-30 10:08:13,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:13,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a87abe430e18c5d16949 HTTP/1.1" 200 None 2026-03-30 10:08:13,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:13,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-30 10:08:13,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:13,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a87abe430e18c5d16949 HTTP/1.1" 200 None 2026-03-30 10:08:13,056 - lava_callback - ERROR - Error processing callback for node 69c7a87abe430e18c5d16949: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37158 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:08:39,998 - lava_callback - INFO - Received callback for node 69ca49220e33d030878a96ef INFO: 172.21.0.1:45468 - "POST /node/69ca49220e33d030878a96ef HTTP/1.0" 202 Accepted 2026-03-30 10:08:40,004 - lava_callback - INFO - Starting async job submit for node 69ca49220e33d030878a96ef 2026-03-30 10:08:40,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:40,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca49220e33d030878a96ef HTTP/1.1" 200 None 2026-03-30 10:08:40,043 - lava_callback - DEBUG - Processing node 69ca49220e33d030878a96ef: baseline-arm64 2026-03-30 10:08:40,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:08:40,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:08:40,896 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca49220e33d030878a96ef/log.txt.gz 2026-03-30 10:08:41,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:08:41,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:08:41,405 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca49220e33d030878a96ef/lava-logs.yaml 2026-03-30 10:08:41,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:08:41,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:08:41,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca49220e33d030878a96ef/lava_callback.json.gz 2026-03-30 10:08:41,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:41,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca49220e33d030878a96ef HTTP/1.1" 200 None 2026-03-30 10:08:41,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:41,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-30 10:08:41,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:08:41,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca49220e33d030878a96ef HTTP/1.1" 200 None 2026-03-30 10:08:41,921 - lava_callback - INFO - Completed processing callback for node 69ca49220e33d030878a96ef INFO: 172.21.0.1:45482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43980 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:09:07,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:09:07,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48070 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:09:28,945 - lava_callback - INFO - Received callback for node 69c7a8cfbe430e18c5d1695d INFO: 172.21.0.1:48086 - "POST /node/69c7a8cfbe430e18c5d1695d HTTP/1.0" 202 Accepted 2026-03-30 10:09:28,951 - lava_callback - INFO - Starting async job submit for node 69c7a8cfbe430e18c5d1695d 2026-03-30 10:09:28,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:09:28,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a8cfbe430e18c5d1695d HTTP/1.1" 200 None 2026-03-30 10:09:28,967 - lava_callback - DEBUG - Processing node 69c7a8cfbe430e18c5d1695d: baseline-arm64-clabbe 2026-03-30 10:09:28,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a8cfbe430e18c5d1695d 2026-03-30 10:09:28,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:09:28,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a8cfbe430e18c5d1695d HTTP/1.1" 200 None 2026-03-30 10:09:28,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:09:29,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-30 10:09:29,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:09:29,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a8cfbe430e18c5d1695d HTTP/1.1" 200 None 2026-03-30 10:09:29,036 - lava_callback - ERROR - Error processing callback for node 69c7a8cfbe430e18c5d1695d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60196 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:10:49,261 - lava_callback - INFO - Received callback for node 69c7a91fbe430e18c5d16967 INFO: 172.21.0.1:60200 - "POST /node/69c7a91fbe430e18c5d16967 HTTP/1.0" 202 Accepted 2026-03-30 10:10:49,266 - lava_callback - INFO - Starting async job submit for node 69c7a91fbe430e18c5d16967 2026-03-30 10:10:49,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:10:49,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a91fbe430e18c5d16967 HTTP/1.1" 200 None 2026-03-30 10:10:49,281 - lava_callback - DEBUG - Processing node 69c7a91fbe430e18c5d16967: baseline-arm64-clabbe 2026-03-30 10:10:49,282 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a91fbe430e18c5d16967 2026-03-30 10:10:49,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:10:49,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a91fbe430e18c5d16967 HTTP/1.1" 200 None 2026-03-30 10:10:49,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:10:49,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-30 10:10:49,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:10:49,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a91fbe430e18c5d16967 HTTP/1.1" 200 None 2026-03-30 10:10:49,333 - lava_callback - ERROR - Error processing callback for node 69c7a91fbe430e18c5d16967: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41758 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:12:17,856 - lava_callback - INFO - Received callback for node 69ca4c6e0e33d030878a976c INFO: 172.21.0.1:41772 - "POST /node/69ca4c6e0e33d030878a976c HTTP/1.0" 202 Accepted 2026-03-30 10:12:17,861 - lava_callback - INFO - Starting async job submit for node 69ca4c6e0e33d030878a976c 2026-03-30 10:12:17,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:12:17,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4c6e0e33d030878a976c HTTP/1.1" 200 None 2026-03-30 10:12:17,879 - lava_callback - DEBUG - Processing node 69ca4c6e0e33d030878a976c: baseline-nfs-arm64 2026-03-30 10:12:17,881 - lava_callback - WARNING - Failed to upload log 2026-03-30 10:12:17,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:12:18,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:12:18,048 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca4c6e0e33d030878a976c/lava-logs.yaml 2026-03-30 10:12:18,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:12:18,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:12:18,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca4c6e0e33d030878a976c/lava_callback.json.gz 2026-03-30 10:12:18,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:12:18,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4c6e0e33d030878a976c HTTP/1.1" 200 None 2026-03-30 10:12:18,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:12:18,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:12:18,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:12:18,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4c6e0e33d030878a976c HTTP/1.1" 200 None 2026-03-30 10:12:18,266 - lava_callback - INFO - Completed processing callback for node 69ca4c6e0e33d030878a976c INFO: 172.21.0.1:45594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56278 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:12:37,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:12:37,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59130 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:13:29,113 - lava_callback - INFO - Received callback for node 69c7a9bfbe430e18c5d16978 INFO: 172.21.0.1:59132 - "POST /node/69c7a9bfbe430e18c5d16978 HTTP/1.0" 202 Accepted 2026-03-30 10:13:29,119 - lava_callback - INFO - Starting async job submit for node 69c7a9bfbe430e18c5d16978 2026-03-30 10:13:29,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:13:29,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a9bfbe430e18c5d16978 HTTP/1.1" 200 None 2026-03-30 10:13:29,138 - lava_callback - DEBUG - Processing node 69c7a9bfbe430e18c5d16978: baseline-arm64-clabbe 2026-03-30 10:13:29,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7a9bfbe430e18c5d16978 2026-03-30 10:13:29,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:13:29,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a9bfbe430e18c5d16978 HTTP/1.1" 200 None 2026-03-30 10:13:29,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:13:29,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-30 10:13:29,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:13:29,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a9bfbe430e18c5d16978 HTTP/1.1" 200 None 2026-03-30 10:13:29,191 - lava_callback - ERROR - Error processing callback for node 69c7a9bfbe430e18c5d16978: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37550 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:14:40,110 - lava_callback - INFO - Received callback for node 69ca4bd20e33d030878a9758 INFO: 172.21.0.1:56936 - "POST /node/69ca4bd20e33d030878a9758 HTTP/1.0" 202 Accepted 2026-03-30 10:14:40,118 - lava_callback - INFO - Starting async job submit for node 69ca4bd20e33d030878a9758 2026-03-30 10:14:40,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:40,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4bd20e33d030878a9758 HTTP/1.1" 200 None 2026-03-30 10:14:40,182 - lava_callback - DEBUG - Processing node 69ca4bd20e33d030878a9758: baseline-arm64 2026-03-30 10:14:40,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:14:40,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:14:40,744 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4bd20e33d030878a9758/log.txt.gz 2026-03-30 10:14:40,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:14:41,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:14:41,195 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4bd20e33d030878a9758/lava-logs.yaml 2026-03-30 10:14:41,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:14:41,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:14:41,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4bd20e33d030878a9758/lava_callback.json.gz DEBUG: Setup failed for 69ca4bd20e33d030878a9758. Transitting job node result: incomplete -> fail 2026-03-30 10:14:41,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:41,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4bd20e33d030878a9758 HTTP/1.1" 200 None 2026-03-30 10:14:41,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:41,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:14:41,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:41,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4bd20e33d030878a9758 HTTP/1.1" 200 None 2026-03-30 10:14:41,592 - lava_callback - INFO - Completed processing callback for node 69ca4bd20e33d030878a9758 INFO: 172.21.0.1:56944 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:14:49,209 - lava_callback - INFO - Received callback for node 69c7aa0ebe430e18c5d16981 INFO: 172.21.0.1:56960 - "POST /node/69c7aa0ebe430e18c5d16981 HTTP/1.0" 202 Accepted 2026-03-30 10:14:49,215 - lava_callback - INFO - Starting async job submit for node 69c7aa0ebe430e18c5d16981 2026-03-30 10:14:49,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:49,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aa0ebe430e18c5d16981 HTTP/1.1" 200 None 2026-03-30 10:14:49,241 - lava_callback - DEBUG - Processing node 69c7aa0ebe430e18c5d16981: baseline-arm64-clabbe 2026-03-30 10:14:49,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7aa0ebe430e18c5d16981 2026-03-30 10:14:49,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:49,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aa0ebe430e18c5d16981 HTTP/1.1" 200 None 2026-03-30 10:14:49,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:49,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-30 10:14:49,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:14:49,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7aa0ebe430e18c5d16981 HTTP/1.1" 200 None 2026-03-30 10:14:49,303 - lava_callback - ERROR - Error processing callback for node 69c7aa0ebe430e18c5d16981: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35932 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:15:07,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:15:07,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51714 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:15:19,671 - lava_callback - INFO - Received callback for node 69ca4cfc0e33d030878a9782 INFO: 172.21.0.1:51730 - "POST /node/69ca4cfc0e33d030878a9782 HTTP/1.0" 202 Accepted 2026-03-30 10:15:19,677 - lava_callback - INFO - Starting async job submit for node 69ca4cfc0e33d030878a9782 2026-03-30 10:15:19,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:15:19,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4cfc0e33d030878a9782 HTTP/1.1" 200 None 2026-03-30 10:15:19,729 - lava_callback - DEBUG - Processing node 69ca4cfc0e33d030878a9782: baseline-arm64 2026-03-30 10:15:19,862 - lava_callback - WARNING - Failed to upload log 2026-03-30 10:15:19,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:15:20,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:15:20,129 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4cfc0e33d030878a9782/lava-logs.yaml 2026-03-30 10:15:20,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:15:20,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:15:20,262 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4cfc0e33d030878a9782/lava_callback.json.gz 2026-03-30 10:15:20,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:15:20,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4cfc0e33d030878a9782 HTTP/1.1" 200 None 2026-03-30 10:15:20,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:15:20,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:15:20,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:15:20,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4cfc0e33d030878a9782 HTTP/1.1" 200 None 2026-03-30 10:15:20,438 - lava_callback - INFO - Completed processing callback for node 69ca4cfc0e33d030878a9782 INFO: 172.21.0.1:49114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37068 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:15:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:15:37,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:17:29,357 - lava_callback - INFO - Received callback for node 69c7aaabbe430e18c5d1699e INFO: 172.21.0.1:49136 - "POST /node/69c7aaabbe430e18c5d1699e HTTP/1.0" 202 Accepted 2026-03-30 10:17:29,363 - lava_callback - INFO - Starting async job submit for node 69c7aaabbe430e18c5d1699e 2026-03-30 10:17:29,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:17:29,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aaabbe430e18c5d1699e HTTP/1.1" 200 None 2026-03-30 10:17:29,379 - lava_callback - DEBUG - Processing node 69c7aaabbe430e18c5d1699e: baseline-arm64-clabbe 2026-03-30 10:17:29,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7aaabbe430e18c5d1699e 2026-03-30 10:17:29,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:17:29,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aaabbe430e18c5d1699e HTTP/1.1" 200 None 2026-03-30 10:17:29,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:17:29,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-30 10:17:29,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:17:29,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7aaabbe430e18c5d1699e HTTP/1.1" 200 None 2026-03-30 10:17:29,454 - lava_callback - ERROR - Error processing callback for node 69c7aaabbe430e18c5d1699e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55642 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:18:00,546 - lava_callback - INFO - Received callback for node 69ca4d930e33d030878a979f INFO: 172.21.0.1:43674 - "POST /node/69ca4d930e33d030878a979f HTTP/1.0" 202 Accepted 2026-03-30 10:18:00,553 - lava_callback - INFO - Starting async job submit for node 69ca4d930e33d030878a979f 2026-03-30 10:18:00,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:18:00,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4d930e33d030878a979f HTTP/1.1" 200 None 2026-03-30 10:18:00,601 - lava_callback - DEBUG - Processing node 69ca4d930e33d030878a979f: baseline-arm64 2026-03-30 10:18:00,761 - lava_callback - WARNING - Failed to upload log 2026-03-30 10:18:00,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:18:01,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:18:01,170 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4d930e33d030878a979f/lava-logs.yaml 2026-03-30 10:18:01,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:18:01,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:18:01,450 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4d930e33d030878a979f/lava_callback.json.gz 2026-03-30 10:18:01,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:18:01,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4d930e33d030878a979f HTTP/1.1" 200 None 2026-03-30 10:18:01,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:18:01,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:18:01,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:18:01,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4d930e33d030878a979f HTTP/1.1" 200 None 2026-03-30 10:18:01,608 - lava_callback - INFO - Completed processing callback for node 69ca4d930e33d030878a979f INFO: 172.21.0.1:43680 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:18:07,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:18:07,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41836 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:20:09,353 - lava_callback - INFO - Received callback for node 69c7ab4abe430e18c5d169ae INFO: 172.21.0.1:41838 - "POST /node/69c7ab4abe430e18c5d169ae HTTP/1.0" 202 Accepted 2026-03-30 10:20:09,358 - lava_callback - INFO - Starting async job submit for node 69c7ab4abe430e18c5d169ae 2026-03-30 10:20:09,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:20:09,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ab4abe430e18c5d169ae HTTP/1.1" 200 None 2026-03-30 10:20:09,374 - lava_callback - DEBUG - Processing node 69c7ab4abe430e18c5d169ae: baseline-arm64-clabbe 2026-03-30 10:20:09,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ab4abe430e18c5d169ae 2026-03-30 10:20:09,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:20:09,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ab4abe430e18c5d169ae HTTP/1.1" 200 None 2026-03-30 10:20:09,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:20:09,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-30 10:20:09,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:20:09,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ab4abe430e18c5d169ae HTTP/1.1" 200 None 2026-03-30 10:20:09,427 - lava_callback - ERROR - Error processing callback for node 69c7ab4abe430e18c5d169ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:21:03,379 - lava_callback - INFO - Received callback for node 69ca4c1e0e33d030878a9761 INFO: 172.21.0.1:53602 - "POST /node/69ca4c1e0e33d030878a9761 HTTP/1.0" 202 Accepted 2026-03-30 10:21:03,385 - lava_callback - INFO - Starting async job submit for node 69ca4c1e0e33d030878a9761 2026-03-30 10:21:03,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:21:03,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4c1e0e33d030878a9761 HTTP/1.1" 200 None 2026-03-30 10:21:03,425 - lava_callback - DEBUG - Processing node 69ca4c1e0e33d030878a9761: baseline-arm64 2026-03-30 10:21:03,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:21:04,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:21:04,258 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4c1e0e33d030878a9761/log.txt.gz 2026-03-30 10:21:04,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:21:04,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:21:04,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4c1e0e33d030878a9761/lava-logs.yaml 2026-03-30 10:21:04,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:21:04,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:21:04,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4c1e0e33d030878a9761/lava_callback.json.gz 2026-03-30 10:21:04,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:21:04,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4c1e0e33d030878a9761 HTTP/1.1" 200 None 2026-03-30 10:21:04,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:21:05,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:21:05,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:21:05,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4c1e0e33d030878a9761 HTTP/1.1" 200 None 2026-03-30 10:21:05,107 - lava_callback - INFO - Completed processing callback for node 69ca4c1e0e33d030878a9761 INFO: 172.21.0.1:53608 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:21:07,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:21:07,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38308 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:22:51,662 - lava_callback - INFO - Received callback for node 69ca4e790e33d030878a97c1 INFO: 172.21.0.1:39628 - "POST /node/69ca4e790e33d030878a97c1 HTTP/1.0" 202 Accepted 2026-03-30 10:22:51,668 - lava_callback - INFO - Starting async job submit for node 69ca4e790e33d030878a97c1 2026-03-30 10:22:51,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:22:51,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4e790e33d030878a97c1 HTTP/1.1" 200 None 2026-03-30 10:22:51,714 - lava_callback - DEBUG - Processing node 69ca4e790e33d030878a97c1: baseline-arm64 2026-03-30 10:22:52,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:22:52,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:22:52,319 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4e790e33d030878a97c1/log.txt.gz 2026-03-30 10:22:52,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:22:52,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:22:52,747 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4e790e33d030878a97c1/lava-logs.yaml 2026-03-30 10:22:52,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:22:52,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:22:52,928 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca4e790e33d030878a97c1/lava_callback.json.gz 2026-03-30 10:22:52,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:22:52,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4e790e33d030878a97c1 HTTP/1.1" 200 None 2026-03-30 10:22:52,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:22:53,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:22:53,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:22:53,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4e790e33d030878a97c1 HTTP/1.1" 200 None 2026-03-30 10:22:53,110 - lava_callback - INFO - Completed processing callback for node 69ca4e790e33d030878a97c1 INFO: 172.21.0.1:39634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:23:07,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:23:07,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35436 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:25:14,039 - lava_callback - INFO - Received callback for node 69c7ac76be430e18c5d169d9 INFO: 172.21.0.1:48916 - "POST /node/69c7ac76be430e18c5d169d9 HTTP/1.0" 202 Accepted 2026-03-30 10:25:14,045 - lava_callback - INFO - Starting async job submit for node 69c7ac76be430e18c5d169d9 2026-03-30 10:25:14,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:25:14,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ac76be430e18c5d169d9 HTTP/1.1" 200 None 2026-03-30 10:25:14,233 - lava_callback - DEBUG - Processing node 69c7ac76be430e18c5d169d9: baseline-arm-baylibre 2026-03-30 10:25:14,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ac76be430e18c5d169d9 2026-03-30 10:25:14,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:25:14,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ac76be430e18c5d169d9 HTTP/1.1" 200 None 2026-03-30 10:25:14,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:25:14,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:25:14,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:25:14,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ac76be430e18c5d169d9 HTTP/1.1" 200 None 2026-03-30 10:25:14,298 - lava_callback - ERROR - Error processing callback for node 69c7ac76be430e18c5d169d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40164 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:26:30,981 - lava_callback - INFO - Received callback for node 69ca4d500e33d030878a9794 INFO: 172.21.0.1:42538 - "POST /node/69ca4d500e33d030878a9794 HTTP/1.0" 202 Accepted 2026-03-30 10:26:30,987 - lava_callback - INFO - Starting async job submit for node 69ca4d500e33d030878a9794 2026-03-30 10:26:31,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:31,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4d500e33d030878a9794 HTTP/1.1" 200 None 2026-03-30 10:26:31,033 - lava_callback - DEBUG - Processing node 69ca4d500e33d030878a9794: baseline-nfs-arm64 2026-03-30 10:26:31,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:26:31,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:26:31,867 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca4d500e33d030878a9794/log.txt.gz 2026-03-30 10:26:32,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:26:32,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:26:32,359 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca4d500e33d030878a9794/lava-logs.yaml 2026-03-30 10:26:32,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:26:32,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:26:32,548 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca4d500e33d030878a9794/lava_callback.json.gz 2026-03-30 10:26:32,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:32,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4d500e33d030878a9794 HTTP/1.1" 200 None 2026-03-30 10:26:32,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:32,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:26:32,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:32,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4d500e33d030878a9794 HTTP/1.1" 200 None 2026-03-30 10:26:32,721 - lava_callback - INFO - Completed processing callback for node 69ca4d500e33d030878a9794 INFO: 172.21.0.1:42550 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:26:37,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:37,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57848 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:26:54,058 - lava_callback - INFO - Received callback for node 69c7acecbe430e18c5d169f7 INFO: 172.21.0.1:35044 - "POST /node/69c7acecbe430e18c5d169f7 HTTP/1.0" 202 Accepted 2026-03-30 10:26:54,064 - lava_callback - INFO - Starting async job submit for node 69c7acecbe430e18c5d169f7 2026-03-30 10:26:54,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:54,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7acecbe430e18c5d169f7 HTTP/1.1" 200 None 2026-03-30 10:26:54,103 - lava_callback - DEBUG - Processing node 69c7acecbe430e18c5d169f7: baseline-arm-baylibre 2026-03-30 10:26:54,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7acecbe430e18c5d169f7 2026-03-30 10:26:54,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:54,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7acecbe430e18c5d169f7 HTTP/1.1" 200 None 2026-03-30 10:26:54,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:54,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:26:54,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:26:54,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7acecbe430e18c5d169f7 HTTP/1.1" 200 None 2026-03-30 10:26:54,163 - lava_callback - ERROR - Error processing callback for node 69c7acecbe430e18c5d169f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60790 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:28:14,151 - lava_callback - INFO - Received callback for node 69c7ad34be430e18c5d16a12 INFO: 172.21.0.1:47244 - "POST /node/69c7ad34be430e18c5d16a12 HTTP/1.0" 202 Accepted 2026-03-30 10:28:14,156 - lava_callback - INFO - Starting async job submit for node 69c7ad34be430e18c5d16a12 2026-03-30 10:28:14,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:28:14,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ad34be430e18c5d16a12 HTTP/1.1" 200 None 2026-03-30 10:28:14,180 - lava_callback - DEBUG - Processing node 69c7ad34be430e18c5d16a12: baseline-arm-baylibre 2026-03-30 10:28:14,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ad34be430e18c5d16a12 2026-03-30 10:28:14,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:28:14,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ad34be430e18c5d16a12 HTTP/1.1" 200 None 2026-03-30 10:28:14,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:28:14,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:28:14,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:28:14,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ad34be430e18c5d16a12 HTTP/1.1" 200 None 2026-03-30 10:28:14,253 - lava_callback - ERROR - Error processing callback for node 69c7ad34be430e18c5d16a12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40200 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:30:09,855 - lava_callback - INFO - Received callback for node 69c7ada7be430e18c5d16a2a INFO: 172.21.0.1:49078 - "POST /node/69c7ada7be430e18c5d16a2a HTTP/1.0" 202 Accepted 2026-03-30 10:30:09,861 - lava_callback - INFO - Starting async job submit for node 69c7ada7be430e18c5d16a2a 2026-03-30 10:30:09,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:09,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ada7be430e18c5d16a2a HTTP/1.1" 200 None 2026-03-30 10:30:09,879 - lava_callback - DEBUG - Processing node 69c7ada7be430e18c5d16a2a: baseline-arm-clabbe 2026-03-30 10:30:09,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ada7be430e18c5d16a2a 2026-03-30 10:30:09,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:09,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ada7be430e18c5d16a2a HTTP/1.1" 200 None 2026-03-30 10:30:09,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:09,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:30:09,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:09,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ada7be430e18c5d16a2a HTTP/1.1" 200 None 2026-03-30 10:30:09,942 - lava_callback - ERROR - Error processing callback for node 69c7ada7be430e18c5d16a2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:30:49,893 - lava_callback - INFO - Received callback for node 69c7adcdbe430e18c5d16a39 INFO: 172.21.0.1:51198 - "POST /node/69c7adcdbe430e18c5d16a39 HTTP/1.0" 202 Accepted 2026-03-30 10:30:49,899 - lava_callback - INFO - Starting async job submit for node 69c7adcdbe430e18c5d16a39 2026-03-30 10:30:49,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:49,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7adcdbe430e18c5d16a39 HTTP/1.1" 200 None 2026-03-30 10:30:49,915 - lava_callback - DEBUG - Processing node 69c7adcdbe430e18c5d16a39: baseline-arm-clabbe 2026-03-30 10:30:49,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7adcdbe430e18c5d16a39 2026-03-30 10:30:49,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:49,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7adcdbe430e18c5d16a39 HTTP/1.1" 200 None 2026-03-30 10:30:49,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:49,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:30:49,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:30:49,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7adcdbe430e18c5d16a39 HTTP/1.1" 200 None 2026-03-30 10:30:49,982 - lava_callback - ERROR - Error processing callback for node 69c7adcdbe430e18c5d16a39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40528 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:31:30,222 - lava_callback - INFO - Received callback for node 69c7adf2be430e18c5d16a49 INFO: 172.21.0.1:35032 - "POST /node/69c7adf2be430e18c5d16a49 HTTP/1.0" 202 Accepted 2026-03-30 10:31:30,231 - lava_callback - INFO - Starting async job submit for node 69c7adf2be430e18c5d16a49 2026-03-30 10:31:30,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:31:30,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7adf2be430e18c5d16a49 HTTP/1.1" 200 None 2026-03-30 10:31:30,253 - lava_callback - DEBUG - Processing node 69c7adf2be430e18c5d16a49: baseline-arm-clabbe 2026-03-30 10:31:30,253 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7adf2be430e18c5d16a49 2026-03-30 10:31:30,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:31:30,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7adf2be430e18c5d16a49 HTTP/1.1" 200 None 2026-03-30 10:31:30,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:31:30,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:31:30,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:31:30,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7adf2be430e18c5d16a49 HTTP/1.1" 200 None 2026-03-30 10:31:30,314 - lava_callback - ERROR - Error processing callback for node 69c7adf2be430e18c5d16a49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:33:10,037 - lava_callback - INFO - Received callback for node 69c7ae59be430e18c5d16a58 INFO: 172.21.0.1:60670 - "POST /node/69c7ae59be430e18c5d16a58 HTTP/1.0" 202 Accepted 2026-03-30 10:33:10,042 - lava_callback - INFO - Starting async job submit for node 69c7ae59be430e18c5d16a58 2026-03-30 10:33:10,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:10,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae59be430e18c5d16a58 HTTP/1.1" 200 None 2026-03-30 10:33:10,059 - lava_callback - DEBUG - Processing node 69c7ae59be430e18c5d16a58: baseline-arm-clabbe 2026-03-30 10:33:10,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ae59be430e18c5d16a58 2026-03-30 10:33:10,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:10,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae59be430e18c5d16a58 HTTP/1.1" 200 None 2026-03-30 10:33:10,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:10,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:33:10,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:10,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ae59be430e18c5d16a58 HTTP/1.1" 200 None 2026-03-30 10:33:10,122 - lava_callback - ERROR - Error processing callback for node 69c7ae59be430e18c5d16a58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:33:50,009 - lava_callback - INFO - Received callback for node 69c7ae7fbe430e18c5d16a6b INFO: 172.21.0.1:45922 - "POST /node/69c7ae7fbe430e18c5d16a6b HTTP/1.0" 202 Accepted 2026-03-30 10:33:50,014 - lava_callback - INFO - Starting async job submit for node 69c7ae7fbe430e18c5d16a6b 2026-03-30 10:33:50,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:50,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae7fbe430e18c5d16a6b HTTP/1.1" 200 None 2026-03-30 10:33:50,031 - lava_callback - DEBUG - Processing node 69c7ae7fbe430e18c5d16a6b: baseline-arm-clabbe 2026-03-30 10:33:50,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ae7fbe430e18c5d16a6b 2026-03-30 10:33:50,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:50,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae7fbe430e18c5d16a6b HTTP/1.1" 200 None 2026-03-30 10:33:50,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:50,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-30 10:33:50,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:33:50,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ae7fbe430e18c5d16a6b HTTP/1.1" 200 None 2026-03-30 10:33:50,096 - lava_callback - ERROR - Error processing callback for node 69c7ae7fbe430e18c5d16a6b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49652 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:40:34,214 - lava_callback - INFO - Received callback for node 69ca51760e33d030878a9860 INFO: 172.21.0.1:55136 - "POST /node/69ca51760e33d030878a9860 HTTP/1.0" 202 Accepted 2026-03-30 10:40:34,221 - lava_callback - INFO - Starting async job submit for node 69ca51760e33d030878a9860 2026-03-30 10:40:34,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:34,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca51760e33d030878a9860 HTTP/1.1" 200 None 2026-03-30 10:40:34,281 - lava_callback - DEBUG - Processing node 69ca51760e33d030878a9860: baseline-arm64 2026-03-30 10:40:34,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:40:34,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:40:34,960 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca51760e33d030878a9860/log.txt.gz 2026-03-30 10:40:35,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:40:35,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:40:35,420 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca51760e33d030878a9860/lava-logs.yaml 2026-03-30 10:40:35,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:40:35,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:40:35,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca51760e33d030878a9860/lava_callback.json.gz DEBUG: Setup failed for 69ca51760e33d030878a9860. Transitting job node result: incomplete -> fail 2026-03-30 10:40:35,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:35,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca51760e33d030878a9860 HTTP/1.1" 200 None 2026-03-30 10:40:35,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:35,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:40:35,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:35,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca51760e33d030878a9860 HTTP/1.1" 200 None 2026-03-30 10:40:35,838 - lava_callback - INFO - Completed processing callback for node 69ca51760e33d030878a9860 INFO: 172.21.0.1:55140 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:40:37,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:37,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 10:40:39,447 - lava_callback - INFO - Received callback for node 69ca4f0f0e33d030878a97e3 INFO: 172.21.0.1:55146 - "POST /node/69ca4f0f0e33d030878a97e3 HTTP/1.0" 202 Accepted 2026-03-30 10:40:39,454 - lava_callback - INFO - Starting async job submit for node 69ca4f0f0e33d030878a97e3 2026-03-30 10:40:39,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:39,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4f0f0e33d030878a97e3 HTTP/1.1" 200 None 2026-03-30 10:40:39,523 - lava_callback - DEBUG - Processing node 69ca4f0f0e33d030878a97e3: baseline-arm 2026-03-30 10:40:40,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:40:40,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:40:40,417 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca4f0f0e33d030878a97e3/log.txt.gz 2026-03-30 10:40:40,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:40:41,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:40:41,372 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca4f0f0e33d030878a97e3/lava-logs.yaml 2026-03-30 10:40:41,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:40:41,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:40:41,605 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca4f0f0e33d030878a97e3/lava_callback.json.gz DEBUG: Setup failed for 69ca4f0f0e33d030878a97e3. Transitting job node result: incomplete -> fail 2026-03-30 10:40:41,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:41,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca4f0f0e33d030878a97e3 HTTP/1.1" 200 None 2026-03-30 10:40:41,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:41,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 10:40:41,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:40:41,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca4f0f0e33d030878a97e3 HTTP/1.1" 200 None 2026-03-30 10:40:41,888 - lava_callback - INFO - Completed processing callback for node 69ca4f0f0e33d030878a97e3 INFO: 172.21.0.1:56508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:41:07,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:41:07,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:44:47,140 - lava_callback - INFO - Received callback for node 69ca51c00e33d030878a986c INFO: 172.21.0.1:59624 - "POST /node/69ca51c00e33d030878a986c HTTP/1.0" 202 Accepted 2026-03-30 10:44:47,147 - lava_callback - INFO - Starting async job submit for node 69ca51c00e33d030878a986c 2026-03-30 10:44:47,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:44:47,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca51c00e33d030878a986c HTTP/1.1" 200 None 2026-03-30 10:44:47,187 - lava_callback - DEBUG - Processing node 69ca51c00e33d030878a986c: baseline-arm64 2026-03-30 10:44:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:44:47,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:44:47,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca51c00e33d030878a986c/log.txt.gz 2026-03-30 10:44:48,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:44:48,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:44:48,444 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca51c00e33d030878a986c/lava-logs.yaml 2026-03-30 10:44:48,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:44:48,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:44:48,621 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca51c00e33d030878a986c/lava_callback.json.gz 2026-03-30 10:44:48,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:44:48,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca51c00e33d030878a986c HTTP/1.1" 200 None 2026-03-30 10:44:48,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:44:48,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:44:48,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:44:48,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca51c00e33d030878a986c HTTP/1.1" 200 None 2026-03-30 10:44:48,750 - lava_callback - INFO - Completed processing callback for node 69ca51c00e33d030878a986c INFO: 172.21.0.1:49204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38832 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:45:07,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:45:07,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36700 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:46:39,669 - lava_callback - INFO - Received callback for node 69ca50660e33d030878a9828 INFO: 172.21.0.1:36702 - "POST /node/69ca50660e33d030878a9828 HTTP/1.0" 202 Accepted 2026-03-30 10:46:39,675 - lava_callback - INFO - Starting async job submit for node 69ca50660e33d030878a9828 2026-03-30 10:46:39,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:46:39,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca50660e33d030878a9828 HTTP/1.1" 200 None 2026-03-30 10:46:39,752 - lava_callback - DEBUG - Processing node 69ca50660e33d030878a9828: ltp-smoke 2026-03-30 10:46:40,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:46:40,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:46:40,375 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca50660e33d030878a9828/log.txt.gz 2026-03-30 10:46:40,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:46:40,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:46:40,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca50660e33d030878a9828/lava-logs.yaml 2026-03-30 10:46:40,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:46:41,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:46:41,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ca50660e33d030878a9828/lava_callback.json.gz 2026-03-30 10:46:41,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:46:41,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca50660e33d030878a9828 HTTP/1.1" 200 None 2026-03-30 10:46:41,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:46:41,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31f70e33d030878a87f5 HTTP/1.1" 200 None 2026-03-30 10:46:41,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:46:41,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca50660e33d030878a9828 HTTP/1.1" 200 None 2026-03-30 10:46:41,489 - lava_callback - INFO - Completed processing callback for node 69ca50660e33d030878a9828 INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45840 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:47:07,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:47:07,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-30 10:47:12,702 - lava_callback - INFO - Received callback for node 69ca52140e33d030878a9879 INFO: 172.21.0.1:55338 - "POST /node/69ca52140e33d030878a9879 HTTP/1.0" 202 Accepted 2026-03-30 10:47:12,709 - lava_callback - INFO - Starting async job submit for node 69ca52140e33d030878a9879 2026-03-30 10:47:12,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:47:12,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca52140e33d030878a9879 HTTP/1.1" 200 None 2026-03-30 10:47:12,751 - lava_callback - DEBUG - Processing node 69ca52140e33d030878a9879: baseline-nfs-arm64 2026-03-30 10:47:13,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:47:13,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:47:13,597 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca52140e33d030878a9879/log.txt.gz 2026-03-30 10:47:13,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:47:14,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:47:14,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca52140e33d030878a9879/lava-logs.yaml 2026-03-30 10:47:14,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:47:14,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:47:14,247 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca52140e33d030878a9879/lava_callback.json.gz 2026-03-30 10:47:14,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:47:14,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca52140e33d030878a9879 HTTP/1.1" 200 None 2026-03-30 10:47:14,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:47:14,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:47:14,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:47:14,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca52140e33d030878a9879 HTTP/1.1" 200 None 2026-03-30 10:47:14,417 - lava_callback - INFO - Completed processing callback for node 69ca52140e33d030878a9879 INFO: 172.21.0.1:55352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54916 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:47:37,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:47:37,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53840 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:50:24,119 - lava_callback - INFO - Received callback for node 69ca50bd0e33d030878a983a INFO: 172.21.0.1:39190 - "POST /node/69ca50bd0e33d030878a983a HTTP/1.0" 202 Accepted 2026-03-30 10:50:24,127 - lava_callback - INFO - Starting async job submit for node 69ca50bd0e33d030878a983a 2026-03-30 10:50:24,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:50:24,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca50bd0e33d030878a983a HTTP/1.1" 200 None 2026-03-30 10:50:24,185 - lava_callback - DEBUG - Processing node 69ca50bd0e33d030878a983a: baseline-nfs-arm64 2026-03-30 10:50:25,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:50:25,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:50:25,504 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca50bd0e33d030878a983a/log.txt.gz 2026-03-30 10:50:26,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:50:26,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:50:26,368 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca50bd0e33d030878a983a/lava-logs.yaml 2026-03-30 10:50:26,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:50:26,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:50:26,595 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca50bd0e33d030878a983a/lava_callback.json.gz DEBUG: Setup failed for 69ca50bd0e33d030878a983a. Transitting job node result: incomplete -> fail 2026-03-30 10:50:26,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:50:26,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca50bd0e33d030878a983a HTTP/1.1" 200 None 2026-03-30 10:50:26,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:50:26,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 10:50:26,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:50:26,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca50bd0e33d030878a983a HTTP/1.1" 200 None 2026-03-30 10:50:26,851 - lava_callback - INFO - Completed processing callback for node 69ca50bd0e33d030878a983a INFO: 172.21.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50230 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:50:37,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:50:37,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45808 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:52:18,234 - lava_callback - INFO - Received callback for node 69ca51580e33d030878a9855 INFO: 172.21.0.1:45824 - "POST /node/69ca51580e33d030878a9855 HTTP/1.0" 202 Accepted 2026-03-30 10:52:18,281 - lava_callback - INFO - Starting async job submit for node 69ca51580e33d030878a9855 2026-03-30 10:52:18,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:52:18,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca51580e33d030878a9855 HTTP/1.1" 200 None 2026-03-30 10:52:18,321 - lava_callback - DEBUG - Processing node 69ca51580e33d030878a9855: baseline-arm-pengutronix 2026-03-30 10:52:18,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:52:18,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:52:18,556 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca51580e33d030878a9855/log.txt.gz 2026-03-30 10:52:18,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:52:18,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:52:18,985 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca51580e33d030878a9855/lava-logs.yaml 2026-03-30 10:52:18,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:52:19,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:52:19,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca51580e33d030878a9855/lava_callback.json.gz DEBUG: Setup failed for 69ca51580e33d030878a9855. Transitting job node result: incomplete -> fail 2026-03-30 10:52:19,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:52:19,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca51580e33d030878a9855 HTTP/1.1" 200 None 2026-03-30 10:52:19,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:52:19,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 10:52:19,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:52:19,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca51580e33d030878a9855 HTTP/1.1" 200 None 2026-03-30 10:52:19,361 - lava_callback - INFO - Completed processing callback for node 69ca51580e33d030878a9855 INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43938 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:52:37,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:52:37,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:58:24,564 - lava_callback - INFO - Received callback for node 69ca532a0e33d030878a988f INFO: 172.21.0.1:35426 - "POST /node/69ca532a0e33d030878a988f HTTP/1.0" 202 Accepted 2026-03-30 10:58:24,571 - lava_callback - INFO - Starting async job submit for node 69ca532a0e33d030878a988f 2026-03-30 10:58:24,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:58:24,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca532a0e33d030878a988f HTTP/1.1" 200 None 2026-03-30 10:58:24,647 - lava_callback - DEBUG - Processing node 69ca532a0e33d030878a988f: baseline-arm 2026-03-30 10:58:25,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:58:25,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:58:25,439 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ca532a0e33d030878a988f/log.txt.gz 2026-03-30 10:58:25,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:58:26,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:58:26,075 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ca532a0e33d030878a988f/lava-logs.yaml 2026-03-30 10:58:26,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 10:58:26,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 10:58:26,276 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ca532a0e33d030878a988f/lava_callback.json.gz DEBUG: Setup failed for 69ca532a0e33d030878a988f. Transitting job node result: incomplete -> fail 2026-03-30 10:58:26,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:58:26,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca532a0e33d030878a988f HTTP/1.1" 200 None 2026-03-30 10:58:26,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:58:26,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 10:58:26,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:58:26,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca532a0e33d030878a988f HTTP/1.1" 200 None INFO: 172.21.0.1:35434 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:58:26,585 - lava_callback - INFO - Completed processing callback for node 69ca532a0e33d030878a988f INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33622 - "GET / HTTP/1.0" 200 OK 2026-03-30 10:58:37,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 10:58:37,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51378 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:09:36,461 - lava_callback - INFO - Received callback for node 69c7b6eebe430e18c5d16b7a INFO: 172.21.0.1:59940 - "POST /node/69c7b6eebe430e18c5d16b7a HTTP/1.0" 202 Accepted 2026-03-30 11:09:36,466 - lava_callback - INFO - Starting async job submit for node 69c7b6eebe430e18c5d16b7a 2026-03-30 11:09:36,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:09:36,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b6eebe430e18c5d16b7a HTTP/1.1" 200 None 2026-03-30 11:09:36,483 - lava_callback - DEBUG - Processing node 69c7b6eebe430e18c5d16b7a: baseline-arm64-baylibre 2026-03-30 11:09:36,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7b6eebe430e18c5d16b7a 2026-03-30 11:09:36,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:09:36,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b6eebe430e18c5d16b7a HTTP/1.1" 200 None 2026-03-30 11:09:36,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:09:36,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:09:36,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:09:36,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7b6eebe430e18c5d16b7a HTTP/1.1" 200 None 2026-03-30 11:09:36,541 - lava_callback - ERROR - Error processing callback for node 69c7b6eebe430e18c5d16b7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33072 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:12:56,589 - lava_callback - INFO - Received callback for node 69c7b7a5be430e18c5d16b87 INFO: 172.21.0.1:33080 - "POST /node/69c7b7a5be430e18c5d16b87 HTTP/1.0" 202 Accepted 2026-03-30 11:12:56,595 - lava_callback - INFO - Starting async job submit for node 69c7b7a5be430e18c5d16b87 2026-03-30 11:12:56,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:12:56,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b7a5be430e18c5d16b87 HTTP/1.1" 200 None 2026-03-30 11:12:56,641 - lava_callback - DEBUG - Processing node 69c7b7a5be430e18c5d16b87: baseline-arm64-baylibre 2026-03-30 11:12:56,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7b7a5be430e18c5d16b87 2026-03-30 11:12:56,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:12:56,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b7a5be430e18c5d16b87 HTTP/1.1" 200 None 2026-03-30 11:12:56,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:12:56,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:12:56,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:12:56,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7b7a5be430e18c5d16b87 HTTP/1.1" 200 None 2026-03-30 11:12:56,694 - lava_callback - ERROR - Error processing callback for node 69c7b7a5be430e18c5d16b87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39028 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:13:25,722 - lava_callback - INFO - Received callback for node 69ca56220e33d030878a996a INFO: 172.21.0.1:56426 - "POST /node/69ca56220e33d030878a996a HTTP/1.0" 202 Accepted 2026-03-30 11:13:25,730 - lava_callback - INFO - Starting async job submit for node 69ca56220e33d030878a996a 2026-03-30 11:13:25,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:13:25,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca56220e33d030878a996a HTTP/1.1" 200 None 2026-03-30 11:13:25,788 - lava_callback - DEBUG - Processing node 69ca56220e33d030878a996a: baseline-nfs-arm64 INFO: 172.21.0.1:56438 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:13:26,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:13:27,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:13:27,203 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca56220e33d030878a996a/log.txt.gz INFO: 172.21.0.1:56442 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:13:27,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:13:28,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:13:28,067 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca56220e33d030878a996a/lava-logs.yaml 2026-03-30 11:13:28,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:13:28,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:13:28,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca56220e33d030878a996a/lava_callback.json.gz DEBUG: Setup failed for 69ca56220e33d030878a996a. Transitting job node result: incomplete -> fail 2026-03-30 11:13:28,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:13:28,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca56220e33d030878a996a HTTP/1.1" 200 None 2026-03-30 11:13:28,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:13:28,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 11:13:28,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:13:28,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca56220e33d030878a996a HTTP/1.1" 200 None 2026-03-30 11:13:28,558 - lava_callback - INFO - Completed processing callback for node 69ca56220e33d030878a996a INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:13:37,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:13:37,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:16:56,786 - lava_callback - INFO - Received callback for node 69c7b895be430e18c5d16b9c INFO: 172.21.0.1:57984 - "POST /node/69c7b895be430e18c5d16b9c HTTP/1.0" 202 Accepted 2026-03-30 11:16:56,791 - lava_callback - INFO - Starting async job submit for node 69c7b895be430e18c5d16b9c 2026-03-30 11:16:56,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:16:56,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b895be430e18c5d16b9c HTTP/1.1" 200 None 2026-03-30 11:16:56,807 - lava_callback - DEBUG - Processing node 69c7b895be430e18c5d16b9c: baseline-arm64-baylibre 2026-03-30 11:16:56,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7b895be430e18c5d16b9c 2026-03-30 11:16:56,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:16:56,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b895be430e18c5d16b9c HTTP/1.1" 200 None 2026-03-30 11:16:56,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:16:56,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:16:56,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:16:56,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7b895be430e18c5d16b9c HTTP/1.1" 200 None 2026-03-30 11:16:56,873 - lava_callback - ERROR - Error processing callback for node 69c7b895be430e18c5d16b9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51664 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:18:56,889 - lava_callback - INFO - Received callback for node 69c7b90dbe430e18c5d16bad INFO: 172.21.0.1:51668 - "POST /node/69c7b90dbe430e18c5d16bad HTTP/1.0" 202 Accepted 2026-03-30 11:18:56,899 - lava_callback - INFO - Starting async job submit for node 69c7b90dbe430e18c5d16bad 2026-03-30 11:18:56,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:18:56,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b90dbe430e18c5d16bad HTTP/1.1" 200 None 2026-03-30 11:18:56,937 - lava_callback - DEBUG - Processing node 69c7b90dbe430e18c5d16bad: baseline-arm64-baylibre 2026-03-30 11:18:56,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7b90dbe430e18c5d16bad 2026-03-30 11:18:56,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:18:56,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b90dbe430e18c5d16bad HTTP/1.1" 200 None 2026-03-30 11:18:56,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:18:56,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:18:56,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:18:57,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7b90dbe430e18c5d16bad HTTP/1.1" 200 None 2026-03-30 11:18:57,014 - lava_callback - ERROR - Error processing callback for node 69c7b90dbe430e18c5d16bad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54410 - "GET /node/69a16198a41e9451dcf412c4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:54426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53086 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:24:52,260 - lava_callback - INFO - Received callback for node 69c7ba7cbe430e18c5d16bc6 INFO: 172.21.0.1:55300 - "POST /node/69c7ba7cbe430e18c5d16bc6 HTTP/1.0" 202 Accepted 2026-03-30 11:24:52,266 - lava_callback - INFO - Starting async job submit for node 69c7ba7cbe430e18c5d16bc6 2026-03-30 11:24:52,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:24:52,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ba7cbe430e18c5d16bc6 HTTP/1.1" 200 None 2026-03-30 11:24:52,303 - lava_callback - DEBUG - Processing node 69c7ba7cbe430e18c5d16bc6: baseline-arm64-clabbe 2026-03-30 11:24:52,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ba7cbe430e18c5d16bc6 2026-03-30 11:24:52,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:24:52,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ba7cbe430e18c5d16bc6 HTTP/1.1" 200 None 2026-03-30 11:24:52,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:24:52,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:24:52,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:24:52,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ba7cbe430e18c5d16bc6 HTTP/1.1" 200 None 2026-03-30 11:24:52,386 - lava_callback - ERROR - Error processing callback for node 69c7ba7cbe430e18c5d16bc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:26:12,283 - lava_callback - INFO - Received callback for node 69c7bacbbe430e18c5d16bcf INFO: 172.21.0.1:45576 - "POST /node/69c7bacbbe430e18c5d16bcf HTTP/1.0" 202 Accepted 2026-03-30 11:26:12,288 - lava_callback - INFO - Starting async job submit for node 69c7bacbbe430e18c5d16bcf 2026-03-30 11:26:12,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:26:12,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bacbbe430e18c5d16bcf HTTP/1.1" 200 None 2026-03-30 11:26:12,316 - lava_callback - DEBUG - Processing node 69c7bacbbe430e18c5d16bcf: baseline-arm64-clabbe 2026-03-30 11:26:12,316 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7bacbbe430e18c5d16bcf 2026-03-30 11:26:12,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:26:12,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bacbbe430e18c5d16bcf HTTP/1.1" 200 None 2026-03-30 11:26:12,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:26:12,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:26:12,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:26:12,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7bacbbe430e18c5d16bcf HTTP/1.1" 200 None 2026-03-30 11:26:12,369 - lava_callback - ERROR - Error processing callback for node 69c7bacbbe430e18c5d16bcf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48078 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:27:52,346 - lava_callback - INFO - Received callback for node 69c7bb30be430e18c5d16be0 INFO: 172.21.0.1:35658 - "POST /node/69c7bb30be430e18c5d16be0 HTTP/1.0" 202 Accepted 2026-03-30 11:27:52,351 - lava_callback - INFO - Starting async job submit for node 69c7bb30be430e18c5d16be0 2026-03-30 11:27:52,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:27:52,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bb30be430e18c5d16be0 HTTP/1.1" 200 None 2026-03-30 11:27:52,375 - lava_callback - DEBUG - Processing node 69c7bb30be430e18c5d16be0: baseline-arm64-clabbe 2026-03-30 11:27:52,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7bb30be430e18c5d16be0 2026-03-30 11:27:52,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:27:52,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bb30be430e18c5d16be0 HTTP/1.1" 200 None 2026-03-30 11:27:52,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:27:52,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:27:52,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:27:52,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7bb30be430e18c5d16be0 HTTP/1.1" 200 None 2026-03-30 11:27:52,453 - lava_callback - ERROR - Error processing callback for node 69c7bb30be430e18c5d16be0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34424 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:29:12,372 - lava_callback - INFO - Received callback for node 69c7bb7ebe430e18c5d16be9 INFO: 172.21.0.1:46288 - "POST /node/69c7bb7ebe430e18c5d16be9 HTTP/1.0" 202 Accepted 2026-03-30 11:29:12,378 - lava_callback - INFO - Starting async job submit for node 69c7bb7ebe430e18c5d16be9 2026-03-30 11:29:12,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:29:12,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bb7ebe430e18c5d16be9 HTTP/1.1" 200 None 2026-03-30 11:29:12,403 - lava_callback - DEBUG - Processing node 69c7bb7ebe430e18c5d16be9: baseline-arm64-clabbe 2026-03-30 11:29:12,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7bb7ebe430e18c5d16be9 2026-03-30 11:29:12,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:29:12,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bb7ebe430e18c5d16be9 HTTP/1.1" 200 None 2026-03-30 11:29:12,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:29:12,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:29:12,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:29:12,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7bb7ebe430e18c5d16be9 HTTP/1.1" 200 None 2026-03-30 11:29:12,504 - lava_callback - ERROR - Error processing callback for node 69c7bb7ebe430e18c5d16be9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60564 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:31:52,508 - lava_callback - INFO - Received callback for node 69c7bc21be430e18c5d16bfd INFO: 172.21.0.1:36714 - "POST /node/69c7bc21be430e18c5d16bfd HTTP/1.0" 202 Accepted 2026-03-30 11:31:52,513 - lava_callback - INFO - Starting async job submit for node 69c7bc21be430e18c5d16bfd 2026-03-30 11:31:52,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:31:52,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bc21be430e18c5d16bfd HTTP/1.1" 200 None 2026-03-30 11:31:52,550 - lava_callback - DEBUG - Processing node 69c7bc21be430e18c5d16bfd: baseline-arm64-clabbe 2026-03-30 11:31:52,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7bc21be430e18c5d16bfd 2026-03-30 11:31:52,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:31:52,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bc21be430e18c5d16bfd HTTP/1.1" 200 None 2026-03-30 11:31:52,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:31:52,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:31:52,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:31:52,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7bc21be430e18c5d16bfd HTTP/1.1" 200 None 2026-03-30 11:31:52,646 - lava_callback - ERROR - Error processing callback for node 69c7bc21be430e18c5d16bfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41990 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:33:12,530 - lava_callback - INFO - Received callback for node 69c7bc70be430e18c5d16c05 INFO: 172.21.0.1:47782 - "POST /node/69c7bc70be430e18c5d16c05 HTTP/1.0" 202 Accepted 2026-03-30 11:33:12,536 - lava_callback - INFO - Starting async job submit for node 69c7bc70be430e18c5d16c05 2026-03-30 11:33:12,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:12,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bc70be430e18c5d16c05 HTTP/1.1" 200 None 2026-03-30 11:33:12,552 - lava_callback - DEBUG - Processing node 69c7bc70be430e18c5d16c05: baseline-arm64-clabbe 2026-03-30 11:33:12,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7bc70be430e18c5d16c05 2026-03-30 11:33:12,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:12,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7bc70be430e18c5d16c05 HTTP/1.1" 200 None 2026-03-30 11:33:12,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:12,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-30 11:33:12,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:12,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7bc70be430e18c5d16c05 HTTP/1.1" 200 None 2026-03-30 11:33:12,611 - lava_callback - ERROR - Error processing callback for node 69c7bc70be430e18c5d16c05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50540 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:33:41,960 - lava_callback - INFO - Received callback for node 69ca54250e33d030878a98b3 INFO: 172.21.0.1:59070 - "POST /node/69ca54250e33d030878a98b3 HTTP/1.0" 202 Accepted 2026-03-30 11:33:41,967 - lava_callback - INFO - Starting async job submit for node 69ca54250e33d030878a98b3 2026-03-30 11:33:41,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:42,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca54250e33d030878a98b3 HTTP/1.1" 200 None 2026-03-30 11:33:42,007 - lava_callback - DEBUG - Processing node 69ca54250e33d030878a98b3: baseline-arm64 2026-03-30 11:33:42,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:33:42,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:33:42,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca54250e33d030878a98b3/log.txt.gz 2026-03-30 11:33:42,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:33:43,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:33:43,213 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca54250e33d030878a98b3/lava-logs.yaml 2026-03-30 11:33:43,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:33:43,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:33:43,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ca54250e33d030878a98b3/lava_callback.json.gz 2026-03-30 11:33:43,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:43,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca54250e33d030878a98b3 HTTP/1.1" 200 None 2026-03-30 11:33:43,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:43,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 11:33:43,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:33:43,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca54250e33d030878a98b3 HTTP/1.1" 200 None 2026-03-30 11:33:43,695 - lava_callback - INFO - Completed processing callback for node 69ca54250e33d030878a98b3 INFO: 172.21.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59262 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:34:07,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:34:07,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:41:21,919 - lava_callback - INFO - Received callback for node 69ca552f0e33d030878a990d INFO: 172.21.0.1:36634 - "POST /node/69ca552f0e33d030878a990d HTTP/1.0" 202 Accepted 2026-03-30 11:41:21,926 - lava_callback - INFO - Starting async job submit for node 69ca552f0e33d030878a990d 2026-03-30 11:41:21,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:41:21,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca552f0e33d030878a990d HTTP/1.1" 200 None 2026-03-30 11:41:21,967 - lava_callback - DEBUG - Processing node 69ca552f0e33d030878a990d: baseline-nfs-arm64 2026-03-30 11:41:22,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:41:22,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:41:22,663 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca552f0e33d030878a990d/log.txt.gz 2026-03-30 11:41:22,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:41:23,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:41:23,189 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca552f0e33d030878a990d/lava-logs.yaml 2026-03-30 11:41:23,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:41:23,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:41:23,383 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ca552f0e33d030878a990d/lava_callback.json.gz 2026-03-30 11:41:23,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:41:23,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca552f0e33d030878a990d HTTP/1.1" 200 None 2026-03-30 11:41:23,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:41:23,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 11:41:23,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:41:23,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca552f0e33d030878a990d HTTP/1.1" 200 None 2026-03-30 11:41:23,571 - lava_callback - INFO - Completed processing callback for node 69ca552f0e33d030878a990d INFO: 172.21.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45688 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:41:37,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:41:37,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59556 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:50:20,859 - lava_callback - INFO - Received callback for node 69ca56e40e33d030878a999c INFO: 172.21.0.1:47102 - "POST /node/69ca56e40e33d030878a999c HTTP/1.0" 202 Accepted 2026-03-30 11:50:20,906 - lava_callback - INFO - Starting async job submit for node 69ca56e40e33d030878a999c 2026-03-30 11:50:20,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:50:20,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca56e40e33d030878a999c HTTP/1.1" 200 None 2026-03-30 11:50:20,954 - lava_callback - DEBUG - Processing node 69ca56e40e33d030878a999c: baseline-arm-pengutronix 2026-03-30 11:50:21,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:50:21,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:50:21,232 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca56e40e33d030878a999c/log.txt.gz 2026-03-30 11:50:21,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:50:21,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:50:21,635 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca56e40e33d030878a999c/lava-logs.yaml 2026-03-30 11:50:21,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 11:50:21,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 11:50:21,791 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ca56e40e33d030878a999c/lava_callback.json.gz DEBUG: Setup failed for 69ca56e40e33d030878a999c. Transitting job node result: incomplete -> fail 2026-03-30 11:50:21,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:50:21,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca56e40e33d030878a999c HTTP/1.1" 200 None 2026-03-30 11:50:21,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:50:21,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-03-30 11:50:21,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:50:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca56e40e33d030878a999c HTTP/1.1" 200 None 2026-03-30 11:50:21,968 - lava_callback - INFO - Completed processing callback for node 69ca56e40e33d030878a999c INFO: 172.21.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43772 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:50:37,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:50:37,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43778 - "GET /node/699399c0cebe99dd8e074f60%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51234 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:56:53,624 - lava_callback - INFO - Received callback for node 69c7c1f1be430e18c5d16c87 INFO: 172.21.0.1:50848 - "POST /node/69c7c1f1be430e18c5d16c87 HTTP/1.0" 202 Accepted 2026-03-30 11:56:53,630 - lava_callback - INFO - Starting async job submit for node 69c7c1f1be430e18c5d16c87 2026-03-30 11:56:53,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:56:53,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c1f1be430e18c5d16c87 HTTP/1.1" 200 None 2026-03-30 11:56:53,656 - lava_callback - DEBUG - Processing node 69c7c1f1be430e18c5d16c87: baseline-arm-clabbe 2026-03-30 11:56:53,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7c1f1be430e18c5d16c87 2026-03-30 11:56:53,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:56:53,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c1f1be430e18c5d16c87 HTTP/1.1" 200 None 2026-03-30 11:56:53,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:56:53,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-30 11:56:53,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:56:53,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7c1f1be430e18c5d16c87 HTTP/1.1" 200 None 2026-03-30 11:56:53,715 - lava_callback - ERROR - Error processing callback for node 69c7c1f1be430e18c5d16c87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36484 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:57:13,644 - lava_callback - INFO - Received callback for node 69c7c216be430e18c5d16c9c INFO: 172.21.0.1:45410 - "POST /node/69c7c216be430e18c5d16c9c HTTP/1.0" 202 Accepted 2026-03-30 11:57:13,650 - lava_callback - INFO - Starting async job submit for node 69c7c216be430e18c5d16c9c 2026-03-30 11:57:13,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:13,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c216be430e18c5d16c9c HTTP/1.1" 200 None 2026-03-30 11:57:13,666 - lava_callback - DEBUG - Processing node 69c7c216be430e18c5d16c9c: baseline-arm-clabbe 2026-03-30 11:57:13,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7c216be430e18c5d16c9c 2026-03-30 11:57:13,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:13,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c216be430e18c5d16c9c HTTP/1.1" 200 None 2026-03-30 11:57:13,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:13,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-30 11:57:13,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:13,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7c216be430e18c5d16c9c HTTP/1.1" 200 None 2026-03-30 11:57:13,721 - lava_callback - ERROR - Error processing callback for node 69c7c216be430e18c5d16c9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51236 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:57:53,630 - lava_callback - INFO - Received callback for node 69c7c23bbe430e18c5d16caa INFO: 172.21.0.1:40238 - "POST /node/69c7c23bbe430e18c5d16caa HTTP/1.0" 202 Accepted 2026-03-30 11:57:53,636 - lava_callback - INFO - Starting async job submit for node 69c7c23bbe430e18c5d16caa 2026-03-30 11:57:53,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:53,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c23bbe430e18c5d16caa HTTP/1.1" 200 None 2026-03-30 11:57:53,674 - lava_callback - DEBUG - Processing node 69c7c23bbe430e18c5d16caa: baseline-arm-clabbe 2026-03-30 11:57:53,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7c23bbe430e18c5d16caa 2026-03-30 11:57:53,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:53,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c23bbe430e18c5d16caa HTTP/1.1" 200 None 2026-03-30 11:57:53,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:53,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-30 11:57:53,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:57:53,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7c23bbe430e18c5d16caa HTTP/1.1" 200 None 2026-03-30 11:57:53,775 - lava_callback - ERROR - Error processing callback for node 69c7c23bbe430e18c5d16caa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48970 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:58:33,618 - lava_callback - INFO - Received callback for node 69c7c260be430e18c5d16cb6 INFO: 172.21.0.1:37292 - "POST /node/69c7c260be430e18c5d16cb6 HTTP/1.0" 202 Accepted 2026-03-30 11:58:33,624 - lava_callback - INFO - Starting async job submit for node 69c7c260be430e18c5d16cb6 2026-03-30 11:58:33,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:58:33,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c260be430e18c5d16cb6 HTTP/1.1" 200 None 2026-03-30 11:58:33,640 - lava_callback - DEBUG - Processing node 69c7c260be430e18c5d16cb6: baseline-arm-clabbe 2026-03-30 11:58:33,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7c260be430e18c5d16cb6 2026-03-30 11:58:33,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:58:33,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c260be430e18c5d16cb6 HTTP/1.1" 200 None 2026-03-30 11:58:33,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:58:33,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-30 11:58:33,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:58:33,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7c260be430e18c5d16cb6 HTTP/1.1" 200 None 2026-03-30 11:58:33,694 - lava_callback - ERROR - Error processing callback for node 69c7c260be430e18c5d16cb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55270 - "GET / HTTP/1.0" 200 OK 2026-03-30 11:59:13,678 - lava_callback - INFO - Received callback for node 69c7c285be430e18c5d16cc6 INFO: 172.21.0.1:40358 - "POST /node/69c7c285be430e18c5d16cc6 HTTP/1.0" 202 Accepted 2026-03-30 11:59:13,684 - lava_callback - INFO - Starting async job submit for node 69c7c285be430e18c5d16cc6 2026-03-30 11:59:13,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:59:13,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c285be430e18c5d16cc6 HTTP/1.1" 200 None 2026-03-30 11:59:13,878 - lava_callback - DEBUG - Processing node 69c7c285be430e18c5d16cc6: baseline-arm-clabbe 2026-03-30 11:59:13,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7c285be430e18c5d16cc6 2026-03-30 11:59:13,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:59:13,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7c285be430e18c5d16cc6 HTTP/1.1" 200 None 2026-03-30 11:59:13,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:59:13,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-30 11:59:13,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 11:59:13,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7c285be430e18c5d16cc6 HTTP/1.1" 200 None 2026-03-30 11:59:13,932 - lava_callback - ERROR - Error processing callback for node 69c7c285be430e18c5d16cc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39888 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:51:42,118 - lava_callback - INFO - Received callback for node 69c7ced5be430e18c5d16d3d INFO: 172.21.0.1:34014 - "POST /node/69c7ced5be430e18c5d16d3d HTTP/1.0" 202 Accepted 2026-03-30 12:51:42,124 - lava_callback - INFO - Starting async job submit for node 69c7ced5be430e18c5d16d3d 2026-03-30 12:51:42,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:51:42,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ced5be430e18c5d16d3d HTTP/1.1" 200 None 2026-03-30 12:51:42,141 - lava_callback - DEBUG - Processing node 69c7ced5be430e18c5d16d3d: baseline-arm-baylibre 2026-03-30 12:51:42,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ced5be430e18c5d16d3d 2026-03-30 12:51:42,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:51:42,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ced5be430e18c5d16d3d HTTP/1.1" 200 None 2026-03-30 12:51:42,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:51:42,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-30 12:51:42,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:51:42,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ced5be430e18c5d16d3d HTTP/1.1" 200 None 2026-03-30 12:51:42,262 - lava_callback - ERROR - Error processing callback for node 69c7ced5be430e18c5d16d3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49486 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:52:36,095 - lava_callback - INFO - Received callback for node 69c7cf06be430e18c5d16d4b INFO: 172.21.0.1:53756 - "POST /node/69c7cf06be430e18c5d16d4b HTTP/1.0" 202 Accepted 2026-03-30 12:52:36,101 - lava_callback - INFO - Starting async job submit for node 69c7cf06be430e18c5d16d4b 2026-03-30 12:52:36,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:52:36,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cf06be430e18c5d16d4b HTTP/1.1" 200 None 2026-03-30 12:52:36,117 - lava_callback - DEBUG - Processing node 69c7cf06be430e18c5d16d4b: baseline-arm-clabbe 2026-03-30 12:52:36,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7cf06be430e18c5d16d4b 2026-03-30 12:52:36,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:52:36,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cf06be430e18c5d16d4b HTTP/1.1" 200 None 2026-03-30 12:52:36,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:52:36,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 12:52:36,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:52:36,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7cf06be430e18c5d16d4b HTTP/1.1" 200 None 2026-03-30 12:52:36,179 - lava_callback - ERROR - Error processing callback for node 69c7cf06be430e18c5d16d4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET /node/6991f80101ff1e8e0859d4b3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37806 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:53:02,025 - lava_callback - INFO - Received callback for node 69c7cf27be430e18c5d16d5a INFO: 172.21.0.1:45458 - "POST /node/69c7cf27be430e18c5d16d5a HTTP/1.0" 202 Accepted 2026-03-30 12:53:02,031 - lava_callback - INFO - Starting async job submit for node 69c7cf27be430e18c5d16d5a 2026-03-30 12:53:02,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:53:02,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cf27be430e18c5d16d5a HTTP/1.1" 200 None 2026-03-30 12:53:02,048 - lava_callback - DEBUG - Processing node 69c7cf27be430e18c5d16d5a: baseline-arm-baylibre 2026-03-30 12:53:02,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7cf27be430e18c5d16d5a 2026-03-30 12:53:02,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:53:02,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cf27be430e18c5d16d5a HTTP/1.1" 200 None 2026-03-30 12:53:02,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:53:02,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-30 12:53:02,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:53:02,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7cf27be430e18c5d16d5a HTTP/1.1" 200 None 2026-03-30 12:53:02,110 - lava_callback - ERROR - Error processing callback for node 69c7cf27be430e18c5d16d5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 12:54:16,004 - lava_callback - INFO - Received callback for node 69c7cf6abe430e18c5d16d69 INFO: 172.21.0.1:57100 - "POST /node/69c7cf6abe430e18c5d16d69 HTTP/1.0" 202 Accepted 2026-03-30 12:54:16,010 - lava_callback - INFO - Starting async job submit for node 69c7cf6abe430e18c5d16d69 2026-03-30 12:54:16,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:54:16,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cf6abe430e18c5d16d69 HTTP/1.1" 200 None 2026-03-30 12:54:16,056 - lava_callback - DEBUG - Processing node 69c7cf6abe430e18c5d16d69: baseline-arm-clabbe 2026-03-30 12:54:16,056 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7cf6abe430e18c5d16d69 2026-03-30 12:54:16,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:54:16,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cf6abe430e18c5d16d69 HTTP/1.1" 200 None 2026-03-30 12:54:16,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:54:16,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 12:54:16,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:54:16,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7cf6abe430e18c5d16d69 HTTP/1.1" 200 None 2026-03-30 12:54:16,122 - lava_callback - ERROR - Error processing callback for node 69c7cf6abe430e18c5d16d69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46638 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:55:36,066 - lava_callback - INFO - Received callback for node 69c7cfb2be430e18c5d16d8e INFO: 172.21.0.1:56624 - "POST /node/69c7cfb2be430e18c5d16d8e HTTP/1.0" 202 Accepted 2026-03-30 12:55:36,072 - lava_callback - INFO - Starting async job submit for node 69c7cfb2be430e18c5d16d8e 2026-03-30 12:55:36,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:36,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cfb2be430e18c5d16d8e HTTP/1.1" 200 None 2026-03-30 12:55:36,105 - lava_callback - DEBUG - Processing node 69c7cfb2be430e18c5d16d8e: baseline-arm-clabbe 2026-03-30 12:55:36,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7cfb2be430e18c5d16d8e 2026-03-30 12:55:36,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:36,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cfb2be430e18c5d16d8e HTTP/1.1" 200 None 2026-03-30 12:55:36,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:36,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 12:55:36,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:36,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7cfb2be430e18c5d16d8e HTTP/1.1" 200 None 2026-03-30 12:55:36,214 - lava_callback - ERROR - Error processing callback for node 69c7cfb2be430e18c5d16d8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41838 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:55:56,031 - lava_callback - INFO - Received callback for node 69c7cfd8be430e18c5d16d9a INFO: 172.21.0.1:33028 - "POST /node/69c7cfd8be430e18c5d16d9a HTTP/1.0" 202 Accepted 2026-03-30 12:55:56,037 - lava_callback - INFO - Starting async job submit for node 69c7cfd8be430e18c5d16d9a 2026-03-30 12:55:56,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:56,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cfd8be430e18c5d16d9a HTTP/1.1" 200 None 2026-03-30 12:55:56,053 - lava_callback - DEBUG - Processing node 69c7cfd8be430e18c5d16d9a: baseline-arm-clabbe 2026-03-30 12:55:56,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7cfd8be430e18c5d16d9a 2026-03-30 12:55:56,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:56,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cfd8be430e18c5d16d9a HTTP/1.1" 200 None 2026-03-30 12:55:56,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:56,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 12:55:56,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:55:56,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7cfd8be430e18c5d16d9a HTTP/1.1" 200 None 2026-03-30 12:55:56,106 - lava_callback - ERROR - Error processing callback for node 69c7cfd8be430e18c5d16d9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET /node/698f5cd63013abef95cf8c01%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:47508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59904 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 12:57:16,176 - lava_callback - INFO - Received callback for node 69c7d020be430e18c5d16dba INFO: 172.21.0.1:59908 - "POST /node/69c7d020be430e18c5d16dba HTTP/1.0" 202 Accepted 2026-03-30 12:57:16,182 - lava_callback - INFO - Starting async job submit for node 69c7d020be430e18c5d16dba 2026-03-30 12:57:16,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:16,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d020be430e18c5d16dba HTTP/1.1" 200 None 2026-03-30 12:57:16,218 - lava_callback - DEBUG - Processing node 69c7d020be430e18c5d16dba: baseline-arm-clabbe 2026-03-30 12:57:16,219 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d020be430e18c5d16dba 2026-03-30 12:57:16,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:16,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d020be430e18c5d16dba HTTP/1.1" 200 None 2026-03-30 12:57:16,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:16,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 12:57:16,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:16,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d020be430e18c5d16dba HTTP/1.1" 200 None 2026-03-30 12:57:16,315 - lava_callback - ERROR - Error processing callback for node 69c7d020be430e18c5d16dba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45694 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:57:56,120 - lava_callback - INFO - Received callback for node 69c7d044be430e18c5d16dd4 INFO: 172.21.0.1:52412 - "POST /node/69c7d044be430e18c5d16dd4 HTTP/1.0" 202 Accepted 2026-03-30 12:57:56,126 - lava_callback - INFO - Starting async job submit for node 69c7d044be430e18c5d16dd4 2026-03-30 12:57:56,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:56,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d044be430e18c5d16dd4 HTTP/1.1" 200 None 2026-03-30 12:57:56,142 - lava_callback - DEBUG - Processing node 69c7d044be430e18c5d16dd4: baseline-arm-clabbe 2026-03-30 12:57:56,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d044be430e18c5d16dd4 2026-03-30 12:57:56,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:56,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d044be430e18c5d16dd4 HTTP/1.1" 200 None 2026-03-30 12:57:56,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:56,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-30 12:57:56,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:57:56,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d044be430e18c5d16dd4 HTTP/1.1" 200 None 2026-03-30 12:57:56,210 - lava_callback - ERROR - Error processing callback for node 69c7d044be430e18c5d16dd4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47762 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:58:36,196 - lava_callback - INFO - Received callback for node 69c7d068be430e18c5d16dde INFO: 172.21.0.1:40070 - "POST /node/69c7d068be430e18c5d16dde HTTP/1.0" 202 Accepted 2026-03-30 12:58:36,202 - lava_callback - INFO - Starting async job submit for node 69c7d068be430e18c5d16dde 2026-03-30 12:58:36,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:36,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d068be430e18c5d16dde HTTP/1.1" 200 None 2026-03-30 12:58:36,225 - lava_callback - DEBUG - Processing node 69c7d068be430e18c5d16dde: baseline-arm-clabbe 2026-03-30 12:58:36,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d068be430e18c5d16dde 2026-03-30 12:58:36,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:36,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d068be430e18c5d16dde HTTP/1.1" 200 None 2026-03-30 12:58:36,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:36,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-30 12:58:36,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:36,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d068be430e18c5d16dde HTTP/1.1" 200 None 2026-03-30 12:58:36,282 - lava_callback - ERROR - Error processing callback for node 69c7d068be430e18c5d16dde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49350 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:58:56,160 - lava_callback - INFO - Received callback for node 69c7d08dbe430e18c5d16dec INFO: 172.21.0.1:36568 - "POST /node/69c7d08dbe430e18c5d16dec HTTP/1.0" 202 Accepted 2026-03-30 12:58:56,165 - lava_callback - INFO - Starting async job submit for node 69c7d08dbe430e18c5d16dec 2026-03-30 12:58:56,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:56,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d08dbe430e18c5d16dec HTTP/1.1" 200 None 2026-03-30 12:58:56,182 - lava_callback - DEBUG - Processing node 69c7d08dbe430e18c5d16dec: baseline-arm-clabbe 2026-03-30 12:58:56,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d08dbe430e18c5d16dec 2026-03-30 12:58:56,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:56,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d08dbe430e18c5d16dec HTTP/1.1" 200 None 2026-03-30 12:58:56,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:56,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-30 12:58:56,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:58:56,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d08dbe430e18c5d16dec HTTP/1.1" 200 None 2026-03-30 12:58:56,249 - lava_callback - ERROR - Error processing callback for node 69c7d08dbe430e18c5d16dec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54418 - "GET / HTTP/1.0" 200 OK 2026-03-30 12:59:36,258 - lava_callback - INFO - Received callback for node 69c7d0b2be430e18c5d16dfa INFO: 172.21.0.1:43316 - "POST /node/69c7d0b2be430e18c5d16dfa HTTP/1.0" 202 Accepted 2026-03-30 12:59:36,264 - lava_callback - INFO - Starting async job submit for node 69c7d0b2be430e18c5d16dfa 2026-03-30 12:59:36,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:59:36,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d0b2be430e18c5d16dfa HTTP/1.1" 200 None 2026-03-30 12:59:36,279 - lava_callback - DEBUG - Processing node 69c7d0b2be430e18c5d16dfa: baseline-arm-clabbe 2026-03-30 12:59:36,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d0b2be430e18c5d16dfa 2026-03-30 12:59:36,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:59:36,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d0b2be430e18c5d16dfa HTTP/1.1" 200 None 2026-03-30 12:59:36,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:59:36,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-30 12:59:36,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 12:59:36,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d0b2be430e18c5d16dfa HTTP/1.1" 200 None 2026-03-30 12:59:36,339 - lava_callback - ERROR - Error processing callback for node 69c7d0b2be430e18c5d16dfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53866 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-30 13:00:16,399 - lava_callback - INFO - Received callback for node 69c7d0d6be430e18c5d16e0e INFO: 172.21.0.1:53874 - "POST /node/69c7d0d6be430e18c5d16e0e HTTP/1.0" 202 Accepted 2026-03-30 13:00:16,404 - lava_callback - INFO - Starting async job submit for node 69c7d0d6be430e18c5d16e0e 2026-03-30 13:00:16,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:16,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d0d6be430e18c5d16e0e HTTP/1.1" 200 None 2026-03-30 13:00:16,421 - lava_callback - DEBUG - Processing node 69c7d0d6be430e18c5d16e0e: baseline-arm-clabbe 2026-03-30 13:00:16,421 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d0d6be430e18c5d16e0e 2026-03-30 13:00:16,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:16,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d0d6be430e18c5d16e0e HTTP/1.1" 200 None 2026-03-30 13:00:16,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:16,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-30 13:00:16,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:16,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d0d6be430e18c5d16e0e HTTP/1.1" 200 None 2026-03-30 13:00:16,477 - lava_callback - ERROR - Error processing callback for node 69c7d0d6be430e18c5d16e0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60034 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:00:56,277 - lava_callback - INFO - Received callback for node 69c7d0fbbe430e18c5d16e25 INFO: 172.21.0.1:37814 - "POST /node/69c7d0fbbe430e18c5d16e25 HTTP/1.0" 202 Accepted 2026-03-30 13:00:56,283 - lava_callback - INFO - Starting async job submit for node 69c7d0fbbe430e18c5d16e25 2026-03-30 13:00:56,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:56,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d0fbbe430e18c5d16e25 HTTP/1.1" 200 None 2026-03-30 13:00:56,307 - lava_callback - DEBUG - Processing node 69c7d0fbbe430e18c5d16e25: baseline-arm-clabbe 2026-03-30 13:00:56,307 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d0fbbe430e18c5d16e25 2026-03-30 13:00:56,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:56,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d0fbbe430e18c5d16e25 HTTP/1.1" 200 None 2026-03-30 13:00:56,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:56,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:00:56,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:00:56,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d0fbbe430e18c5d16e25 HTTP/1.1" 200 None 2026-03-30 13:00:56,367 - lava_callback - ERROR - Error processing callback for node 69c7d0fbbe430e18c5d16e25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41040 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:01:36,266 - lava_callback - INFO - Received callback for node 69c7d120be430e18c5d16e2d INFO: 172.21.0.1:59034 - "POST /node/69c7d120be430e18c5d16e2d HTTP/1.0" 202 Accepted 2026-03-30 13:01:36,271 - lava_callback - INFO - Starting async job submit for node 69c7d120be430e18c5d16e2d 2026-03-30 13:01:36,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:01:36,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d120be430e18c5d16e2d HTTP/1.1" 200 None 2026-03-30 13:01:36,288 - lava_callback - DEBUG - Processing node 69c7d120be430e18c5d16e2d: baseline-arm-clabbe 2026-03-30 13:01:36,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d120be430e18c5d16e2d 2026-03-30 13:01:36,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:01:36,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d120be430e18c5d16e2d HTTP/1.1" 200 None 2026-03-30 13:01:36,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:01:36,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:01:36,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:01:36,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d120be430e18c5d16e2d HTTP/1.1" 200 None 2026-03-30 13:01:36,363 - lava_callback - ERROR - Error processing callback for node 69c7d120be430e18c5d16e2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50072 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:02:56,367 - lava_callback - INFO - Received callback for node 69c7d16fbe430e18c5d16e41 INFO: 172.21.0.1:34940 - "POST /node/69c7d16fbe430e18c5d16e41 HTTP/1.0" 202 Accepted 2026-03-30 13:02:56,372 - lava_callback - INFO - Starting async job submit for node 69c7d16fbe430e18c5d16e41 2026-03-30 13:02:56,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:02:56,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d16fbe430e18c5d16e41 HTTP/1.1" 200 None 2026-03-30 13:02:56,394 - lava_callback - DEBUG - Processing node 69c7d16fbe430e18c5d16e41: baseline-arm-clabbe 2026-03-30 13:02:56,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d16fbe430e18c5d16e41 2026-03-30 13:02:56,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:02:56,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d16fbe430e18c5d16e41 HTTP/1.1" 200 None 2026-03-30 13:02:56,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:02:56,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:02:56,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:02:56,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d16fbe430e18c5d16e41 HTTP/1.1" 200 None 2026-03-30 13:02:56,446 - lava_callback - ERROR - Error processing callback for node 69c7d16fbe430e18c5d16e41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:03:56,401 - lava_callback - INFO - Received callback for node 69c7d1abbe430e18c5d16e51 INFO: 172.21.0.1:56554 - "POST /node/69c7d1abbe430e18c5d16e51 HTTP/1.0" 202 Accepted 2026-03-30 13:03:56,406 - lava_callback - INFO - Starting async job submit for node 69c7d1abbe430e18c5d16e51 2026-03-30 13:03:56,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:03:56,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d1abbe430e18c5d16e51 HTTP/1.1" 200 None 2026-03-30 13:03:56,423 - lava_callback - DEBUG - Processing node 69c7d1abbe430e18c5d16e51: baseline-arm-clabbe 2026-03-30 13:03:56,424 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d1abbe430e18c5d16e51 2026-03-30 13:03:56,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:03:56,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d1abbe430e18c5d16e51 HTTP/1.1" 200 None 2026-03-30 13:03:56,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:03:56,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:03:56,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:03:56,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d1abbe430e18c5d16e51 HTTP/1.1" 200 None 2026-03-30 13:03:56,488 - lava_callback - ERROR - Error processing callback for node 69c7d1abbe430e18c5d16e51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34606 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:04:56,441 - lava_callback - INFO - Received callback for node 69c7d1e7be430e18c5d16e63 INFO: 172.21.0.1:60904 - "POST /node/69c7d1e7be430e18c5d16e63 HTTP/1.0" 202 Accepted 2026-03-30 13:04:56,447 - lava_callback - INFO - Starting async job submit for node 69c7d1e7be430e18c5d16e63 2026-03-30 13:04:56,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:04:56,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d1e7be430e18c5d16e63 HTTP/1.1" 200 None 2026-03-30 13:04:56,463 - lava_callback - DEBUG - Processing node 69c7d1e7be430e18c5d16e63: baseline-arm-clabbe 2026-03-30 13:04:56,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d1e7be430e18c5d16e63 2026-03-30 13:04:56,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:04:56,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d1e7be430e18c5d16e63 HTTP/1.1" 200 None 2026-03-30 13:04:56,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:04:56,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:04:56,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:04:56,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d1e7be430e18c5d16e63 HTTP/1.1" 200 None 2026-03-30 13:04:56,523 - lava_callback - ERROR - Error processing callback for node 69c7d1e7be430e18c5d16e63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41908 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:05:56,501 - lava_callback - INFO - Received callback for node 69c7d224be430e18c5d16e76 INFO: 172.21.0.1:45808 - "POST /node/69c7d224be430e18c5d16e76 HTTP/1.0" 202 Accepted 2026-03-30 13:05:56,507 - lava_callback - INFO - Starting async job submit for node 69c7d224be430e18c5d16e76 2026-03-30 13:05:56,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:05:56,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d224be430e18c5d16e76 HTTP/1.1" 200 None 2026-03-30 13:05:56,529 - lava_callback - DEBUG - Processing node 69c7d224be430e18c5d16e76: baseline-arm-clabbe 2026-03-30 13:05:56,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d224be430e18c5d16e76 2026-03-30 13:05:56,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:05:56,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d224be430e18c5d16e76 HTTP/1.1" 200 None 2026-03-30 13:05:56,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:05:56,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:05:56,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:05:56,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d224be430e18c5d16e76 HTTP/1.1" 200 None 2026-03-30 13:05:56,591 - lava_callback - ERROR - Error processing callback for node 69c7d224be430e18c5d16e76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40062 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:06:36,504 - lava_callback - INFO - Received callback for node 69c7d248be430e18c5d16e85 INFO: 172.21.0.1:33446 - "POST /node/69c7d248be430e18c5d16e85 HTTP/1.0" 202 Accepted 2026-03-30 13:06:36,510 - lava_callback - INFO - Starting async job submit for node 69c7d248be430e18c5d16e85 2026-03-30 13:06:36,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:33454 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:06:36,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d248be430e18c5d16e85 HTTP/1.1" 200 None 2026-03-30 13:06:36,530 - lava_callback - DEBUG - Processing node 69c7d248be430e18c5d16e85: baseline-arm-clabbe 2026-03-30 13:06:36,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d248be430e18c5d16e85 2026-03-30 13:06:36,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:06:36,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d248be430e18c5d16e85 HTTP/1.1" 200 None 2026-03-30 13:06:36,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:06:36,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:06:36,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:06:36,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d248be430e18c5d16e85 HTTP/1.1" 200 None 2026-03-30 13:06:36,587 - lava_callback - ERROR - Error processing callback for node 69c7d248be430e18c5d16e85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:06:56,519 - lava_callback - INFO - Received callback for node 69c7d26dbe430e18c5d16e93 INFO: 172.21.0.1:50450 - "POST /node/69c7d26dbe430e18c5d16e93 HTTP/1.0" 202 Accepted 2026-03-30 13:06:56,524 - lava_callback - INFO - Starting async job submit for node 69c7d26dbe430e18c5d16e93 2026-03-30 13:06:56,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:06:56,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d26dbe430e18c5d16e93 HTTP/1.1" 200 None 2026-03-30 13:06:56,549 - lava_callback - DEBUG - Processing node 69c7d26dbe430e18c5d16e93: baseline-arm-clabbe 2026-03-30 13:06:56,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d26dbe430e18c5d16e93 2026-03-30 13:06:56,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:06:56,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d26dbe430e18c5d16e93 HTTP/1.1" 200 None 2026-03-30 13:06:56,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:06:56,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:06:56,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:06:56,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d26dbe430e18c5d16e93 HTTP/1.1" 200 None 2026-03-30 13:06:56,621 - lava_callback - ERROR - Error processing callback for node 69c7d26dbe430e18c5d16e93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47720 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:07:56,573 - lava_callback - INFO - Received callback for node 69c7d29bbe430e18c5d16ea6 INFO: 172.21.0.1:47722 - "POST /node/69c7d29bbe430e18c5d16ea6 HTTP/1.0" 202 Accepted 2026-03-30 13:07:56,578 - lava_callback - INFO - Starting async job submit for node 69c7d29bbe430e18c5d16ea6 2026-03-30 13:07:56,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:07:56,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d29bbe430e18c5d16ea6 HTTP/1.1" 200 None 2026-03-30 13:07:56,613 - lava_callback - DEBUG - Processing node 69c7d29bbe430e18c5d16ea6: baseline-arm-clabbe 2026-03-30 13:07:56,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d29bbe430e18c5d16ea6 2026-03-30 13:07:56,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:07:56,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d29bbe430e18c5d16ea6 HTTP/1.1" 200 None 2026-03-30 13:07:56,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:07:56,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:07:56,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:07:56,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d29bbe430e18c5d16ea6 HTTP/1.1" 200 None 2026-03-30 13:07:56,669 - lava_callback - ERROR - Error processing callback for node 69c7d29bbe430e18c5d16ea6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:08:36,547 - lava_callback - INFO - Received callback for node 69c7d2c1be430e18c5d16eb5 INFO: 172.21.0.1:39012 - "POST /node/69c7d2c1be430e18c5d16eb5 HTTP/1.0" 202 Accepted 2026-03-30 13:08:36,553 - lava_callback - INFO - Starting async job submit for node 69c7d2c1be430e18c5d16eb5 2026-03-30 13:08:36,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:08:36,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d2c1be430e18c5d16eb5 HTTP/1.1" 200 None 2026-03-30 13:08:36,569 - lava_callback - DEBUG - Processing node 69c7d2c1be430e18c5d16eb5: baseline-arm-clabbe 2026-03-30 13:08:36,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d2c1be430e18c5d16eb5 2026-03-30 13:08:36,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:08:36,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d2c1be430e18c5d16eb5 HTTP/1.1" 200 None 2026-03-30 13:08:36,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:08:36,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-30 13:08:36,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:08:36,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d2c1be430e18c5d16eb5 HTTP/1.1" 200 None 2026-03-30 13:08:36,623 - lava_callback - ERROR - Error processing callback for node 69c7d2c1be430e18c5d16eb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40496 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:31:04,079 - lava_callback - INFO - Received callback for node 69c7d807be430e18c5d16f26 INFO: 172.21.0.1:54086 - "POST /node/69c7d807be430e18c5d16f26 HTTP/1.0" 202 Accepted 2026-03-30 13:31:04,085 - lava_callback - INFO - Starting async job submit for node 69c7d807be430e18c5d16f26 2026-03-30 13:31:04,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:31:04,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d807be430e18c5d16f26 HTTP/1.1" 200 None 2026-03-30 13:31:04,102 - lava_callback - DEBUG - Processing node 69c7d807be430e18c5d16f26: baseline-arm64-baylibre 2026-03-30 13:31:04,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d807be430e18c5d16f26 2026-03-30 13:31:04,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:31:04,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d807be430e18c5d16f26 HTTP/1.1" 200 None 2026-03-30 13:31:04,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:31:04,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:31:04,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:31:04,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d807be430e18c5d16f26 HTTP/1.1" 200 None 2026-03-30 13:31:04,163 - lava_callback - ERROR - Error processing callback for node 69c7d807be430e18c5d16f26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35174 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:32:24,217 - lava_callback - INFO - Received callback for node 69c7d855be430e18c5d16f2e INFO: 172.21.0.1:57786 - "POST /node/69c7d855be430e18c5d16f2e HTTP/1.0" 202 Accepted 2026-03-30 13:32:24,223 - lava_callback - INFO - Starting async job submit for node 69c7d855be430e18c5d16f2e 2026-03-30 13:32:24,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:32:24,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d855be430e18c5d16f2e HTTP/1.1" 200 None 2026-03-30 13:32:24,238 - lava_callback - DEBUG - Processing node 69c7d855be430e18c5d16f2e: baseline-arm64-baylibre 2026-03-30 13:32:24,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d855be430e18c5d16f2e 2026-03-30 13:32:24,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:32:24,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d855be430e18c5d16f2e HTTP/1.1" 200 None 2026-03-30 13:32:24,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:32:24,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:32:24,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:32:24,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d855be430e18c5d16f2e HTTP/1.1" 200 None 2026-03-30 13:32:24,293 - lava_callback - ERROR - Error processing callback for node 69c7d855be430e18c5d16f2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47164 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:33:24,129 - lava_callback - INFO - Received callback for node 69c7d8a1be430e18c5d16f3d INFO: 172.21.0.1:45738 - "POST /node/69c7d8a1be430e18c5d16f3d HTTP/1.0" 202 Accepted 2026-03-30 13:33:24,134 - lava_callback - INFO - Starting async job submit for node 69c7d8a1be430e18c5d16f3d 2026-03-30 13:33:24,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:33:24,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d8a1be430e18c5d16f3d HTTP/1.1" 200 None 2026-03-30 13:33:24,150 - lava_callback - DEBUG - Processing node 69c7d8a1be430e18c5d16f3d: baseline-arm64-baylibre 2026-03-30 13:33:24,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d8a1be430e18c5d16f3d 2026-03-30 13:33:24,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:33:24,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d8a1be430e18c5d16f3d HTTP/1.1" 200 None 2026-03-30 13:33:24,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:33:24,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:33:24,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:33:24,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d8a1be430e18c5d16f3d HTTP/1.1" 200 None 2026-03-30 13:33:24,204 - lava_callback - ERROR - Error processing callback for node 69c7d8a1be430e18c5d16f3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52018 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:35:04,231 - lava_callback - INFO - Received callback for node 69c7d8fabe430e18c5d16f4a INFO: 172.21.0.1:40768 - "POST /node/69c7d8fabe430e18c5d16f4a HTTP/1.0" 202 Accepted 2026-03-30 13:35:04,237 - lava_callback - INFO - Starting async job submit for node 69c7d8fabe430e18c5d16f4a 2026-03-30 13:35:04,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:35:04,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d8fabe430e18c5d16f4a HTTP/1.1" 200 None 2026-03-30 13:35:04,256 - lava_callback - DEBUG - Processing node 69c7d8fabe430e18c5d16f4a: baseline-arm64-baylibre 2026-03-30 13:35:04,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d8fabe430e18c5d16f4a 2026-03-30 13:35:04,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:35:04,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d8fabe430e18c5d16f4a HTTP/1.1" 200 None 2026-03-30 13:35:04,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:35:04,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:35:04,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:35:04,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d8fabe430e18c5d16f4a HTTP/1.1" 200 None 2026-03-30 13:35:04,485 - lava_callback - ERROR - Error processing callback for node 69c7d8fabe430e18c5d16f4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48600 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:36:24,426 - lava_callback - INFO - Received callback for node 69c7d947be430e18c5d16f52 INFO: 172.21.0.1:57592 - "POST /node/69c7d947be430e18c5d16f52 HTTP/1.0" 202 Accepted 2026-03-30 13:36:24,432 - lava_callback - INFO - Starting async job submit for node 69c7d947be430e18c5d16f52 2026-03-30 13:36:24,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:36:24,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d947be430e18c5d16f52 HTTP/1.1" 200 None 2026-03-30 13:36:24,447 - lava_callback - DEBUG - Processing node 69c7d947be430e18c5d16f52: baseline-arm64-baylibre 2026-03-30 13:36:24,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d947be430e18c5d16f52 2026-03-30 13:36:24,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:36:24,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d947be430e18c5d16f52 HTTP/1.1" 200 None 2026-03-30 13:36:24,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:36:24,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:36:24,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:36:24,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d947be430e18c5d16f52 HTTP/1.1" 200 None 2026-03-30 13:36:24,519 - lava_callback - ERROR - Error processing callback for node 69c7d947be430e18c5d16f52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:37:44,398 - lava_callback - INFO - Received callback for node 69c7d993be430e18c5d16f64 INFO: 172.21.0.1:53652 - "POST /node/69c7d993be430e18c5d16f64 HTTP/1.0" 202 Accepted 2026-03-30 13:37:44,404 - lava_callback - INFO - Starting async job submit for node 69c7d993be430e18c5d16f64 2026-03-30 13:37:44,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:37:44,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d993be430e18c5d16f64 HTTP/1.1" 200 None 2026-03-30 13:37:44,420 - lava_callback - DEBUG - Processing node 69c7d993be430e18c5d16f64: baseline-arm64-baylibre 2026-03-30 13:37:44,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d993be430e18c5d16f64 2026-03-30 13:37:44,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:37:44,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d993be430e18c5d16f64 HTTP/1.1" 200 None 2026-03-30 13:37:44,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:37:44,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:37:44,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:37:44,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d993be430e18c5d16f64 HTTP/1.1" 200 None 2026-03-30 13:37:44,492 - lava_callback - ERROR - Error processing callback for node 69c7d993be430e18c5d16f64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52106 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:38:44,437 - lava_callback - INFO - Received callback for node 69c7d9dfbe430e18c5d16f6a INFO: 172.21.0.1:43714 - "POST /node/69c7d9dfbe430e18c5d16f6a HTTP/1.0" 202 Accepted 2026-03-30 13:38:44,443 - lava_callback - INFO - Starting async job submit for node 69c7d9dfbe430e18c5d16f6a 2026-03-30 13:38:44,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:38:44,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d9dfbe430e18c5d16f6a HTTP/1.1" 200 None 2026-03-30 13:38:44,459 - lava_callback - DEBUG - Processing node 69c7d9dfbe430e18c5d16f6a: baseline-arm64-baylibre 2026-03-30 13:38:44,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7d9dfbe430e18c5d16f6a 2026-03-30 13:38:44,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:38:44,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7d9dfbe430e18c5d16f6a HTTP/1.1" 200 None 2026-03-30 13:38:44,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:38:44,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:38:44,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:38:44,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7d9dfbe430e18c5d16f6a HTTP/1.1" 200 None 2026-03-30 13:38:44,517 - lava_callback - ERROR - Error processing callback for node 69c7d9dfbe430e18c5d16f6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35198 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:40:18,070 - lava_callback - INFO - Received callback for node 69c7da31be430e18c5d16f76 INFO: 172.21.0.1:35202 - "POST /node/69c7da31be430e18c5d16f76 HTTP/1.0" 202 Accepted 2026-03-30 13:40:18,075 - lava_callback - INFO - Starting async job submit for node 69c7da31be430e18c5d16f76 2026-03-30 13:40:18,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:40:18,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7da31be430e18c5d16f76 HTTP/1.1" 200 None 2026-03-30 13:40:18,092 - lava_callback - DEBUG - Processing node 69c7da31be430e18c5d16f76: baseline-arm64-clabbe 2026-03-30 13:40:18,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7da31be430e18c5d16f76 2026-03-30 13:40:18,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:40:18,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7da31be430e18c5d16f76 HTTP/1.1" 200 None 2026-03-30 13:40:18,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:40:18,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:40:18,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:40:18,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7da31be430e18c5d16f76 HTTP/1.1" 200 None 2026-03-30 13:40:18,165 - lava_callback - ERROR - Error processing callback for node 69c7da31be430e18c5d16f76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:41:38,095 - lava_callback - INFO - Received callback for node 69c7da7ebe430e18c5d16f83 INFO: 172.21.0.1:38036 - "POST /node/69c7da7ebe430e18c5d16f83 HTTP/1.0" 202 Accepted 2026-03-30 13:41:38,101 - lava_callback - INFO - Starting async job submit for node 69c7da7ebe430e18c5d16f83 2026-03-30 13:41:38,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:41:38,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7da7ebe430e18c5d16f83 HTTP/1.1" 200 None 2026-03-30 13:41:38,117 - lava_callback - DEBUG - Processing node 69c7da7ebe430e18c5d16f83: baseline-arm64-clabbe 2026-03-30 13:41:38,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7da7ebe430e18c5d16f83 2026-03-30 13:41:38,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:41:38,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7da7ebe430e18c5d16f83 HTTP/1.1" 200 None 2026-03-30 13:41:38,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:41:38,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:41:38,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:41:38,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7da7ebe430e18c5d16f83 HTTP/1.1" 200 None 2026-03-30 13:41:38,176 - lava_callback - ERROR - Error processing callback for node 69c7da7ebe430e18c5d16f83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:42:58,142 - lava_callback - INFO - Received callback for node 69c7dacdbe430e18c5d16f8f INFO: 172.21.0.1:48784 - "POST /node/69c7dacdbe430e18c5d16f8f HTTP/1.0" 202 Accepted 2026-03-30 13:42:58,147 - lava_callback - INFO - Starting async job submit for node 69c7dacdbe430e18c5d16f8f 2026-03-30 13:42:58,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:42:58,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dacdbe430e18c5d16f8f HTTP/1.1" 200 None 2026-03-30 13:42:58,167 - lava_callback - DEBUG - Processing node 69c7dacdbe430e18c5d16f8f: baseline-arm64-clabbe 2026-03-30 13:42:58,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dacdbe430e18c5d16f8f 2026-03-30 13:42:58,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:42:58,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dacdbe430e18c5d16f8f HTTP/1.1" 200 None 2026-03-30 13:42:58,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:42:58,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:42:58,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:42:58,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dacdbe430e18c5d16f8f HTTP/1.1" 200 None 2026-03-30 13:42:58,219 - lava_callback - ERROR - Error processing callback for node 69c7dacdbe430e18c5d16f8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47814 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:43:58,161 - lava_callback - INFO - Received callback for node 69c7db1bbe430e18c5d16f9c INFO: 172.21.0.1:47826 - "POST /node/69c7db1bbe430e18c5d16f9c HTTP/1.0" 202 Accepted 2026-03-30 13:43:58,166 - lava_callback - INFO - Starting async job submit for node 69c7db1bbe430e18c5d16f9c 2026-03-30 13:43:58,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:43:58,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7db1bbe430e18c5d16f9c HTTP/1.1" 200 None 2026-03-30 13:43:58,186 - lava_callback - DEBUG - Processing node 69c7db1bbe430e18c5d16f9c: baseline-arm64-clabbe 2026-03-30 13:43:58,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7db1bbe430e18c5d16f9c 2026-03-30 13:43:58,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:43:58,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7db1bbe430e18c5d16f9c HTTP/1.1" 200 None 2026-03-30 13:43:58,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:43:58,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:43:58,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:43:58,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7db1bbe430e18c5d16f9c HTTP/1.1" 200 None 2026-03-30 13:43:58,242 - lava_callback - ERROR - Error processing callback for node 69c7db1bbe430e18c5d16f9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38754 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:45:18,245 - lava_callback - INFO - Received callback for node 69c7db67be430e18c5d16fa4 INFO: 172.21.0.1:38768 - "POST /node/69c7db67be430e18c5d16fa4 HTTP/1.0" 202 Accepted 2026-03-30 13:45:18,250 - lava_callback - INFO - Starting async job submit for node 69c7db67be430e18c5d16fa4 2026-03-30 13:45:18,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:45:18,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7db67be430e18c5d16fa4 HTTP/1.1" 200 None 2026-03-30 13:45:18,268 - lava_callback - DEBUG - Processing node 69c7db67be430e18c5d16fa4: baseline-arm64-clabbe 2026-03-30 13:45:18,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7db67be430e18c5d16fa4 2026-03-30 13:45:18,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:45:18,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7db67be430e18c5d16fa4 HTTP/1.1" 200 None 2026-03-30 13:45:18,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:45:18,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:45:18,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:45:18,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7db67be430e18c5d16fa4 HTTP/1.1" 200 None 2026-03-30 13:45:18,333 - lava_callback - ERROR - Error processing callback for node 69c7db67be430e18c5d16fa4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:46:38,286 - lava_callback - INFO - Received callback for node 69c7dbb3be430e18c5d16fb2 INFO: 172.21.0.1:58266 - "POST /node/69c7dbb3be430e18c5d16fb2 HTTP/1.0" 202 Accepted 2026-03-30 13:46:38,292 - lava_callback - INFO - Starting async job submit for node 69c7dbb3be430e18c5d16fb2 2026-03-30 13:46:38,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:46:38,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dbb3be430e18c5d16fb2 HTTP/1.1" 200 None 2026-03-30 13:46:38,308 - lava_callback - DEBUG - Processing node 69c7dbb3be430e18c5d16fb2: baseline-arm64-clabbe 2026-03-30 13:46:38,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dbb3be430e18c5d16fb2 2026-03-30 13:46:38,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:46:38,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dbb3be430e18c5d16fb2 HTTP/1.1" 200 None 2026-03-30 13:46:38,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:46:38,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:46:38,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:46:38,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dbb3be430e18c5d16fb2 HTTP/1.1" 200 None 2026-03-30 13:46:38,369 - lava_callback - ERROR - Error processing callback for node 69c7dbb3be430e18c5d16fb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:47:58,481 - lava_callback - INFO - Received callback for node 69c7dbffbe430e18c5d16fc3 INFO: 172.21.0.1:50590 - "POST /node/69c7dbffbe430e18c5d16fc3 HTTP/1.0" 202 Accepted 2026-03-30 13:47:58,487 - lava_callback - INFO - Starting async job submit for node 69c7dbffbe430e18c5d16fc3 2026-03-30 13:47:58,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:47:58,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dbffbe430e18c5d16fc3 HTTP/1.1" 200 None 2026-03-30 13:47:58,502 - lava_callback - DEBUG - Processing node 69c7dbffbe430e18c5d16fc3: baseline-arm64-clabbe 2026-03-30 13:47:58,502 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dbffbe430e18c5d16fc3 2026-03-30 13:47:58,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:47:58,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dbffbe430e18c5d16fc3 HTTP/1.1" 200 None 2026-03-30 13:47:58,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:47:58,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:47:58,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:47:58,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dbffbe430e18c5d16fc3 HTTP/1.1" 200 None 2026-03-30 13:47:58,560 - lava_callback - ERROR - Error processing callback for node 69c7dbffbe430e18c5d16fc3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56550 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:49:18,425 - lava_callback - INFO - Received callback for node 69c7dc4bbe430e18c5d16fce INFO: 172.21.0.1:56552 - "POST /node/69c7dc4bbe430e18c5d16fce HTTP/1.0" 202 Accepted 2026-03-30 13:49:18,431 - lava_callback - INFO - Starting async job submit for node 69c7dc4bbe430e18c5d16fce 2026-03-30 13:49:18,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:49:18,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dc4bbe430e18c5d16fce HTTP/1.1" 200 None 2026-03-30 13:49:18,447 - lava_callback - DEBUG - Processing node 69c7dc4bbe430e18c5d16fce: baseline-arm64-clabbe 2026-03-30 13:49:18,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dc4bbe430e18c5d16fce 2026-03-30 13:49:18,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:49:18,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dc4bbe430e18c5d16fce HTTP/1.1" 200 None 2026-03-30 13:49:18,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:49:18,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:49:18,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:49:18,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dc4bbe430e18c5d16fce HTTP/1.1" 200 None 2026-03-30 13:49:18,514 - lava_callback - ERROR - Error processing callback for node 69c7dc4bbe430e18c5d16fce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60716 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:50:38,453 - lava_callback - INFO - Received callback for node 69c7dc98be430e18c5d16fd8 INFO: 172.21.0.1:60730 - "POST /node/69c7dc98be430e18c5d16fd8 HTTP/1.0" 202 Accepted 2026-03-30 13:50:38,459 - lava_callback - INFO - Starting async job submit for node 69c7dc98be430e18c5d16fd8 2026-03-30 13:50:38,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:50:38,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dc98be430e18c5d16fd8 HTTP/1.1" 200 None 2026-03-30 13:50:38,474 - lava_callback - DEBUG - Processing node 69c7dc98be430e18c5d16fd8: baseline-arm64-clabbe 2026-03-30 13:50:38,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dc98be430e18c5d16fd8 2026-03-30 13:50:38,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:50:38,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dc98be430e18c5d16fd8 HTTP/1.1" 200 None 2026-03-30 13:50:38,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:50:38,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:50:38,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:50:38,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dc98be430e18c5d16fd8 HTTP/1.1" 200 None 2026-03-30 13:50:38,542 - lava_callback - ERROR - Error processing callback for node 69c7dc98be430e18c5d16fd8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:51:38,507 - lava_callback - INFO - Received callback for node 69c7dce5be430e18c5d16fe1 INFO: 172.21.0.1:39424 - "POST /node/69c7dce5be430e18c5d16fe1 HTTP/1.0" 202 Accepted 2026-03-30 13:51:38,513 - lava_callback - INFO - Starting async job submit for node 69c7dce5be430e18c5d16fe1 2026-03-30 13:51:38,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:51:38,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dce5be430e18c5d16fe1 HTTP/1.1" 200 None 2026-03-30 13:51:38,528 - lava_callback - DEBUG - Processing node 69c7dce5be430e18c5d16fe1: baseline-arm64-clabbe 2026-03-30 13:51:38,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dce5be430e18c5d16fe1 2026-03-30 13:51:38,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:51:38,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dce5be430e18c5d16fe1 HTTP/1.1" 200 None 2026-03-30 13:51:38,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:51:38,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:51:38,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:51:38,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dce5be430e18c5d16fe1 HTTP/1.1" 200 None 2026-03-30 13:51:38,611 - lava_callback - ERROR - Error processing callback for node 69c7dce5be430e18c5d16fe1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:52:58,536 - lava_callback - INFO - Received callback for node 69c7dd31be430e18c5d16ff1 INFO: 172.21.0.1:50638 - "POST /node/69c7dd31be430e18c5d16ff1 HTTP/1.0" 202 Accepted 2026-03-30 13:52:58,541 - lava_callback - INFO - Starting async job submit for node 69c7dd31be430e18c5d16ff1 2026-03-30 13:52:58,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:52:58,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dd31be430e18c5d16ff1 HTTP/1.1" 200 None 2026-03-30 13:52:58,563 - lava_callback - DEBUG - Processing node 69c7dd31be430e18c5d16ff1: baseline-arm64-clabbe 2026-03-30 13:52:58,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dd31be430e18c5d16ff1 2026-03-30 13:52:58,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:52:58,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dd31be430e18c5d16ff1 HTTP/1.1" 200 None 2026-03-30 13:52:58,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:52:58,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-30 13:52:58,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:52:58,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dd31be430e18c5d16ff1 HTTP/1.1" 200 None 2026-03-30 13:52:58,627 - lava_callback - ERROR - Error processing callback for node 69c7dd31be430e18c5d16ff1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38912 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:54:18,518 - lava_callback - INFO - Received callback for node 69c7dd7dbe430e18c5d16ff7 INFO: 172.21.0.1:38914 - "POST /node/69c7dd7dbe430e18c5d16ff7 HTTP/1.0" 202 Accepted 2026-03-30 13:54:18,524 - lava_callback - INFO - Starting async job submit for node 69c7dd7dbe430e18c5d16ff7 2026-03-30 13:54:18,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:54:18,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dd7dbe430e18c5d16ff7 HTTP/1.1" 200 None 2026-03-30 13:54:18,539 - lava_callback - DEBUG - Processing node 69c7dd7dbe430e18c5d16ff7: baseline-arm64-clabbe 2026-03-30 13:54:18,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dd7dbe430e18c5d16ff7 2026-03-30 13:54:18,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:54:18,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dd7dbe430e18c5d16ff7 HTTP/1.1" 200 None 2026-03-30 13:54:18,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:54:18,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:54:18,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:54:18,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dd7dbe430e18c5d16ff7 HTTP/1.1" 200 None 2026-03-30 13:54:18,595 - lava_callback - ERROR - Error processing callback for node 69c7dd7dbe430e18c5d16ff7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:55:25,521 - lava_callback - INFO - Received callback for node 69c7ddc3be430e18c5d17006 INFO: 172.21.0.1:40614 - "POST /node/69c7ddc3be430e18c5d17006 HTTP/1.0" 202 Accepted 2026-03-30 13:55:25,526 - lava_callback - INFO - Starting async job submit for node 69c7ddc3be430e18c5d17006 2026-03-30 13:55:25,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:55:25,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ddc3be430e18c5d17006 HTTP/1.1" 200 None 2026-03-30 13:55:25,542 - lava_callback - DEBUG - Processing node 69c7ddc3be430e18c5d17006: baseline-arm64-baylibre 2026-03-30 13:55:25,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ddc3be430e18c5d17006 2026-03-30 13:55:25,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:55:25,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ddc3be430e18c5d17006 HTTP/1.1" 200 None 2026-03-30 13:55:25,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:55:25,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:55:25,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:55:25,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ddc3be430e18c5d17006 HTTP/1.1" 200 None 2026-03-30 13:55:25,600 - lava_callback - ERROR - Error processing callback for node 69c7ddc3be430e18c5d17006: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49648 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:56:38,697 - lava_callback - INFO - Received callback for node 69c7de14be430e18c5d1700f INFO: 172.21.0.1:49664 - "POST /node/69c7de14be430e18c5d1700f HTTP/1.0" 202 Accepted 2026-03-30 13:56:38,703 - lava_callback - INFO - Starting async job submit for node 69c7de14be430e18c5d1700f 2026-03-30 13:56:38,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:56:38,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7de14be430e18c5d1700f HTTP/1.1" 200 None 2026-03-30 13:56:38,728 - lava_callback - DEBUG - Processing node 69c7de14be430e18c5d1700f: baseline-arm64-clabbe 2026-03-30 13:56:38,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7de14be430e18c5d1700f 2026-03-30 13:56:38,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:56:38,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7de14be430e18c5d1700f HTTP/1.1" 200 None 2026-03-30 13:56:38,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:56:38,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:56:38,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:56:38,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7de14be430e18c5d1700f HTTP/1.1" 200 None 2026-03-30 13:56:38,787 - lava_callback - ERROR - Error processing callback for node 69c7de14be430e18c5d1700f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52232 - "GET / HTTP/1.0" 200 OK 2026-03-30 13:58:18,829 - lava_callback - INFO - Received callback for node 69c7de67be430e18c5d1701d INFO: 172.21.0.1:52244 - "POST /node/69c7de67be430e18c5d1701d HTTP/1.0" 202 Accepted 2026-03-30 13:58:18,835 - lava_callback - INFO - Starting async job submit for node 69c7de67be430e18c5d1701d 2026-03-30 13:58:18,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:58:18,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7de67be430e18c5d1701d HTTP/1.1" 200 None 2026-03-30 13:58:18,852 - lava_callback - DEBUG - Processing node 69c7de67be430e18c5d1701d: baseline-arm64-clabbe 2026-03-30 13:58:18,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7de67be430e18c5d1701d 2026-03-30 13:58:18,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:58:18,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7de67be430e18c5d1701d HTTP/1.1" 200 None 2026-03-30 13:58:18,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:58:18,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 13:58:18,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 13:58:18,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7de67be430e18c5d1701d HTTP/1.1" 200 None 2026-03-30 13:58:18,922 - lava_callback - ERROR - Error processing callback for node 69c7de67be430e18c5d1701d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37434 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:00:19,041 - lava_callback - INFO - Received callback for node 69c7dedebe430e18c5d17033 INFO: 172.21.0.1:37440 - "POST /node/69c7dedebe430e18c5d17033 HTTP/1.0" 202 Accepted 2026-03-30 14:00:19,047 - lava_callback - INFO - Starting async job submit for node 69c7dedebe430e18c5d17033 2026-03-30 14:00:19,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:00:19,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dedebe430e18c5d17033 HTTP/1.1" 200 None 2026-03-30 14:00:19,074 - lava_callback - DEBUG - Processing node 69c7dedebe430e18c5d17033: baseline-arm64-clabbe 2026-03-30 14:00:19,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dedebe430e18c5d17033 2026-03-30 14:00:19,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:00:19,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dedebe430e18c5d17033 HTTP/1.1" 200 None 2026-03-30 14:00:19,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:00:19,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:00:19,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:00:19,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dedebe430e18c5d17033 HTTP/1.1" 200 None 2026-03-30 14:00:19,149 - lava_callback - ERROR - Error processing callback for node 69c7dedebe430e18c5d17033: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41718 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:03:05,839 - lava_callback - INFO - Received callback for node 69c7df8dbe430e18c5d1704c INFO: 172.21.0.1:52262 - "POST /node/69c7df8dbe430e18c5d1704c HTTP/1.0" 202 Accepted 2026-03-30 14:03:05,845 - lava_callback - INFO - Starting async job submit for node 69c7df8dbe430e18c5d1704c 2026-03-30 14:03:05,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:03:05,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7df8dbe430e18c5d1704c HTTP/1.1" 200 None 2026-03-30 14:03:05,861 - lava_callback - DEBUG - Processing node 69c7df8dbe430e18c5d1704c: baseline-arm64-baylibre 2026-03-30 14:03:05,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7df8dbe430e18c5d1704c 2026-03-30 14:03:05,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:03:05,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7df8dbe430e18c5d1704c HTTP/1.1" 200 None 2026-03-30 14:03:05,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:03:05,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:03:05,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:03:05,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7df8dbe430e18c5d1704c HTTP/1.1" 200 None 2026-03-30 14:03:05,920 - lava_callback - ERROR - Error processing callback for node 69c7df8dbe430e18c5d1704c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:04:19,123 - lava_callback - INFO - Received callback for node 69c7dfdfbe430e18c5d17064 INFO: 172.21.0.1:41798 - "POST /node/69c7dfdfbe430e18c5d17064 HTTP/1.0" 202 Accepted 2026-03-30 14:04:19,129 - lava_callback - INFO - Starting async job submit for node 69c7dfdfbe430e18c5d17064 2026-03-30 14:04:19,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:04:19,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dfdfbe430e18c5d17064 HTTP/1.1" 200 None 2026-03-30 14:04:19,145 - lava_callback - DEBUG - Processing node 69c7dfdfbe430e18c5d17064: baseline-arm64-clabbe 2026-03-30 14:04:19,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7dfdfbe430e18c5d17064 2026-03-30 14:04:19,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:04:19,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7dfdfbe430e18c5d17064 HTTP/1.1" 200 None 2026-03-30 14:04:19,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:04:19,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:04:19,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:04:19,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7dfdfbe430e18c5d17064 HTTP/1.1" 200 None 2026-03-30 14:04:19,197 - lava_callback - ERROR - Error processing callback for node 69c7dfdfbe430e18c5d17064: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45346 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:06:19,199 - lava_callback - INFO - Received callback for node 69c7e047be430e18c5d1706e INFO: 172.21.0.1:45352 - "POST /node/69c7e047be430e18c5d1706e HTTP/1.0" 202 Accepted 2026-03-30 14:06:19,204 - lava_callback - INFO - Starting async job submit for node 69c7e047be430e18c5d1706e 2026-03-30 14:06:19,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:06:19,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e047be430e18c5d1706e HTTP/1.1" 200 None 2026-03-30 14:06:19,221 - lava_callback - DEBUG - Processing node 69c7e047be430e18c5d1706e: baseline-arm64-clabbe 2026-03-30 14:06:19,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e047be430e18c5d1706e 2026-03-30 14:06:19,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:06:19,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e047be430e18c5d1706e HTTP/1.1" 200 None 2026-03-30 14:06:19,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:06:19,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:06:19,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:06:19,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e047be430e18c5d1706e HTTP/1.1" 200 None 2026-03-30 14:06:19,273 - lava_callback - ERROR - Error processing callback for node 69c7e047be430e18c5d1706e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49650 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:07:19,176 - lava_callback - INFO - Received callback for node 69c7e094be430e18c5d1707c INFO: 172.21.0.1:49664 - "POST /node/69c7e094be430e18c5d1707c HTTP/1.0" 202 Accepted 2026-03-30 14:07:19,182 - lava_callback - INFO - Starting async job submit for node 69c7e094be430e18c5d1707c 2026-03-30 14:07:19,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:07:19,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e094be430e18c5d1707c HTTP/1.1" 200 None 2026-03-30 14:07:19,198 - lava_callback - DEBUG - Processing node 69c7e094be430e18c5d1707c: baseline-arm64-clabbe 2026-03-30 14:07:19,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e094be430e18c5d1707c 2026-03-30 14:07:19,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:07:19,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e094be430e18c5d1707c HTTP/1.1" 200 None 2026-03-30 14:07:19,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:07:19,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:07:19,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:07:19,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e094be430e18c5d1707c HTTP/1.1" 200 None 2026-03-30 14:07:19,273 - lava_callback - ERROR - Error processing callback for node 69c7e094be430e18c5d1707c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:08:39,294 - lava_callback - INFO - Received callback for node 69c7e0dfbe430e18c5d17086 INFO: 172.21.0.1:39452 - "POST /node/69c7e0dfbe430e18c5d17086 HTTP/1.0" 202 Accepted 2026-03-30 14:08:39,299 - lava_callback - INFO - Starting async job submit for node 69c7e0dfbe430e18c5d17086 2026-03-30 14:08:39,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:08:39,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e0dfbe430e18c5d17086 HTTP/1.1" 200 None 2026-03-30 14:08:39,321 - lava_callback - DEBUG - Processing node 69c7e0dfbe430e18c5d17086: baseline-arm64-clabbe 2026-03-30 14:08:39,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e0dfbe430e18c5d17086 2026-03-30 14:08:39,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:08:39,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e0dfbe430e18c5d17086 HTTP/1.1" 200 None 2026-03-30 14:08:39,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:08:39,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:08:39,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:08:39,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e0dfbe430e18c5d17086 HTTP/1.1" 200 None 2026-03-30 14:08:39,386 - lava_callback - ERROR - Error processing callback for node 69c7e0dfbe430e18c5d17086: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39320 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:10:19,353 - lava_callback - INFO - Received callback for node 69c7e138be430e18c5d17092 INFO: 172.21.0.1:39332 - "POST /node/69c7e138be430e18c5d17092 HTTP/1.0" 202 Accepted 2026-03-30 14:10:19,359 - lava_callback - INFO - Starting async job submit for node 69c7e138be430e18c5d17092 2026-03-30 14:10:19,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:10:19,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e138be430e18c5d17092 HTTP/1.1" 200 None 2026-03-30 14:10:19,375 - lava_callback - DEBUG - Processing node 69c7e138be430e18c5d17092: baseline-arm64-clabbe 2026-03-30 14:10:19,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e138be430e18c5d17092 2026-03-30 14:10:19,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:10:19,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e138be430e18c5d17092 HTTP/1.1" 200 None 2026-03-30 14:10:19,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:10:19,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:10:19,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:10:19,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e138be430e18c5d17092 HTTP/1.1" 200 None 2026-03-30 14:10:19,427 - lava_callback - ERROR - Error processing callback for node 69c7e138be430e18c5d17092: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43944 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:12:19,375 - lava_callback - INFO - Received callback for node 69c7e1afbe430e18c5d170a1 INFO: 172.21.0.1:43958 - "POST /node/69c7e1afbe430e18c5d170a1 HTTP/1.0" 202 Accepted 2026-03-30 14:12:19,381 - lava_callback - INFO - Starting async job submit for node 69c7e1afbe430e18c5d170a1 2026-03-30 14:12:19,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:12:19,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e1afbe430e18c5d170a1 HTTP/1.1" 200 None 2026-03-30 14:12:19,402 - lava_callback - DEBUG - Processing node 69c7e1afbe430e18c5d170a1: baseline-arm64-clabbe 2026-03-30 14:12:19,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e1afbe430e18c5d170a1 2026-03-30 14:12:19,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:12:19,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e1afbe430e18c5d170a1 HTTP/1.1" 200 None 2026-03-30 14:12:19,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:12:19,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-30 14:12:19,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:12:19,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e1afbe430e18c5d170a1 HTTP/1.1" 200 None 2026-03-30 14:12:19,462 - lava_callback - ERROR - Error processing callback for node 69c7e1afbe430e18c5d170a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38480 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:42:20,623 - lava_callback - INFO - Received callback for node 69c7e8c4be430e18c5d17103 INFO: 172.21.0.1:39720 - "POST /node/69c7e8c4be430e18c5d17103 HTTP/1.0" 202 Accepted 2026-03-30 14:42:20,629 - lava_callback - INFO - Starting async job submit for node 69c7e8c4be430e18c5d17103 2026-03-30 14:42:20,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:42:20,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e8c4be430e18c5d17103 HTTP/1.1" 200 None 2026-03-30 14:42:20,645 - lava_callback - DEBUG - Processing node 69c7e8c4be430e18c5d17103: baseline-arm-clabbe 2026-03-30 14:42:20,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e8c4be430e18c5d17103 2026-03-30 14:42:20,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:42:20,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e8c4be430e18c5d17103 HTTP/1.1" 200 None 2026-03-30 14:42:20,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:42:20,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 14:42:20,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:42:20,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e8c4be430e18c5d17103 HTTP/1.1" 200 None 2026-03-30 14:42:20,733 - lava_callback - ERROR - Error processing callback for node 69c7e8c4be430e18c5d17103: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53964 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:43:00,574 - lava_callback - INFO - Received callback for node 69c7e8e9be430e18c5d17112 INFO: 172.21.0.1:46426 - "POST /node/69c7e8e9be430e18c5d17112 HTTP/1.0" 202 Accepted 2026-03-30 14:43:00,580 - lava_callback - INFO - Starting async job submit for node 69c7e8e9be430e18c5d17112 2026-03-30 14:43:00,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:00,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e8e9be430e18c5d17112 HTTP/1.1" 200 None 2026-03-30 14:43:00,596 - lava_callback - DEBUG - Processing node 69c7e8e9be430e18c5d17112: baseline-arm-clabbe 2026-03-30 14:43:00,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e8e9be430e18c5d17112 2026-03-30 14:43:00,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:00,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e8e9be430e18c5d17112 HTTP/1.1" 200 None 2026-03-30 14:43:00,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:00,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 14:43:00,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:00,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e8e9be430e18c5d17112 HTTP/1.1" 200 None 2026-03-30 14:43:00,659 - lava_callback - ERROR - Error processing callback for node 69c7e8e9be430e18c5d17112: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49756 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:43:40,678 - lava_callback - INFO - Received callback for node 69c7e90ebe430e18c5d1711d INFO: 172.21.0.1:51322 - "POST /node/69c7e90ebe430e18c5d1711d HTTP/1.0" 202 Accepted 2026-03-30 14:43:40,684 - lava_callback - INFO - Starting async job submit for node 69c7e90ebe430e18c5d1711d 2026-03-30 14:43:40,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:40,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e90ebe430e18c5d1711d HTTP/1.1" 200 None 2026-03-30 14:43:40,700 - lava_callback - DEBUG - Processing node 69c7e90ebe430e18c5d1711d: baseline-arm-clabbe 2026-03-30 14:43:40,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e90ebe430e18c5d1711d 2026-03-30 14:43:40,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:40,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e90ebe430e18c5d1711d HTTP/1.1" 200 None 2026-03-30 14:43:40,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:40,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 14:43:40,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:43:40,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e90ebe430e18c5d1711d HTTP/1.1" 200 None 2026-03-30 14:43:40,766 - lava_callback - ERROR - Error processing callback for node 69c7e90ebe430e18c5d1711d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39702 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:44:20,643 - lava_callback - INFO - Received callback for node 69c7e933be430e18c5d1712a INFO: 172.21.0.1:60666 - "POST /node/69c7e933be430e18c5d1712a HTTP/1.0" 202 Accepted 2026-03-30 14:44:20,650 - lava_callback - INFO - Starting async job submit for node 69c7e933be430e18c5d1712a 2026-03-30 14:44:20,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:44:20,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e933be430e18c5d1712a HTTP/1.1" 200 None 2026-03-30 14:44:20,666 - lava_callback - DEBUG - Processing node 69c7e933be430e18c5d1712a: baseline-arm-clabbe 2026-03-30 14:44:20,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e933be430e18c5d1712a 2026-03-30 14:44:20,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:44:20,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e933be430e18c5d1712a HTTP/1.1" 200 None 2026-03-30 14:44:20,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:44:20,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 14:44:20,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:44:20,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e933be430e18c5d1712a HTTP/1.1" 200 None 2026-03-30 14:44:20,726 - lava_callback - ERROR - Error processing callback for node 69c7e933be430e18c5d1712a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48326 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:45:00,716 - lava_callback - INFO - Received callback for node 69c7e958be430e18c5d17137 INFO: 172.21.0.1:39280 - "POST /node/69c7e958be430e18c5d17137 HTTP/1.0" 202 Accepted 2026-03-30 14:45:00,722 - lava_callback - INFO - Starting async job submit for node 69c7e958be430e18c5d17137 2026-03-30 14:45:00,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:00,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e958be430e18c5d17137 HTTP/1.1" 200 None 2026-03-30 14:45:00,738 - lava_callback - DEBUG - Processing node 69c7e958be430e18c5d17137: baseline-arm-clabbe 2026-03-30 14:45:00,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e958be430e18c5d17137 2026-03-30 14:45:00,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:00,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e958be430e18c5d17137 HTTP/1.1" 200 None 2026-03-30 14:45:00,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:00,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-30 14:45:00,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:00,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e958be430e18c5d17137 HTTP/1.1" 200 None 2026-03-30 14:45:00,798 - lava_callback - ERROR - Error processing callback for node 69c7e958be430e18c5d17137: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48490 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:45:20,683 - lava_callback - INFO - Received callback for node 69c7e97dbe430e18c5d1714a INFO: 172.21.0.1:57378 - "POST /node/69c7e97dbe430e18c5d1714a HTTP/1.0" 202 Accepted 2026-03-30 14:45:20,688 - lava_callback - INFO - Starting async job submit for node 69c7e97dbe430e18c5d1714a 2026-03-30 14:45:20,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:20,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e97dbe430e18c5d1714a HTTP/1.1" 200 None 2026-03-30 14:45:20,705 - lava_callback - DEBUG - Processing node 69c7e97dbe430e18c5d1714a: baseline-arm-clabbe 2026-03-30 14:45:20,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e97dbe430e18c5d1714a 2026-03-30 14:45:20,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:20,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e97dbe430e18c5d1714a HTTP/1.1" 200 None 2026-03-30 14:45:20,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:20,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:45:20,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:45:20,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e97dbe430e18c5d1714a HTTP/1.1" 200 None 2026-03-30 14:45:20,809 - lava_callback - ERROR - Error processing callback for node 69c7e97dbe430e18c5d1714a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39996 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:46:00,744 - lava_callback - INFO - Received callback for node 69c7e9a1be430e18c5d17159 INFO: 172.21.0.1:60228 - "POST /node/69c7e9a1be430e18c5d17159 HTTP/1.0" 202 Accepted 2026-03-30 14:46:00,749 - lava_callback - INFO - Starting async job submit for node 69c7e9a1be430e18c5d17159 2026-03-30 14:46:00,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:46:00,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e9a1be430e18c5d17159 HTTP/1.1" 200 None 2026-03-30 14:46:00,765 - lava_callback - DEBUG - Processing node 69c7e9a1be430e18c5d17159: baseline-arm-clabbe 2026-03-30 14:46:00,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e9a1be430e18c5d17159 2026-03-30 14:46:00,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:46:00,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e9a1be430e18c5d17159 HTTP/1.1" 200 None 2026-03-30 14:46:00,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:46:00,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:46:00,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:46:00,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e9a1be430e18c5d17159 HTTP/1.1" 200 None 2026-03-30 14:46:00,818 - lava_callback - ERROR - Error processing callback for node 69c7e9a1be430e18c5d17159: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60590 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:47:20,793 - lava_callback - INFO - Received callback for node 69c7e9e9be430e18c5d17175 INFO: 172.21.0.1:47402 - "POST /node/69c7e9e9be430e18c5d17175 HTTP/1.0" 202 Accepted 2026-03-30 14:47:20,799 - lava_callback - INFO - Starting async job submit for node 69c7e9e9be430e18c5d17175 2026-03-30 14:47:20,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:47:20,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e9e9be430e18c5d17175 HTTP/1.1" 200 None 2026-03-30 14:47:20,815 - lava_callback - DEBUG - Processing node 69c7e9e9be430e18c5d17175: baseline-arm-clabbe 2026-03-30 14:47:20,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7e9e9be430e18c5d17175 2026-03-30 14:47:20,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:47:20,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7e9e9be430e18c5d17175 HTTP/1.1" 200 None 2026-03-30 14:47:20,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:47:20,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:47:20,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:47:20,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7e9e9be430e18c5d17175 HTTP/1.1" 200 None 2026-03-30 14:47:20,877 - lava_callback - ERROR - Error processing callback for node 69c7e9e9be430e18c5d17175: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:48:00,789 - lava_callback - INFO - Received callback for node 69c7ea0fbe430e18c5d17191 INFO: 172.21.0.1:46424 - "POST /node/69c7ea0fbe430e18c5d17191 HTTP/1.0" 202 Accepted 2026-03-30 14:48:00,794 - lava_callback - INFO - Starting async job submit for node 69c7ea0fbe430e18c5d17191 2026-03-30 14:48:00,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:00,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea0fbe430e18c5d17191 HTTP/1.1" 200 None 2026-03-30 14:48:00,810 - lava_callback - DEBUG - Processing node 69c7ea0fbe430e18c5d17191: baseline-arm-clabbe 2026-03-30 14:48:00,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ea0fbe430e18c5d17191 2026-03-30 14:48:00,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:00,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea0fbe430e18c5d17191 HTTP/1.1" 200 None 2026-03-30 14:48:00,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:00,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:48:00,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:00,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ea0fbe430e18c5d17191 HTTP/1.1" 200 None 2026-03-30 14:48:00,869 - lava_callback - ERROR - Error processing callback for node 69c7ea0fbe430e18c5d17191: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:48:40,850 - lava_callback - INFO - Received callback for node 69c7ea33be430e18c5d1719c INFO: 172.21.0.1:36196 - "POST /node/69c7ea33be430e18c5d1719c HTTP/1.0" 202 Accepted 2026-03-30 14:48:40,855 - lava_callback - INFO - Starting async job submit for node 69c7ea33be430e18c5d1719c 2026-03-30 14:48:40,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:40,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea33be430e18c5d1719c HTTP/1.1" 200 None 2026-03-30 14:48:40,879 - lava_callback - DEBUG - Processing node 69c7ea33be430e18c5d1719c: baseline-arm-clabbe 2026-03-30 14:48:40,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ea33be430e18c5d1719c 2026-03-30 14:48:40,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:40,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea33be430e18c5d1719c HTTP/1.1" 200 None 2026-03-30 14:48:40,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:40,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:48:40,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:48:40,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ea33be430e18c5d1719c HTTP/1.1" 200 None 2026-03-30 14:48:40,937 - lava_callback - ERROR - Error processing callback for node 69c7ea33be430e18c5d1719c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47804 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:49:00,834 - lava_callback - INFO - Received callback for node 69c7ea57be430e18c5d171a9 INFO: 172.21.0.1:49726 - "POST /node/69c7ea57be430e18c5d171a9 HTTP/1.0" 202 Accepted 2026-03-30 14:49:00,840 - lava_callback - INFO - Starting async job submit for node 69c7ea57be430e18c5d171a9 2026-03-30 14:49:00,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:00,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea57be430e18c5d171a9 HTTP/1.1" 200 None 2026-03-30 14:49:00,856 - lava_callback - DEBUG - Processing node 69c7ea57be430e18c5d171a9: baseline-arm-clabbe 2026-03-30 14:49:00,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ea57be430e18c5d171a9 2026-03-30 14:49:00,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:00,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea57be430e18c5d171a9 HTTP/1.1" 200 None 2026-03-30 14:49:00,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:00,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:49:00,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:00,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ea57be430e18c5d171a9 HTTP/1.1" 200 None 2026-03-30 14:49:00,918 - lava_callback - ERROR - Error processing callback for node 69c7ea57be430e18c5d171a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46342 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:49:40,912 - lava_callback - INFO - Received callback for node 69c7ea7cbe430e18c5d171b7 INFO: 172.21.0.1:35776 - "POST /node/69c7ea7cbe430e18c5d171b7 HTTP/1.0" 202 Accepted 2026-03-30 14:49:40,918 - lava_callback - INFO - Starting async job submit for node 69c7ea7cbe430e18c5d171b7 2026-03-30 14:49:40,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:40,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea7cbe430e18c5d171b7 HTTP/1.1" 200 None 2026-03-30 14:49:40,933 - lava_callback - DEBUG - Processing node 69c7ea7cbe430e18c5d171b7: baseline-arm-clabbe 2026-03-30 14:49:40,934 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ea7cbe430e18c5d171b7 2026-03-30 14:49:40,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:40,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ea7cbe430e18c5d171b7 HTTP/1.1" 200 None 2026-03-30 14:49:40,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:40,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:49:40,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:49:41,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ea7cbe430e18c5d171b7 HTTP/1.1" 200 None 2026-03-30 14:49:41,019 - lava_callback - ERROR - Error processing callback for node 69c7ea7cbe430e18c5d171b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:50:20,901 - lava_callback - INFO - Received callback for node 69c7eaa0be430e18c5d171ca INFO: 172.21.0.1:44562 - "POST /node/69c7eaa0be430e18c5d171ca HTTP/1.0" 202 Accepted 2026-03-30 14:50:20,907 - lava_callback - INFO - Starting async job submit for node 69c7eaa0be430e18c5d171ca 2026-03-30 14:50:20,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:50:20,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eaa0be430e18c5d171ca HTTP/1.1" 200 None 2026-03-30 14:50:20,936 - lava_callback - DEBUG - Processing node 69c7eaa0be430e18c5d171ca: baseline-arm-clabbe 2026-03-30 14:50:20,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eaa0be430e18c5d171ca 2026-03-30 14:50:20,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:50:20,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eaa0be430e18c5d171ca HTTP/1.1" 200 None 2026-03-30 14:50:20,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:50:20,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:50:20,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:50:20,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eaa0be430e18c5d171ca HTTP/1.1" 200 None 2026-03-30 14:50:20,992 - lava_callback - ERROR - Error processing callback for node 69c7eaa0be430e18c5d171ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47042 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:51:00,980 - lava_callback - INFO - Received callback for node 69c7eac5be430e18c5d171d8 INFO: 172.21.0.1:34276 - "POST /node/69c7eac5be430e18c5d171d8 HTTP/1.0" 202 Accepted 2026-03-30 14:51:00,985 - lava_callback - INFO - Starting async job submit for node 69c7eac5be430e18c5d171d8 2026-03-30 14:51:00,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:01,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eac5be430e18c5d171d8 HTTP/1.1" 200 None 2026-03-30 14:51:01,011 - lava_callback - DEBUG - Processing node 69c7eac5be430e18c5d171d8: baseline-arm-clabbe 2026-03-30 14:51:01,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eac5be430e18c5d171d8 2026-03-30 14:51:01,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:01,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eac5be430e18c5d171d8 HTTP/1.1" 200 None 2026-03-30 14:51:01,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:01,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:51:01,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:01,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eac5be430e18c5d171d8 HTTP/1.1" 200 None 2026-03-30 14:51:01,080 - lava_callback - ERROR - Error processing callback for node 69c7eac5be430e18c5d171d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59972 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:51:28,573 - lava_callback - INFO - Received callback for node 69c7eae3be430e18c5d171e1 INFO: 172.21.0.1:59976 - "POST /node/69c7eae3be430e18c5d171e1 HTTP/1.0" 202 Accepted 2026-03-30 14:51:28,579 - lava_callback - INFO - Starting async job submit for node 69c7eae3be430e18c5d171e1 2026-03-30 14:51:28,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:28,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eae3be430e18c5d171e1 HTTP/1.1" 200 None 2026-03-30 14:51:28,598 - lava_callback - DEBUG - Processing node 69c7eae3be430e18c5d171e1: baseline-arm-baylibre 2026-03-30 14:51:28,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eae3be430e18c5d171e1 2026-03-30 14:51:28,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:28,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eae3be430e18c5d171e1 HTTP/1.1" 200 None 2026-03-30 14:51:28,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:28,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 14:51:28,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:51:28,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eae3be430e18c5d171e1 HTTP/1.1" 200 None 2026-03-30 14:51:28,656 - lava_callback - ERROR - Error processing callback for node 69c7eae3be430e18c5d171e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46862 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:52:08,362 - lava_callback - INFO - Received callback for node 69c7eb08be430e18c5d171ea INFO: 172.21.0.1:46864 - "POST /node/69c7eb08be430e18c5d171ea HTTP/1.0" 202 Accepted 2026-03-30 14:52:08,367 - lava_callback - INFO - Starting async job submit for node 69c7eb08be430e18c5d171ea 2026-03-30 14:52:08,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:08,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eb08be430e18c5d171ea HTTP/1.1" 200 None 2026-03-30 14:52:08,383 - lava_callback - DEBUG - Processing node 69c7eb08be430e18c5d171ea: baseline-arm-baylibre 2026-03-30 14:52:08,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eb08be430e18c5d171ea 2026-03-30 14:52:08,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:08,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eb08be430e18c5d171ea HTTP/1.1" 200 None 2026-03-30 14:52:08,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:08,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 14:52:08,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:08,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eb08be430e18c5d171ea HTTP/1.1" 200 None 2026-03-30 14:52:08,450 - lava_callback - ERROR - Error processing callback for node 69c7eb08be430e18c5d171ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38782 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:52:41,011 - lava_callback - INFO - Received callback for node 69c7eb30be430e18c5d171f6 INFO: 172.21.0.1:38988 - "POST /node/69c7eb30be430e18c5d171f6 HTTP/1.0" 202 Accepted 2026-03-30 14:52:41,017 - lava_callback - INFO - Starting async job submit for node 69c7eb30be430e18c5d171f6 2026-03-30 14:52:41,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:41,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eb30be430e18c5d171f6 HTTP/1.1" 200 None 2026-03-30 14:52:41,032 - lava_callback - DEBUG - Processing node 69c7eb30be430e18c5d171f6: baseline-arm-clabbe 2026-03-30 14:52:41,032 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eb30be430e18c5d171f6 2026-03-30 14:52:41,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:41,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eb30be430e18c5d171f6 HTTP/1.1" 200 None 2026-03-30 14:52:41,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:41,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-30 14:52:41,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:52:41,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eb30be430e18c5d171f6 HTTP/1.1" 200 None 2026-03-30 14:52:41,099 - lava_callback - ERROR - Error processing callback for node 69c7eb30be430e18c5d171f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38258 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:54:28,586 - lava_callback - INFO - Received callback for node 69c7eb90be430e18c5d17216 INFO: 172.21.0.1:38262 - "POST /node/69c7eb90be430e18c5d17216 HTTP/1.0" 202 Accepted 2026-03-30 14:54:28,592 - lava_callback - INFO - Starting async job submit for node 69c7eb90be430e18c5d17216 2026-03-30 14:54:28,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:28,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eb90be430e18c5d17216 HTTP/1.1" 200 None 2026-03-30 14:54:28,608 - lava_callback - DEBUG - Processing node 69c7eb90be430e18c5d17216: baseline-arm-baylibre 2026-03-30 14:54:28,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eb90be430e18c5d17216 2026-03-30 14:54:28,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:28,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eb90be430e18c5d17216 HTTP/1.1" 200 None 2026-03-30 14:54:28,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:28,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 14:54:28,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:28,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eb90be430e18c5d17216 HTTP/1.1" 200 None 2026-03-30 14:54:28,664 - lava_callback - ERROR - Error processing callback for node 69c7eb90be430e18c5d17216: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54800 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:54:48,507 - lava_callback - INFO - Received callback for node 69c7ebb4be430e18c5d17236 INFO: 172.21.0.1:54812 - "POST /node/69c7ebb4be430e18c5d17236 HTTP/1.0" 202 Accepted 2026-03-30 14:54:48,512 - lava_callback - INFO - Starting async job submit for node 69c7ebb4be430e18c5d17236 2026-03-30 14:54:48,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:48,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ebb4be430e18c5d17236 HTTP/1.1" 200 None 2026-03-30 14:54:48,528 - lava_callback - DEBUG - Processing node 69c7ebb4be430e18c5d17236: baseline-arm-baylibre 2026-03-30 14:54:48,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ebb4be430e18c5d17236 2026-03-30 14:54:48,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:48,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ebb4be430e18c5d17236 HTTP/1.1" 200 None 2026-03-30 14:54:48,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:48,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 14:54:48,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:54:48,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ebb4be430e18c5d17236 HTTP/1.1" 200 None 2026-03-30 14:54:48,583 - lava_callback - ERROR - Error processing callback for node 69c7ebb4be430e18c5d17236: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:55:28,584 - lava_callback - INFO - Received callback for node 69c7ebd9be430e18c5d17248 INFO: 172.21.0.1:47110 - "POST /node/69c7ebd9be430e18c5d17248 HTTP/1.0" 202 Accepted 2026-03-30 14:55:28,589 - lava_callback - INFO - Starting async job submit for node 69c7ebd9be430e18c5d17248 2026-03-30 14:55:28,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:55:28,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ebd9be430e18c5d17248 HTTP/1.1" 200 None 2026-03-30 14:55:28,611 - lava_callback - DEBUG - Processing node 69c7ebd9be430e18c5d17248: baseline-arm-baylibre 2026-03-30 14:55:28,611 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ebd9be430e18c5d17248 2026-03-30 14:55:28,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:55:28,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ebd9be430e18c5d17248 HTTP/1.1" 200 None 2026-03-30 14:55:28,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:55:28,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 14:55:28,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:55:28,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ebd9be430e18c5d17248 HTTP/1.1" 200 None 2026-03-30 14:55:28,674 - lava_callback - ERROR - Error processing callback for node 69c7ebd9be430e18c5d17248: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44420 - "GET / HTTP/1.0" 200 OK 2026-03-30 14:56:08,644 - lava_callback - INFO - Received callback for node 69c7ebfcbe430e18c5d17263 INFO: 172.21.0.1:44422 - "POST /node/69c7ebfcbe430e18c5d17263 HTTP/1.0" 202 Accepted 2026-03-30 14:56:08,650 - lava_callback - INFO - Starting async job submit for node 69c7ebfcbe430e18c5d17263 2026-03-30 14:56:08,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:56:08,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ebfcbe430e18c5d17263 HTTP/1.1" 200 None 2026-03-30 14:56:08,667 - lava_callback - DEBUG - Processing node 69c7ebfcbe430e18c5d17263: baseline-arm-baylibre 2026-03-30 14:56:08,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ebfcbe430e18c5d17263 2026-03-30 14:56:08,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:56:08,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ebfcbe430e18c5d17263 HTTP/1.1" 200 None 2026-03-30 14:56:08,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:56:08,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 14:56:08,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 14:56:08,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ebfcbe430e18c5d17263 HTTP/1.1" 200 None 2026-03-30 14:56:08,720 - lava_callback - ERROR - Error processing callback for node 69c7ebfcbe430e18c5d17263: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49692 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:02:21,423 - lava_callback - INFO - Received callback for node 69c7ed6bbe430e18c5d172bd INFO: 172.21.0.1:50918 - "POST /node/69c7ed6bbe430e18c5d172bd HTTP/1.0" 202 Accepted 2026-03-30 15:02:21,428 - lava_callback - INFO - Starting async job submit for node 69c7ed6bbe430e18c5d172bd 2026-03-30 15:02:21,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:02:21,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ed6bbe430e18c5d172bd HTTP/1.1" 200 None 2026-03-30 15:02:21,444 - lava_callback - DEBUG - Processing node 69c7ed6bbe430e18c5d172bd: baseline-arm-clabbe 2026-03-30 15:02:21,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ed6bbe430e18c5d172bd 2026-03-30 15:02:21,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:02:21,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ed6bbe430e18c5d172bd HTTP/1.1" 200 None 2026-03-30 15:02:21,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:02:21,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:02:21,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:02:21,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ed6bbe430e18c5d172bd HTTP/1.1" 200 None 2026-03-30 15:02:21,506 - lava_callback - ERROR - Error processing callback for node 69c7ed6bbe430e18c5d172bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:03:29,090 - lava_callback - INFO - Received callback for node 69c7edb8be430e18c5d172da INFO: 172.21.0.1:43884 - "POST /node/69c7edb8be430e18c5d172da HTTP/1.0" 202 Accepted 2026-03-30 15:03:29,096 - lava_callback - INFO - Starting async job submit for node 69c7edb8be430e18c5d172da 2026-03-30 15:03:29,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:03:29,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7edb8be430e18c5d172da HTTP/1.1" 200 None 2026-03-30 15:03:29,115 - lava_callback - DEBUG - Processing node 69c7edb8be430e18c5d172da: baseline-arm64-baylibre 2026-03-30 15:03:29,115 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7edb8be430e18c5d172da 2026-03-30 15:03:29,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:03:29,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7edb8be430e18c5d172da HTTP/1.1" 200 None 2026-03-30 15:03:29,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:03:29,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:03:29,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:03:29,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7edb8be430e18c5d172da HTTP/1.1" 200 None 2026-03-30 15:03:29,170 - lava_callback - ERROR - Error processing callback for node 69c7edb8be430e18c5d172da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:04:49,073 - lava_callback - INFO - Received callback for node 69c7ee03be430e18c5d172e3 INFO: 172.21.0.1:43116 - "POST /node/69c7ee03be430e18c5d172e3 HTTP/1.0" 202 Accepted 2026-03-30 15:04:49,079 - lava_callback - INFO - Starting async job submit for node 69c7ee03be430e18c5d172e3 2026-03-30 15:04:49,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:04:49,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ee03be430e18c5d172e3 HTTP/1.1" 200 None 2026-03-30 15:04:49,095 - lava_callback - DEBUG - Processing node 69c7ee03be430e18c5d172e3: baseline-arm64-baylibre 2026-03-30 15:04:49,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ee03be430e18c5d172e3 2026-03-30 15:04:49,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:04:49,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ee03be430e18c5d172e3 HTTP/1.1" 200 None 2026-03-30 15:04:49,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:04:49,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:04:49,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:04:49,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ee03be430e18c5d172e3 HTTP/1.1" 200 None 2026-03-30 15:04:49,154 - lava_callback - ERROR - Error processing callback for node 69c7ee03be430e18c5d172e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37086 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:06:09,259 - lava_callback - INFO - Received callback for node 69c7ee4ebe430e18c5d172ef INFO: 172.21.0.1:37100 - "POST /node/69c7ee4ebe430e18c5d172ef HTTP/1.0" 202 Accepted 2026-03-30 15:06:09,265 - lava_callback - INFO - Starting async job submit for node 69c7ee4ebe430e18c5d172ef 2026-03-30 15:06:09,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:06:09,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ee4ebe430e18c5d172ef HTTP/1.1" 200 None 2026-03-30 15:06:09,281 - lava_callback - DEBUG - Processing node 69c7ee4ebe430e18c5d172ef: baseline-arm64-baylibre 2026-03-30 15:06:09,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ee4ebe430e18c5d172ef 2026-03-30 15:06:09,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:06:09,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ee4ebe430e18c5d172ef HTTP/1.1" 200 None 2026-03-30 15:06:09,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:06:09,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:06:09,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:06:09,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ee4ebe430e18c5d172ef HTTP/1.1" 200 None 2026-03-30 15:06:09,349 - lava_callback - ERROR - Error processing callback for node 69c7ee4ebe430e18c5d172ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53700 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:07:09,187 - lava_callback - INFO - Received callback for node 69c7ee9abe430e18c5d172f6 INFO: 172.21.0.1:53716 - "POST /node/69c7ee9abe430e18c5d172f6 HTTP/1.0" 202 Accepted 2026-03-30 15:07:09,193 - lava_callback - INFO - Starting async job submit for node 69c7ee9abe430e18c5d172f6 2026-03-30 15:07:09,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:07:09,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ee9abe430e18c5d172f6 HTTP/1.1" 200 None 2026-03-30 15:07:09,210 - lava_callback - DEBUG - Processing node 69c7ee9abe430e18c5d172f6: baseline-arm64-baylibre 2026-03-30 15:07:09,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ee9abe430e18c5d172f6 2026-03-30 15:07:09,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:07:09,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ee9abe430e18c5d172f6 HTTP/1.1" 200 None 2026-03-30 15:07:09,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:07:09,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:07:09,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:07:09,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ee9abe430e18c5d172f6 HTTP/1.1" 200 None 2026-03-30 15:07:09,270 - lava_callback - ERROR - Error processing callback for node 69c7ee9abe430e18c5d172f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57238 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:08:29,438 - lava_callback - INFO - Received callback for node 69c7eee5be430e18c5d17300 INFO: 172.21.0.1:57248 - "POST /node/69c7eee5be430e18c5d17300 HTTP/1.0" 202 Accepted 2026-03-30 15:08:29,443 - lava_callback - INFO - Starting async job submit for node 69c7eee5be430e18c5d17300 2026-03-30 15:08:29,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:08:29,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eee5be430e18c5d17300 HTTP/1.1" 200 None 2026-03-30 15:08:29,460 - lava_callback - DEBUG - Processing node 69c7eee5be430e18c5d17300: baseline-arm64-baylibre 2026-03-30 15:08:29,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eee5be430e18c5d17300 2026-03-30 15:08:29,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:08:29,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eee5be430e18c5d17300 HTTP/1.1" 200 None 2026-03-30 15:08:29,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:08:29,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:08:29,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:08:29,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eee5be430e18c5d17300 HTTP/1.1" 200 None 2026-03-30 15:08:29,518 - lava_callback - ERROR - Error processing callback for node 69c7eee5be430e18c5d17300: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45092 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:09:49,433 - lava_callback - INFO - Received callback for node 69c7ef30be430e18c5d17306 INFO: 172.21.0.1:45104 - "POST /node/69c7ef30be430e18c5d17306 HTTP/1.0" 202 Accepted 2026-03-30 15:09:49,439 - lava_callback - INFO - Starting async job submit for node 69c7ef30be430e18c5d17306 2026-03-30 15:09:49,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:09:49,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ef30be430e18c5d17306 HTTP/1.1" 200 None 2026-03-30 15:09:49,464 - lava_callback - DEBUG - Processing node 69c7ef30be430e18c5d17306: baseline-arm64-baylibre 2026-03-30 15:09:49,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ef30be430e18c5d17306 2026-03-30 15:09:49,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:09:49,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ef30be430e18c5d17306 HTTP/1.1" 200 None 2026-03-30 15:09:49,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:09:49,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:09:49,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:09:49,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ef30be430e18c5d17306 HTTP/1.1" 200 None 2026-03-30 15:09:49,542 - lava_callback - ERROR - Error processing callback for node 69c7ef30be430e18c5d17306: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:11:01,798 - lava_callback - INFO - Received callback for node 69c7ef73be430e18c5d17310 INFO: 172.21.0.1:45814 - "POST /node/69c7ef73be430e18c5d17310 HTTP/1.0" 202 Accepted 2026-03-30 15:11:01,803 - lava_callback - INFO - Starting async job submit for node 69c7ef73be430e18c5d17310 2026-03-30 15:11:01,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:01,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ef73be430e18c5d17310 HTTP/1.1" 200 None 2026-03-30 15:11:01,820 - lava_callback - DEBUG - Processing node 69c7ef73be430e18c5d17310: baseline-arm-clabbe 2026-03-30 15:11:01,820 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7ef73be430e18c5d17310 2026-03-30 15:11:01,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:01,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ef73be430e18c5d17310 HTTP/1.1" 200 None 2026-03-30 15:11:01,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:01,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:11:01,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:01,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ef73be430e18c5d17310 HTTP/1.1" 200 None 2026-03-30 15:11:01,901 - lava_callback - ERROR - Error processing callback for node 69c7ef73be430e18c5d17310: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:11:41,808 - lava_callback - INFO - Received callback for node 69c7efa7be430e18c5d1731e INFO: 172.21.0.1:33182 - "POST /node/69c7efa7be430e18c5d1731e HTTP/1.0" 202 Accepted 2026-03-30 15:11:41,813 - lava_callback - INFO - Starting async job submit for node 69c7efa7be430e18c5d1731e 2026-03-30 15:11:41,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:41,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7efa7be430e18c5d1731e HTTP/1.1" 200 None 2026-03-30 15:11:41,838 - lava_callback - DEBUG - Processing node 69c7efa7be430e18c5d1731e: baseline-arm64-clabbe 2026-03-30 15:11:41,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7efa7be430e18c5d1731e 2026-03-30 15:11:41,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:41,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7efa7be430e18c5d1731e HTTP/1.1" 200 None 2026-03-30 15:11:41,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:41,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:11:41,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:11:41,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7efa7be430e18c5d1731e HTTP/1.1" 200 None 2026-03-30 15:11:41,894 - lava_callback - ERROR - Error processing callback for node 69c7efa7be430e18c5d1731e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39294 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:13:01,893 - lava_callback - INFO - Received callback for node 69c7eff2be430e18c5d1732c INFO: 172.21.0.1:33622 - "POST /node/69c7eff2be430e18c5d1732c HTTP/1.0" 202 Accepted 2026-03-30 15:13:01,899 - lava_callback - INFO - Starting async job submit for node 69c7eff2be430e18c5d1732c 2026-03-30 15:13:01,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:13:01,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eff2be430e18c5d1732c HTTP/1.1" 200 None 2026-03-30 15:13:01,923 - lava_callback - DEBUG - Processing node 69c7eff2be430e18c5d1732c: baseline-arm64-clabbe 2026-03-30 15:13:01,923 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7eff2be430e18c5d1732c 2026-03-30 15:13:01,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:13:01,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7eff2be430e18c5d1732c HTTP/1.1" 200 None 2026-03-30 15:13:01,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:13:01,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:13:01,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:13:01,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7eff2be430e18c5d1732c HTTP/1.1" 200 None 2026-03-30 15:13:01,982 - lava_callback - ERROR - Error processing callback for node 69c7eff2be430e18c5d1732c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36362 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:14:21,916 - lava_callback - INFO - Received callback for node 69c7f03dbe430e18c5d17335 INFO: 172.21.0.1:59256 - "POST /node/69c7f03dbe430e18c5d17335 HTTP/1.0" 202 Accepted 2026-03-30 15:14:21,922 - lava_callback - INFO - Starting async job submit for node 69c7f03dbe430e18c5d17335 2026-03-30 15:14:21,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:14:21,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f03dbe430e18c5d17335 HTTP/1.1" 200 None 2026-03-30 15:14:21,942 - lava_callback - DEBUG - Processing node 69c7f03dbe430e18c5d17335: baseline-arm64-clabbe 2026-03-30 15:14:21,942 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f03dbe430e18c5d17335 2026-03-30 15:14:21,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:14:21,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f03dbe430e18c5d17335 HTTP/1.1" 200 None 2026-03-30 15:14:21,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:14:21,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:14:21,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:14:22,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f03dbe430e18c5d17335 HTTP/1.1" 200 None 2026-03-30 15:14:22,030 - lava_callback - ERROR - Error processing callback for node 69c7f03dbe430e18c5d17335: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41970 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:15:41,938 - lava_callback - INFO - Received callback for node 69c7f08abe430e18c5d17342 INFO: 172.21.0.1:47886 - "POST /node/69c7f08abe430e18c5d17342 HTTP/1.0" 202 Accepted 2026-03-30 15:15:41,944 - lava_callback - INFO - Starting async job submit for node 69c7f08abe430e18c5d17342 2026-03-30 15:15:41,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:15:41,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f08abe430e18c5d17342 HTTP/1.1" 200 None 2026-03-30 15:15:41,960 - lava_callback - DEBUG - Processing node 69c7f08abe430e18c5d17342: baseline-arm64-clabbe 2026-03-30 15:15:41,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f08abe430e18c5d17342 2026-03-30 15:15:41,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:15:41,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f08abe430e18c5d17342 HTTP/1.1" 200 None 2026-03-30 15:15:41,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:15:41,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:15:41,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:15:42,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f08abe430e18c5d17342 HTTP/1.1" 200 None 2026-03-30 15:15:42,016 - lava_callback - ERROR - Error processing callback for node 69c7f08abe430e18c5d17342: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:16:42,016 - lava_callback - INFO - Received callback for node 69c7f0d6be430e18c5d17349 INFO: 172.21.0.1:40300 - "POST /node/69c7f0d6be430e18c5d17349 HTTP/1.0" 202 Accepted 2026-03-30 15:16:42,022 - lava_callback - INFO - Starting async job submit for node 69c7f0d6be430e18c5d17349 2026-03-30 15:16:42,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:16:42,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f0d6be430e18c5d17349 HTTP/1.1" 200 None 2026-03-30 15:16:42,039 - lava_callback - DEBUG - Processing node 69c7f0d6be430e18c5d17349: baseline-arm64-clabbe 2026-03-30 15:16:42,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f0d6be430e18c5d17349 2026-03-30 15:16:42,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:16:42,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f0d6be430e18c5d17349 HTTP/1.1" 200 None 2026-03-30 15:16:42,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:16:42,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:16:42,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:16:42,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f0d6be430e18c5d17349 HTTP/1.1" 200 None 2026-03-30 15:16:42,110 - lava_callback - ERROR - Error processing callback for node 69c7f0d6be430e18c5d17349: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58496 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:18:02,085 - lava_callback - INFO - Received callback for node 69c7f120be430e18c5d17356 INFO: 172.21.0.1:37054 - "POST /node/69c7f120be430e18c5d17356 HTTP/1.0" 202 Accepted 2026-03-30 15:18:02,090 - lava_callback - INFO - Starting async job submit for node 69c7f120be430e18c5d17356 2026-03-30 15:18:02,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:18:02,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f120be430e18c5d17356 HTTP/1.1" 200 None 2026-03-30 15:18:02,109 - lava_callback - DEBUG - Processing node 69c7f120be430e18c5d17356: baseline-arm64-clabbe 2026-03-30 15:18:02,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f120be430e18c5d17356 2026-03-30 15:18:02,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:18:02,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f120be430e18c5d17356 HTTP/1.1" 200 None 2026-03-30 15:18:02,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:18:02,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-30 15:18:02,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:18:02,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f120be430e18c5d17356 HTTP/1.1" 200 None 2026-03-30 15:18:02,167 - lava_callback - ERROR - Error processing callback for node 69c7f120be430e18c5d17356: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43130 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:19:42,166 - lava_callback - INFO - Received callback for node 69c7f181be430e18c5d17368 INFO: 172.21.0.1:49106 - "POST /node/69c7f181be430e18c5d17368 HTTP/1.0" 202 Accepted 2026-03-30 15:19:42,172 - lava_callback - INFO - Starting async job submit for node 69c7f181be430e18c5d17368 2026-03-30 15:19:42,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:19:42,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f181be430e18c5d17368 HTTP/1.1" 200 None 2026-03-30 15:19:42,188 - lava_callback - DEBUG - Processing node 69c7f181be430e18c5d17368: baseline-arm-clabbe 2026-03-30 15:19:42,188 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f181be430e18c5d17368 2026-03-30 15:19:42,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:19:42,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f181be430e18c5d17368 HTTP/1.1" 200 None 2026-03-30 15:19:42,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:19:42,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:19:42,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:19:42,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f181be430e18c5d17368 HTTP/1.1" 200 None 2026-03-30 15:19:42,246 - lava_callback - ERROR - Error processing callback for node 69c7f181be430e18c5d17368: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39558 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:20:10,023 - lava_callback - INFO - Received callback for node 69c7f1a1be430e18c5d17376 INFO: 172.21.0.1:45640 - "POST /node/69c7f1a1be430e18c5d17376 HTTP/1.0" 202 Accepted 2026-03-30 15:20:10,029 - lava_callback - INFO - Starting async job submit for node 69c7f1a1be430e18c5d17376 2026-03-30 15:20:10,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:20:10,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f1a1be430e18c5d17376 HTTP/1.1" 200 None 2026-03-30 15:20:10,045 - lava_callback - DEBUG - Processing node 69c7f1a1be430e18c5d17376: baseline-arm-baylibre 2026-03-30 15:20:10,046 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f1a1be430e18c5d17376 2026-03-30 15:20:10,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:20:10,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f1a1be430e18c5d17376 HTTP/1.1" 200 None 2026-03-30 15:20:10,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:20:10,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:20:10,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:20:10,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f1a1be430e18c5d17376 HTTP/1.1" 200 None 2026-03-30 15:20:10,105 - lava_callback - ERROR - Error processing callback for node 69c7f1a1be430e18c5d17376: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58722 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:21:22,224 - lava_callback - INFO - Received callback for node 69c7f1ebbe430e18c5d17397 INFO: 172.21.0.1:40464 - "POST /node/69c7f1ebbe430e18c5d17397 HTTP/1.0" 202 Accepted 2026-03-30 15:21:22,231 - lava_callback - INFO - Starting async job submit for node 69c7f1ebbe430e18c5d17397 2026-03-30 15:21:22,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:21:22,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f1ebbe430e18c5d17397 HTTP/1.1" 200 None 2026-03-30 15:21:22,247 - lava_callback - DEBUG - Processing node 69c7f1ebbe430e18c5d17397: baseline-arm-clabbe 2026-03-30 15:21:22,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f1ebbe430e18c5d17397 2026-03-30 15:21:22,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:21:22,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f1ebbe430e18c5d17397 HTTP/1.1" 200 None 2026-03-30 15:21:22,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:21:22,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:21:22,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:21:22,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f1ebbe430e18c5d17397 HTTP/1.1" 200 None 2026-03-30 15:21:22,317 - lava_callback - ERROR - Error processing callback for node 69c7f1ebbe430e18c5d17397: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41968 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:22:07,569 - lava_callback - INFO - Received callback for node 69ca94ef0e33d030878aa371 INFO: 172.21.0.1:41984 - "POST /node/69ca94ef0e33d030878aa371 HTTP/1.0" 202 Accepted 2026-03-30 15:22:07,576 - lava_callback - INFO - Starting async job submit for node 69ca94ef0e33d030878aa371 2026-03-30 15:22:07,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:07,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca94ef0e33d030878aa371 HTTP/1.1" 200 None 2026-03-30 15:22:07,619 - lava_callback - DEBUG - Processing node 69ca94ef0e33d030878aa371: baseline-arm64-foundriesio 2026-03-30 15:22:07,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 15:22:08,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 15:22:08,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69ca94ef0e33d030878aa371/log.txt.gz 2026-03-30 15:22:08,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 15:22:08,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 15:22:08,651 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69ca94ef0e33d030878aa371/lava-logs.yaml 2026-03-30 15:22:08,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-30 15:22:08,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-30 15:22:08,860 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69ca94ef0e33d030878aa371/lava_callback.json.gz 2026-03-30 15:22:08,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:08,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca94ef0e33d030878aa371 HTTP/1.1" 200 None 2026-03-30 15:22:08,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:08,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-03-30 15:22:08,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:08,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ca94ef0e33d030878aa371 HTTP/1.1" 200 None 2026-03-30 15:22:08,961 - lava_callback - INFO - Completed processing callback for node 69ca94ef0e33d030878aa371 2026-03-30 15:22:10,160 - lava_callback - INFO - Received callback for node 69c7f20fbe430e18c5d173a0 INFO: 172.21.0.1:46118 - "POST /node/69c7f20fbe430e18c5d173a0 HTTP/1.0" 202 Accepted 2026-03-30 15:22:10,165 - lava_callback - INFO - Starting async job submit for node 69c7f20fbe430e18c5d173a0 2026-03-30 15:22:10,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:10,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f20fbe430e18c5d173a0 HTTP/1.1" 200 None 2026-03-30 15:22:10,182 - lava_callback - DEBUG - Processing node 69c7f20fbe430e18c5d173a0: baseline-arm-baylibre 2026-03-30 15:22:10,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f20fbe430e18c5d173a0 2026-03-30 15:22:10,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:10,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f20fbe430e18c5d173a0 HTTP/1.1" 200 None 2026-03-30 15:22:10,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:10,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:22:10,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:10,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f20fbe430e18c5d173a0 HTTP/1.1" 200 None 2026-03-30 15:22:10,249 - lava_callback - ERROR - Error processing callback for node 69c7f20fbe430e18c5d173a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33336 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:22:30,013 - lava_callback - INFO - Received callback for node 69c7f232be430e18c5d173b0 INFO: 172.21.0.1:56766 - "POST /node/69c7f232be430e18c5d173b0 HTTP/1.0" 202 Accepted 2026-03-30 15:22:30,018 - lava_callback - INFO - Starting async job submit for node 69c7f232be430e18c5d173b0 2026-03-30 15:22:30,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:30,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f232be430e18c5d173b0 HTTP/1.1" 200 None 2026-03-30 15:22:30,034 - lava_callback - DEBUG - Processing node 69c7f232be430e18c5d173b0: baseline-arm-baylibre 2026-03-30 15:22:30,034 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f232be430e18c5d173b0 2026-03-30 15:22:30,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:30,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f232be430e18c5d173b0 HTTP/1.1" 200 None 2026-03-30 15:22:30,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:30,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:22:30,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:30,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f232be430e18c5d173b0 HTTP/1.1" 200 None 2026-03-30 15:22:30,093 - lava_callback - ERROR - Error processing callback for node 69c7f232be430e18c5d173b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56780 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:22:37,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:22:37,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41470 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:23:22,348 - lava_callback - INFO - Received callback for node 69c7f259be430e18c5d173bd INFO: 172.21.0.1:43608 - "POST /node/69c7f259be430e18c5d173bd HTTP/1.0" 202 Accepted 2026-03-30 15:23:22,353 - lava_callback - INFO - Starting async job submit for node 69c7f259be430e18c5d173bd 2026-03-30 15:23:22,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:22,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f259be430e18c5d173bd HTTP/1.1" 200 None 2026-03-30 15:23:22,370 - lava_callback - DEBUG - Processing node 69c7f259be430e18c5d173bd: baseline-arm-clabbe 2026-03-30 15:23:22,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f259be430e18c5d173bd 2026-03-30 15:23:22,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:22,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f259be430e18c5d173bd HTTP/1.1" 200 None 2026-03-30 15:23:22,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:22,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:23:22,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:22,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f259be430e18c5d173bd HTTP/1.1" 200 None 2026-03-30 15:23:22,433 - lava_callback - ERROR - Error processing callback for node 69c7f259be430e18c5d173bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38142 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:23:50,435 - lava_callback - INFO - Received callback for node 69c7f27bbe430e18c5d173cd INFO: 172.21.0.1:47422 - "POST /node/69c7f27bbe430e18c5d173cd HTTP/1.0" 202 Accepted 2026-03-30 15:23:50,440 - lava_callback - INFO - Starting async job submit for node 69c7f27bbe430e18c5d173cd 2026-03-30 15:23:50,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:50,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f27bbe430e18c5d173cd HTTP/1.1" 200 None 2026-03-30 15:23:50,459 - lava_callback - DEBUG - Processing node 69c7f27bbe430e18c5d173cd: baseline-arm-baylibre 2026-03-30 15:23:50,460 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f27bbe430e18c5d173cd 2026-03-30 15:23:50,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:50,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f27bbe430e18c5d173cd HTTP/1.1" 200 None 2026-03-30 15:23:50,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:50,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:23:50,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:23:50,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f27bbe430e18c5d173cd HTTP/1.1" 200 None 2026-03-30 15:23:50,529 - lava_callback - ERROR - Error processing callback for node 69c7f27bbe430e18c5d173cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52236 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:24:22,420 - lava_callback - INFO - Received callback for node 69c7f2a1be430e18c5d173e4 INFO: 172.21.0.1:39168 - "POST /node/69c7f2a1be430e18c5d173e4 HTTP/1.0" 202 Accepted 2026-03-30 15:24:22,425 - lava_callback - INFO - Starting async job submit for node 69c7f2a1be430e18c5d173e4 2026-03-30 15:24:22,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:24:22,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f2a1be430e18c5d173e4 HTTP/1.1" 200 None 2026-03-30 15:24:22,442 - lava_callback - DEBUG - Processing node 69c7f2a1be430e18c5d173e4: baseline-arm-clabbe 2026-03-30 15:24:22,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f2a1be430e18c5d173e4 2026-03-30 15:24:22,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:24:22,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f2a1be430e18c5d173e4 HTTP/1.1" 200 None 2026-03-30 15:24:22,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:24:22,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:24:22,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:24:22,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f2a1be430e18c5d173e4 HTTP/1.1" 200 None 2026-03-30 15:24:22,518 - lava_callback - ERROR - Error processing callback for node 69c7f2a1be430e18c5d173e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47506 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:25:02,363 - lava_callback - INFO - Received callback for node 69c7f2c5be430e18c5d173f1 INFO: 172.21.0.1:57316 - "POST /node/69c7f2c5be430e18c5d173f1 HTTP/1.0" 202 Accepted 2026-03-30 15:25:02,368 - lava_callback - INFO - Starting async job submit for node 69c7f2c5be430e18c5d173f1 2026-03-30 15:25:02,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:02,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f2c5be430e18c5d173f1 HTTP/1.1" 200 None 2026-03-30 15:25:02,384 - lava_callback - DEBUG - Processing node 69c7f2c5be430e18c5d173f1: baseline-arm-clabbe 2026-03-30 15:25:02,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f2c5be430e18c5d173f1 2026-03-30 15:25:02,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:02,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f2c5be430e18c5d173f1 HTTP/1.1" 200 None 2026-03-30 15:25:02,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:02,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:25:02,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:02,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f2c5be430e18c5d173f1 HTTP/1.1" 200 None 2026-03-30 15:25:02,445 - lava_callback - ERROR - Error processing callback for node 69c7f2c5be430e18c5d173f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36182 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:25:42,457 - lava_callback - INFO - Received callback for node 69c7f2e9be430e18c5d17401 INFO: 172.21.0.1:50442 - "POST /node/69c7f2e9be430e18c5d17401 HTTP/1.0" 202 Accepted 2026-03-30 15:25:42,462 - lava_callback - INFO - Starting async job submit for node 69c7f2e9be430e18c5d17401 2026-03-30 15:25:42,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:42,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f2e9be430e18c5d17401 HTTP/1.1" 200 None 2026-03-30 15:25:42,483 - lava_callback - DEBUG - Processing node 69c7f2e9be430e18c5d17401: baseline-arm-clabbe 2026-03-30 15:25:42,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f2e9be430e18c5d17401 2026-03-30 15:25:42,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:42,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f2e9be430e18c5d17401 HTTP/1.1" 200 None 2026-03-30 15:25:42,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:42,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:25:42,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:25:42,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f2e9be430e18c5d17401 HTTP/1.1" 200 None 2026-03-30 15:25:42,548 - lava_callback - ERROR - Error processing callback for node 69c7f2e9be430e18c5d17401: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54764 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:26:22,441 - lava_callback - INFO - Received callback for node 69c7f30ebe430e18c5d17412 INFO: 172.21.0.1:51304 - "POST /node/69c7f30ebe430e18c5d17412 HTTP/1.0" 202 Accepted 2026-03-30 15:26:22,447 - lava_callback - INFO - Starting async job submit for node 69c7f30ebe430e18c5d17412 2026-03-30 15:26:22,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:26:22,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f30ebe430e18c5d17412 HTTP/1.1" 200 None 2026-03-30 15:26:22,462 - lava_callback - DEBUG - Processing node 69c7f30ebe430e18c5d17412: baseline-arm-clabbe 2026-03-30 15:26:22,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f30ebe430e18c5d17412 2026-03-30 15:26:22,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:26:22,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f30ebe430e18c5d17412 HTTP/1.1" 200 None 2026-03-30 15:26:22,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:26:22,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:26:22,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:26:22,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f30ebe430e18c5d17412 HTTP/1.1" 200 None 2026-03-30 15:26:22,520 - lava_callback - ERROR - Error processing callback for node 69c7f30ebe430e18c5d17412: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60020 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:27:02,487 - lava_callback - INFO - Received callback for node 69c7f333be430e18c5d17421 INFO: 172.21.0.1:32872 - "POST /node/69c7f333be430e18c5d17421 HTTP/1.0" 202 Accepted 2026-03-30 15:27:02,492 - lava_callback - INFO - Starting async job submit for node 69c7f333be430e18c5d17421 2026-03-30 15:27:02,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:02,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f333be430e18c5d17421 HTTP/1.1" 200 None 2026-03-30 15:27:02,508 - lava_callback - DEBUG - Processing node 69c7f333be430e18c5d17421: baseline-arm-clabbe 2026-03-30 15:27:02,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f333be430e18c5d17421 2026-03-30 15:27:02,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:02,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f333be430e18c5d17421 HTTP/1.1" 200 None 2026-03-30 15:27:02,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:02,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:27:02,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:02,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f333be430e18c5d17421 HTTP/1.1" 200 None 2026-03-30 15:27:02,567 - lava_callback - ERROR - Error processing callback for node 69c7f333be430e18c5d17421: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38064 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:27:42,481 - lava_callback - INFO - Received callback for node 69c7f358be430e18c5d17434 INFO: 172.21.0.1:51394 - "POST /node/69c7f358be430e18c5d17434 HTTP/1.0" 202 Accepted 2026-03-30 15:27:42,487 - lava_callback - INFO - Starting async job submit for node 69c7f358be430e18c5d17434 2026-03-30 15:27:42,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:42,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f358be430e18c5d17434 HTTP/1.1" 200 None 2026-03-30 15:27:42,503 - lava_callback - DEBUG - Processing node 69c7f358be430e18c5d17434: baseline-arm-clabbe 2026-03-30 15:27:42,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f358be430e18c5d17434 2026-03-30 15:27:42,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:42,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f358be430e18c5d17434 HTTP/1.1" 200 None 2026-03-30 15:27:42,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:42,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:27:42,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:27:42,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f358be430e18c5d17434 HTTP/1.1" 200 None 2026-03-30 15:27:42,595 - lava_callback - ERROR - Error processing callback for node 69c7f358be430e18c5d17434: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37344 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:28:02,527 - lava_callback - INFO - Received callback for node 69c7f37dbe430e18c5d17443 INFO: 172.21.0.1:40434 - "POST /node/69c7f37dbe430e18c5d17443 HTTP/1.0" 202 Accepted 2026-03-30 15:28:02,533 - lava_callback - INFO - Starting async job submit for node 69c7f37dbe430e18c5d17443 2026-03-30 15:28:02,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:02,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f37dbe430e18c5d17443 HTTP/1.1" 200 None 2026-03-30 15:28:02,550 - lava_callback - DEBUG - Processing node 69c7f37dbe430e18c5d17443: baseline-arm-clabbe 2026-03-30 15:28:02,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f37dbe430e18c5d17443 2026-03-30 15:28:02,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:02,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f37dbe430e18c5d17443 HTTP/1.1" 200 None 2026-03-30 15:28:02,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:02,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:28:02,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:02,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f37dbe430e18c5d17443 HTTP/1.1" 200 None 2026-03-30 15:28:02,610 - lava_callback - ERROR - Error processing callback for node 69c7f37dbe430e18c5d17443: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56482 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:28:42,552 - lava_callback - INFO - Received callback for node 69c7f3a2be430e18c5d17455 INFO: 172.21.0.1:57376 - "POST /node/69c7f3a2be430e18c5d17455 HTTP/1.0" 202 Accepted 2026-03-30 15:28:42,558 - lava_callback - INFO - Starting async job submit for node 69c7f3a2be430e18c5d17455 2026-03-30 15:28:42,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:42,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f3a2be430e18c5d17455 HTTP/1.1" 200 None 2026-03-30 15:28:42,577 - lava_callback - DEBUG - Processing node 69c7f3a2be430e18c5d17455: baseline-arm-clabbe 2026-03-30 15:28:42,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f3a2be430e18c5d17455 2026-03-30 15:28:42,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:42,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f3a2be430e18c5d17455 HTTP/1.1" 200 None 2026-03-30 15:28:42,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:42,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:28:42,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:28:42,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f3a2be430e18c5d17455 HTTP/1.1" 200 None 2026-03-30 15:28:42,643 - lava_callback - ERROR - Error processing callback for node 69c7f3a2be430e18c5d17455: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43166 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:29:22,597 - lava_callback - INFO - Received callback for node 69c7f3c8be430e18c5d17467 INFO: 172.21.0.1:43490 - "POST /node/69c7f3c8be430e18c5d17467 HTTP/1.0" 202 Accepted 2026-03-30 15:29:22,603 - lava_callback - INFO - Starting async job submit for node 69c7f3c8be430e18c5d17467 2026-03-30 15:29:22,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:29:22,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f3c8be430e18c5d17467 HTTP/1.1" 200 None 2026-03-30 15:29:22,624 - lava_callback - DEBUG - Processing node 69c7f3c8be430e18c5d17467: baseline-arm-clabbe 2026-03-30 15:29:22,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f3c8be430e18c5d17467 2026-03-30 15:29:22,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:29:22,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f3c8be430e18c5d17467 HTTP/1.1" 200 None 2026-03-30 15:29:22,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:29:22,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:29:22,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:29:22,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f3c8be430e18c5d17467 HTTP/1.1" 200 None 2026-03-30 15:29:22,691 - lava_callback - ERROR - Error processing callback for node 69c7f3c8be430e18c5d17467: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51360 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:30:02,541 - lava_callback - INFO - Received callback for node 69c7f3ecbe430e18c5d1747c INFO: 172.21.0.1:37692 - "POST /node/69c7f3ecbe430e18c5d1747c HTTP/1.0" 202 Accepted 2026-03-30 15:30:02,547 - lava_callback - INFO - Starting async job submit for node 69c7f3ecbe430e18c5d1747c 2026-03-30 15:30:02,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:02,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f3ecbe430e18c5d1747c HTTP/1.1" 200 None 2026-03-30 15:30:02,570 - lava_callback - DEBUG - Processing node 69c7f3ecbe430e18c5d1747c: baseline-arm-clabbe 2026-03-30 15:30:02,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f3ecbe430e18c5d1747c 2026-03-30 15:30:02,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:02,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f3ecbe430e18c5d1747c HTTP/1.1" 200 None 2026-03-30 15:30:02,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:02,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-30 15:30:02,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:02,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f3ecbe430e18c5d1747c HTTP/1.1" 200 None 2026-03-30 15:30:02,631 - lava_callback - ERROR - Error processing callback for node 69c7f3ecbe430e18c5d1747c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:30:50,617 - lava_callback - INFO - Received callback for node 69c7f415be430e18c5d17488 INFO: 172.21.0.1:37152 - "POST /node/69c7f415be430e18c5d17488 HTTP/1.0" 202 Accepted 2026-03-30 15:30:50,623 - lava_callback - INFO - Starting async job submit for node 69c7f415be430e18c5d17488 2026-03-30 15:30:50,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:50,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f415be430e18c5d17488 HTTP/1.1" 200 None 2026-03-30 15:30:50,640 - lava_callback - DEBUG - Processing node 69c7f415be430e18c5d17488: baseline-arm64-baylibre 2026-03-30 15:30:50,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f415be430e18c5d17488 2026-03-30 15:30:50,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:50,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f415be430e18c5d17488 HTTP/1.1" 200 None 2026-03-30 15:30:50,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:50,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:30:50,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:30:50,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f415be430e18c5d17488 HTTP/1.1" 200 None 2026-03-30 15:30:50,697 - lava_callback - ERROR - Error processing callback for node 69c7f415be430e18c5d17488: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53622 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:31:50,549 - lava_callback - INFO - Received callback for node 69c7f45fbe430e18c5d17492 INFO: 172.21.0.1:60734 - "POST /node/69c7f45fbe430e18c5d17492 HTTP/1.0" 202 Accepted 2026-03-30 15:31:50,554 - lava_callback - INFO - Starting async job submit for node 69c7f45fbe430e18c5d17492 2026-03-30 15:31:50,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:31:50,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f45fbe430e18c5d17492 HTTP/1.1" 200 None 2026-03-30 15:31:50,571 - lava_callback - DEBUG - Processing node 69c7f45fbe430e18c5d17492: baseline-arm64-baylibre 2026-03-30 15:31:50,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f45fbe430e18c5d17492 2026-03-30 15:31:50,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:31:50,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f45fbe430e18c5d17492 HTTP/1.1" 200 None 2026-03-30 15:31:50,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:31:50,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:31:50,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:31:50,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f45fbe430e18c5d17492 HTTP/1.1" 200 None 2026-03-30 15:31:50,626 - lava_callback - ERROR - Error processing callback for node 69c7f45fbe430e18c5d17492: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37372 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:33:10,658 - lava_callback - INFO - Received callback for node 69c7f4aabe430e18c5d174a3 INFO: 172.21.0.1:59170 - "POST /node/69c7f4aabe430e18c5d174a3 HTTP/1.0" 202 Accepted 2026-03-30 15:33:10,664 - lava_callback - INFO - Starting async job submit for node 69c7f4aabe430e18c5d174a3 2026-03-30 15:33:10,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:33:10,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f4aabe430e18c5d174a3 HTTP/1.1" 200 None 2026-03-30 15:33:10,681 - lava_callback - DEBUG - Processing node 69c7f4aabe430e18c5d174a3: baseline-arm64-baylibre 2026-03-30 15:33:10,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f4aabe430e18c5d174a3 2026-03-30 15:33:10,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:33:10,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f4aabe430e18c5d174a3 HTTP/1.1" 200 None 2026-03-30 15:33:10,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:33:10,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:33:10,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:33:10,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f4aabe430e18c5d174a3 HTTP/1.1" 200 None 2026-03-30 15:33:10,742 - lava_callback - ERROR - Error processing callback for node 69c7f4aabe430e18c5d174a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58570 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:34:30,739 - lava_callback - INFO - Received callback for node 69c7f4f6be430e18c5d174b0 INFO: 172.21.0.1:59566 - "POST /node/69c7f4f6be430e18c5d174b0 HTTP/1.0" 202 Accepted 2026-03-30 15:34:30,745 - lava_callback - INFO - Starting async job submit for node 69c7f4f6be430e18c5d174b0 2026-03-30 15:34:30,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:34:30,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f4f6be430e18c5d174b0 HTTP/1.1" 200 None 2026-03-30 15:34:30,762 - lava_callback - DEBUG - Processing node 69c7f4f6be430e18c5d174b0: baseline-arm64-baylibre 2026-03-30 15:34:30,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f4f6be430e18c5d174b0 2026-03-30 15:34:30,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:34:30,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f4f6be430e18c5d174b0 HTTP/1.1" 200 None 2026-03-30 15:34:30,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:34:30,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:34:30,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:34:30,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f4f6be430e18c5d174b0 HTTP/1.1" 200 None 2026-03-30 15:34:30,834 - lava_callback - ERROR - Error processing callback for node 69c7f4f6be430e18c5d174b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38516 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:35:30,754 - lava_callback - INFO - Received callback for node 69c7f540be430e18c5d174bb INFO: 172.21.0.1:43360 - "POST /node/69c7f540be430e18c5d174bb HTTP/1.0" 202 Accepted 2026-03-30 15:35:30,760 - lava_callback - INFO - Starting async job submit for node 69c7f540be430e18c5d174bb 2026-03-30 15:35:30,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:35:30,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f540be430e18c5d174bb HTTP/1.1" 200 None 2026-03-30 15:35:30,784 - lava_callback - DEBUG - Processing node 69c7f540be430e18c5d174bb: baseline-arm64-baylibre 2026-03-30 15:35:30,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f540be430e18c5d174bb 2026-03-30 15:35:30,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:35:30,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f540be430e18c5d174bb HTTP/1.1" 200 None 2026-03-30 15:35:30,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:35:30,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:35:30,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:35:30,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f540be430e18c5d174bb HTTP/1.1" 200 None 2026-03-30 15:35:30,855 - lava_callback - ERROR - Error processing callback for node 69c7f540be430e18c5d174bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34614 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:36:51,042 - lava_callback - INFO - Received callback for node 69c7f58dbe430e18c5d174c6 INFO: 172.21.0.1:40668 - "POST /node/69c7f58dbe430e18c5d174c6 HTTP/1.0" 202 Accepted 2026-03-30 15:36:51,048 - lava_callback - INFO - Starting async job submit for node 69c7f58dbe430e18c5d174c6 2026-03-30 15:36:51,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:36:51,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f58dbe430e18c5d174c6 HTTP/1.1" 200 None 2026-03-30 15:36:51,075 - lava_callback - DEBUG - Processing node 69c7f58dbe430e18c5d174c6: baseline-arm64-baylibre 2026-03-30 15:36:51,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f58dbe430e18c5d174c6 2026-03-30 15:36:51,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:36:51,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f58dbe430e18c5d174c6 HTTP/1.1" 200 None 2026-03-30 15:36:51,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:36:51,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:36:51,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:36:51,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f58dbe430e18c5d174c6 HTTP/1.1" 200 None 2026-03-30 15:36:51,140 - lava_callback - ERROR - Error processing callback for node 69c7f58dbe430e18c5d174c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52448 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:38:10,981 - lava_callback - INFO - Received callback for node 69c7f5d7be430e18c5d174cf INFO: 172.21.0.1:44806 - "POST /node/69c7f5d7be430e18c5d174cf HTTP/1.0" 202 Accepted 2026-03-30 15:38:10,987 - lava_callback - INFO - Starting async job submit for node 69c7f5d7be430e18c5d174cf 2026-03-30 15:38:10,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:38:11,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f5d7be430e18c5d174cf HTTP/1.1" 200 None 2026-03-30 15:38:11,004 - lava_callback - DEBUG - Processing node 69c7f5d7be430e18c5d174cf: baseline-arm64-baylibre 2026-03-30 15:38:11,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f5d7be430e18c5d174cf 2026-03-30 15:38:11,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:38:11,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f5d7be430e18c5d174cf HTTP/1.1" 200 None 2026-03-30 15:38:11,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:38:11,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:38:11,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:38:11,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f5d7be430e18c5d174cf HTTP/1.1" 200 None 2026-03-30 15:38:11,060 - lava_callback - ERROR - Error processing callback for node 69c7f5d7be430e18c5d174cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42834 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:39:31,130 - lava_callback - INFO - Received callback for node 69c7f61fbe430e18c5d174d7 INFO: 172.21.0.1:51688 - "POST /node/69c7f61fbe430e18c5d174d7 HTTP/1.0" 202 Accepted 2026-03-30 15:39:31,136 - lava_callback - INFO - Starting async job submit for node 69c7f61fbe430e18c5d174d7 2026-03-30 15:39:31,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:39:31,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f61fbe430e18c5d174d7 HTTP/1.1" 200 None 2026-03-30 15:39:31,155 - lava_callback - DEBUG - Processing node 69c7f61fbe430e18c5d174d7: baseline-arm64-baylibre 2026-03-30 15:39:31,156 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f61fbe430e18c5d174d7 2026-03-30 15:39:31,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:39:31,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f61fbe430e18c5d174d7 HTTP/1.1" 200 None 2026-03-30 15:39:31,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:39:31,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:39:31,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:39:31,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f61fbe430e18c5d174d7 HTTP/1.1" 200 None 2026-03-30 15:39:31,210 - lava_callback - ERROR - Error processing callback for node 69c7f61fbe430e18c5d174d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:40:43,071 - lava_callback - INFO - Received callback for node 69c7f675be430e18c5d174e5 INFO: 172.21.0.1:35254 - "POST /node/69c7f675be430e18c5d174e5 HTTP/1.0" 202 Accepted 2026-03-30 15:40:43,078 - lava_callback - INFO - Starting async job submit for node 69c7f675be430e18c5d174e5 2026-03-30 15:40:43,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:40:43,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f675be430e18c5d174e5 HTTP/1.1" 200 None 2026-03-30 15:40:43,099 - lava_callback - DEBUG - Processing node 69c7f675be430e18c5d174e5: baseline-arm64-clabbe 2026-03-30 15:40:43,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f675be430e18c5d174e5 2026-03-30 15:40:43,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:40:43,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f675be430e18c5d174e5 HTTP/1.1" 200 None 2026-03-30 15:40:43,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:40:43,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:40:43,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:40:43,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f675be430e18c5d174e5 HTTP/1.1" 200 None 2026-03-30 15:40:43,156 - lava_callback - ERROR - Error processing callback for node 69c7f675be430e18c5d174e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53376 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:41:51,188 - lava_callback - INFO - Received callback for node 69c7f6b8be430e18c5d174ee INFO: 172.21.0.1:39928 - "POST /node/69c7f6b8be430e18c5d174ee HTTP/1.0" 202 Accepted 2026-03-30 15:41:51,195 - lava_callback - INFO - Starting async job submit for node 69c7f6b8be430e18c5d174ee 2026-03-30 15:41:51,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:41:51,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f6b8be430e18c5d174ee HTTP/1.1" 200 None 2026-03-30 15:41:51,211 - lava_callback - DEBUG - Processing node 69c7f6b8be430e18c5d174ee: baseline-arm64-baylibre 2026-03-30 15:41:51,211 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f6b8be430e18c5d174ee 2026-03-30 15:41:51,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:41:51,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f6b8be430e18c5d174ee HTTP/1.1" 200 None 2026-03-30 15:41:51,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:41:51,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:41:51,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:41:51,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f6b8be430e18c5d174ee HTTP/1.1" 200 None 2026-03-30 15:41:51,264 - lava_callback - ERROR - Error processing callback for node 69c7f6b8be430e18c5d174ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50816 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:43:11,303 - lava_callback - INFO - Received callback for node 69c7f705be430e18c5d174f9 INFO: 172.21.0.1:37234 - "POST /node/69c7f705be430e18c5d174f9 HTTP/1.0" 202 Accepted 2026-03-30 15:43:11,309 - lava_callback - INFO - Starting async job submit for node 69c7f705be430e18c5d174f9 2026-03-30 15:43:11,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:43:11,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f705be430e18c5d174f9 HTTP/1.1" 200 None 2026-03-30 15:43:11,324 - lava_callback - DEBUG - Processing node 69c7f705be430e18c5d174f9: baseline-arm64-baylibre 2026-03-30 15:43:11,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f705be430e18c5d174f9 2026-03-30 15:43:11,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:43:11,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f705be430e18c5d174f9 HTTP/1.1" 200 None 2026-03-30 15:43:11,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:43:11,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:43:11,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:43:11,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f705be430e18c5d174f9 HTTP/1.1" 200 None 2026-03-30 15:43:11,397 - lava_callback - ERROR - Error processing callback for node 69c7f705be430e18c5d174f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:44:23,283 - lava_callback - INFO - Received callback for node 69c7f754be430e18c5d17509 INFO: 172.21.0.1:59668 - "POST /node/69c7f754be430e18c5d17509 HTTP/1.0" 202 Accepted 2026-03-30 15:44:23,289 - lava_callback - INFO - Starting async job submit for node 69c7f754be430e18c5d17509 2026-03-30 15:44:23,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:44:23,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f754be430e18c5d17509 HTTP/1.1" 200 None 2026-03-30 15:44:23,313 - lava_callback - DEBUG - Processing node 69c7f754be430e18c5d17509: baseline-arm64-clabbe 2026-03-30 15:44:23,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f754be430e18c5d17509 2026-03-30 15:44:23,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:44:23,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f754be430e18c5d17509 HTTP/1.1" 200 None 2026-03-30 15:44:23,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:44:23,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:44:23,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:44:23,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f754be430e18c5d17509 HTTP/1.1" 200 None 2026-03-30 15:44:23,381 - lava_callback - ERROR - Error processing callback for node 69c7f754be430e18c5d17509: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41048 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:45:43,290 - lava_callback - INFO - Received callback for node 69c7f79fbe430e18c5d17511 INFO: 172.21.0.1:51440 - "POST /node/69c7f79fbe430e18c5d17511 HTTP/1.0" 202 Accepted 2026-03-30 15:45:43,296 - lava_callback - INFO - Starting async job submit for node 69c7f79fbe430e18c5d17511 2026-03-30 15:45:43,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:45:43,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f79fbe430e18c5d17511 HTTP/1.1" 200 None 2026-03-30 15:45:43,312 - lava_callback - DEBUG - Processing node 69c7f79fbe430e18c5d17511: baseline-arm64-clabbe 2026-03-30 15:45:43,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f79fbe430e18c5d17511 2026-03-30 15:45:43,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:45:43,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f79fbe430e18c5d17511 HTTP/1.1" 200 None 2026-03-30 15:45:43,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:45:43,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:45:43,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:45:43,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f79fbe430e18c5d17511 HTTP/1.1" 200 None 2026-03-30 15:45:43,403 - lava_callback - ERROR - Error processing callback for node 69c7f79fbe430e18c5d17511: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49906 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:49922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39370 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:46:51,428 - lava_callback - INFO - Received callback for node 69c7f7e4be430e18c5d1751c INFO: 172.21.0.1:51734 - "POST /node/69c7f7e4be430e18c5d1751c HTTP/1.0" 202 Accepted 2026-03-30 15:46:51,434 - lava_callback - INFO - Starting async job submit for node 69c7f7e4be430e18c5d1751c 2026-03-30 15:46:51,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:46:51,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f7e4be430e18c5d1751c HTTP/1.1" 200 None 2026-03-30 15:46:51,451 - lava_callback - DEBUG - Processing node 69c7f7e4be430e18c5d1751c: baseline-arm64-baylibre 2026-03-30 15:46:51,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f7e4be430e18c5d1751c 2026-03-30 15:46:51,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:46:51,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f7e4be430e18c5d1751c HTTP/1.1" 200 None 2026-03-30 15:46:51,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:46:51,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:46:51,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:46:51,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f7e4be430e18c5d1751c HTTP/1.1" 200 None 2026-03-30 15:46:51,524 - lava_callback - ERROR - Error processing callback for node 69c7f7e4be430e18c5d1751c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38004 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:48:23,370 - lava_callback - INFO - Received callback for node 69c7f834be430e18c5d17528 INFO: 172.21.0.1:34920 - "POST /node/69c7f834be430e18c5d17528 HTTP/1.0" 202 Accepted 2026-03-30 15:48:23,378 - lava_callback - INFO - Starting async job submit for node 69c7f834be430e18c5d17528 2026-03-30 15:48:23,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:48:23,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f834be430e18c5d17528 HTTP/1.1" 200 None 2026-03-30 15:48:23,401 - lava_callback - DEBUG - Processing node 69c7f834be430e18c5d17528: baseline-arm64-clabbe 2026-03-30 15:48:23,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f834be430e18c5d17528 2026-03-30 15:48:23,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:48:23,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f834be430e18c5d17528 HTTP/1.1" 200 None 2026-03-30 15:48:23,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:48:23,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:48:23,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:48:23,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f834be430e18c5d17528 HTTP/1.1" 200 None 2026-03-30 15:48:23,474 - lava_callback - ERROR - Error processing callback for node 69c7f834be430e18c5d17528: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57474 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:49:31,698 - lava_callback - INFO - Received callback for node 69c7f879be430e18c5d17530 INFO: 172.21.0.1:57488 - "POST /node/69c7f879be430e18c5d17530 HTTP/1.0" 202 Accepted 2026-03-30 15:49:31,704 - lava_callback - INFO - Starting async job submit for node 69c7f879be430e18c5d17530 2026-03-30 15:49:31,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:49:31,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f879be430e18c5d17530 HTTP/1.1" 200 None 2026-03-30 15:49:31,720 - lava_callback - DEBUG - Processing node 69c7f879be430e18c5d17530: baseline-arm64-baylibre 2026-03-30 15:49:31,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f879be430e18c5d17530 2026-03-30 15:49:31,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:49:31,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f879be430e18c5d17530 HTTP/1.1" 200 None 2026-03-30 15:49:31,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:49:31,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:49:31,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:49:31,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f879be430e18c5d17530 HTTP/1.1" 200 None 2026-03-30 15:49:31,787 - lava_callback - ERROR - Error processing callback for node 69c7f879be430e18c5d17530: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57196 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:50:43,488 - lava_callback - INFO - Received callback for node 69c7f8c9be430e18c5d1753f INFO: 172.21.0.1:55608 - "POST /node/69c7f8c9be430e18c5d1753f HTTP/1.0" 202 Accepted 2026-03-30 15:50:43,494 - lava_callback - INFO - Starting async job submit for node 69c7f8c9be430e18c5d1753f 2026-03-30 15:50:43,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:50:43,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f8c9be430e18c5d1753f HTTP/1.1" 200 None 2026-03-30 15:50:43,511 - lava_callback - DEBUG - Processing node 69c7f8c9be430e18c5d1753f: baseline-arm64-clabbe 2026-03-30 15:50:43,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f8c9be430e18c5d1753f 2026-03-30 15:50:43,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:50:43,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f8c9be430e18c5d1753f HTTP/1.1" 200 None 2026-03-30 15:50:43,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:50:43,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:50:43,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:50:43,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f8c9be430e18c5d1753f HTTP/1.1" 200 None 2026-03-30 15:50:43,602 - lava_callback - ERROR - Error processing callback for node 69c7f8c9be430e18c5d1753f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:52:03,522 - lava_callback - INFO - Received callback for node 69c7f915be430e18c5d1754b INFO: 172.21.0.1:59082 - "POST /node/69c7f915be430e18c5d1754b HTTP/1.0" 202 Accepted 2026-03-30 15:52:03,528 - lava_callback - INFO - Starting async job submit for node 69c7f915be430e18c5d1754b 2026-03-30 15:52:03,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:52:03,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f915be430e18c5d1754b HTTP/1.1" 200 None 2026-03-30 15:52:03,544 - lava_callback - DEBUG - Processing node 69c7f915be430e18c5d1754b: baseline-arm64-clabbe 2026-03-30 15:52:03,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f915be430e18c5d1754b 2026-03-30 15:52:03,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:52:03,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f915be430e18c5d1754b HTTP/1.1" 200 None 2026-03-30 15:52:03,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:52:03,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:52:03,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:52:03,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f915be430e18c5d1754b HTTP/1.1" 200 None 2026-03-30 15:52:03,617 - lava_callback - ERROR - Error processing callback for node 69c7f915be430e18c5d1754b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33914 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:53:23,608 - lava_callback - INFO - Received callback for node 69c7f960be430e18c5d17555 INFO: 172.21.0.1:49794 - "POST /node/69c7f960be430e18c5d17555 HTTP/1.0" 202 Accepted 2026-03-30 15:53:23,618 - lava_callback - INFO - Starting async job submit for node 69c7f960be430e18c5d17555 2026-03-30 15:53:23,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:53:23,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f960be430e18c5d17555 HTTP/1.1" 200 None 2026-03-30 15:53:23,652 - lava_callback - DEBUG - Processing node 69c7f960be430e18c5d17555: baseline-arm64-clabbe 2026-03-30 15:53:23,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f960be430e18c5d17555 2026-03-30 15:53:23,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:53:23,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f960be430e18c5d17555 HTTP/1.1" 200 None 2026-03-30 15:53:23,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:53:23,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:53:23,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:53:23,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f960be430e18c5d17555 HTTP/1.1" 200 None 2026-03-30 15:53:23,748 - lava_callback - ERROR - Error processing callback for node 69c7f960be430e18c5d17555: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33390 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:54:31,931 - lava_callback - INFO - Received callback for node 69c7f9a4be430e18c5d1755e INFO: 172.21.0.1:33404 - "POST /node/69c7f9a4be430e18c5d1755e HTTP/1.0" 202 Accepted 2026-03-30 15:54:31,937 - lava_callback - INFO - Starting async job submit for node 69c7f9a4be430e18c5d1755e 2026-03-30 15:54:31,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:54:31,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f9a4be430e18c5d1755e HTTP/1.1" 200 None 2026-03-30 15:54:31,954 - lava_callback - DEBUG - Processing node 69c7f9a4be430e18c5d1755e: baseline-arm64-baylibre 2026-03-30 15:54:31,954 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f9a4be430e18c5d1755e 2026-03-30 15:54:31,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:54:31,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f9a4be430e18c5d1755e HTTP/1.1" 200 None 2026-03-30 15:54:31,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:54:31,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 15:54:31,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:54:32,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f9a4be430e18c5d1755e HTTP/1.1" 200 None 2026-03-30 15:54:32,013 - lava_callback - ERROR - Error processing callback for node 69c7f9a4be430e18c5d1755e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50274 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:55:43,718 - lava_callback - INFO - Received callback for node 69c7f9f8be430e18c5d1756a INFO: 172.21.0.1:57586 - "POST /node/69c7f9f8be430e18c5d1756a HTTP/1.0" 202 Accepted 2026-03-30 15:55:43,723 - lava_callback - INFO - Starting async job submit for node 69c7f9f8be430e18c5d1756a 2026-03-30 15:55:43,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:55:43,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f9f8be430e18c5d1756a HTTP/1.1" 200 None 2026-03-30 15:55:43,754 - lava_callback - DEBUG - Processing node 69c7f9f8be430e18c5d1756a: baseline-arm64-clabbe 2026-03-30 15:55:43,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7f9f8be430e18c5d1756a 2026-03-30 15:55:43,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:55:43,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7f9f8be430e18c5d1756a HTTP/1.1" 200 None 2026-03-30 15:55:43,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:55:43,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:55:43,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:55:43,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7f9f8be430e18c5d1756a HTTP/1.1" 200 None 2026-03-30 15:55:43,811 - lava_callback - ERROR - Error processing callback for node 69c7f9f8be430e18c5d1756a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38380 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:57:03,771 - lava_callback - INFO - Received callback for node 69c7fa45be430e18c5d17578 INFO: 172.21.0.1:40534 - "POST /node/69c7fa45be430e18c5d17578 HTTP/1.0" 202 Accepted 2026-03-30 15:57:03,777 - lava_callback - INFO - Starting async job submit for node 69c7fa45be430e18c5d17578 2026-03-30 15:57:03,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:57:03,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fa45be430e18c5d17578 HTTP/1.1" 200 None 2026-03-30 15:57:03,801 - lava_callback - DEBUG - Processing node 69c7fa45be430e18c5d17578: baseline-arm64-clabbe 2026-03-30 15:57:03,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fa45be430e18c5d17578 2026-03-30 15:57:03,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:57:03,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fa45be430e18c5d17578 HTTP/1.1" 200 None 2026-03-30 15:57:03,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:57:03,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:57:03,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:57:03,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fa45be430e18c5d17578 HTTP/1.1" 200 None 2026-03-30 15:57:03,853 - lava_callback - ERROR - Error processing callback for node 69c7fa45be430e18c5d17578: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:58:23,835 - lava_callback - INFO - Received callback for node 69c7fa91be430e18c5d1757e INFO: 172.21.0.1:42874 - "POST /node/69c7fa91be430e18c5d1757e HTTP/1.0" 202 Accepted 2026-03-30 15:58:23,840 - lava_callback - INFO - Starting async job submit for node 69c7fa91be430e18c5d1757e 2026-03-30 15:58:23,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:58:23,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fa91be430e18c5d1757e HTTP/1.1" 200 None 2026-03-30 15:58:23,857 - lava_callback - DEBUG - Processing node 69c7fa91be430e18c5d1757e: baseline-arm64-clabbe 2026-03-30 15:58:23,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fa91be430e18c5d1757e 2026-03-30 15:58:23,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:58:23,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fa91be430e18c5d1757e HTTP/1.1" 200 None 2026-03-30 15:58:23,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:58:23,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 15:58:23,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:58:23,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fa91be430e18c5d1757e HTTP/1.1" 200 None 2026-03-30 15:58:23,909 - lava_callback - ERROR - Error processing callback for node 69c7fa91be430e18c5d1757e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40996 - "GET / HTTP/1.0" 200 OK 2026-03-30 15:59:32,070 - lava_callback - INFO - Received callback for node 69c7fad4be430e18c5d1758c INFO: 172.21.0.1:41002 - "POST /node/69c7fad4be430e18c5d1758c HTTP/1.0" 202 Accepted 2026-03-30 15:59:32,075 - lava_callback - INFO - Starting async job submit for node 69c7fad4be430e18c5d1758c 2026-03-30 15:59:32,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:59:32,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fad4be430e18c5d1758c HTTP/1.1" 200 None 2026-03-30 15:59:32,092 - lava_callback - DEBUG - Processing node 69c7fad4be430e18c5d1758c: baseline-arm64-baylibre 2026-03-30 15:59:32,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fad4be430e18c5d1758c 2026-03-30 15:59:32,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:59:32,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fad4be430e18c5d1758c HTTP/1.1" 200 None 2026-03-30 15:59:32,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:59:32,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 15:59:32,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 15:59:32,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fad4be430e18c5d1758c HTTP/1.1" 200 None 2026-03-30 15:59:32,158 - lava_callback - ERROR - Error processing callback for node 69c7fad4be430e18c5d1758c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45436 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:00:43,967 - lava_callback - INFO - Received callback for node 69c7fb27be430e18c5d17592 INFO: 172.21.0.1:33992 - "POST /node/69c7fb27be430e18c5d17592 HTTP/1.0" 202 Accepted 2026-03-30 16:00:43,972 - lava_callback - INFO - Starting async job submit for node 69c7fb27be430e18c5d17592 2026-03-30 16:00:43,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:00:43,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fb27be430e18c5d17592 HTTP/1.1" 200 None 2026-03-30 16:00:43,989 - lava_callback - DEBUG - Processing node 69c7fb27be430e18c5d17592: baseline-arm64-clabbe 2026-03-30 16:00:43,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fb27be430e18c5d17592 2026-03-30 16:00:43,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:00:44,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fb27be430e18c5d17592 HTTP/1.1" 200 None 2026-03-30 16:00:44,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:00:44,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 16:00:44,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:00:44,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fb27be430e18c5d17592 HTTP/1.1" 200 None 2026-03-30 16:00:44,063 - lava_callback - ERROR - Error processing callback for node 69c7fb27be430e18c5d17592: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:01:52,152 - lava_callback - INFO - Received callback for node 69c7fb6dbe430e18c5d175a0 INFO: 172.21.0.1:58348 - "POST /node/69c7fb6dbe430e18c5d175a0 HTTP/1.0" 202 Accepted 2026-03-30 16:01:52,157 - lava_callback - INFO - Starting async job submit for node 69c7fb6dbe430e18c5d175a0 2026-03-30 16:01:52,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:01:52,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fb6dbe430e18c5d175a0 HTTP/1.1" 200 None 2026-03-30 16:01:52,173 - lava_callback - DEBUG - Processing node 69c7fb6dbe430e18c5d175a0: baseline-arm64-baylibre 2026-03-30 16:01:52,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fb6dbe430e18c5d175a0 2026-03-30 16:01:52,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:01:52,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fb6dbe430e18c5d175a0 HTTP/1.1" 200 None 2026-03-30 16:01:52,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:01:52,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:01:52,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:01:52,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fb6dbe430e18c5d175a0 HTTP/1.1" 200 None 2026-03-30 16:01:52,249 - lava_callback - ERROR - Error processing callback for node 69c7fb6dbe430e18c5d175a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40118 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:03:24,052 - lava_callback - INFO - Received callback for node 69c7fbbdbe430e18c5d175aa INFO: 172.21.0.1:54358 - "POST /node/69c7fbbdbe430e18c5d175aa HTTP/1.0" 202 Accepted 2026-03-30 16:03:24,057 - lava_callback - INFO - Starting async job submit for node 69c7fbbdbe430e18c5d175aa 2026-03-30 16:03:24,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:03:24,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fbbdbe430e18c5d175aa HTTP/1.1" 200 None 2026-03-30 16:03:24,074 - lava_callback - DEBUG - Processing node 69c7fbbdbe430e18c5d175aa: baseline-arm64-clabbe 2026-03-30 16:03:24,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fbbdbe430e18c5d175aa 2026-03-30 16:03:24,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:03:24,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fbbdbe430e18c5d175aa HTTP/1.1" 200 None 2026-03-30 16:03:24,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:03:24,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-30 16:03:24,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:03:24,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fbbdbe430e18c5d175aa HTTP/1.1" 200 None 2026-03-30 16:03:24,126 - lava_callback - ERROR - Error processing callback for node 69c7fbbdbe430e18c5d175aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57782 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:04:32,536 - lava_callback - INFO - Received callback for node 69c7fc04be430e18c5d175b6 INFO: 172.21.0.1:57784 - "POST /node/69c7fc04be430e18c5d175b6 HTTP/1.0" 202 Accepted 2026-03-30 16:04:32,541 - lava_callback - INFO - Starting async job submit for node 69c7fc04be430e18c5d175b6 2026-03-30 16:04:32,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:04:32,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fc04be430e18c5d175b6 HTTP/1.1" 200 None 2026-03-30 16:04:32,560 - lava_callback - DEBUG - Processing node 69c7fc04be430e18c5d175b6: baseline-arm64-baylibre 2026-03-30 16:04:32,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fc04be430e18c5d175b6 2026-03-30 16:04:32,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:04:32,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fc04be430e18c5d175b6 HTTP/1.1" 200 None 2026-03-30 16:04:32,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:04:32,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:04:32,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:04:32,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fc04be430e18c5d175b6 HTTP/1.1" 200 None 2026-03-30 16:04:32,627 - lava_callback - ERROR - Error processing callback for node 69c7fc04be430e18c5d175b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34072 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:05:52,388 - lava_callback - INFO - Received callback for node 69c7fc4cbe430e18c5d175bc INFO: 172.21.0.1:40790 - "POST /node/69c7fc4cbe430e18c5d175bc HTTP/1.0" 202 Accepted 2026-03-30 16:05:52,393 - lava_callback - INFO - Starting async job submit for node 69c7fc4cbe430e18c5d175bc 2026-03-30 16:05:52,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:05:52,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fc4cbe430e18c5d175bc HTTP/1.1" 200 None 2026-03-30 16:05:52,410 - lava_callback - DEBUG - Processing node 69c7fc4cbe430e18c5d175bc: baseline-arm64-baylibre 2026-03-30 16:05:52,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fc4cbe430e18c5d175bc 2026-03-30 16:05:52,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:05:52,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fc4cbe430e18c5d175bc HTTP/1.1" 200 None 2026-03-30 16:05:52,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:05:52,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:05:52,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:05:52,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fc4cbe430e18c5d175bc HTTP/1.1" 200 None 2026-03-30 16:05:52,461 - lava_callback - ERROR - Error processing callback for node 69c7fc4cbe430e18c5d175bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42064 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:06:52,406 - lava_callback - INFO - Received callback for node 69c7fc98be430e18c5d175cc INFO: 172.21.0.1:40432 - "POST /node/69c7fc98be430e18c5d175cc HTTP/1.0" 202 Accepted 2026-03-30 16:06:52,411 - lava_callback - INFO - Starting async job submit for node 69c7fc98be430e18c5d175cc 2026-03-30 16:06:52,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:06:52,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fc98be430e18c5d175cc HTTP/1.1" 200 None 2026-03-30 16:06:52,428 - lava_callback - DEBUG - Processing node 69c7fc98be430e18c5d175cc: baseline-arm64-baylibre 2026-03-30 16:06:52,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fc98be430e18c5d175cc 2026-03-30 16:06:52,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:06:52,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fc98be430e18c5d175cc HTTP/1.1" 200 None 2026-03-30 16:06:52,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:06:52,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:06:52,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:06:52,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fc98be430e18c5d175cc HTTP/1.1" 200 None 2026-03-30 16:06:52,487 - lava_callback - ERROR - Error processing callback for node 69c7fc98be430e18c5d175cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36288 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:08:12,980 - lava_callback - INFO - Received callback for node 69c7fcdfbe430e18c5d175d2 INFO: 172.21.0.1:46150 - "POST /node/69c7fcdfbe430e18c5d175d2 HTTP/1.0" 202 Accepted 2026-03-30 16:08:12,986 - lava_callback - INFO - Starting async job submit for node 69c7fcdfbe430e18c5d175d2 2026-03-30 16:08:12,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:08:13,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fcdfbe430e18c5d175d2 HTTP/1.1" 200 None 2026-03-30 16:08:13,011 - lava_callback - DEBUG - Processing node 69c7fcdfbe430e18c5d175d2: baseline-arm64-baylibre 2026-03-30 16:08:13,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fcdfbe430e18c5d175d2 2026-03-30 16:08:13,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:08:13,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fcdfbe430e18c5d175d2 HTTP/1.1" 200 None 2026-03-30 16:08:13,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:08:13,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:08:13,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:08:13,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fcdfbe430e18c5d175d2 HTTP/1.1" 200 None 2026-03-30 16:08:13,072 - lava_callback - ERROR - Error processing callback for node 69c7fcdfbe430e18c5d175d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60436 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:09:24,299 - lava_callback - INFO - Received callback for node 69c7fd31be430e18c5d175e5 INFO: 172.21.0.1:54650 - "POST /node/69c7fd31be430e18c5d175e5 HTTP/1.0" 202 Accepted 2026-03-30 16:09:24,305 - lava_callback - INFO - Starting async job submit for node 69c7fd31be430e18c5d175e5 2026-03-30 16:09:24,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:09:24,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fd31be430e18c5d175e5 HTTP/1.1" 200 None 2026-03-30 16:09:24,321 - lava_callback - DEBUG - Processing node 69c7fd31be430e18c5d175e5: baseline-arm64-clabbe 2026-03-30 16:09:24,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fd31be430e18c5d175e5 2026-03-30 16:09:24,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:09:24,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fd31be430e18c5d175e5 HTTP/1.1" 200 None 2026-03-30 16:09:24,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:09:24,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:09:24,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:09:24,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fd31be430e18c5d175e5 HTTP/1.1" 200 None 2026-03-30 16:09:24,372 - lava_callback - ERROR - Error processing callback for node 69c7fd31be430e18c5d175e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56476 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:10:44,410 - lava_callback - INFO - Received callback for node 69c7fd7cbe430e18c5d175ee INFO: 172.21.0.1:45756 - "POST /node/69c7fd7cbe430e18c5d175ee HTTP/1.0" 202 Accepted 2026-03-30 16:10:44,415 - lava_callback - INFO - Starting async job submit for node 69c7fd7cbe430e18c5d175ee 2026-03-30 16:10:44,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:10:44,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fd7cbe430e18c5d175ee HTTP/1.1" 200 None 2026-03-30 16:10:44,431 - lava_callback - DEBUG - Processing node 69c7fd7cbe430e18c5d175ee: baseline-arm64-clabbe 2026-03-30 16:10:44,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fd7cbe430e18c5d175ee 2026-03-30 16:10:44,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:10:44,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fd7cbe430e18c5d175ee HTTP/1.1" 200 None 2026-03-30 16:10:44,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:10:44,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:10:44,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:10:44,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fd7cbe430e18c5d175ee HTTP/1.1" 200 None 2026-03-30 16:10:44,485 - lava_callback - ERROR - Error processing callback for node 69c7fd7cbe430e18c5d175ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42626 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:11:52,779 - lava_callback - INFO - Received callback for node 69c7fdc1be430e18c5d175fa INFO: 172.21.0.1:43136 - "POST /node/69c7fdc1be430e18c5d175fa HTTP/1.0" 202 Accepted 2026-03-30 16:11:52,785 - lava_callback - INFO - Starting async job submit for node 69c7fdc1be430e18c5d175fa 2026-03-30 16:11:52,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:11:52,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fdc1be430e18c5d175fa HTTP/1.1" 200 None 2026-03-30 16:11:52,801 - lava_callback - DEBUG - Processing node 69c7fdc1be430e18c5d175fa: baseline-arm64-baylibre 2026-03-30 16:11:52,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fdc1be430e18c5d175fa 2026-03-30 16:11:52,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:11:52,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fdc1be430e18c5d175fa HTTP/1.1" 200 None 2026-03-30 16:11:52,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:11:52,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:11:52,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:11:52,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fdc1be430e18c5d175fa HTTP/1.1" 200 None 2026-03-30 16:11:52,860 - lava_callback - ERROR - Error processing callback for node 69c7fdc1be430e18c5d175fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40166 - "GET / HTTP/1.0" 200 OK 2026-03-30 16:13:24,497 - lava_callback - INFO - Received callback for node 69c7fe11be430e18c5d17608 INFO: 172.21.0.1:40130 - "POST /node/69c7fe11be430e18c5d17608 HTTP/1.0" 202 Accepted 2026-03-30 16:13:24,504 - lava_callback - INFO - Starting async job submit for node 69c7fe11be430e18c5d17608 2026-03-30 16:13:24,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:13:24,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fe11be430e18c5d17608 HTTP/1.1" 200 None 2026-03-30 16:13:24,523 - lava_callback - DEBUG - Processing node 69c7fe11be430e18c5d17608: baseline-arm64-clabbe 2026-03-30 16:13:24,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7fe11be430e18c5d17608 2026-03-30 16:13:24,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:13:24,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7fe11be430e18c5d17608 HTTP/1.1" 200 None 2026-03-30 16:13:24,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:13:24,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-30 16:13:24,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 16:13:24,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7fe11be430e18c5d17608 HTTP/1.1" 200 None 2026-03-30 16:13:24,600 - lava_callback - ERROR - Error processing callback for node 69c7fe11be430e18c5d17608: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40144 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54970 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]