INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:46730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37728 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:31:28,876 - lava_callback - INFO - Received callback for node 69be56f2448e074b3ffe1675 2026-03-21 08:31:28,888 - lava_callback - INFO - Starting async job submit for node 69be56f2448e074b3ffe1675 INFO: 172.20.0.1:37736 - "POST /node/69be56f2448e074b3ffe1675 HTTP/1.0" 202 Accepted 2026-03-21 08:31:28,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:31:28,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be56f2448e074b3ffe1675 HTTP/1.1" 200 None 2026-03-21 08:31:28,934 - lava_callback - DEBUG - Processing node 69be56f2448e074b3ffe1675: baseline-arm 2026-03-21 08:31:29,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:31:29,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:31:29,478 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be56f2448e074b3ffe1675/log.txt.gz 2026-03-21 08:31:29,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:31:29,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:31:29,814 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be56f2448e074b3ffe1675/lava-logs.yaml 2026-03-21 08:31:29,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:31:29,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:31:29,994 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be56f2448e074b3ffe1675/lava_callback.json.gz 2026-03-21 08:31:30,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:31:30,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be56f2448e074b3ffe1675 HTTP/1.1" 200 None 2026-03-21 08:31:30,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:31:30,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5456448e074b3ffe1633 HTTP/1.1" 200 None 2026-03-21 08:31:30,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:31:30,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be56f2448e074b3ffe1675 HTTP/1.1" 200 None 2026-03-21 08:31:30,088 - lava_callback - INFO - Completed processing callback for node 69be56f2448e074b3ffe1675 INFO: 172.20.0.1:57734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:32:00,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:32:00,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37332 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:38:02,844 - lava_callback - INFO - Received callback for node 69bbb5ddec3f435d20d51afd INFO: 172.20.0.1:50958 - "POST /node/69bbb5ddec3f435d20d51afd HTTP/1.0" 202 Accepted 2026-03-21 08:38:02,850 - lava_callback - INFO - Starting async job submit for node 69bbb5ddec3f435d20d51afd 2026-03-21 08:38:02,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:02,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb5ddec3f435d20d51afd HTTP/1.1" 200 None 2026-03-21 08:38:02,866 - lava_callback - DEBUG - Processing node 69bbb5ddec3f435d20d51afd: baseline-arm-baylibre 2026-03-21 08:38:02,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb5ddec3f435d20d51afd 2026-03-21 08:38:02,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:02,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb5ddec3f435d20d51afd HTTP/1.1" 200 None 2026-03-21 08:38:02,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:02,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:38:02,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:02,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb5ddec3f435d20d51afd HTTP/1.1" 200 None 2026-03-21 08:38:02,938 - lava_callback - ERROR - Error processing callback for node 69bbb5ddec3f435d20d51afd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59362 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:38:42,663 - lava_callback - INFO - Received callback for node 69bbb605ec3f435d20d51b16 INFO: 172.20.0.1:56328 - "POST /node/69bbb605ec3f435d20d51b16 HTTP/1.0" 202 Accepted 2026-03-21 08:38:42,672 - lava_callback - INFO - Starting async job submit for node 69bbb605ec3f435d20d51b16 2026-03-21 08:38:42,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:42,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb605ec3f435d20d51b16 HTTP/1.1" 200 None 2026-03-21 08:38:42,707 - lava_callback - DEBUG - Processing node 69bbb605ec3f435d20d51b16: baseline-arm-baylibre 2026-03-21 08:38:42,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb605ec3f435d20d51b16 2026-03-21 08:38:42,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:42,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb605ec3f435d20d51b16 HTTP/1.1" 200 None 2026-03-21 08:38:42,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:42,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:38:42,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:38:42,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb605ec3f435d20d51b16 HTTP/1.1" 200 None 2026-03-21 08:38:42,824 - lava_callback - ERROR - Error processing callback for node 69bbb605ec3f435d20d51b16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60268 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:39:22,929 - lava_callback - INFO - Received callback for node 69bbb628ec3f435d20d51b2a INFO: 172.20.0.1:57646 - "POST /node/69bbb628ec3f435d20d51b2a HTTP/1.0" 202 Accepted 2026-03-21 08:39:22,934 - lava_callback - INFO - Starting async job submit for node 69bbb628ec3f435d20d51b2a 2026-03-21 08:39:22,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:22,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb628ec3f435d20d51b2a HTTP/1.1" 200 None 2026-03-21 08:39:22,950 - lava_callback - DEBUG - Processing node 69bbb628ec3f435d20d51b2a: baseline-arm-baylibre 2026-03-21 08:39:22,951 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb628ec3f435d20d51b2a 2026-03-21 08:39:22,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:22,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb628ec3f435d20d51b2a HTTP/1.1" 200 None 2026-03-21 08:39:22,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:22,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:39:22,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:23,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb628ec3f435d20d51b2a HTTP/1.1" 200 None 2026-03-21 08:39:23,014 - lava_callback - ERROR - Error processing callback for node 69bbb628ec3f435d20d51b2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57648 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:39:28,698 - lava_callback - INFO - Received callback for node 69be5938448e074b3ffe169b INFO: 172.20.0.1:57650 - "POST /node/69be5938448e074b3ffe169b HTTP/1.0" 202 Accepted 2026-03-21 08:39:28,704 - lava_callback - INFO - Starting async job submit for node 69be5938448e074b3ffe169b 2026-03-21 08:39:28,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:28,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5938448e074b3ffe169b HTTP/1.1" 200 None 2026-03-21 08:39:28,723 - lava_callback - DEBUG - Processing node 69be5938448e074b3ffe169b: baseline-arm 2026-03-21 08:39:28,728 - lava_callback - WARNING - Failed to upload log 2026-03-21 08:39:28,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:39:28,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:39:28,848 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be5938448e074b3ffe169b/lava-logs.yaml 2026-03-21 08:39:28,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:39:28,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:39:28,980 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be5938448e074b3ffe169b/lava_callback.json.gz 2026-03-21 08:39:28,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:28,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5938448e074b3ffe169b HTTP/1.1" 200 None 2026-03-21 08:39:28,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:29,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:39:29,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:29,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5938448e074b3ffe169b HTTP/1.1" 200 None 2026-03-21 08:39:29,057 - lava_callback - INFO - Completed processing callback for node 69be5938448e074b3ffe169b 2026-03-21 08:39:30,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:30,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45804 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:39:59,340 - lava_callback - INFO - Received callback for node 69be5939448e074b3ffe169e INFO: 172.20.0.1:45820 - "POST /node/69be5939448e074b3ffe169e HTTP/1.0" 202 Accepted 2026-03-21 08:39:59,390 - lava_callback - INFO - Starting async job submit for node 69be5939448e074b3ffe169e 2026-03-21 08:39:59,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:39:59,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5939448e074b3ffe169e HTTP/1.1" 200 None 2026-03-21 08:39:59,426 - lava_callback - DEBUG - Processing node 69be5939448e074b3ffe169e: baseline-arm 2026-03-21 08:39:59,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:39:59,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:39:59,677 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be5939448e074b3ffe169e/log.txt.gz 2026-03-21 08:39:59,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:39:59,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:39:59,931 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be5939448e074b3ffe169e/lava-logs.yaml 2026-03-21 08:39:59,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:00,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:00,110 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be5939448e074b3ffe169e/lava_callback.json.gz 2026-03-21 08:40:00,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:00,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5939448e074b3ffe169e HTTP/1.1" 200 None 2026-03-21 08:40:00,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:00,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:40:00,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:00,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5939448e074b3ffe169e HTTP/1.1" 200 None 2026-03-21 08:40:00,191 - lava_callback - INFO - Completed processing callback for node 69be5939448e074b3ffe169e 2026-03-21 08:40:02,737 - lava_callback - INFO - Received callback for node 69bbb64dec3f435d20d51b3b INFO: 172.20.0.1:44564 - "POST /node/69bbb64dec3f435d20d51b3b HTTP/1.0" 202 Accepted 2026-03-21 08:40:02,742 - lava_callback - INFO - Starting async job submit for node 69bbb64dec3f435d20d51b3b 2026-03-21 08:40:02,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:02,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb64dec3f435d20d51b3b HTTP/1.1" 200 None 2026-03-21 08:40:02,759 - lava_callback - DEBUG - Processing node 69bbb64dec3f435d20d51b3b: baseline-arm-baylibre 2026-03-21 08:40:02,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb64dec3f435d20d51b3b 2026-03-21 08:40:02,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:02,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb64dec3f435d20d51b3b HTTP/1.1" 200 None 2026-03-21 08:40:02,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:02,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:40:02,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:02,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb64dec3f435d20d51b3b HTTP/1.1" 200 None 2026-03-21 08:40:02,831 - lava_callback - ERROR - Error processing callback for node 69bbb64dec3f435d20d51b3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:40:14,648 - lava_callback - INFO - Received callback for node 69be593c448e074b3ffe16a9 INFO: 172.20.0.1:35222 - "POST /node/69be593c448e074b3ffe16a9 HTTP/1.0" 202 Accepted 2026-03-21 08:40:14,698 - lava_callback - INFO - Starting async job submit for node 69be593c448e074b3ffe16a9 2026-03-21 08:40:14,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:14,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be593c448e074b3ffe16a9 HTTP/1.1" 200 None 2026-03-21 08:40:14,734 - lava_callback - DEBUG - Processing node 69be593c448e074b3ffe16a9: baseline-arm 2026-03-21 08:40:14,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:14,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:14,973 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be593c448e074b3ffe16a9/log.txt.gz 2026-03-21 08:40:15,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:35232 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 08:40:15,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:15,205 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be593c448e074b3ffe16a9/lava-logs.yaml 2026-03-21 08:40:15,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:15,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:15,326 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be593c448e074b3ffe16a9/lava_callback.json.gz 2026-03-21 08:40:15,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:15,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be593c448e074b3ffe16a9 HTTP/1.1" 200 None 2026-03-21 08:40:15,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:15,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:40:15,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:15,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be593c448e074b3ffe16a9 HTTP/1.1" 200 None 2026-03-21 08:40:15,403 - lava_callback - INFO - Completed processing callback for node 69be593c448e074b3ffe16a9 INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:40:22,801 - lava_callback - INFO - Received callback for node 69bbb672ec3f435d20d51b45 INFO: 172.20.0.1:43684 - "POST /node/69bbb672ec3f435d20d51b45 HTTP/1.0" 202 Accepted 2026-03-21 08:40:22,807 - lava_callback - INFO - Starting async job submit for node 69bbb672ec3f435d20d51b45 2026-03-21 08:40:22,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:22,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb672ec3f435d20d51b45 HTTP/1.1" 200 None 2026-03-21 08:40:22,823 - lava_callback - DEBUG - Processing node 69bbb672ec3f435d20d51b45: baseline-arm-baylibre 2026-03-21 08:40:22,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb672ec3f435d20d51b45 2026-03-21 08:40:22,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:22,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb672ec3f435d20d51b45 HTTP/1.1" 200 None 2026-03-21 08:40:22,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:22,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:40:22,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:22,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb672ec3f435d20d51b45 HTTP/1.1" 200 None 2026-03-21 08:40:22,885 - lava_callback - ERROR - Error processing callback for node 69bbb672ec3f435d20d51b45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:40:23,297 - lava_callback - INFO - Received callback for node 69be5961448e074b3ffe1735 INFO: 172.20.0.1:43690 - "POST /node/69be5961448e074b3ffe1735 HTTP/1.0" 202 Accepted 2026-03-21 08:40:23,303 - lava_callback - INFO - Starting async job submit for node 69be5961448e074b3ffe1735 2026-03-21 08:40:23,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:23,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5961448e074b3ffe1735 HTTP/1.1" 200 None 2026-03-21 08:40:23,321 - lava_callback - DEBUG - Processing node 69be5961448e074b3ffe1735: baseline-arm 2026-03-21 08:40:23,326 - lava_callback - WARNING - Failed to upload log 2026-03-21 08:40:23,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:23,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:23,466 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be5961448e074b3ffe1735/lava-logs.yaml 2026-03-21 08:40:23,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:23,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:23,607 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be5961448e074b3ffe1735/lava_callback.json.gz 2026-03-21 08:40:23,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:23,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5961448e074b3ffe1735 HTTP/1.1" 200 None 2026-03-21 08:40:23,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:23,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:40:23,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:23,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5961448e074b3ffe1735 HTTP/1.1" 200 None 2026-03-21 08:40:23,672 - lava_callback - INFO - Completed processing callback for node 69be5961448e074b3ffe1735 INFO: 172.20.0.1:43696 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:40:30,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:30,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:40:35,912 - lava_callback - INFO - Received callback for node 69be593b448e074b3ffe16a6 INFO: 172.20.0.1:46914 - "POST /node/69be593b448e074b3ffe16a6 HTTP/1.0" 202 Accepted 2026-03-21 08:40:35,918 - lava_callback - INFO - Starting async job submit for node 69be593b448e074b3ffe16a6 2026-03-21 08:40:35,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:35,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be593b448e074b3ffe16a6 HTTP/1.1" 200 None 2026-03-21 08:40:35,959 - lava_callback - DEBUG - Processing node 69be593b448e074b3ffe16a6: baseline-arm 2026-03-21 08:40:36,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:36,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:36,342 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be593b448e074b3ffe16a6/log.txt.gz 2026-03-21 08:40:36,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:46930 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:40:36,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:36,646 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be593b448e074b3ffe16a6/lava-logs.yaml 2026-03-21 08:40:36,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:36,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:36,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be593b448e074b3ffe16a6/lava_callback.json.gz 2026-03-21 08:40:36,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:36,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be593b448e074b3ffe16a6 HTTP/1.1" 200 None 2026-03-21 08:40:36,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:36,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:40:36,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:36,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be593b448e074b3ffe16a6 HTTP/1.1" 200 None 2026-03-21 08:40:36,930 - lava_callback - INFO - Completed processing callback for node 69be593b448e074b3ffe16a6 2026-03-21 08:40:38,531 - lava_callback - INFO - Received callback for node 69be5938448e074b3ffe1698 INFO: 172.20.0.1:46936 - "POST /node/69be5938448e074b3ffe1698 HTTP/1.0" 202 Accepted 2026-03-21 08:40:38,540 - lava_callback - INFO - Starting async job submit for node 69be5938448e074b3ffe1698 2026-03-21 08:40:38,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:38,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5938448e074b3ffe1698 HTTP/1.1" 200 None 2026-03-21 08:40:38,590 - lava_callback - DEBUG - Processing node 69be5938448e074b3ffe1698: baseline-arm 2026-03-21 08:40:38,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:38,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:38,955 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be5938448e074b3ffe1698/log.txt.gz 2026-03-21 08:40:39,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:39,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:39,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be5938448e074b3ffe1698/lava-logs.yaml 2026-03-21 08:40:39,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:39,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:39,503 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be5938448e074b3ffe1698/lava_callback.json.gz 2026-03-21 08:40:39,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:39,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5938448e074b3ffe1698 HTTP/1.1" 200 None 2026-03-21 08:40:39,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:39,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:40:39,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:39,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5938448e074b3ffe1698 HTTP/1.1" 200 None 2026-03-21 08:40:39,599 - lava_callback - INFO - Completed processing callback for node 69be5938448e074b3ffe1698 INFO: 172.20.0.1:47780 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:40:48,862 - lava_callback - INFO - Received callback for node 69be594a448e074b3ffe16cf INFO: 172.20.0.1:47796 - "POST /node/69be594a448e074b3ffe16cf HTTP/1.0" 202 Accepted 2026-03-21 08:40:48,871 - lava_callback - INFO - Starting async job submit for node 69be594a448e074b3ffe16cf 2026-03-21 08:40:48,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:48,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be594a448e074b3ffe16cf HTTP/1.1" 200 None 2026-03-21 08:40:48,940 - lava_callback - DEBUG - Processing node 69be594a448e074b3ffe16cf: baseline-arm-broonie 2026-03-21 08:40:49,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:49,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:49,467 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594a448e074b3ffe16cf/log.txt.gz 2026-03-21 08:40:49,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:49,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:49,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594a448e074b3ffe16cf/lava-logs.yaml 2026-03-21 08:40:49,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:40:50,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:40:50,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594a448e074b3ffe16cf/lava_callback.json.gz 2026-03-21 08:40:50,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:50,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be594a448e074b3ffe16cf HTTP/1.1" 200 None 2026-03-21 08:40:50,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:50,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:40:50,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:40:50,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be594a448e074b3ffe16cf HTTP/1.1" 200 None 2026-03-21 08:40:50,203 - lava_callback - INFO - Completed processing callback for node 69be594a448e074b3ffe16cf INFO: 172.20.0.1:43736 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:41:00,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:00,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:41:02,935 - lava_callback - INFO - Received callback for node 69bbb697ec3f435d20d51b60 INFO: 172.20.0.1:43988 - "POST /node/69bbb697ec3f435d20d51b60 HTTP/1.0" 202 Accepted 2026-03-21 08:41:02,941 - lava_callback - INFO - Starting async job submit for node 69bbb697ec3f435d20d51b60 2026-03-21 08:41:02,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:02,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb697ec3f435d20d51b60 HTTP/1.1" 200 None 2026-03-21 08:41:02,957 - lava_callback - DEBUG - Processing node 69bbb697ec3f435d20d51b60: baseline-arm-baylibre 2026-03-21 08:41:02,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb697ec3f435d20d51b60 2026-03-21 08:41:02,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:02,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb697ec3f435d20d51b60 HTTP/1.1" 200 None 2026-03-21 08:41:02,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:02,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:41:02,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:03,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb697ec3f435d20d51b60 HTTP/1.1" 200 None 2026-03-21 08:41:03,019 - lava_callback - ERROR - Error processing callback for node 69bbb697ec3f435d20d51b60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:41:06,267 - lava_callback - INFO - Received callback for node 69be595f448e074b3ffe172a INFO: 172.20.0.1:43998 - "POST /node/69be595f448e074b3ffe172a HTTP/1.0" 202 Accepted 2026-03-21 08:41:06,313 - lava_callback - INFO - Starting async job submit for node 69be595f448e074b3ffe172a 2026-03-21 08:41:06,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:06,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595f448e074b3ffe172a HTTP/1.1" 200 None 2026-03-21 08:41:06,349 - lava_callback - DEBUG - Processing node 69be595f448e074b3ffe172a: baseline-arm-pengutronix 2026-03-21 08:41:06,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:44010 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:41:06,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:06,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595f448e074b3ffe172a/log.txt.gz 2026-03-21 08:41:06,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:06,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:06,825 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595f448e074b3ffe172a/lava-logs.yaml 2026-03-21 08:41:06,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:06,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:06,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595f448e074b3ffe172a/lava_callback.json.gz 2026-03-21 08:41:06,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:06,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595f448e074b3ffe172a HTTP/1.1" 200 None 2026-03-21 08:41:06,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:06,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:41:06,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:07,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be595f448e074b3ffe172a HTTP/1.1" 200 None 2026-03-21 08:41:07,021 - lava_callback - INFO - Completed processing callback for node 69be595f448e074b3ffe172a 2026-03-21 08:41:08,613 - lava_callback - INFO - Received callback for node 69be594b448e074b3ffe16d2 INFO: 172.20.0.1:44014 - "POST /node/69be594b448e074b3ffe16d2 HTTP/1.0" 202 Accepted 2026-03-21 08:41:08,619 - lava_callback - INFO - Starting async job submit for node 69be594b448e074b3ffe16d2 2026-03-21 08:41:08,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:08,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be594b448e074b3ffe16d2 HTTP/1.1" 200 None 2026-03-21 08:41:08,656 - lava_callback - DEBUG - Processing node 69be594b448e074b3ffe16d2: baseline-arm-broonie 2026-03-21 08:41:08,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:08,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:08,997 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594b448e074b3ffe16d2/log.txt.gz 2026-03-21 08:41:09,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:09,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:09,320 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594b448e074b3ffe16d2/lava-logs.yaml 2026-03-21 08:41:09,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:09,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:09,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594b448e074b3ffe16d2/lava_callback.json.gz 2026-03-21 08:41:09,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:09,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be594b448e074b3ffe16d2 HTTP/1.1" 200 None 2026-03-21 08:41:09,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:09,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:41:09,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:09,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be594b448e074b3ffe16d2 HTTP/1.1" 200 None 2026-03-21 08:41:09,547 - lava_callback - INFO - Completed processing callback for node 69be594b448e074b3ffe16d2 INFO: 172.20.0.1:50058 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:41:12,572 - lava_callback - INFO - Received callback for node 69be595b448e074b3ffe1721 INFO: 172.20.0.1:50066 - "POST /node/69be595b448e074b3ffe1721 HTTP/1.0" 202 Accepted 2026-03-21 08:41:12,577 - lava_callback - INFO - Starting async job submit for node 69be595b448e074b3ffe1721 2026-03-21 08:41:12,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:12,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595b448e074b3ffe1721 HTTP/1.1" 200 None 2026-03-21 08:41:12,610 - lava_callback - DEBUG - Processing node 69be595b448e074b3ffe1721: baseline-arm-pengutronix 2026-03-21 08:41:12,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:12,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:12,861 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595b448e074b3ffe1721/log.txt.gz 2026-03-21 08:41:12,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:13,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:13,113 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595b448e074b3ffe1721/lava-logs.yaml 2026-03-21 08:41:13,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:13,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:13,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595b448e074b3ffe1721/lava_callback.json.gz 2026-03-21 08:41:13,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:13,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595b448e074b3ffe1721 HTTP/1.1" 200 None 2026-03-21 08:41:13,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:13,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:41:13,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:13,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be595b448e074b3ffe1721 HTTP/1.1" 200 None 2026-03-21 08:41:13,331 - lava_callback - INFO - Completed processing callback for node 69be595b448e074b3ffe1721 2026-03-21 08:41:14,031 - lava_callback - INFO - Received callback for node 69be595c448e074b3ffe1724 INFO: 172.20.0.1:50076 - "POST /node/69be595c448e074b3ffe1724 HTTP/1.0" 202 Accepted 2026-03-21 08:41:14,037 - lava_callback - INFO - Starting async job submit for node 69be595c448e074b3ffe1724 2026-03-21 08:41:14,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:14,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595c448e074b3ffe1724 HTTP/1.1" 200 None 2026-03-21 08:41:14,072 - lava_callback - DEBUG - Processing node 69be595c448e074b3ffe1724: baseline-arm-pengutronix 2026-03-21 08:41:14,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:14,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:14,336 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595c448e074b3ffe1724/log.txt.gz 2026-03-21 08:41:14,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:14,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:14,614 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595c448e074b3ffe1724/lava-logs.yaml 2026-03-21 08:41:14,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:14,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:14,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595c448e074b3ffe1724/lava_callback.json.gz 2026-03-21 08:41:14,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:14,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595c448e074b3ffe1724 HTTP/1.1" 200 None 2026-03-21 08:41:14,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:14,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:41:14,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:14,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be595c448e074b3ffe1724 HTTP/1.1" 200 None 2026-03-21 08:41:14,848 - lava_callback - INFO - Completed processing callback for node 69be595c448e074b3ffe1724 INFO: 172.20.0.1:50084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50090 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:41:16,772 - lava_callback - INFO - Received callback for node 69be594d448e074b3ffe16d5 INFO: 172.20.0.1:50096 - "POST /node/69be594d448e074b3ffe16d5 HTTP/1.0" 202 Accepted 2026-03-21 08:41:16,777 - lava_callback - INFO - Starting async job submit for node 69be594d448e074b3ffe16d5 2026-03-21 08:41:16,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:16,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be594d448e074b3ffe16d5 HTTP/1.1" 200 None 2026-03-21 08:41:16,814 - lava_callback - DEBUG - Processing node 69be594d448e074b3ffe16d5: baseline-arm-broonie 2026-03-21 08:41:17,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:17,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:17,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594d448e074b3ffe16d5/log.txt.gz 2026-03-21 08:41:17,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:17,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:17,660 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594d448e074b3ffe16d5/lava-logs.yaml 2026-03-21 08:41:17,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:17,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:17,803 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be594d448e074b3ffe16d5/lava_callback.json.gz 2026-03-21 08:41:17,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:17,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be594d448e074b3ffe16d5 HTTP/1.1" 200 None 2026-03-21 08:41:17,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:17,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:41:17,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:17,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be594d448e074b3ffe16d5 HTTP/1.1" 200 None 2026-03-21 08:41:17,887 - lava_callback - INFO - Completed processing callback for node 69be594d448e074b3ffe16d5 2026-03-21 08:41:20,019 - lava_callback - INFO - Received callback for node 69be5951448e074b3ffe16ff INFO: 172.20.0.1:41596 - "POST /node/69be5951448e074b3ffe16ff HTTP/1.0" 202 Accepted 2026-03-21 08:41:20,025 - lava_callback - INFO - Starting async job submit for node 69be5951448e074b3ffe16ff 2026-03-21 08:41:20,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:20,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5951448e074b3ffe16ff HTTP/1.1" 200 None 2026-03-21 08:41:20,083 - lava_callback - DEBUG - Processing node 69be5951448e074b3ffe16ff: baseline-arm-clabbe 2026-03-21 08:41:20,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:20,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:20,432 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5951448e074b3ffe16ff/log.txt.gz 2026-03-21 08:41:20,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:20,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:20,744 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5951448e074b3ffe16ff/lava-logs.yaml 2026-03-21 08:41:20,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:20,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:20,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5951448e074b3ffe16ff/lava_callback.json.gz 2026-03-21 08:41:20,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:20,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5951448e074b3ffe16ff HTTP/1.1" 200 None 2026-03-21 08:41:20,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:20,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:41:20,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:20,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5951448e074b3ffe16ff HTTP/1.1" 200 None 2026-03-21 08:41:20,987 - lava_callback - INFO - Completed processing callback for node 69be5951448e074b3ffe16ff INFO: 172.20.0.1:41600 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:41:30,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:30,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:41:37,022 - lava_callback - INFO - Received callback for node 69be5948448e074b3ffe16cc INFO: 172.20.0.1:49082 - "POST /node/69be5948448e074b3ffe16cc HTTP/1.0" 202 Accepted 2026-03-21 08:41:37,033 - lava_callback - INFO - Starting async job submit for node 69be5948448e074b3ffe16cc 2026-03-21 08:41:37,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:37,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5948448e074b3ffe16cc HTTP/1.1" 200 None 2026-03-21 08:41:37,094 - lava_callback - DEBUG - Processing node 69be5948448e074b3ffe16cc: baseline-arm-broonie 2026-03-21 08:41:37,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:37,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:37,577 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be5948448e074b3ffe16cc/log.txt.gz 2026-03-21 08:41:37,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:38,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:38,162 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be5948448e074b3ffe16cc/lava-logs.yaml 2026-03-21 08:41:38,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:41:38,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:41:38,379 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69be5948448e074b3ffe16cc/lava_callback.json.gz 2026-03-21 08:41:38,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:38,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5948448e074b3ffe16cc HTTP/1.1" 200 None 2026-03-21 08:41:38,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:38,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:41:38,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:38,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5948448e074b3ffe16cc HTTP/1.1" 200 None 2026-03-21 08:41:38,514 - lava_callback - INFO - Completed processing callback for node 69be5948448e074b3ffe16cc 2026-03-21 08:41:42,864 - lava_callback - INFO - Received callback for node 69bbb6bcec3f435d20d51b74 INFO: 172.20.0.1:43500 - "POST /node/69bbb6bcec3f435d20d51b74 HTTP/1.0" 202 Accepted 2026-03-21 08:41:42,869 - lava_callback - INFO - Starting async job submit for node 69bbb6bcec3f435d20d51b74 2026-03-21 08:41:42,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:42,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb6bcec3f435d20d51b74 HTTP/1.1" 200 None 2026-03-21 08:41:42,885 - lava_callback - DEBUG - Processing node 69bbb6bcec3f435d20d51b74: baseline-arm-baylibre 2026-03-21 08:41:42,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb6bcec3f435d20d51b74 2026-03-21 08:41:42,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:42,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb6bcec3f435d20d51b74 HTTP/1.1" 200 None 2026-03-21 08:41:42,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:42,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:41:42,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:41:42,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb6bcec3f435d20d51b74 HTTP/1.1" 200 None 2026-03-21 08:41:42,949 - lava_callback - ERROR - Error processing callback for node 69bbb6bcec3f435d20d51b74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52190 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:42:00,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:00,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:42:23,006 - lava_callback - INFO - Received callback for node 69bbb6e1ec3f435d20d51b87 INFO: 172.20.0.1:34788 - "POST /node/69bbb6e1ec3f435d20d51b87 HTTP/1.0" 202 Accepted 2026-03-21 08:42:23,012 - lava_callback - INFO - Starting async job submit for node 69bbb6e1ec3f435d20d51b87 2026-03-21 08:42:23,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:23,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb6e1ec3f435d20d51b87 HTTP/1.1" 200 None 2026-03-21 08:42:23,050 - lava_callback - DEBUG - Processing node 69bbb6e1ec3f435d20d51b87: baseline-arm-baylibre 2026-03-21 08:42:23,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb6e1ec3f435d20d51b87 2026-03-21 08:42:23,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:23,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb6e1ec3f435d20d51b87 HTTP/1.1" 200 None 2026-03-21 08:42:23,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:23,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:42:23,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:23,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb6e1ec3f435d20d51b87 HTTP/1.1" 200 None 2026-03-21 08:42:23,171 - lava_callback - ERROR - Error processing callback for node 69bbb6e1ec3f435d20d51b87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34790 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:42:34,931 - lava_callback - INFO - Received callback for node 69be597d448e074b3ffe176b INFO: 172.20.0.1:42850 - "POST /node/69be597d448e074b3ffe176b HTTP/1.0" 202 Accepted 2026-03-21 08:42:34,937 - lava_callback - INFO - Starting async job submit for node 69be597d448e074b3ffe176b 2026-03-21 08:42:34,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:35,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be597d448e074b3ffe176b HTTP/1.1" 200 None 2026-03-21 08:42:35,005 - lava_callback - DEBUG - Processing node 69be597d448e074b3ffe176b: ltp-dio 2026-03-21 08:42:35,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:42:35,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:42:35,990 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69be597d448e074b3ffe176b/log.txt.gz 2026-03-21 08:42:36,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:42860 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:42:36,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:42:36,633 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69be597d448e074b3ffe176b/lava-logs.yaml 2026-03-21 08:42:36,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:42:36,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:42:36,879 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69be597d448e074b3ffe176b/lava_callback.json.gz 2026-03-21 08:42:36,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:36,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be597d448e074b3ffe176b HTTP/1.1" 200 None 2026-03-21 08:42:36,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:36,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:42:36,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:36,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be597d448e074b3ffe176b HTTP/1.1" 200 None 2026-03-21 08:42:36,984 - lava_callback - INFO - Completed processing callback for node 69be597d448e074b3ffe176b INFO: 172.20.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46440 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:42:59,102 - lava_callback - INFO - Received callback for node 69be5952448e074b3ffe1703 INFO: 172.20.0.1:46452 - "POST /node/69be5952448e074b3ffe1703 HTTP/1.0" 202 Accepted 2026-03-21 08:42:59,108 - lava_callback - INFO - Starting async job submit for node 69be5952448e074b3ffe1703 2026-03-21 08:42:59,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:42:59,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5952448e074b3ffe1703 HTTP/1.1" 200 None 2026-03-21 08:42:59,164 - lava_callback - DEBUG - Processing node 69be5952448e074b3ffe1703: baseline-arm-clabbe 2026-03-21 08:42:59,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:42:59,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:42:59,681 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5952448e074b3ffe1703/log.txt.gz 2026-03-21 08:42:59,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:43:00,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:00,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:43:00,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:43:00,281 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5952448e074b3ffe1703/lava-logs.yaml 2026-03-21 08:43:00,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:43:00,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:43:00,436 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5952448e074b3ffe1703/lava_callback.json.gz 2026-03-21 08:43:00,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:00,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5952448e074b3ffe1703 HTTP/1.1" 200 None 2026-03-21 08:43:00,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:00,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:43:00,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:00,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5952448e074b3ffe1703 HTTP/1.1" 200 None 2026-03-21 08:43:00,524 - lava_callback - INFO - Completed processing callback for node 69be5952448e074b3ffe1703 INFO: 172.20.0.1:59724 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:43:10,901 - lava_callback - INFO - Received callback for node 69bbb709ec3f435d20d51b98 INFO: 172.20.0.1:55666 - "POST /node/69bbb709ec3f435d20d51b98 HTTP/1.0" 202 Accepted 2026-03-21 08:43:10,907 - lava_callback - INFO - Starting async job submit for node 69bbb709ec3f435d20d51b98 2026-03-21 08:43:10,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:10,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb709ec3f435d20d51b98 HTTP/1.1" 200 None 2026-03-21 08:43:10,926 - lava_callback - DEBUG - Processing node 69bbb709ec3f435d20d51b98: baseline-arm-clabbe 2026-03-21 08:43:10,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb709ec3f435d20d51b98 2026-03-21 08:43:10,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:10,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb709ec3f435d20d51b98 HTTP/1.1" 200 None 2026-03-21 08:43:10,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:10,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:43:10,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:10,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb709ec3f435d20d51b98 HTTP/1.1" 200 None 2026-03-21 08:43:10,986 - lava_callback - ERROR - Error processing callback for node 69bbb709ec3f435d20d51b98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36854 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:43:30,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:30,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:43:30,723 - lava_callback - INFO - Received callback for node 69bbb72fec3f435d20d51ba4 INFO: 172.20.0.1:46922 - "POST /node/69bbb72fec3f435d20d51ba4 HTTP/1.0" 202 Accepted 2026-03-21 08:43:30,728 - lava_callback - INFO - Starting async job submit for node 69bbb72fec3f435d20d51ba4 2026-03-21 08:43:30,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:30,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb72fec3f435d20d51ba4 HTTP/1.1" 200 None 2026-03-21 08:43:30,744 - lava_callback - DEBUG - Processing node 69bbb72fec3f435d20d51ba4: baseline-arm-clabbe 2026-03-21 08:43:30,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb72fec3f435d20d51ba4 2026-03-21 08:43:30,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:30,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb72fec3f435d20d51ba4 HTTP/1.1" 200 None 2026-03-21 08:43:30,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:30,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:43:30,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:43:30,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb72fec3f435d20d51ba4 HTTP/1.1" 200 None 2026-03-21 08:43:30,808 - lava_callback - ERROR - Error processing callback for node 69bbb72fec3f435d20d51ba4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35602 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:44:10,826 - lava_callback - INFO - Received callback for node 69bbb755ec3f435d20d51bb8 INFO: 172.20.0.1:50522 - "POST /node/69bbb755ec3f435d20d51bb8 HTTP/1.0" 202 Accepted 2026-03-21 08:44:10,831 - lava_callback - INFO - Starting async job submit for node 69bbb755ec3f435d20d51bb8 2026-03-21 08:44:10,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:10,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb755ec3f435d20d51bb8 HTTP/1.1" 200 None 2026-03-21 08:44:10,847 - lava_callback - DEBUG - Processing node 69bbb755ec3f435d20d51bb8: baseline-arm-clabbe 2026-03-21 08:44:10,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb755ec3f435d20d51bb8 2026-03-21 08:44:10,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:10,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb755ec3f435d20d51bb8 HTTP/1.1" 200 None 2026-03-21 08:44:10,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:10,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:44:10,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:10,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb755ec3f435d20d51bb8 HTTP/1.1" 200 None 2026-03-21 08:44:10,910 - lava_callback - ERROR - Error processing callback for node 69bbb755ec3f435d20d51bb8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50542 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:44:17,057 - lava_callback - INFO - Received callback for node 69be5971448e074b3ffe175e INFO: 172.20.0.1:50546 - "POST /node/69be5971448e074b3ffe175e HTTP/1.0" 202 Accepted 2026-03-21 08:44:17,064 - lava_callback - INFO - Starting async job submit for node 69be5971448e074b3ffe175e 2026-03-21 08:44:17,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:17,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5971448e074b3ffe175e HTTP/1.1" 200 None 2026-03-21 08:44:17,103 - lava_callback - DEBUG - Processing node 69be5971448e074b3ffe175e: ltp-containers 2026-03-21 08:44:17,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:17,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:17,600 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69be5971448e074b3ffe175e/log.txt.gz 2026-03-21 08:44:17,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:18,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:18,011 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69be5971448e074b3ffe175e/lava-logs.yaml 2026-03-21 08:44:18,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:18,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:18,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69be5971448e074b3ffe175e/lava_callback.json.gz 2026-03-21 08:44:18,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:18,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5971448e074b3ffe175e HTTP/1.1" 200 None 2026-03-21 08:44:18,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:18,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:44:18,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:18,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5971448e074b3ffe175e HTTP/1.1" 200 None 2026-03-21 08:44:18,306 - lava_callback - INFO - Completed processing callback for node 69be5971448e074b3ffe175e INFO: 172.20.0.1:56410 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:44:29,230 - lava_callback - INFO - Received callback for node 69be596f448e074b3ffe175b INFO: 172.20.0.1:56420 - "POST /node/69be596f448e074b3ffe175b HTTP/1.0" 202 Accepted 2026-03-21 08:44:29,237 - lava_callback - INFO - Starting async job submit for node 69be596f448e074b3ffe175b 2026-03-21 08:44:29,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:29,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be596f448e074b3ffe175b HTTP/1.1" 200 None 2026-03-21 08:44:29,287 - lava_callback - DEBUG - Processing node 69be596f448e074b3ffe175b: ltp-cap_bounds 2026-03-21 08:44:29,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:29,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:29,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69be596f448e074b3ffe175b/log.txt.gz 2026-03-21 08:44:30,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:30,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:30,239 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69be596f448e074b3ffe175b/lava-logs.yaml 2026-03-21 08:44:30,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:30,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:30,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:44:30,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:30,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69be596f448e074b3ffe175b/lava_callback.json.gz 2026-03-21 08:44:30,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:30,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be596f448e074b3ffe175b HTTP/1.1" 200 None 2026-03-21 08:44:30,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:30,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:44:30,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:30,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be596f448e074b3ffe175b HTTP/1.1" 200 None 2026-03-21 08:44:30,531 - lava_callback - INFO - Completed processing callback for node 69be596f448e074b3ffe175b INFO: 172.20.0.1:43248 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:44:38,127 - lava_callback - INFO - Received callback for node 69be5955448e074b3ffe1710 INFO: 172.20.0.1:43252 - "POST /node/69be5955448e074b3ffe1710 HTTP/1.0" 202 Accepted 2026-03-21 08:44:38,133 - lava_callback - INFO - Starting async job submit for node 69be5955448e074b3ffe1710 2026-03-21 08:44:38,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:38,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5955448e074b3ffe1710 HTTP/1.1" 200 None 2026-03-21 08:44:38,169 - lava_callback - DEBUG - Processing node 69be5955448e074b3ffe1710: baseline-arm-clabbe 2026-03-21 08:44:38,361 - lava_callback - INFO - Received callback for node 69be5972448e074b3ffe1761 2026-03-21 08:44:38,378 - lava_callback - INFO - Starting async job submit for node 69be5972448e074b3ffe1761 INFO: 172.20.0.1:43268 - "POST /node/69be5972448e074b3ffe1761 HTTP/1.0" 202 Accepted 2026-03-21 08:44:38,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:38,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:38,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5972448e074b3ffe1761 HTTP/1.1" 200 None 2026-03-21 08:44:38,425 - lava_callback - DEBUG - Processing node 69be5972448e074b3ffe1761: ltp-controllers 2026-03-21 08:44:38,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:38,621 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5955448e074b3ffe1710/log.txt.gz 2026-03-21 08:44:38,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:39,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:39,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:39,208 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5955448e074b3ffe1710/lava-logs.yaml 2026-03-21 08:44:39,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:39,211 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69be5972448e074b3ffe1761/log.txt.gz 2026-03-21 08:44:39,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:39,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:39,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:39,483 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5955448e074b3ffe1710/lava_callback.json.gz 2026-03-21 08:44:39,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:39,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5955448e074b3ffe1710 HTTP/1.1" 200 None 2026-03-21 08:44:39,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:39,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:44:39,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:39,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5955448e074b3ffe1710 HTTP/1.1" 200 None 2026-03-21 08:44:39,590 - lava_callback - INFO - Completed processing callback for node 69be5955448e074b3ffe1710 2026-03-21 08:44:39,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:39,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69be5972448e074b3ffe1761/lava-logs.yaml 2026-03-21 08:44:39,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:40,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:40,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69be5972448e074b3ffe1761/lava_callback.json.gz 2026-03-21 08:44:40,027 - lava_callback - INFO - Received callback for node 69be5989448e074b3ffe17a2 INFO: 172.20.0.1:34592 - "POST /node/69be5989448e074b3ffe17a2 HTTP/1.0" 202 Accepted 2026-03-21 08:44:40,040 - lava_callback - INFO - Starting async job submit for node 69be5989448e074b3ffe17a2 2026-03-21 08:44:40,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:40,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:40,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5972448e074b3ffe1761 HTTP/1.1" 200 None 2026-03-21 08:44:40,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:40,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5989448e074b3ffe17a2 HTTP/1.1" 200 None 2026-03-21 08:44:40,085 - lava_callback - DEBUG - Processing node 69be5989448e074b3ffe17a2: ltp-syscalls 2026-03-21 08:44:40,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:44:40,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:40,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5972448e074b3ffe1761 HTTP/1.1" 200 None 2026-03-21 08:44:40,304 - lava_callback - INFO - Completed processing callback for node 69be5972448e074b3ffe1761 2026-03-21 08:44:40,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:41,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:41,015 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69be5989448e074b3ffe17a2/log.txt.gz 2026-03-21 08:44:41,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:41,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:41,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69be5989448e074b3ffe17a2/lava-logs.yaml 2026-03-21 08:44:41,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:44:41,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:44:41,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69be5989448e074b3ffe17a2/lava_callback.json.gz 2026-03-21 08:44:41,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:41,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5989448e074b3ffe17a2 HTTP/1.1" 200 None 2026-03-21 08:44:41,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:42,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:44:42,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:42,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5989448e074b3ffe17a2 HTTP/1.1" 200 None 2026-03-21 08:44:42,054 - lava_callback - INFO - Completed processing callback for node 69be5989448e074b3ffe17a2 INFO: 172.20.0.1:34602 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:44:51,014 - lava_callback - INFO - Received callback for node 69bbb77cec3f435d20d51bc7 INFO: 172.20.0.1:46728 - "POST /node/69bbb77cec3f435d20d51bc7 HTTP/1.0" 202 Accepted 2026-03-21 08:44:51,020 - lava_callback - INFO - Starting async job submit for node 69bbb77cec3f435d20d51bc7 2026-03-21 08:44:51,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:51,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb77cec3f435d20d51bc7 HTTP/1.1" 200 None 2026-03-21 08:44:51,036 - lava_callback - DEBUG - Processing node 69bbb77cec3f435d20d51bc7: baseline-arm-clabbe 2026-03-21 08:44:51,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb77cec3f435d20d51bc7 2026-03-21 08:44:51,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:51,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb77cec3f435d20d51bc7 HTTP/1.1" 200 None 2026-03-21 08:44:51,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:51,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:44:51,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:44:51,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb77cec3f435d20d51bc7 HTTP/1.1" 200 None 2026-03-21 08:44:51,091 - lava_callback - ERROR - Error processing callback for node 69bbb77cec3f435d20d51bc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46734 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:45:00,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:00,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46442 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:45:23,101 - lava_callback - INFO - Received callback for node 69bbb799ec3f435d20d51bf0 INFO: 172.20.0.1:56472 - "POST /node/69bbb799ec3f435d20d51bf0 HTTP/1.0" 202 Accepted 2026-03-21 08:45:23,106 - lava_callback - INFO - Starting async job submit for node 69bbb799ec3f435d20d51bf0 2026-03-21 08:45:23,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb799ec3f435d20d51bf0 HTTP/1.1" 200 None 2026-03-21 08:45:23,123 - lava_callback - DEBUG - Processing node 69bbb799ec3f435d20d51bf0: baseline-arm-baylibre 2026-03-21 08:45:23,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb799ec3f435d20d51bf0 2026-03-21 08:45:23,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb799ec3f435d20d51bf0 HTTP/1.1" 200 None 2026-03-21 08:45:23,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:23,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb799ec3f435d20d51bf0 HTTP/1.1" 200 None 2026-03-21 08:45:23,178 - lava_callback - ERROR - Error processing callback for node 69bbb799ec3f435d20d51bf0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:23,744 - lava_callback - INFO - Received callback for node 69bbb79aec3f435d20d51bf5 INFO: 172.20.0.1:56486 - "POST /node/69bbb79aec3f435d20d51bf5 HTTP/1.0" 202 Accepted 2026-03-21 08:45:23,750 - lava_callback - INFO - Starting async job submit for node 69bbb79aec3f435d20d51bf5 2026-03-21 08:45:23,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb79aec3f435d20d51bf5 HTTP/1.1" 200 None 2026-03-21 08:45:23,765 - lava_callback - DEBUG - Processing node 69bbb79aec3f435d20d51bf5: baseline-arm-baylibre 2026-03-21 08:45:23,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb79aec3f435d20d51bf5 2026-03-21 08:45:23,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb79aec3f435d20d51bf5 HTTP/1.1" 200 None 2026-03-21 08:45:23,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:23,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:23,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb79aec3f435d20d51bf5 HTTP/1.1" 200 None 2026-03-21 08:45:23,823 - lava_callback - ERROR - Error processing callback for node 69bbb79aec3f435d20d51bf5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:24,387 - lava_callback - INFO - Received callback for node 69bbb79cec3f435d20d51bf8 INFO: 172.20.0.1:56492 - "POST /node/69bbb79cec3f435d20d51bf8 HTTP/1.0" 202 Accepted 2026-03-21 08:45:24,393 - lava_callback - INFO - Starting async job submit for node 69bbb79cec3f435d20d51bf8 2026-03-21 08:45:24,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:24,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb79cec3f435d20d51bf8 HTTP/1.1" 200 None 2026-03-21 08:45:24,409 - lava_callback - DEBUG - Processing node 69bbb79cec3f435d20d51bf8: baseline-arm-baylibre 2026-03-21 08:45:24,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb79cec3f435d20d51bf8 2026-03-21 08:45:24,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:24,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb79cec3f435d20d51bf8 HTTP/1.1" 200 None 2026-03-21 08:45:24,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:24,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:24,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:24,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb79cec3f435d20d51bf8 HTTP/1.1" 200 None 2026-03-21 08:45:24,464 - lava_callback - ERROR - Error processing callback for node 69bbb79cec3f435d20d51bf8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:25,032 - lava_callback - INFO - Received callback for node 69bbb79dec3f435d20d51bfb INFO: 172.20.0.1:56502 - "POST /node/69bbb79dec3f435d20d51bfb HTTP/1.0" 202 Accepted 2026-03-21 08:45:25,038 - lava_callback - INFO - Starting async job submit for node 69bbb79dec3f435d20d51bfb 2026-03-21 08:45:25,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb79dec3f435d20d51bfb HTTP/1.1" 200 None 2026-03-21 08:45:25,056 - lava_callback - DEBUG - Processing node 69bbb79dec3f435d20d51bfb: baseline-arm-baylibre 2026-03-21 08:45:25,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb79dec3f435d20d51bfb 2026-03-21 08:45:25,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb79dec3f435d20d51bfb HTTP/1.1" 200 None 2026-03-21 08:45:25,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:25,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb79dec3f435d20d51bfb HTTP/1.1" 200 None 2026-03-21 08:45:25,123 - lava_callback - ERROR - Error processing callback for node 69bbb79dec3f435d20d51bfb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:25,701 - lava_callback - INFO - Received callback for node 69bbb7a0ec3f435d20d51c06 INFO: 172.20.0.1:56504 - "POST /node/69bbb7a0ec3f435d20d51c06 HTTP/1.0" 202 Accepted 2026-03-21 08:45:25,706 - lava_callback - INFO - Starting async job submit for node 69bbb7a0ec3f435d20d51c06 2026-03-21 08:45:25,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a0ec3f435d20d51c06 HTTP/1.1" 200 None 2026-03-21 08:45:25,722 - lava_callback - DEBUG - Processing node 69bbb7a0ec3f435d20d51c06: baseline-arm-baylibre 2026-03-21 08:45:25,722 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7a0ec3f435d20d51c06 2026-03-21 08:45:25,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a0ec3f435d20d51c06 HTTP/1.1" 200 None 2026-03-21 08:45:25,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:25,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:25,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7a0ec3f435d20d51c06 HTTP/1.1" 200 None 2026-03-21 08:45:25,783 - lava_callback - ERROR - Error processing callback for node 69bbb7a0ec3f435d20d51c06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56520 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:45:30,965 - lava_callback - INFO - Received callback for node 69bbb7a1ec3f435d20d51c09 INFO: 172.20.0.1:43754 - "POST /node/69bbb7a1ec3f435d20d51c09 HTTP/1.0" 202 Accepted 2026-03-21 08:45:30,973 - lava_callback - INFO - Starting async job submit for node 69bbb7a1ec3f435d20d51c09 2026-03-21 08:45:30,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:30,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a1ec3f435d20d51c09 HTTP/1.1" 200 None 2026-03-21 08:45:30,993 - lava_callback - DEBUG - Processing node 69bbb7a1ec3f435d20d51c09: baseline-arm-clabbe 2026-03-21 08:45:30,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7a1ec3f435d20d51c09 2026-03-21 08:45:30,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:31,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a1ec3f435d20d51c09 HTTP/1.1" 200 None 2026-03-21 08:45:31,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:31,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 08:45:31,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:31,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7a1ec3f435d20d51c09 HTTP/1.1" 200 None 2026-03-21 08:45:31,056 - lava_callback - ERROR - Error processing callback for node 69bbb7a1ec3f435d20d51c09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:31,627 - lava_callback - INFO - Received callback for node 69bbb7a9ec3f435d20d51c21 INFO: 172.20.0.1:43756 - "POST /node/69bbb7a9ec3f435d20d51c21 HTTP/1.0" 202 Accepted 2026-03-21 08:45:31,633 - lava_callback - INFO - Starting async job submit for node 69bbb7a9ec3f435d20d51c21 2026-03-21 08:45:31,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:31,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a9ec3f435d20d51c21 HTTP/1.1" 200 None 2026-03-21 08:45:31,649 - lava_callback - DEBUG - Processing node 69bbb7a9ec3f435d20d51c21: baseline-arm-clabbe 2026-03-21 08:45:31,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7a9ec3f435d20d51c21 2026-03-21 08:45:31,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:31,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a9ec3f435d20d51c21 HTTP/1.1" 200 None 2026-03-21 08:45:31,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:31,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:31,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:31,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7a9ec3f435d20d51c21 HTTP/1.1" 200 None 2026-03-21 08:45:31,712 - lava_callback - ERROR - Error processing callback for node 69bbb7a9ec3f435d20d51c21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43760 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:45:37,960 - lava_callback - INFO - Received callback for node 69be597b448e074b3ffe1768 INFO: 172.20.0.1:43774 - "POST /node/69be597b448e074b3ffe1768 HTTP/1.0" 202 Accepted 2026-03-21 08:45:37,967 - lava_callback - INFO - Starting async job submit for node 69be597b448e074b3ffe1768 2026-03-21 08:45:37,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:38,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be597b448e074b3ffe1768 HTTP/1.1" 200 None 2026-03-21 08:45:38,021 - lava_callback - DEBUG - Processing node 69be597b448e074b3ffe1768: ltp-cve 2026-03-21 08:45:38,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:45:38,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:45:38,571 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69be597b448e074b3ffe1768/log.txt.gz 2026-03-21 08:45:38,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:45:39,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:45:39,029 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69be597b448e074b3ffe1768/lava-logs.yaml 2026-03-21 08:45:39,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:45:39,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:45:39,190 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69be597b448e074b3ffe1768/lava_callback.json.gz 2026-03-21 08:45:39,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:39,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be597b448e074b3ffe1768 HTTP/1.1" 200 None 2026-03-21 08:45:39,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:39,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:45:39,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:39,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be597b448e074b3ffe1768 HTTP/1.1" 200 None 2026-03-21 08:45:39,280 - lava_callback - INFO - Completed processing callback for node 69be597b448e074b3ffe1768 2026-03-21 08:45:43,073 - lava_callback - INFO - Received callback for node 69bbb7a2ec3f435d20d51c0b INFO: 172.20.0.1:50864 - "POST /node/69bbb7a2ec3f435d20d51c0b HTTP/1.0" 202 Accepted 2026-03-21 08:45:43,079 - lava_callback - INFO - Starting async job submit for node 69bbb7a2ec3f435d20d51c0b 2026-03-21 08:45:43,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:43,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a2ec3f435d20d51c0b HTTP/1.1" 200 None 2026-03-21 08:45:43,110 - lava_callback - DEBUG - Processing node 69bbb7a2ec3f435d20d51c0b: baseline-arm-baylibre 2026-03-21 08:45:43,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7a2ec3f435d20d51c0b 2026-03-21 08:45:43,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:43,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7a2ec3f435d20d51c0b HTTP/1.1" 200 None 2026-03-21 08:45:43,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:43,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:43,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:43,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7a2ec3f435d20d51c0b HTTP/1.1" 200 None 2026-03-21 08:45:43,175 - lava_callback - ERROR - Error processing callback for node 69bbb7a2ec3f435d20d51c0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50866 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:45:51,131 - lava_callback - INFO - Received callback for node 69bbb7acec3f435d20d51c2e INFO: 172.20.0.1:58860 - "POST /node/69bbb7acec3f435d20d51c2e HTTP/1.0" 202 Accepted 2026-03-21 08:45:51,136 - lava_callback - INFO - Starting async job submit for node 69bbb7acec3f435d20d51c2e 2026-03-21 08:45:51,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7acec3f435d20d51c2e HTTP/1.1" 200 None 2026-03-21 08:45:51,152 - lava_callback - DEBUG - Processing node 69bbb7acec3f435d20d51c2e: baseline-arm-clabbe 2026-03-21 08:45:51,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7acec3f435d20d51c2e 2026-03-21 08:45:51,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7acec3f435d20d51c2e HTTP/1.1" 200 None 2026-03-21 08:45:51,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:51,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7acec3f435d20d51c2e HTTP/1.1" 200 None 2026-03-21 08:45:51,218 - lava_callback - ERROR - Error processing callback for node 69bbb7acec3f435d20d51c2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:51,830 - lava_callback - INFO - Received callback for node 69bbb7adec3f435d20d51c31 INFO: 172.20.0.1:58864 - "POST /node/69bbb7adec3f435d20d51c31 HTTP/1.0" 202 Accepted 2026-03-21 08:45:51,835 - lava_callback - INFO - Starting async job submit for node 69bbb7adec3f435d20d51c31 2026-03-21 08:45:51,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7adec3f435d20d51c31 HTTP/1.1" 200 None 2026-03-21 08:45:51,852 - lava_callback - DEBUG - Processing node 69bbb7adec3f435d20d51c31: baseline-arm-clabbe 2026-03-21 08:45:51,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7adec3f435d20d51c31 2026-03-21 08:45:51,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7adec3f435d20d51c31 HTTP/1.1" 200 None 2026-03-21 08:45:51,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:51,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:51,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7adec3f435d20d51c31 HTTP/1.1" 200 None 2026-03-21 08:45:51,912 - lava_callback - ERROR - Error processing callback for node 69bbb7adec3f435d20d51c31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:52,469 - lava_callback - INFO - Received callback for node 69bbb7aeec3f435d20d51c34 INFO: 172.20.0.1:58868 - "POST /node/69bbb7aeec3f435d20d51c34 HTTP/1.0" 202 Accepted 2026-03-21 08:45:52,475 - lava_callback - INFO - Starting async job submit for node 69bbb7aeec3f435d20d51c34 2026-03-21 08:45:52,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:52,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7aeec3f435d20d51c34 HTTP/1.1" 200 None 2026-03-21 08:45:52,496 - lava_callback - DEBUG - Processing node 69bbb7aeec3f435d20d51c34: baseline-arm-clabbe 2026-03-21 08:45:52,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7aeec3f435d20d51c34 2026-03-21 08:45:52,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:52,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7aeec3f435d20d51c34 HTTP/1.1" 200 None 2026-03-21 08:45:52,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:52,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:52,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:52,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7aeec3f435d20d51c34 HTTP/1.1" 200 None 2026-03-21 08:45:52,557 - lava_callback - ERROR - Error processing callback for node 69bbb7aeec3f435d20d51c34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:45:53,090 - lava_callback - INFO - Received callback for node 69bbb7b0ec3f435d20d51c3d INFO: 172.20.0.1:58882 - "POST /node/69bbb7b0ec3f435d20d51c3d HTTP/1.0" 202 Accepted 2026-03-21 08:45:53,096 - lava_callback - INFO - Starting async job submit for node 69bbb7b0ec3f435d20d51c3d 2026-03-21 08:45:53,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:53,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7b0ec3f435d20d51c3d HTTP/1.1" 200 None 2026-03-21 08:45:53,111 - lava_callback - DEBUG - Processing node 69bbb7b0ec3f435d20d51c3d: baseline-arm-clabbe 2026-03-21 08:45:53,112 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb7b0ec3f435d20d51c3d 2026-03-21 08:45:53,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:53,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb7b0ec3f435d20d51c3d HTTP/1.1" 200 None 2026-03-21 08:45:53,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:53,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 08:45:53,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:45:53,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb7b0ec3f435d20d51c3d HTTP/1.1" 200 None 2026-03-21 08:45:53,164 - lava_callback - ERROR - Error processing callback for node 69bbb7b0ec3f435d20d51c3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:46:00,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:00,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40478 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:46:10,810 - lava_callback - INFO - Received callback for node 69be5ab4448e074b3ffe188c INFO: 172.20.0.1:36924 - "POST /node/69be5ab4448e074b3ffe188c HTTP/1.0" 202 Accepted 2026-03-21 08:46:10,822 - lava_callback - INFO - Starting async job submit for node 69be5ab4448e074b3ffe188c 2026-03-21 08:46:10,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:10,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5ab4448e074b3ffe188c HTTP/1.1" 200 None 2026-03-21 08:46:10,857 - lava_callback - DEBUG - Processing node 69be5ab4448e074b3ffe188c: baseline-arm 2026-03-21 08:46:10,865 - lava_callback - WARNING - Failed to upload log 2026-03-21 08:46:10,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:46:11,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:46:11,337 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be5ab4448e074b3ffe188c/lava-logs.yaml 2026-03-21 08:46:11,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36928 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:46:11,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:46:11,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be5ab4448e074b3ffe188c/lava_callback.json.gz 2026-03-21 08:46:11,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:11,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5ab4448e074b3ffe188c HTTP/1.1" 200 None 2026-03-21 08:46:11,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:11,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:46:11,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:11,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5ab4448e074b3ffe188c HTTP/1.1" 200 None 2026-03-21 08:46:11,710 - lava_callback - INFO - Completed processing callback for node 69be5ab4448e074b3ffe188c INFO: 172.20.0.1:36944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36956 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:46:21,203 - lava_callback - INFO - Received callback for node 69be5956448e074b3ffe1713 INFO: 172.20.0.1:43960 - "POST /node/69be5956448e074b3ffe1713 HTTP/1.0" 202 Accepted 2026-03-21 08:46:21,213 - lava_callback - INFO - Starting async job submit for node 69be5956448e074b3ffe1713 2026-03-21 08:46:21,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:21,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5956448e074b3ffe1713 HTTP/1.1" 200 None 2026-03-21 08:46:21,307 - lava_callback - DEBUG - Processing node 69be5956448e074b3ffe1713: baseline-arm-clabbe 2026-03-21 08:46:21,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:46:22,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:46:22,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5956448e074b3ffe1713/log.txt.gz 2026-03-21 08:46:22,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:46:23,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:46:23,354 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5956448e074b3ffe1713/lava-logs.yaml 2026-03-21 08:46:23,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:46:23,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:46:23,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69be5956448e074b3ffe1713/lava_callback.json.gz 2026-03-21 08:46:23,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:23,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5956448e074b3ffe1713 HTTP/1.1" 200 None 2026-03-21 08:46:23,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:23,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:46:23,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:24,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5956448e074b3ffe1713 HTTP/1.1" 200 None 2026-03-21 08:46:24,024 - lava_callback - INFO - Completed processing callback for node 69be5956448e074b3ffe1713 INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:46:30,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:46:30,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49334 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:47:23,233 - lava_callback - INFO - Received callback for node 69bbb806ec3f435d20d51cfe INFO: 172.20.0.1:49634 - "POST /node/69bbb806ec3f435d20d51cfe HTTP/1.0" 202 Accepted 2026-03-21 08:47:23,242 - lava_callback - INFO - Starting async job submit for node 69bbb806ec3f435d20d51cfe 2026-03-21 08:47:23,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:47:23,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb806ec3f435d20d51cfe HTTP/1.1" 200 None 2026-03-21 08:47:23,275 - lava_callback - DEBUG - Processing node 69bbb806ec3f435d20d51cfe: baseline-arm-baylibre 2026-03-21 08:47:23,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb806ec3f435d20d51cfe 2026-03-21 08:47:23,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:47:23,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb806ec3f435d20d51cfe HTTP/1.1" 200 None 2026-03-21 08:47:23,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:47:23,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:47:23,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:47:23,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb806ec3f435d20d51cfe HTTP/1.1" 200 None 2026-03-21 08:47:23,375 - lava_callback - ERROR - Error processing callback for node 69bbb806ec3f435d20d51cfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45012 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:48:03,243 - lava_callback - INFO - Received callback for node 69bbb82eec3f435d20d51d3a INFO: 172.20.0.1:41210 - "POST /node/69bbb82eec3f435d20d51d3a HTTP/1.0" 202 Accepted 2026-03-21 08:48:03,250 - lava_callback - INFO - Starting async job submit for node 69bbb82eec3f435d20d51d3a 2026-03-21 08:48:03,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:03,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb82eec3f435d20d51d3a HTTP/1.1" 200 None 2026-03-21 08:48:03,266 - lava_callback - DEBUG - Processing node 69bbb82eec3f435d20d51d3a: baseline-arm-baylibre 2026-03-21 08:48:03,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb82eec3f435d20d51d3a 2026-03-21 08:48:03,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:03,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb82eec3f435d20d51d3a HTTP/1.1" 200 None 2026-03-21 08:48:03,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:03,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:48:03,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:03,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb82eec3f435d20d51d3a HTTP/1.1" 200 None 2026-03-21 08:48:03,329 - lava_callback - ERROR - Error processing callback for node 69bbb82eec3f435d20d51d3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60434 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:48:17,133 - lava_callback - INFO - Received callback for node 69be597f448e074b3ffe1775 INFO: 172.20.0.1:60440 - "POST /node/69be597f448e074b3ffe1775 HTTP/1.0" 202 Accepted 2026-03-21 08:48:17,143 - lava_callback - INFO - Starting async job submit for node 69be597f448e074b3ffe1775 2026-03-21 08:48:17,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:17,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be597f448e074b3ffe1775 HTTP/1.1" 200 None 2026-03-21 08:48:17,231 - lava_callback - DEBUG - Processing node 69be597f448e074b3ffe1775: ltp-fs_bind 2026-03-21 08:48:17,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:18,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:18,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69be597f448e074b3ffe1775/log.txt.gz 2026-03-21 08:48:18,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:19,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:19,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69be597f448e074b3ffe1775/lava-logs.yaml 2026-03-21 08:48:19,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:19,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:19,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69be597f448e074b3ffe1775/lava_callback.json.gz 2026-03-21 08:48:19,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:19,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be597f448e074b3ffe1775 HTTP/1.1" 200 None 2026-03-21 08:48:19,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:19,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:48:19,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:19,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be597f448e074b3ffe1775 HTTP/1.1" 200 None 2026-03-21 08:48:19,496 - lava_callback - INFO - Completed processing callback for node 69be597f448e074b3ffe1775 INFO: 172.20.0.1:42548 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:48:29,133 - lava_callback - INFO - Received callback for node 69be5982448e074b3ffe177b INFO: 172.20.0.1:42552 - "POST /node/69be5982448e074b3ffe177b HTTP/1.0" 202 Accepted 2026-03-21 08:48:29,144 - lava_callback - INFO - Starting async job submit for node 69be5982448e074b3ffe177b 2026-03-21 08:48:29,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:29,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5982448e074b3ffe177b HTTP/1.1" 200 None 2026-03-21 08:48:29,214 - lava_callback - DEBUG - Processing node 69be5982448e074b3ffe177b: ltp-input 2026-03-21 08:48:29,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:30,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:30,312 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69be5982448e074b3ffe177b/log.txt.gz 2026-03-21 08:48:30,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:30,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:48:30,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:31,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:31,434 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69be5982448e074b3ffe177b/lava-logs.yaml 2026-03-21 08:48:31,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:31,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:31,824 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69be5982448e074b3ffe177b/lava_callback.json.gz 2026-03-21 08:48:31,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:31,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5982448e074b3ffe177b HTTP/1.1" 200 None 2026-03-21 08:48:31,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:31,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:48:31,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:31,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5982448e074b3ffe177b HTTP/1.1" 200 None 2026-03-21 08:48:31,985 - lava_callback - INFO - Completed processing callback for node 69be5982448e074b3ffe177b INFO: 172.20.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52886 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:48:57,406 - lava_callback - INFO - Received callback for node 69be5980448e074b3ffe1778 INFO: 172.20.0.1:52902 - "POST /node/69be5980448e074b3ffe1778 HTTP/1.0" 202 Accepted 2026-03-21 08:48:57,416 - lava_callback - INFO - Starting async job submit for node 69be5980448e074b3ffe1778 2026-03-21 08:48:57,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:48:57,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5980448e074b3ffe1778 HTTP/1.1" 200 None 2026-03-21 08:48:57,498 - lava_callback - DEBUG - Processing node 69be5980448e074b3ffe1778: ltp-fsx 2026-03-21 08:48:58,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:58,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:58,665 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69be5980448e074b3ffe1778/log.txt.gz 2026-03-21 08:48:58,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:48:59,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:48:59,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69be5980448e074b3ffe1778/lava-logs.yaml 2026-03-21 08:48:59,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:49:00,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:00,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:49:00,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:49:00,507 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69be5980448e074b3ffe1778/lava_callback.json.gz 2026-03-21 08:49:00,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:00,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5980448e074b3ffe1778 HTTP/1.1" 200 None 2026-03-21 08:49:00,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:00,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:49:00,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:00,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5980448e074b3ffe1778 HTTP/1.1" 200 None 2026-03-21 08:49:00,654 - lava_callback - INFO - Completed processing callback for node 69be5980448e074b3ffe1778 INFO: 172.20.0.1:33910 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:49:09,226 - lava_callback - INFO - Received callback for node 69be56f3448e074b3ffe1678 INFO: 172.20.0.1:33924 - "POST /node/69be56f3448e074b3ffe1678 HTTP/1.0" 202 Accepted 2026-03-21 08:49:09,232 - lava_callback - INFO - Starting async job submit for node 69be56f3448e074b3ffe1678 2026-03-21 08:49:09,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:09,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be56f3448e074b3ffe1678 HTTP/1.1" 200 None 2026-03-21 08:49:09,306 - lava_callback - DEBUG - Processing node 69be56f3448e074b3ffe1678: ltp-smoke 2026-03-21 08:49:09,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:49:09,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:49:09,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69be56f3448e074b3ffe1678/log.txt.gz 2026-03-21 08:49:10,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:49:10,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:49:10,482 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69be56f3448e074b3ffe1678/lava-logs.yaml 2026-03-21 08:49:10,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:49:10,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:49:10,647 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69be56f3448e074b3ffe1678/lava_callback.json.gz 2026-03-21 08:49:10,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:10,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be56f3448e074b3ffe1678 HTTP/1.1" 200 None 2026-03-21 08:49:10,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:10,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5456448e074b3ffe1633 HTTP/1.1" 200 None 2026-03-21 08:49:10,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:10,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be56f3448e074b3ffe1678 HTTP/1.1" 200 None 2026-03-21 08:49:10,964 - lava_callback - INFO - Completed processing callback for node 69be56f3448e074b3ffe1678 INFO: 172.20.0.1:55516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48622 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:49:30,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:30,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43738 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:49:37,200 - lava_callback - INFO - Received callback for node 69be5983448e074b3ffe177e INFO: 172.20.0.1:43740 - "POST /node/69be5983448e074b3ffe177e HTTP/1.0" 202 Accepted 2026-03-21 08:49:37,206 - lava_callback - INFO - Starting async job submit for node 69be5983448e074b3ffe177e 2026-03-21 08:49:37,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:37,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5983448e074b3ffe177e HTTP/1.1" 200 None 2026-03-21 08:49:37,246 - lava_callback - DEBUG - Processing node 69be5983448e074b3ffe177e: ltp-ipc 2026-03-21 08:49:37,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:49:37,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:49:37,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69be5983448e074b3ffe177e/log.txt.gz 2026-03-21 08:49:37,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:49:38,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:49:38,169 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69be5983448e074b3ffe177e/lava-logs.yaml 2026-03-21 08:49:38,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:49:38,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:49:38,354 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69be5983448e074b3ffe177e/lava_callback.json.gz 2026-03-21 08:49:38,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:38,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5983448e074b3ffe177e HTTP/1.1" 200 None 2026-03-21 08:49:38,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:38,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:49:38,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:49:38,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5983448e074b3ffe177e HTTP/1.1" 200 None 2026-03-21 08:49:38,435 - lava_callback - INFO - Completed processing callback for node 69be5983448e074b3ffe177e INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46962 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:50:00,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:50:00,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54346 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:52:17,238 - lava_callback - INFO - Received callback for node 69be5985448e074b3ffe1785 INFO: 172.20.0.1:54362 - "POST /node/69be5985448e074b3ffe1785 HTTP/1.0" 202 Accepted 2026-03-21 08:52:17,249 - lava_callback - INFO - Starting async job submit for node 69be5985448e074b3ffe1785 2026-03-21 08:52:17,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:17,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5985448e074b3ffe1785 HTTP/1.1" 200 None 2026-03-21 08:52:17,327 - lava_callback - DEBUG - Processing node 69be5985448e074b3ffe1785: ltp-pty 2026-03-21 08:52:17,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:52:18,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:52:18,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69be5985448e074b3ffe1785/log.txt.gz 2026-03-21 08:52:18,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:52:19,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:52:19,119 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69be5985448e074b3ffe1785/lava-logs.yaml 2026-03-21 08:52:19,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:52:19,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:52:19,843 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69be5985448e074b3ffe1785/lava_callback.json.gz 2026-03-21 08:52:19,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:19,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5985448e074b3ffe1785 HTTP/1.1" 200 None 2026-03-21 08:52:19,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:52:19,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:19,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5985448e074b3ffe1785 HTTP/1.1" 200 None 2026-03-21 08:52:19,993 - lava_callback - INFO - Completed processing callback for node 69be5985448e074b3ffe1785 INFO: 172.20.0.1:39070 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:52:29,180 - lava_callback - INFO - Received callback for node 69be5987448e074b3ffe1788 INFO: 172.20.0.1:39082 - "POST /node/69be5987448e074b3ffe1788 HTTP/1.0" 202 Accepted 2026-03-21 08:52:29,196 - lava_callback - INFO - Starting async job submit for node 69be5987448e074b3ffe1788 2026-03-21 08:52:29,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:29,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5987448e074b3ffe1788 HTTP/1.1" 200 None 2026-03-21 08:52:29,274 - lava_callback - DEBUG - Processing node 69be5987448e074b3ffe1788: ltp-sched 2026-03-21 08:52:29,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:52:30,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:52:30,470 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69be5987448e074b3ffe1788/log.txt.gz 2026-03-21 08:52:30,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:30,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:52:30,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:52:31,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:52:31,554 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69be5987448e074b3ffe1788/lava-logs.yaml 2026-03-21 08:52:31,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:52:32,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:52:32,428 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69be5987448e074b3ffe1788/lava_callback.json.gz 2026-03-21 08:52:32,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:32,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5987448e074b3ffe1788 HTTP/1.1" 200 None 2026-03-21 08:52:32,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:32,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:52:32,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:52:32,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5987448e074b3ffe1788 HTTP/1.1" 200 None 2026-03-21 08:52:32,531 - lava_callback - INFO - Completed processing callback for node 69be5987448e074b3ffe1788 INFO: 172.20.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54850 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:53:00,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:00,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58592 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:53:11,369 - lava_callback - INFO - Received callback for node 69bbb961ec3f435d20d523d3 INFO: 172.20.0.1:40970 - "POST /node/69bbb961ec3f435d20d523d3 HTTP/1.0" 202 Accepted 2026-03-21 08:53:11,374 - lava_callback - INFO - Starting async job submit for node 69bbb961ec3f435d20d523d3 2026-03-21 08:53:11,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:11,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb961ec3f435d20d523d3 HTTP/1.1" 200 None 2026-03-21 08:53:11,391 - lava_callback - DEBUG - Processing node 69bbb961ec3f435d20d523d3: baseline-arm-clabbe 2026-03-21 08:53:11,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb961ec3f435d20d523d3 2026-03-21 08:53:11,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:11,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb961ec3f435d20d523d3 HTTP/1.1" 200 None 2026-03-21 08:53:11,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:11,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:53:11,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:11,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb961ec3f435d20d523d3 HTTP/1.1" 200 None 2026-03-21 08:53:11,452 - lava_callback - ERROR - Error processing callback for node 69bbb961ec3f435d20d523d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41000 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:53:17,380 - lava_callback - INFO - Received callback for node 69be5988448e074b3ffe178f INFO: 172.20.0.1:41008 - "POST /node/69be5988448e074b3ffe178f HTTP/1.0" 202 Accepted 2026-03-21 08:53:17,386 - lava_callback - INFO - Starting async job submit for node 69be5988448e074b3ffe178f 2026-03-21 08:53:17,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:17,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5988448e074b3ffe178f HTTP/1.1" 200 None 2026-03-21 08:53:17,428 - lava_callback - DEBUG - Processing node 69be5988448e074b3ffe178f: ltp-smoke 2026-03-21 08:53:17,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:53:17,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:53:17,904 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69be5988448e074b3ffe178f/log.txt.gz 2026-03-21 08:53:18,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:53:18,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:53:18,417 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69be5988448e074b3ffe178f/lava-logs.yaml 2026-03-21 08:53:18,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:53:18,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:53:18,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69be5988448e074b3ffe178f/lava_callback.json.gz 2026-03-21 08:53:18,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:18,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5988448e074b3ffe178f HTTP/1.1" 200 None 2026-03-21 08:53:18,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:18,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:53:18,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:18,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5988448e074b3ffe178f HTTP/1.1" 200 None 2026-03-21 08:53:18,697 - lava_callback - INFO - Completed processing callback for node 69be5988448e074b3ffe178f INFO: 172.20.0.1:54638 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:53:30,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:30,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43272 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:53:37,356 - lava_callback - INFO - Received callback for node 69be598b448e074b3ffe17a5 INFO: 172.20.0.1:43280 - "POST /node/69be598b448e074b3ffe17a5 HTTP/1.0" 202 Accepted 2026-03-21 08:53:37,363 - lava_callback - INFO - Starting async job submit for node 69be598b448e074b3ffe17a5 2026-03-21 08:53:37,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:37,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be598b448e074b3ffe17a5 HTTP/1.1" 200 None 2026-03-21 08:53:37,402 - lava_callback - DEBUG - Processing node 69be598b448e074b3ffe17a5: ltp-syscalls-ipc 2026-03-21 08:53:37,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:53:37,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:53:37,881 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69be598b448e074b3ffe17a5/log.txt.gz 2026-03-21 08:53:38,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:53:38,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:53:38,321 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69be598b448e074b3ffe17a5/lava-logs.yaml 2026-03-21 08:53:38,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:53:38,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:53:38,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69be598b448e074b3ffe17a5/lava_callback.json.gz 2026-03-21 08:53:38,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:38,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be598b448e074b3ffe17a5 HTTP/1.1" 200 None 2026-03-21 08:53:38,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:38,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:53:38,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:38,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be598b448e074b3ffe17a5 HTTP/1.1" 200 None 2026-03-21 08:53:38,617 - lava_callback - INFO - Completed processing callback for node 69be598b448e074b3ffe17a5 2026-03-21 08:53:43,599 - lava_callback - INFO - Received callback for node 69bbb987ec3f435d20d523e2 INFO: 172.20.0.1:36448 - "POST /node/69bbb987ec3f435d20d523e2 HTTP/1.0" 202 Accepted 2026-03-21 08:53:43,604 - lava_callback - INFO - Starting async job submit for node 69bbb987ec3f435d20d523e2 2026-03-21 08:53:43,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:43,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb987ec3f435d20d523e2 HTTP/1.1" 200 None 2026-03-21 08:53:43,625 - lava_callback - DEBUG - Processing node 69bbb987ec3f435d20d523e2: baseline-arm-baylibre 2026-03-21 08:53:43,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbb987ec3f435d20d523e2 2026-03-21 08:53:43,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:43,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb987ec3f435d20d523e2 HTTP/1.1" 200 None 2026-03-21 08:53:43,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:43,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:53:43,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:53:43,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbb987ec3f435d20d523e2 HTTP/1.1" 200 None 2026-03-21 08:53:43,692 - lava_callback - ERROR - Error processing callback for node 69bbb987ec3f435d20d523e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47786 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:54:00,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:54:00,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:55:49,146 - lava_callback - INFO - Received callback for node 69be593a448e074b3ffe16a1 INFO: 172.20.0.1:44150 - "POST /node/69be593a448e074b3ffe16a1 HTTP/1.0" 202 Accepted 2026-03-21 08:55:49,157 - lava_callback - INFO - Starting async job submit for node 69be593a448e074b3ffe16a1 2026-03-21 08:55:49,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:55:49,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be593a448e074b3ffe16a1 HTTP/1.1" 200 None 2026-03-21 08:55:49,250 - lava_callback - DEBUG - Processing node 69be593a448e074b3ffe16a1: baseline-arm 2026-03-21 08:55:50,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:55:50,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:55:50,157 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be593a448e074b3ffe16a1/log.txt.gz 2026-03-21 08:55:50,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:55:50,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:55:50,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be593a448e074b3ffe16a1/lava-logs.yaml 2026-03-21 08:55:50,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:55:50,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:55:50,958 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be593a448e074b3ffe16a1/lava_callback.json.gz DEBUG: Setup failed for 69be593a448e074b3ffe16a1. Transitting job node result: incomplete -> fail 2026-03-21 08:55:51,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:55:51,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be593a448e074b3ffe16a1 HTTP/1.1" 200 None 2026-03-21 08:55:51,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:55:51,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:55:51,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:55:51,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be593a448e074b3ffe16a1 HTTP/1.1" 200 None 2026-03-21 08:55:51,236 - lava_callback - INFO - Completed processing callback for node 69be593a448e074b3ffe16a1 INFO: 172.20.0.1:57612 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:56:00,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:00,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:56:03,732 - lava_callback - INFO - Received callback for node 69bbba1aec3f435d20d524ab INFO: 172.20.0.1:35168 - "POST /node/69bbba1aec3f435d20d524ab HTTP/1.0" 202 Accepted 2026-03-21 08:56:03,738 - lava_callback - INFO - Starting async job submit for node 69bbba1aec3f435d20d524ab 2026-03-21 08:56:03,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:03,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1aec3f435d20d524ab HTTP/1.1" 200 None 2026-03-21 08:56:03,754 - lava_callback - DEBUG - Processing node 69bbba1aec3f435d20d524ab: baseline-arm64-baylibre 2026-03-21 08:56:03,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba1aec3f435d20d524ab 2026-03-21 08:56:03,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:03,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1aec3f435d20d524ab HTTP/1.1" 200 None 2026-03-21 08:56:03,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:03,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:03,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:03,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba1aec3f435d20d524ab HTTP/1.1" 200 None 2026-03-21 08:56:03,811 - lava_callback - ERROR - Error processing callback for node 69bbba1aec3f435d20d524ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:04,409 - lava_callback - INFO - Received callback for node 69bbba1fec3f435d20d524bc INFO: 172.20.0.1:35184 - "POST /node/69bbba1fec3f435d20d524bc HTTP/1.0" 202 Accepted 2026-03-21 08:56:04,418 - lava_callback - INFO - Starting async job submit for node 69bbba1fec3f435d20d524bc 2026-03-21 08:56:04,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:04,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1fec3f435d20d524bc HTTP/1.1" 200 None 2026-03-21 08:56:04,435 - lava_callback - DEBUG - Processing node 69bbba1fec3f435d20d524bc: baseline-arm64-baylibre 2026-03-21 08:56:04,436 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba1fec3f435d20d524bc 2026-03-21 08:56:04,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:04,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1fec3f435d20d524bc HTTP/1.1" 200 None 2026-03-21 08:56:04,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:04,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:04,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:04,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba1fec3f435d20d524bc HTTP/1.1" 200 None 2026-03-21 08:56:04,488 - lava_callback - ERROR - Error processing callback for node 69bbba1fec3f435d20d524bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:05,056 - lava_callback - INFO - Received callback for node 69bbba1cec3f435d20d524b2 INFO: 172.20.0.1:35190 - "POST /node/69bbba1cec3f435d20d524b2 HTTP/1.0" 202 Accepted 2026-03-21 08:56:05,062 - lava_callback - INFO - Starting async job submit for node 69bbba1cec3f435d20d524b2 2026-03-21 08:56:05,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1cec3f435d20d524b2 HTTP/1.1" 200 None 2026-03-21 08:56:05,082 - lava_callback - DEBUG - Processing node 69bbba1cec3f435d20d524b2: baseline-arm64-baylibre 2026-03-21 08:56:05,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba1cec3f435d20d524b2 2026-03-21 08:56:05,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1cec3f435d20d524b2 HTTP/1.1" 200 None 2026-03-21 08:56:05,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:05,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba1cec3f435d20d524b2 HTTP/1.1" 200 None 2026-03-21 08:56:05,142 - lava_callback - ERROR - Error processing callback for node 69bbba1cec3f435d20d524b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:05,718 - lava_callback - INFO - Received callback for node 69bbba1eec3f435d20d524b9 INFO: 172.20.0.1:35202 - "POST /node/69bbba1eec3f435d20d524b9 HTTP/1.0" 202 Accepted 2026-03-21 08:56:05,723 - lava_callback - INFO - Starting async job submit for node 69bbba1eec3f435d20d524b9 2026-03-21 08:56:05,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1eec3f435d20d524b9 HTTP/1.1" 200 None 2026-03-21 08:56:05,741 - lava_callback - DEBUG - Processing node 69bbba1eec3f435d20d524b9: baseline-arm64-baylibre 2026-03-21 08:56:05,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba1eec3f435d20d524b9 2026-03-21 08:56:05,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba1eec3f435d20d524b9 HTTP/1.1" 200 None 2026-03-21 08:56:05,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:05,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:05,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba1eec3f435d20d524b9 HTTP/1.1" 200 None 2026-03-21 08:56:05,805 - lava_callback - ERROR - Error processing callback for node 69bbba1eec3f435d20d524b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38030 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:56:17,391 - lava_callback - INFO - Received callback for node 69be598d448e074b3ffe17ac INFO: 172.20.0.1:38034 - "POST /node/69be598d448e074b3ffe17ac HTTP/1.0" 202 Accepted 2026-03-21 08:56:17,398 - lava_callback - INFO - Starting async job submit for node 69be598d448e074b3ffe17ac 2026-03-21 08:56:17,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:17,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be598d448e074b3ffe17ac HTTP/1.1" 200 None 2026-03-21 08:56:17,447 - lava_callback - DEBUG - Processing node 69be598d448e074b3ffe17ac: ltp-timers 2026-03-21 08:56:17,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:56:17,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:56:17,956 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69be598d448e074b3ffe17ac/log.txt.gz 2026-03-21 08:56:18,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:56:18,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:56:18,353 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69be598d448e074b3ffe17ac/lava-logs.yaml 2026-03-21 08:56:18,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:56:18,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:56:18,525 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69be598d448e074b3ffe17ac/lava_callback.json.gz 2026-03-21 08:56:18,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:18,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be598d448e074b3ffe17ac HTTP/1.1" 200 None 2026-03-21 08:56:18,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:18,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:56:18,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:18,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be598d448e074b3ffe17ac HTTP/1.1" 200 None 2026-03-21 08:56:18,615 - lava_callback - INFO - Completed processing callback for node 69be598d448e074b3ffe17ac INFO: 172.20.0.1:59242 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:56:29,362 - lava_callback - INFO - Received callback for node 69be598e448e074b3ffe17af INFO: 172.20.0.1:59246 - "POST /node/69be598e448e074b3ffe17af HTTP/1.0" 202 Accepted 2026-03-21 08:56:29,368 - lava_callback - INFO - Starting async job submit for node 69be598e448e074b3ffe17af 2026-03-21 08:56:29,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:29,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be598e448e074b3ffe17af HTTP/1.1" 200 None 2026-03-21 08:56:29,420 - lava_callback - DEBUG - Processing node 69be598e448e074b3ffe17af: ltp-watchqueue 2026-03-21 08:56:29,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:56:29,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:56:29,880 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69be598e448e074b3ffe17af/log.txt.gz 2026-03-21 08:56:30,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:56:30,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:56:30,302 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69be598e448e074b3ffe17af/lava-logs.yaml 2026-03-21 08:56:30,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:56:30,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:56:30,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69be598e448e074b3ffe17af/lava_callback.json.gz 2026-03-21 08:56:30,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:30,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be598e448e074b3ffe17af HTTP/1.1" 200 None 2026-03-21 08:56:30,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:30,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:56:30,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:30,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be598e448e074b3ffe17af HTTP/1.1" 200 None 2026-03-21 08:56:30,552 - lava_callback - INFO - Completed processing callback for node 69be598e448e074b3ffe17af 2026-03-21 08:56:30,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:30,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 08:56:31,449 - lava_callback - INFO - Received callback for node 69bbba30ec3f435d20d52508 INFO: 172.20.0.1:49882 - "POST /node/69bbba30ec3f435d20d52508 HTTP/1.0" 202 Accepted 2026-03-21 08:56:31,454 - lava_callback - INFO - Starting async job submit for node 69bbba30ec3f435d20d52508 2026-03-21 08:56:31,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:31,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba30ec3f435d20d52508 HTTP/1.1" 200 None 2026-03-21 08:56:31,470 - lava_callback - DEBUG - Processing node 69bbba30ec3f435d20d52508: baseline-arm64-clabbe 2026-03-21 08:56:31,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba30ec3f435d20d52508 2026-03-21 08:56:31,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:31,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba30ec3f435d20d52508 HTTP/1.1" 200 None 2026-03-21 08:56:31,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:31,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:31,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:31,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba30ec3f435d20d52508 HTTP/1.1" 200 None 2026-03-21 08:56:31,534 - lava_callback - ERROR - Error processing callback for node 69bbba30ec3f435d20d52508: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:32,154 - lava_callback - INFO - Received callback for node 69bbba32ec3f435d20d5250f INFO: 172.20.0.1:49884 - "POST /node/69bbba32ec3f435d20d5250f HTTP/1.0" 202 Accepted 2026-03-21 08:56:32,160 - lava_callback - INFO - Starting async job submit for node 69bbba32ec3f435d20d5250f 2026-03-21 08:56:32,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba32ec3f435d20d5250f HTTP/1.1" 200 None 2026-03-21 08:56:32,177 - lava_callback - DEBUG - Processing node 69bbba32ec3f435d20d5250f: baseline-arm64-clabbe 2026-03-21 08:56:32,177 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba32ec3f435d20d5250f 2026-03-21 08:56:32,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba32ec3f435d20d5250f HTTP/1.1" 200 None 2026-03-21 08:56:32,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:32,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba32ec3f435d20d5250f HTTP/1.1" 200 None 2026-03-21 08:56:32,241 - lava_callback - ERROR - Error processing callback for node 69bbba32ec3f435d20d5250f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:32,832 - lava_callback - INFO - Received callback for node 69bbba33ec3f435d20d52512 INFO: 172.20.0.1:49890 - "POST /node/69bbba33ec3f435d20d52512 HTTP/1.0" 202 Accepted 2026-03-21 08:56:32,838 - lava_callback - INFO - Starting async job submit for node 69bbba33ec3f435d20d52512 2026-03-21 08:56:32,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba33ec3f435d20d52512 HTTP/1.1" 200 None 2026-03-21 08:56:32,854 - lava_callback - DEBUG - Processing node 69bbba33ec3f435d20d52512: baseline-arm64-clabbe 2026-03-21 08:56:32,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba33ec3f435d20d52512 2026-03-21 08:56:32,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba33ec3f435d20d52512 HTTP/1.1" 200 None 2026-03-21 08:56:32,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:32,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:32,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba33ec3f435d20d52512 HTTP/1.1" 200 None 2026-03-21 08:56:32,907 - lava_callback - ERROR - Error processing callback for node 69bbba33ec3f435d20d52512: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:33,491 - lava_callback - INFO - Received callback for node 69bbba34ec3f435d20d52515 INFO: 172.20.0.1:49900 - "POST /node/69bbba34ec3f435d20d52515 HTTP/1.0" 202 Accepted 2026-03-21 08:56:33,497 - lava_callback - INFO - Starting async job submit for node 69bbba34ec3f435d20d52515 2026-03-21 08:56:33,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:33,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba34ec3f435d20d52515 HTTP/1.1" 200 None 2026-03-21 08:56:33,515 - lava_callback - DEBUG - Processing node 69bbba34ec3f435d20d52515: baseline-arm64-clabbe 2026-03-21 08:56:33,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba34ec3f435d20d52515 2026-03-21 08:56:33,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:33,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba34ec3f435d20d52515 HTTP/1.1" 200 None 2026-03-21 08:56:33,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:33,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:33,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:33,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba34ec3f435d20d52515 HTTP/1.1" 200 None 2026-03-21 08:56:33,587 - lava_callback - ERROR - Error processing callback for node 69bbba34ec3f435d20d52515: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:34,120 - lava_callback - INFO - Received callback for node 69bbba35ec3f435d20d5251c INFO: 172.20.0.1:49916 - "POST /node/69bbba35ec3f435d20d5251c HTTP/1.0" 202 Accepted 2026-03-21 08:56:34,125 - lava_callback - INFO - Starting async job submit for node 69bbba35ec3f435d20d5251c 2026-03-21 08:56:34,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba35ec3f435d20d5251c HTTP/1.1" 200 None 2026-03-21 08:56:34,140 - lava_callback - DEBUG - Processing node 69bbba35ec3f435d20d5251c: baseline-arm64-clabbe 2026-03-21 08:56:34,141 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba35ec3f435d20d5251c 2026-03-21 08:56:34,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba35ec3f435d20d5251c HTTP/1.1" 200 None 2026-03-21 08:56:34,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:34,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba35ec3f435d20d5251c HTTP/1.1" 200 None 2026-03-21 08:56:34,201 - lava_callback - ERROR - Error processing callback for node 69bbba35ec3f435d20d5251c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 08:56:34,785 - lava_callback - INFO - Received callback for node 69bbba36ec3f435d20d5251f INFO: 172.20.0.1:49932 - "POST /node/69bbba36ec3f435d20d5251f HTTP/1.0" 202 Accepted 2026-03-21 08:56:34,791 - lava_callback - INFO - Starting async job submit for node 69bbba36ec3f435d20d5251f 2026-03-21 08:56:34,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba36ec3f435d20d5251f HTTP/1.1" 200 None 2026-03-21 08:56:34,814 - lava_callback - DEBUG - Processing node 69bbba36ec3f435d20d5251f: baseline-arm64-clabbe 2026-03-21 08:56:34,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba36ec3f435d20d5251f 2026-03-21 08:56:34,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba36ec3f435d20d5251f HTTP/1.1" 200 None 2026-03-21 08:56:34,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 08:56:34,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:56:34,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba36ec3f435d20d5251f HTTP/1.1" 200 None 2026-03-21 08:56:34,881 - lava_callback - ERROR - Error processing callback for node 69bbba36ec3f435d20d5251f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35116 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:57:31,507 - lava_callback - INFO - Received callback for node 69bbba67ec3f435d20d52587 INFO: 172.20.0.1:39888 - "POST /node/69bbba67ec3f435d20d52587 HTTP/1.0" 202 Accepted 2026-03-21 08:57:31,513 - lava_callback - INFO - Starting async job submit for node 69bbba67ec3f435d20d52587 2026-03-21 08:57:31,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:57:31,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba67ec3f435d20d52587 HTTP/1.1" 200 None 2026-03-21 08:57:31,529 - lava_callback - DEBUG - Processing node 69bbba67ec3f435d20d52587: baseline-arm-clabbe 2026-03-21 08:57:31,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbba67ec3f435d20d52587 2026-03-21 08:57:31,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:57:31,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbba67ec3f435d20d52587 HTTP/1.1" 200 None 2026-03-21 08:57:31,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:57:31,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:57:31,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:57:31,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbba67ec3f435d20d52587 HTTP/1.1" 200 None 2026-03-21 08:57:31,631 - lava_callback - ERROR - Error processing callback for node 69bbba67ec3f435d20d52587: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33612 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:59:11,583 - lava_callback - INFO - Received callback for node 69bbbad6ec3f435d20d52682 INFO: 172.20.0.1:51704 - "POST /node/69bbbad6ec3f435d20d52682 HTTP/1.0" 202 Accepted 2026-03-21 08:59:11,589 - lava_callback - INFO - Starting async job submit for node 69bbbad6ec3f435d20d52682 2026-03-21 08:59:11,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:11,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbad6ec3f435d20d52682 HTTP/1.1" 200 None 2026-03-21 08:59:11,608 - lava_callback - DEBUG - Processing node 69bbbad6ec3f435d20d52682: baseline-arm-clabbe 2026-03-21 08:59:11,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbbad6ec3f435d20d52682 INFO: 172.20.0.1:51708 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:59:11,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:11,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbad6ec3f435d20d52682 HTTP/1.1" 200 None 2026-03-21 08:59:11,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:11,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 08:59:11,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:11,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbbad6ec3f435d20d52682 HTTP/1.1" 200 None 2026-03-21 08:59:11,691 - lava_callback - ERROR - Error processing callback for node 69bbbad6ec3f435d20d52682: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36550 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:59:39,193 - lava_callback - INFO - Received callback for node 69be595d448e074b3ffe1727 INFO: 172.20.0.1:36566 - "POST /node/69be595d448e074b3ffe1727 HTTP/1.0" 202 Accepted 2026-03-21 08:59:39,241 - lava_callback - INFO - Starting async job submit for node 69be595d448e074b3ffe1727 2026-03-21 08:59:39,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:39,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595d448e074b3ffe1727 HTTP/1.1" 200 None 2026-03-21 08:59:39,280 - lava_callback - DEBUG - Processing node 69be595d448e074b3ffe1727: baseline-arm-pengutronix 2026-03-21 08:59:39,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:59:39,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:59:39,534 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595d448e074b3ffe1727/log.txt.gz 2026-03-21 08:59:39,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:59:39,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:59:39,788 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595d448e074b3ffe1727/lava-logs.yaml 2026-03-21 08:59:39,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:59:39,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:59:39,941 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be595d448e074b3ffe1727/lava_callback.json.gz DEBUG: Setup failed for 69be595d448e074b3ffe1727. Transitting job node result: incomplete -> fail 2026-03-21 08:59:39,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:40,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be595d448e074b3ffe1727 HTTP/1.1" 200 None 2026-03-21 08:59:40,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:40,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 08:59:40,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:59:40,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be595d448e074b3ffe1727 HTTP/1.1" 200 None 2026-03-21 08:59:40,120 - lava_callback - INFO - Completed processing callback for node 69be595d448e074b3ffe1727 INFO: 172.20.0.1:59174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:00:00,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:00:00,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50678 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:01:11,632 - lava_callback - INFO - Received callback for node 69bbbb45ec3f435d20d526ce INFO: 172.20.0.1:39378 - "POST /node/69bbbb45ec3f435d20d526ce HTTP/1.0" 202 Accepted 2026-03-21 09:01:11,637 - lava_callback - INFO - Starting async job submit for node 69bbbb45ec3f435d20d526ce 2026-03-21 09:01:11,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:39392 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:01:11,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbb45ec3f435d20d526ce HTTP/1.1" 200 None 2026-03-21 09:01:11,657 - lava_callback - DEBUG - Processing node 69bbbb45ec3f435d20d526ce: baseline-arm-clabbe 2026-03-21 09:01:11,657 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbbb45ec3f435d20d526ce 2026-03-21 09:01:11,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:11,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbb45ec3f435d20d526ce HTTP/1.1" 200 None 2026-03-21 09:01:11,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:11,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 09:01:11,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:11,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbbb45ec3f435d20d526ce HTTP/1.1" 200 None 2026-03-21 09:01:11,721 - lava_callback - ERROR - Error processing callback for node 69bbbb45ec3f435d20d526ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:01:51,332 - lava_callback - INFO - Received callback for node 69be5e6a448e074b3ffe1a3a INFO: 172.20.0.1:35478 - "POST /node/69be5e6a448e074b3ffe1a3a HTTP/1.0" 202 Accepted 2026-03-21 09:01:51,338 - lava_callback - INFO - Starting async job submit for node 69be5e6a448e074b3ffe1a3a 2026-03-21 09:01:51,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:51,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5e6a448e074b3ffe1a3a HTTP/1.1" 200 None 2026-03-21 09:01:51,367 - lava_callback - DEBUG - Processing node 69be5e6a448e074b3ffe1a3a: baseline-arm 2026-03-21 09:01:51,372 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:01:51,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:01:51,566 - lava_callback - INFO - Received callback for node 69bbbb6cec3f435d20d526ea INFO: 172.20.0.1:35484 - "POST /node/69bbbb6cec3f435d20d526ea HTTP/1.0" 202 Accepted 2026-03-21 09:01:51,572 - lava_callback - INFO - Starting async job submit for node 69bbbb6cec3f435d20d526ea 2026-03-21 09:01:51,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:51,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbb6cec3f435d20d526ea HTTP/1.1" 200 None 2026-03-21 09:01:51,601 - lava_callback - DEBUG - Processing node 69bbbb6cec3f435d20d526ea: baseline-arm-clabbe 2026-03-21 09:01:51,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbbb6cec3f435d20d526ea 2026-03-21 09:01:51,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:51,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbb6cec3f435d20d526ea HTTP/1.1" 200 None 2026-03-21 09:01:51,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:51,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c7f91727ff69d6f21d89 HTTP/1.1" 200 None 2026-03-21 09:01:51,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:51,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbbb6cec3f435d20d526ea HTTP/1.1" 200 None 2026-03-21 09:01:51,665 - lava_callback - ERROR - Error processing callback for node 69bbbb6cec3f435d20d526ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 09:01:51,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:01:51,843 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be5e6a448e074b3ffe1a3a/lava-logs.yaml 2026-03-21 09:01:51,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:01:51,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:01:51,990 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be5e6a448e074b3ffe1a3a/lava_callback.json.gz 2026-03-21 09:01:51,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:52,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5e6a448e074b3ffe1a3a HTTP/1.1" 200 None 2026-03-21 09:01:52,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:52,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 09:01:52,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:01:52,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5e6a448e074b3ffe1a3a HTTP/1.1" 200 None 2026-03-21 09:01:52,058 - lava_callback - INFO - Completed processing callback for node 69be5e6a448e074b3ffe1a3a INFO: 172.20.0.1:35490 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:02:00,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:02:00,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33938 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:05:53,278 - lava_callback - INFO - Received callback for node 69be5f4c448e074b3ffe2045 INFO: 172.20.0.1:39004 - "POST /node/69be5f4c448e074b3ffe2045 HTTP/1.0" 202 Accepted 2026-03-21 09:05:53,284 - lava_callback - INFO - Starting async job submit for node 69be5f4c448e074b3ffe2045 2026-03-21 09:05:53,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:05:53,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4c448e074b3ffe2045 HTTP/1.1" 200 None 2026-03-21 09:05:53,317 - lava_callback - DEBUG - Processing node 69be5f4c448e074b3ffe2045: baseline-x86 2026-03-21 09:05:53,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:05:53,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:05:53,633 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4c448e074b3ffe2045/log.txt.gz 2026-03-21 09:05:53,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:05:53,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:05:53,942 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4c448e074b3ffe2045/lava-logs.yaml 2026-03-21 09:05:53,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:05:54,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:05:54,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4c448e074b3ffe2045/lava_callback.json.gz 2026-03-21 09:05:54,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:05:54,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4c448e074b3ffe2045 HTTP/1.1" 200 None 2026-03-21 09:05:54,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:05:54,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5478448e074b3ffe1656 HTTP/1.1" 200 None 2026-03-21 09:05:54,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:05:54,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5f4c448e074b3ffe2045 HTTP/1.1" 200 None 2026-03-21 09:05:54,170 - lava_callback - INFO - Completed processing callback for node 69be5f4c448e074b3ffe2045 INFO: 172.20.0.1:39016 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:06:00,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:00,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36758 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:06:09,856 - lava_callback - INFO - Received callback for node 69be5f4e448e074b3ffe2050 INFO: 172.20.0.1:43666 - "POST /node/69be5f4e448e074b3ffe2050 HTTP/1.0" 202 Accepted 2026-03-21 09:06:09,863 - lava_callback - INFO - Starting async job submit for node 69be5f4e448e074b3ffe2050 2026-03-21 09:06:09,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:09,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4e448e074b3ffe2050 HTTP/1.1" 200 None 2026-03-21 09:06:09,900 - lava_callback - DEBUG - Processing node 69be5f4e448e074b3ffe2050: baseline-x86 2026-03-21 09:06:10,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:06:10,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:06:10,199 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4e448e074b3ffe2050/log.txt.gz 2026-03-21 09:06:10,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:06:10,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:06:10,522 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4e448e074b3ffe2050/lava-logs.yaml 2026-03-21 09:06:10,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:06:10,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:06:10,670 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4e448e074b3ffe2050/lava_callback.json.gz 2026-03-21 09:06:10,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:10,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4e448e074b3ffe2050 HTTP/1.1" 200 None 2026-03-21 09:06:10,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:10,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5478448e074b3ffe1656 HTTP/1.1" 200 None 2026-03-21 09:06:10,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:10,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5f4e448e074b3ffe2050 HTTP/1.1" 200 None 2026-03-21 09:06:10,749 - lava_callback - INFO - Completed processing callback for node 69be5f4e448e074b3ffe2050 INFO: 172.20.0.1:43676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50448 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:06:30,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:30,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51730 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:06:42,665 - lava_callback - INFO - Received callback for node 69be5f4d448e074b3ffe204d INFO: 172.20.0.1:47260 - "POST /node/69be5f4d448e074b3ffe204d HTTP/1.0" 202 Accepted 2026-03-21 09:06:42,671 - lava_callback - INFO - Starting async job submit for node 69be5f4d448e074b3ffe204d 2026-03-21 09:06:42,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:42,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4d448e074b3ffe204d HTTP/1.1" 200 None 2026-03-21 09:06:42,704 - lava_callback - DEBUG - Processing node 69be5f4d448e074b3ffe204d: baseline-x86 2026-03-21 09:06:42,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:06:43,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:06:43,130 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4d448e074b3ffe204d/log.txt.gz 2026-03-21 09:06:43,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:06:43,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:06:43,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4d448e074b3ffe204d/lava-logs.yaml 2026-03-21 09:06:43,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:06:43,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:06:43,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4d448e074b3ffe204d/lava_callback.json.gz 2026-03-21 09:06:43,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:43,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4d448e074b3ffe204d HTTP/1.1" 200 None 2026-03-21 09:06:43,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:43,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5478448e074b3ffe1656 HTTP/1.1" 200 None 2026-03-21 09:06:43,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:06:43,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5f4d448e074b3ffe204d HTTP/1.1" 200 None 2026-03-21 09:06:43,713 - lava_callback - INFO - Completed processing callback for node 69be5f4d448e074b3ffe204d INFO: 172.20.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39208 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:07:00,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:07:00,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59978 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:07:08,388 - lava_callback - INFO - Received callback for node 69be5f4d448e074b3ffe204a INFO: 172.20.0.1:59980 - "POST /node/69be5f4d448e074b3ffe204a HTTP/1.0" 202 Accepted 2026-03-21 09:07:08,394 - lava_callback - INFO - Starting async job submit for node 69be5f4d448e074b3ffe204a 2026-03-21 09:07:08,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:07:08,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4d448e074b3ffe204a HTTP/1.1" 200 None 2026-03-21 09:07:08,428 - lava_callback - DEBUG - Processing node 69be5f4d448e074b3ffe204a: baseline-x86 2026-03-21 09:07:08,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:07:08,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:07:08,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4d448e074b3ffe204a/log.txt.gz 2026-03-21 09:07:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:07:09,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:07:09,226 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4d448e074b3ffe204a/lava-logs.yaml 2026-03-21 09:07:09,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:07:09,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:07:09,360 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f4d448e074b3ffe204a/lava_callback.json.gz 2026-03-21 09:07:09,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:07:09,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f4d448e074b3ffe204a HTTP/1.1" 200 None 2026-03-21 09:07:09,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:07:09,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5478448e074b3ffe1656 HTTP/1.1" 200 None 2026-03-21 09:07:09,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:07:09,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5f4d448e074b3ffe204a HTTP/1.1" 200 None 2026-03-21 09:07:09,439 - lava_callback - INFO - Completed processing callback for node 69be5f4d448e074b3ffe204a INFO: 172.20.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45374 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:07:30,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:07:30,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59566 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:18:24,985 - lava_callback - INFO - Received callback for node 69bbbf5dec3f435d20d52846 INFO: 172.20.0.1:44176 - "POST /node/69bbbf5dec3f435d20d52846 HTTP/1.0" 202 Accepted 2026-03-21 09:18:24,991 - lava_callback - INFO - Starting async job submit for node 69bbbf5dec3f435d20d52846 2026-03-21 09:18:24,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:18:25,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbf5dec3f435d20d52846 HTTP/1.1" 200 None 2026-03-21 09:18:25,008 - lava_callback - DEBUG - Processing node 69bbbf5dec3f435d20d52846: baseline-arm64-baylibre 2026-03-21 09:18:25,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbbf5dec3f435d20d52846 2026-03-21 09:18:25,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:18:25,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbbf5dec3f435d20d52846 HTTP/1.1" 200 None 2026-03-21 09:18:25,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:18:25,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:18:25,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:18:25,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbbf5dec3f435d20d52846 HTTP/1.1" 200 None 2026-03-21 09:18:25,067 - lava_callback - ERROR - Error processing callback for node 69bbbf5dec3f435d20d52846: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:21:30,348 - lava_callback - INFO - Received callback for node 69be5ef9448e074b3ffe2022 INFO: 172.20.0.1:37960 - "POST /node/69be5ef9448e074b3ffe2022 HTTP/1.0" 202 Accepted 2026-03-21 09:21:30,355 - lava_callback - INFO - Starting async job submit for node 69be5ef9448e074b3ffe2022 2026-03-21 09:21:30,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:30,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5ef9448e074b3ffe2022 HTTP/1.1" 200 None 2026-03-21 09:21:30,463 - lava_callback - DEBUG - Processing node 69be5ef9448e074b3ffe2022: ltp-smoke 2026-03-21 09:21:31,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:21:31,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:21:31,281 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69be5ef9448e074b3ffe2022/log.txt.gz 2026-03-21 09:21:31,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:21:31,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:21:31,869 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69be5ef9448e074b3ffe2022/lava-logs.yaml 2026-03-21 09:21:31,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:21:32,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:21:32,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69be5ef9448e074b3ffe2022/lava_callback.json.gz 2026-03-21 09:21:32,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:32,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5ef9448e074b3ffe2022 HTTP/1.1" 200 None 2026-03-21 09:21:32,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:32,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5456448e074b3ffe1633 HTTP/1.1" 200 None 2026-03-21 09:21:32,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:32,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5ef9448e074b3ffe2022 HTTP/1.1" 200 None 2026-03-21 09:21:32,410 - lava_callback - INFO - Completed processing callback for node 69be5ef9448e074b3ffe2022 INFO: 172.20.0.1:37974 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:21:45,091 - lava_callback - INFO - Received callback for node 69bbc01cec3f435d20d5286e INFO: 172.20.0.1:48320 - "POST /node/69bbc01cec3f435d20d5286e HTTP/1.0" 202 Accepted 2026-03-21 09:21:45,097 - lava_callback - INFO - Starting async job submit for node 69bbc01cec3f435d20d5286e 2026-03-21 09:21:45,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:45,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc01cec3f435d20d5286e HTTP/1.1" 200 None 2026-03-21 09:21:45,119 - lava_callback - DEBUG - Processing node 69bbc01cec3f435d20d5286e: baseline-arm64-baylibre 2026-03-21 09:21:45,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc01cec3f435d20d5286e 2026-03-21 09:21:45,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:45,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc01cec3f435d20d5286e HTTP/1.1" 200 None 2026-03-21 09:21:45,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:45,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:21:45,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:21:45,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc01cec3f435d20d5286e HTTP/1.1" 200 None 2026-03-21 09:21:45,222 - lava_callback - ERROR - Error processing callback for node 69bbc01cec3f435d20d5286e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:22:00,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:00,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:22:39,003 - lava_callback - INFO - Received callback for node 69be6335448e074b3ffe2103 INFO: 172.20.0.1:58166 - "POST /node/69be6335448e074b3ffe2103 HTTP/1.0" 202 Accepted 2026-03-21 09:22:39,009 - lava_callback - INFO - Starting async job submit for node 69be6335448e074b3ffe2103 2026-03-21 09:22:39,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:39,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6335448e074b3ffe2103 HTTP/1.1" 200 None 2026-03-21 09:22:39,044 - lava_callback - DEBUG - Processing node 69be6335448e074b3ffe2103: baseline-arm64 2026-03-21 09:22:39,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:39,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:39,441 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6335448e074b3ffe2103/log.txt.gz 2026-03-21 09:22:39,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:39,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:39,904 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6335448e074b3ffe2103/lava-logs.yaml 2026-03-21 09:22:39,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:40,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:40,080 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6335448e074b3ffe2103/lava_callback.json.gz 2026-03-21 09:22:40,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:40,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6335448e074b3ffe2103 HTTP/1.1" 200 None 2026-03-21 09:22:40,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:40,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:22:40,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:40,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6335448e074b3ffe2103 HTTP/1.1" 200 None 2026-03-21 09:22:40,194 - lava_callback - INFO - Completed processing callback for node 69be6335448e074b3ffe2103 INFO: 172.20.0.1:34380 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:22:50,361 - lava_callback - INFO - Received callback for node 69be634f448e074b3ffe2168 INFO: 172.20.0.1:46550 - "POST /node/69be634f448e074b3ffe2168 HTTP/1.0" 202 Accepted 2026-03-21 09:22:50,367 - lava_callback - INFO - Starting async job submit for node 69be634f448e074b3ffe2168 2026-03-21 09:22:50,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634f448e074b3ffe2168 HTTP/1.1" 200 None 2026-03-21 09:22:50,383 - lava_callback - DEBUG - Processing node 69be634f448e074b3ffe2168: baseline-nfs-arm64 2026-03-21 09:22:50,385 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:22:50,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:50,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:50,505 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634f448e074b3ffe2168/lava-logs.yaml 2026-03-21 09:22:50,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:50,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:50,687 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634f448e074b3ffe2168/lava_callback.json.gz 2026-03-21 09:22:50,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:50,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634f448e074b3ffe2168 HTTP/1.1" 200 None 2026-03-21 09:22:50,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:50,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:22:50,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:50,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be634f448e074b3ffe2168 HTTP/1.1" 200 None 2026-03-21 09:22:50,747 - lava_callback - INFO - Completed processing callback for node 69be634f448e074b3ffe2168 2026-03-21 09:22:52,511 - lava_callback - INFO - Received callback for node 69be6341448e074b3ffe2127 INFO: 172.20.0.1:46554 - "POST /node/69be6341448e074b3ffe2127 HTTP/1.0" 202 Accepted 2026-03-21 09:22:52,518 - lava_callback - INFO - Starting async job submit for node 69be6341448e074b3ffe2127 2026-03-21 09:22:52,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:52,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6341448e074b3ffe2127 HTTP/1.1" 200 None 2026-03-21 09:22:52,554 - lava_callback - DEBUG - Processing node 69be6341448e074b3ffe2127: baseline-arm64 2026-03-21 09:22:52,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:52,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:52,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6341448e074b3ffe2127/log.txt.gz 2026-03-21 09:22:52,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:53,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:53,163 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6341448e074b3ffe2127/lava-logs.yaml 2026-03-21 09:22:53,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:22:53,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:22:53,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6341448e074b3ffe2127/lava_callback.json.gz 2026-03-21 09:22:53,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:53,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6341448e074b3ffe2127 HTTP/1.1" 200 None 2026-03-21 09:22:53,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:53,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:22:53,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:22:53,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6341448e074b3ffe2127 HTTP/1.1" 200 None 2026-03-21 09:22:53,489 - lava_callback - INFO - Completed processing callback for node 69be6341448e074b3ffe2127 INFO: 172.20.0.1:46566 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:23:00,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:00,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 09:23:04,093 - lava_callback - INFO - Received callback for node 69be6337448e074b3ffe2109 INFO: 172.20.0.1:54110 - "POST /node/69be6337448e074b3ffe2109 HTTP/1.0" 202 Accepted 2026-03-21 09:23:04,099 - lava_callback - INFO - Starting async job submit for node 69be6337448e074b3ffe2109 2026-03-21 09:23:04,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:04,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6337448e074b3ffe2109 HTTP/1.1" 200 None 2026-03-21 09:23:04,138 - lava_callback - DEBUG - Processing node 69be6337448e074b3ffe2109: baseline-arm64 2026-03-21 09:23:04,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:04,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:04,534 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6337448e074b3ffe2109/log.txt.gz 2026-03-21 09:23:04,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:04,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:04,921 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6337448e074b3ffe2109/lava-logs.yaml 2026-03-21 09:23:04,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:05,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:05,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6337448e074b3ffe2109/lava_callback.json.gz 2026-03-21 09:23:05,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:05,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6337448e074b3ffe2109 HTTP/1.1" 200 None 2026-03-21 09:23:05,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:05,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:05,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:05,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6337448e074b3ffe2109 HTTP/1.1" 200 None 2026-03-21 09:23:05,231 - lava_callback - INFO - Completed processing callback for node 69be6337448e074b3ffe2109 INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60562 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 09:23:16,016 - lava_callback - INFO - Received callback for node 69be633a448e074b3ffe2115 INFO: 172.20.0.1:60570 - "POST /node/69be633a448e074b3ffe2115 HTTP/1.0" 202 Accepted 2026-03-21 09:23:16,022 - lava_callback - INFO - Starting async job submit for node 69be633a448e074b3ffe2115 2026-03-21 09:23:16,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:16,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633a448e074b3ffe2115 HTTP/1.1" 200 None 2026-03-21 09:23:16,077 - lava_callback - DEBUG - Processing node 69be633a448e074b3ffe2115: baseline-arm64 2026-03-21 09:23:16,175 - lava_callback - INFO - Received callback for node 69be5f64448e074b3ffe2056 INFO: 172.20.0.1:60584 - "POST /node/69be5f64448e074b3ffe2056 HTTP/1.0" 202 Accepted 2026-03-21 09:23:16,210 - lava_callback - INFO - Starting async job submit for node 69be5f64448e074b3ffe2056 2026-03-21 09:23:16,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:16,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:16,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f64448e074b3ffe2056 HTTP/1.1" 200 None 2026-03-21 09:23:16,336 - lava_callback - DEBUG - Processing node 69be5f64448e074b3ffe2056: baseline-arm 2026-03-21 09:23:16,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:16,490 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633a448e074b3ffe2115/log.txt.gz INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:23:16,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:16,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:17,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633a448e074b3ffe2115/lava-logs.yaml 2026-03-21 09:23:17,053 - lava_callback - INFO - Received callback for node 69be6339448e074b3ffe2112 2026-03-21 09:23:17,068 - lava_callback - INFO - Starting async job submit for node 69be6339448e074b3ffe2112 INFO: 172.20.0.1:60602 - "POST /node/69be6339448e074b3ffe2112 HTTP/1.0" 202 Accepted 2026-03-21 09:23:17,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:17,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:17,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:17,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6339448e074b3ffe2112 HTTP/1.1" 200 None 2026-03-21 09:23:17,141 - lava_callback - DEBUG - Processing node 69be6339448e074b3ffe2112: baseline-arm64 2026-03-21 09:23:17,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:17,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:17,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be5f64448e074b3ffe2056/log.txt.gz 2026-03-21 09:23:17,359 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633a448e074b3ffe2115/lava_callback.json.gz 2026-03-21 09:23:17,410 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:23:17,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:17,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633a448e074b3ffe2115 HTTP/1.1" 200 None 2026-03-21 09:23:17,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:17,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:17,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:17,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:17,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be633a448e074b3ffe2115 HTTP/1.1" 200 None 2026-03-21 09:23:17,883 - lava_callback - INFO - Completed processing callback for node 69be633a448e074b3ffe2115 2026-03-21 09:23:17,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:17,923 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6339448e074b3ffe2112/lava-logs.yaml 2026-03-21 09:23:17,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:18,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:18,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:18,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6339448e074b3ffe2112/lava_callback.json.gz 2026-03-21 09:23:18,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:18,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6339448e074b3ffe2112 HTTP/1.1" 200 None 2026-03-21 09:23:18,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:18,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:18,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:18,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6339448e074b3ffe2112 HTTP/1.1" 200 None 2026-03-21 09:23:18,290 - lava_callback - INFO - Completed processing callback for node 69be6339448e074b3ffe2112 2026-03-21 09:23:18,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:18,329 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be5f64448e074b3ffe2056/lava-logs.yaml 2026-03-21 09:23:18,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:18,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:18,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be5f64448e074b3ffe2056/lava_callback.json.gz DEBUG: Setup failed for 69be5f64448e074b3ffe2056. Transitting job node result: incomplete -> fail 2026-03-21 09:23:18,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:18,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f64448e074b3ffe2056 HTTP/1.1" 200 None 2026-03-21 09:23:18,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:18,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 09:23:18,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:18,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5f64448e074b3ffe2056 HTTP/1.1" 200 None 2026-03-21 09:23:18,855 - lava_callback - INFO - Completed processing callback for node 69be5f64448e074b3ffe2056 2026-03-21 09:23:21,093 - lava_callback - INFO - Received callback for node 69be633e448e074b3ffe211e INFO: 172.20.0.1:36340 - "POST /node/69be633e448e074b3ffe211e HTTP/1.0" 202 Accepted 2026-03-21 09:23:21,100 - lava_callback - INFO - Starting async job submit for node 69be633e448e074b3ffe211e 2026-03-21 09:23:21,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:21,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633e448e074b3ffe211e HTTP/1.1" 200 None 2026-03-21 09:23:21,146 - lava_callback - DEBUG - Processing node 69be633e448e074b3ffe211e: baseline-arm64 2026-03-21 09:23:21,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:21,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:21,616 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633e448e074b3ffe211e/log.txt.gz 2026-03-21 09:23:21,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:22,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:22,092 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633e448e074b3ffe211e/lava-logs.yaml 2026-03-21 09:23:22,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:22,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:22,229 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633e448e074b3ffe211e/lava_callback.json.gz 2026-03-21 09:23:22,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:22,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633e448e074b3ffe211e HTTP/1.1" 200 None 2026-03-21 09:23:22,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:22,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:22,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:22,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be633e448e074b3ffe211e HTTP/1.1" 200 None 2026-03-21 09:23:22,321 - lava_callback - INFO - Completed processing callback for node 69be633e448e074b3ffe211e INFO: 172.20.0.1:36346 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:23:29,599 - lava_callback - INFO - Received callback for node 69be6340448e074b3ffe2124 INFO: 172.20.0.1:36348 - "POST /node/69be6340448e074b3ffe2124 HTTP/1.0" 202 Accepted 2026-03-21 09:23:29,606 - lava_callback - INFO - Starting async job submit for node 69be6340448e074b3ffe2124 2026-03-21 09:23:29,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:29,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6340448e074b3ffe2124 HTTP/1.1" 200 None 2026-03-21 09:23:29,641 - lava_callback - DEBUG - Processing node 69be6340448e074b3ffe2124: baseline-arm64 2026-03-21 09:23:30,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:30,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:30,130 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6340448e074b3ffe2124/log.txt.gz 2026-03-21 09:23:30,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:30,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:30,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6340448e074b3ffe2124/lava-logs.yaml 2026-03-21 09:23:30,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:30,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:30,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6340448e074b3ffe2124/lava_callback.json.gz 2026-03-21 09:23:30,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:30,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6340448e074b3ffe2124 HTTP/1.1" 200 None 2026-03-21 09:23:30,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:30,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:30,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:30,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6340448e074b3ffe2124 HTTP/1.1" 200 None 2026-03-21 09:23:30,798 - lava_callback - INFO - Completed processing callback for node 69be6340448e074b3ffe2124 2026-03-21 09:23:30,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:30,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-21 09:23:31,803 - lava_callback - INFO - Received callback for node 69be633f448e074b3ffe2121 INFO: 172.20.0.1:59834 - "POST /node/69be633f448e074b3ffe2121 HTTP/1.0" 202 Accepted 2026-03-21 09:23:31,809 - lava_callback - INFO - Starting async job submit for node 69be633f448e074b3ffe2121 2026-03-21 09:23:31,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:31,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633f448e074b3ffe2121 HTTP/1.1" 200 None 2026-03-21 09:23:31,849 - lava_callback - DEBUG - Processing node 69be633f448e074b3ffe2121: baseline-arm64 2026-03-21 09:23:32,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:32,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:32,307 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633f448e074b3ffe2121/log.txt.gz 2026-03-21 09:23:32,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:32,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:32,681 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633f448e074b3ffe2121/lava-logs.yaml 2026-03-21 09:23:32,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:32,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:32,823 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633f448e074b3ffe2121/lava_callback.json.gz 2026-03-21 09:23:32,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:32,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633f448e074b3ffe2121 HTTP/1.1" 200 None 2026-03-21 09:23:32,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:32,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:32,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:32,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be633f448e074b3ffe2121 HTTP/1.1" 200 None 2026-03-21 09:23:32,988 - lava_callback - INFO - Completed processing callback for node 69be633f448e074b3ffe2121 2026-03-21 09:23:35,366 - lava_callback - INFO - Received callback for node 69be6342448e074b3ffe212a INFO: 172.20.0.1:59846 - "POST /node/69be6342448e074b3ffe212a HTTP/1.0" 202 Accepted 2026-03-21 09:23:35,372 - lava_callback - INFO - Starting async job submit for node 69be6342448e074b3ffe212a 2026-03-21 09:23:35,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:35,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6342448e074b3ffe212a HTTP/1.1" 200 None 2026-03-21 09:23:35,410 - lava_callback - DEBUG - Processing node 69be6342448e074b3ffe212a: baseline-arm64 2026-03-21 09:23:35,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:35,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:35,743 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6342448e074b3ffe212a/log.txt.gz 2026-03-21 09:23:35,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:36,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:36,178 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6342448e074b3ffe212a/lava-logs.yaml 2026-03-21 09:23:36,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:36,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:36,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6342448e074b3ffe212a/lava_callback.json.gz 2026-03-21 09:23:36,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:36,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6342448e074b3ffe212a HTTP/1.1" 200 None 2026-03-21 09:23:36,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:36,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:36,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:36,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6342448e074b3ffe212a HTTP/1.1" 200 None 2026-03-21 09:23:36,439 - lava_callback - INFO - Completed processing callback for node 69be6342448e074b3ffe212a INFO: 172.20.0.1:59850 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:23:37,151 - lava_callback - INFO - Received callback for node 69be633c448e074b3ffe211b INFO: 172.20.0.1:59864 - "POST /node/69be633c448e074b3ffe211b HTTP/1.0" 202 Accepted 2026-03-21 09:23:37,161 - lava_callback - INFO - Starting async job submit for node 69be633c448e074b3ffe211b 2026-03-21 09:23:37,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:37,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633c448e074b3ffe211b HTTP/1.1" 200 None 2026-03-21 09:23:37,202 - lava_callback - DEBUG - Processing node 69be633c448e074b3ffe211b: baseline-arm64 2026-03-21 09:23:37,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:37,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:37,572 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633c448e074b3ffe211b/log.txt.gz 2026-03-21 09:23:37,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:37,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:37,947 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633c448e074b3ffe211b/lava-logs.yaml 2026-03-21 09:23:37,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:38,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:38,091 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633c448e074b3ffe211b/lava_callback.json.gz 2026-03-21 09:23:38,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:38,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633c448e074b3ffe211b HTTP/1.1" 200 None 2026-03-21 09:23:38,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:38,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:38,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:38,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be633c448e074b3ffe211b HTTP/1.1" 200 None 2026-03-21 09:23:38,196 - lava_callback - INFO - Completed processing callback for node 69be633c448e074b3ffe211b 2026-03-21 09:23:38,272 - lava_callback - INFO - Received callback for node 69be6344448e074b3ffe2130 INFO: 172.20.0.1:59868 - "POST /node/69be6344448e074b3ffe2130 HTTP/1.0" 202 Accepted 2026-03-21 09:23:38,281 - lava_callback - INFO - Starting async job submit for node 69be6344448e074b3ffe2130 2026-03-21 09:23:38,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:38,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6344448e074b3ffe2130 HTTP/1.1" 200 None 2026-03-21 09:23:38,340 - lava_callback - DEBUG - Processing node 69be6344448e074b3ffe2130: baseline-arm64 2026-03-21 09:23:38,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:38,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:38,793 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6344448e074b3ffe2130/log.txt.gz 2026-03-21 09:23:38,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:39,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:39,171 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6344448e074b3ffe2130/lava-logs.yaml 2026-03-21 09:23:39,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:39,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:39,304 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6344448e074b3ffe2130/lava_callback.json.gz 2026-03-21 09:23:39,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:39,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6344448e074b3ffe2130 HTTP/1.1" 200 None 2026-03-21 09:23:39,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:39,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:39,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:39,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6344448e074b3ffe2130 HTTP/1.1" 200 None 2026-03-21 09:23:39,388 - lava_callback - INFO - Completed processing callback for node 69be6344448e074b3ffe2130 2026-03-21 09:23:41,442 - lava_callback - INFO - Received callback for node 69be6336448e074b3ffe2106 INFO: 172.20.0.1:56584 - "POST /node/69be6336448e074b3ffe2106 HTTP/1.0" 202 Accepted 2026-03-21 09:23:41,448 - lava_callback - INFO - Starting async job submit for node 69be6336448e074b3ffe2106 2026-03-21 09:23:41,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:41,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6336448e074b3ffe2106 HTTP/1.1" 200 None 2026-03-21 09:23:41,485 - lava_callback - DEBUG - Processing node 69be6336448e074b3ffe2106: baseline-arm64 2026-03-21 09:23:41,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:41,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:41,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6336448e074b3ffe2106/log.txt.gz 2026-03-21 09:23:41,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:42,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:42,119 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6336448e074b3ffe2106/lava-logs.yaml 2026-03-21 09:23:42,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:42,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:42,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6336448e074b3ffe2106/lava_callback.json.gz 2026-03-21 09:23:42,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:42,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6336448e074b3ffe2106 HTTP/1.1" 200 None 2026-03-21 09:23:42,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:42,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:42,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:42,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6336448e074b3ffe2106 HTTP/1.1" 200 None 2026-03-21 09:23:42,373 - lava_callback - INFO - Completed processing callback for node 69be6336448e074b3ffe2106 2026-03-21 09:23:43,813 - lava_callback - INFO - Received callback for node 69be634d448e074b3ffe2162 INFO: 172.20.0.1:56586 - "POST /node/69be634d448e074b3ffe2162 HTTP/1.0" 202 Accepted 2026-03-21 09:23:43,819 - lava_callback - INFO - Starting async job submit for node 69be634d448e074b3ffe2162 2026-03-21 09:23:43,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:43,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634d448e074b3ffe2162 HTTP/1.1" 200 None 2026-03-21 09:23:43,857 - lava_callback - DEBUG - Processing node 69be634d448e074b3ffe2162: baseline-nfs-arm64 2026-03-21 09:23:44,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:44,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:44,416 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634d448e074b3ffe2162/log.txt.gz 2026-03-21 09:23:44,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:44,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:44,902 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634d448e074b3ffe2162/lava-logs.yaml 2026-03-21 09:23:44,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:45,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:45,051 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634d448e074b3ffe2162/lava_callback.json.gz 2026-03-21 09:23:45,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:45,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634d448e074b3ffe2162 HTTP/1.1" 200 None 2026-03-21 09:23:45,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:45,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:45,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:45,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be634d448e074b3ffe2162 HTTP/1.1" 200 None 2026-03-21 09:23:45,130 - lava_callback - INFO - Completed processing callback for node 69be634d448e074b3ffe2162 INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:23:46,791 - lava_callback - INFO - Received callback for node 69be6346448e074b3ffe214c INFO: 172.20.0.1:56606 - "POST /node/69be6346448e074b3ffe214c HTTP/1.0" 202 Accepted 2026-03-21 09:23:46,797 - lava_callback - INFO - Starting async job submit for node 69be6346448e074b3ffe214c 2026-03-21 09:23:46,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:46,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6346448e074b3ffe214c HTTP/1.1" 200 None 2026-03-21 09:23:46,833 - lava_callback - DEBUG - Processing node 69be6346448e074b3ffe214c: baseline-arm64 2026-03-21 09:23:47,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:47,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:47,223 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6346448e074b3ffe214c/log.txt.gz 2026-03-21 09:23:47,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:47,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:47,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6346448e074b3ffe214c/lava-logs.yaml 2026-03-21 09:23:47,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:23:47,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:23:47,776 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6346448e074b3ffe214c/lava_callback.json.gz 2026-03-21 09:23:47,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:47,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6346448e074b3ffe214c HTTP/1.1" 200 None 2026-03-21 09:23:47,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:47,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:23:47,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:23:47,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6346448e074b3ffe214c HTTP/1.1" 200 None 2026-03-21 09:23:47,861 - lava_callback - INFO - Completed processing callback for node 69be6346448e074b3ffe214c INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:24:00,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:01,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-21 09:24:01,711 - lava_callback - INFO - Received callback for node 69be637c448e074b3ffe2202 INFO: 172.20.0.1:43172 - "POST /node/69be637c448e074b3ffe2202 HTTP/1.0" 202 Accepted 2026-03-21 09:24:01,718 - lava_callback - INFO - Starting async job submit for node 69be637c448e074b3ffe2202 2026-03-21 09:24:01,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:01,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be637c448e074b3ffe2202 HTTP/1.1" 200 None 2026-03-21 09:24:01,766 - lava_callback - DEBUG - Processing node 69be637c448e074b3ffe2202: baseline-arm64-foundriesio 2026-03-21 09:24:01,821 - lava_callback - INFO - Received callback for node 69be6351448e074b3ffe216b INFO: 172.20.0.1:43178 - "POST /node/69be6351448e074b3ffe216b HTTP/1.0" 202 Accepted 2026-03-21 09:24:01,843 - lava_callback - INFO - Starting async job submit for node 69be6351448e074b3ffe216b 2026-03-21 09:24:01,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:01,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6351448e074b3ffe216b HTTP/1.1" 200 None 2026-03-21 09:24:01,975 - lava_callback - DEBUG - Processing node 69be6351448e074b3ffe216b: baseline-nfs-arm64 2026-03-21 09:24:02,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:02,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:02,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:02,711 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69be637c448e074b3ffe2202/log.txt.gz 2026-03-21 09:24:02,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:02,800 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6351448e074b3ffe216b/log.txt.gz 2026-03-21 09:24:02,862 - lava_callback - INFO - Received callback for node 69be637b448e074b3ffe21ff INFO: 172.20.0.1:43192 - "POST /node/69be637b448e074b3ffe21ff HTTP/1.0" 202 Accepted 2026-03-21 09:24:02,957 - lava_callback - INFO - Starting async job submit for node 69be637b448e074b3ffe21ff 2026-03-21 09:24:03,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:03,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be637b448e074b3ffe21ff HTTP/1.1" 200 None 2026-03-21 09:24:03,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:03,186 - lava_callback - DEBUG - Processing node 69be637b448e074b3ffe21ff: baseline-arm64-foundriesio 2026-03-21 09:24:03,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:03,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:03,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:03,499 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69be637c448e074b3ffe2202/lava-logs.yaml 2026-03-21 09:24:03,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:03,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:03,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69be637b448e074b3ffe21ff/log.txt.gz 2026-03-21 09:24:03,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:03,643 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6351448e074b3ffe216b/lava-logs.yaml 2026-03-21 09:24:03,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:03,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:03,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69be637c448e074b3ffe2202/lava_callback.json.gz 2026-03-21 09:24:03,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:03,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:03,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be637c448e074b3ffe2202 HTTP/1.1" 200 None 2026-03-21 09:24:03,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:03,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:03,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:03,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be637c448e074b3ffe2202 HTTP/1.1" 200 None 2026-03-21 09:24:03,839 - lava_callback - INFO - Completed processing callback for node 69be637c448e074b3ffe2202 2026-03-21 09:24:03,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:03,955 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6351448e074b3ffe216b/lava_callback.json.gz 2026-03-21 09:24:03,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:04,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6351448e074b3ffe216b HTTP/1.1" 200 None 2026-03-21 09:24:04,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:04,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:04,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:04,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6351448e074b3ffe216b HTTP/1.1" 200 None 2026-03-21 09:24:04,084 - lava_callback - INFO - Completed processing callback for node 69be6351448e074b3ffe216b 2026-03-21 09:24:04,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:04,096 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69be637b448e074b3ffe21ff/lava-logs.yaml 2026-03-21 09:24:04,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:04,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:04,303 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69be637b448e074b3ffe21ff/lava_callback.json.gz 2026-03-21 09:24:04,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:04,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be637b448e074b3ffe21ff HTTP/1.1" 200 None 2026-03-21 09:24:04,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:04,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:04,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:04,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be637b448e074b3ffe21ff HTTP/1.1" 200 None 2026-03-21 09:24:04,394 - lava_callback - INFO - Completed processing callback for node 69be637b448e074b3ffe21ff 2026-03-21 09:24:05,984 - lava_callback - INFO - Received callback for node 69be6355448e074b3ffe2178 INFO: 172.20.0.1:43204 - "POST /node/69be6355448e074b3ffe2178 HTTP/1.0" 202 Accepted 2026-03-21 09:24:05,991 - lava_callback - INFO - Starting async job submit for node 69be6355448e074b3ffe2178 2026-03-21 09:24:06,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:06,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6355448e074b3ffe2178 HTTP/1.1" 200 None 2026-03-21 09:24:06,032 - lava_callback - DEBUG - Processing node 69be6355448e074b3ffe2178: baseline-nfs-arm64 2026-03-21 09:24:06,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:43212 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:24:06,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:06,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6355448e074b3ffe2178/log.txt.gz 2026-03-21 09:24:06,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:06,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:06,995 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6355448e074b3ffe2178/lava-logs.yaml 2026-03-21 09:24:07,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:07,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:07,158 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6355448e074b3ffe2178/lava_callback.json.gz 2026-03-21 09:24:07,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:07,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6355448e074b3ffe2178 HTTP/1.1" 200 None 2026-03-21 09:24:07,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:07,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:07,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:07,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6355448e074b3ffe2178 HTTP/1.1" 200 None 2026-03-21 09:24:07,242 - lava_callback - INFO - Completed processing callback for node 69be6355448e074b3ffe2178 2026-03-21 09:24:07,278 - lava_callback - INFO - Received callback for node 69be6353448e074b3ffe2171 INFO: 172.20.0.1:43222 - "POST /node/69be6353448e074b3ffe2171 HTTP/1.0" 202 Accepted 2026-03-21 09:24:07,291 - lava_callback - INFO - Starting async job submit for node 69be6353448e074b3ffe2171 2026-03-21 09:24:07,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:07,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6353448e074b3ffe2171 HTTP/1.1" 200 None 2026-03-21 09:24:07,360 - lava_callback - DEBUG - Processing node 69be6353448e074b3ffe2171: baseline-nfs-arm64 2026-03-21 09:24:07,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:07,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:07,906 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6353448e074b3ffe2171/log.txt.gz 2026-03-21 09:24:08,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:08,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:08,302 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6353448e074b3ffe2171/lava-logs.yaml 2026-03-21 09:24:08,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:08,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:08,477 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6353448e074b3ffe2171/lava_callback.json.gz 2026-03-21 09:24:08,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:08,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6353448e074b3ffe2171 HTTP/1.1" 200 None 2026-03-21 09:24:08,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:08,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:08,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:08,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6353448e074b3ffe2171 HTTP/1.1" 200 None 2026-03-21 09:24:08,558 - lava_callback - INFO - Completed processing callback for node 69be6353448e074b3ffe2171 2026-03-21 09:24:10,497 - lava_callback - INFO - Received callback for node 69be6338448e074b3ffe210c INFO: 172.20.0.1:38676 - "POST /node/69be6338448e074b3ffe210c HTTP/1.0" 202 Accepted 2026-03-21 09:24:10,504 - lava_callback - INFO - Starting async job submit for node 69be6338448e074b3ffe210c 2026-03-21 09:24:10,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:10,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6338448e074b3ffe210c HTTP/1.1" 200 None 2026-03-21 09:24:10,543 - lava_callback - DEBUG - Processing node 69be6338448e074b3ffe210c: baseline-arm64 2026-03-21 09:24:10,656 - lava_callback - INFO - Received callback for node 69be6361448e074b3ffe2195 INFO: 172.20.0.1:38684 - "POST /node/69be6361448e074b3ffe2195 HTTP/1.0" 202 Accepted 2026-03-21 09:24:10,683 - lava_callback - INFO - Starting async job submit for node 69be6361448e074b3ffe2195 2026-03-21 09:24:10,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:10,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6361448e074b3ffe2195 HTTP/1.1" 200 None 2026-03-21 09:24:10,848 - lava_callback - DEBUG - Processing node 69be6361448e074b3ffe2195: baseline-arm64-broonie 2026-03-21 09:24:10,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:11,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:11,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:11,106 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6338448e074b3ffe210c/log.txt.gz 2026-03-21 09:24:11,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:11,263 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6361448e074b3ffe2195/log.txt.gz 2026-03-21 09:24:11,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:11,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:11,471 - lava_callback - INFO - Received callback for node 69be6377448e074b3ffe21f2 INFO: 172.20.0.1:38698 - "POST /node/69be6377448e074b3ffe21f2 HTTP/1.0" 202 Accepted 2026-03-21 09:24:11,477 - lava_callback - INFO - Starting async job submit for node 69be6377448e074b3ffe21f2 2026-03-21 09:24:11,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:11,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6377448e074b3ffe21f2 HTTP/1.1" 200 None 2026-03-21 09:24:11,523 - lava_callback - DEBUG - Processing node 69be6377448e074b3ffe21f2: baseline-arm64-pengutronix 2026-03-21 09:24:11,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:11,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:11,606 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6338448e074b3ffe210c/lava-logs.yaml 2026-03-21 09:24:11,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6361448e074b3ffe2195/lava-logs.yaml 2026-03-21 09:24:11,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:11,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:11,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:11,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:11,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6338448e074b3ffe210c/lava_callback.json.gz 2026-03-21 09:24:11,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:11,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be6377448e074b3ffe21f2/log.txt.gz 2026-03-21 09:24:11,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:11,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:11,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6361448e074b3ffe2195/lava_callback.json.gz INFO: 172.20.0.1:38712 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:24:12,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6338448e074b3ffe210c HTTP/1.1" 200 None 2026-03-21 09:24:12,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:12,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6361448e074b3ffe2195 HTTP/1.1" 200 None 2026-03-21 09:24:12,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:12,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:12,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6338448e074b3ffe210c HTTP/1.1" 200 None 2026-03-21 09:24:12,231 - lava_callback - INFO - Completed processing callback for node 69be6338448e074b3ffe210c 2026-03-21 09:24:12,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6361448e074b3ffe2195 HTTP/1.1" 200 None 2026-03-21 09:24:12,234 - lava_callback - INFO - Completed processing callback for node 69be6361448e074b3ffe2195 2026-03-21 09:24:12,263 - lava_callback - INFO - Received callback for node 69be6369448e074b3ffe21c4 INFO: 172.20.0.1:38718 - "POST /node/69be6369448e074b3ffe21c4 HTTP/1.0" 202 Accepted 2026-03-21 09:24:12,278 - lava_callback - INFO - Starting async job submit for node 69be6369448e074b3ffe21c4 2026-03-21 09:24:12,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:12,369 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be6377448e074b3ffe21f2/lava-logs.yaml 2026-03-21 09:24:12,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6369448e074b3ffe21c4 HTTP/1.1" 200 None 2026-03-21 09:24:12,374 - lava_callback - DEBUG - Processing node 69be6369448e074b3ffe21c4: baseline-arm64-broonie 2026-03-21 09:24:12,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:12,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:12,683 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be6377448e074b3ffe21f2/lava_callback.json.gz 2026-03-21 09:24:12,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:12,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6377448e074b3ffe21f2 HTTP/1.1" 200 None 2026-03-21 09:24:12,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:12,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6377448e074b3ffe21f2 HTTP/1.1" 200 None 2026-03-21 09:24:12,841 - lava_callback - INFO - Completed processing callback for node 69be6377448e074b3ffe21f2 2026-03-21 09:24:12,918 - lava_callback - INFO - Received callback for node 69be634b448e074b3ffe2158 INFO: 172.20.0.1:38726 - "POST /node/69be634b448e074b3ffe2158 HTTP/1.0" 202 Accepted 2026-03-21 09:24:12,925 - lava_callback - INFO - Starting async job submit for node 69be634b448e074b3ffe2158 2026-03-21 09:24:12,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:12,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:12,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6369448e074b3ffe21c4/log.txt.gz 2026-03-21 09:24:13,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634b448e074b3ffe2158 HTTP/1.1" 200 None 2026-03-21 09:24:13,058 - lava_callback - DEBUG - Processing node 69be634b448e074b3ffe2158: baseline-nfs-arm64 2026-03-21 09:24:13,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:13,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:13,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:13,522 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6369448e074b3ffe21c4/lava-logs.yaml 2026-03-21 09:24:13,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:13,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:13,610 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634b448e074b3ffe2158/log.txt.gz 2026-03-21 09:24:13,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:13,717 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6369448e074b3ffe21c4/lava_callback.json.gz 2026-03-21 09:24:13,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:13,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6369448e074b3ffe21c4 HTTP/1.1" 200 None 2026-03-21 09:24:13,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:13,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:13,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:13,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:13,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6369448e074b3ffe21c4 HTTP/1.1" 200 None 2026-03-21 09:24:13,906 - lava_callback - INFO - Completed processing callback for node 69be6369448e074b3ffe21c4 2026-03-21 09:24:14,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:14,117 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634b448e074b3ffe2158/lava-logs.yaml 2026-03-21 09:24:14,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:14,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:14,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634b448e074b3ffe2158/lava_callback.json.gz 2026-03-21 09:24:14,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:14,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634b448e074b3ffe2158 HTTP/1.1" 200 None 2026-03-21 09:24:14,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:14,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:14,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:14,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be634b448e074b3ffe2158 HTTP/1.1" 200 None 2026-03-21 09:24:14,380 - lava_callback - INFO - Completed processing callback for node 69be634b448e074b3ffe2158 INFO: 172.20.0.1:38734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38736 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:24:18,014 - lava_callback - INFO - Received callback for node 69be6348448e074b3ffe2152 INFO: 172.20.0.1:38744 - "POST /node/69be6348448e074b3ffe2152 HTTP/1.0" 202 Accepted 2026-03-21 09:24:18,020 - lava_callback - INFO - Starting async job submit for node 69be6348448e074b3ffe2152 2026-03-21 09:24:18,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:18,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6348448e074b3ffe2152 HTTP/1.1" 200 None 2026-03-21 09:24:18,061 - lava_callback - DEBUG - Processing node 69be6348448e074b3ffe2152: baseline-nfs-arm64 2026-03-21 09:24:18,183 - lava_callback - INFO - Received callback for node 69be6378448e074b3ffe21f5 INFO: 172.20.0.1:38754 - "POST /node/69be6378448e074b3ffe21f5 HTTP/1.0" 202 Accepted 2026-03-21 09:24:18,209 - lava_callback - INFO - Starting async job submit for node 69be6378448e074b3ffe21f5 2026-03-21 09:24:18,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:18,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6378448e074b3ffe21f5 HTTP/1.1" 200 None 2026-03-21 09:24:18,353 - lava_callback - DEBUG - Processing node 69be6378448e074b3ffe21f5: baseline-arm64-pengutronix 2026-03-21 09:24:18,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:18,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:18,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:18,763 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6348448e074b3ffe2152/log.txt.gz 2026-03-21 09:24:18,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:18,824 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be6378448e074b3ffe21f5/log.txt.gz 2026-03-21 09:24:19,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:19,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:19,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:19,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be6378448e074b3ffe21f5/lava-logs.yaml 2026-03-21 09:24:19,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:19,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:19,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6348448e074b3ffe2152/lava-logs.yaml 2026-03-21 09:24:19,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:19,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:19,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be6378448e074b3ffe21f5/lava_callback.json.gz 2026-03-21 09:24:19,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:19,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:19,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6378448e074b3ffe21f5 HTTP/1.1" 200 None 2026-03-21 09:24:19,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:19,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6348448e074b3ffe2152/lava_callback.json.gz 2026-03-21 09:24:19,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:19,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:19,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:19,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6348448e074b3ffe2152 HTTP/1.1" 200 None 2026-03-21 09:24:19,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:19,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:19,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:19,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6378448e074b3ffe21f5 HTTP/1.1" 200 None 2026-03-21 09:24:19,600 - lava_callback - INFO - Completed processing callback for node 69be6378448e074b3ffe21f5 2026-03-21 09:24:19,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6348448e074b3ffe2152 HTTP/1.1" 200 None 2026-03-21 09:24:19,678 - lava_callback - INFO - Completed processing callback for node 69be6348448e074b3ffe2152 INFO: 172.20.0.1:37074 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:24:27,053 - lava_callback - INFO - Received callback for node 69be637a448e074b3ffe21fc INFO: 172.20.0.1:37082 - "POST /node/69be637a448e074b3ffe21fc HTTP/1.0" 202 Accepted 2026-03-21 09:24:27,060 - lava_callback - INFO - Starting async job submit for node 69be637a448e074b3ffe21fc 2026-03-21 09:24:27,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:27,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be637a448e074b3ffe21fc HTTP/1.1" 200 None 2026-03-21 09:24:27,094 - lava_callback - DEBUG - Processing node 69be637a448e074b3ffe21fc: baseline-arm64-pengutronix 2026-03-21 09:24:27,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:27,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:27,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be637a448e074b3ffe21fc/log.txt.gz 2026-03-21 09:24:27,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:27,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:27,752 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be637a448e074b3ffe21fc/lava-logs.yaml 2026-03-21 09:24:27,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:27,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:27,889 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69be637a448e074b3ffe21fc/lava_callback.json.gz 2026-03-21 09:24:27,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:27,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be637a448e074b3ffe21fc HTTP/1.1" 200 None 2026-03-21 09:24:27,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:27,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:27,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:27,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be637a448e074b3ffe21fc HTTP/1.1" 200 None 2026-03-21 09:24:27,966 - lava_callback - INFO - Completed processing callback for node 69be637a448e074b3ffe21fc 2026-03-21 09:24:31,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:31,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:60962 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:24:38,319 - lava_callback - INFO - Received callback for node 69be6366448e074b3ffe21be INFO: 172.20.0.1:60974 - "POST /node/69be6366448e074b3ffe21be HTTP/1.0" 202 Accepted 2026-03-21 09:24:38,325 - lava_callback - INFO - Starting async job submit for node 69be6366448e074b3ffe21be 2026-03-21 09:24:38,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:38,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6366448e074b3ffe21be HTTP/1.1" 200 None 2026-03-21 09:24:38,371 - lava_callback - DEBUG - Processing node 69be6366448e074b3ffe21be: baseline-arm64-broonie 2026-03-21 09:24:38,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:38,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:38,755 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6366448e074b3ffe21be/log.txt.gz 2026-03-21 09:24:38,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:39,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:39,077 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6366448e074b3ffe21be/lava-logs.yaml 2026-03-21 09:24:39,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:39,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:39,219 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6366448e074b3ffe21be/lava_callback.json.gz 2026-03-21 09:24:39,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:39,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6366448e074b3ffe21be HTTP/1.1" 200 None 2026-03-21 09:24:39,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:39,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:39,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:39,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6366448e074b3ffe21be HTTP/1.1" 200 None 2026-03-21 09:24:39,301 - lava_callback - INFO - Completed processing callback for node 69be6366448e074b3ffe21be 2026-03-21 09:24:39,748 - lava_callback - INFO - Received callback for node 69be636d448e074b3ffe21cc INFO: 172.20.0.1:48442 - "POST /node/69be636d448e074b3ffe21cc HTTP/1.0" 202 Accepted 2026-03-21 09:24:39,754 - lava_callback - INFO - Starting async job submit for node 69be636d448e074b3ffe21cc 2026-03-21 09:24:39,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:39,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be636d448e074b3ffe21cc HTTP/1.1" 200 None 2026-03-21 09:24:39,792 - lava_callback - DEBUG - Processing node 69be636d448e074b3ffe21cc: baseline-arm64-broonie 2026-03-21 09:24:40,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:40,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:40,241 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636d448e074b3ffe21cc/log.txt.gz 2026-03-21 09:24:40,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:40,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:40,558 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636d448e074b3ffe21cc/lava-logs.yaml 2026-03-21 09:24:40,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:40,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:40,690 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636d448e074b3ffe21cc/lava_callback.json.gz 2026-03-21 09:24:40,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:40,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be636d448e074b3ffe21cc HTTP/1.1" 200 None 2026-03-21 09:24:40,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:40,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:40,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:40,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be636d448e074b3ffe21cc HTTP/1.1" 200 None 2026-03-21 09:24:40,785 - lava_callback - INFO - Completed processing callback for node 69be636d448e074b3ffe21cc 2026-03-21 09:24:42,586 - lava_callback - INFO - Received callback for node 69be6373448e074b3ffe21e2 INFO: 172.20.0.1:48444 - "POST /node/69be6373448e074b3ffe21e2 HTTP/1.0" 202 Accepted 2026-03-21 09:24:42,592 - lava_callback - INFO - Starting async job submit for node 69be6373448e074b3ffe21e2 2026-03-21 09:24:42,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:42,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6373448e074b3ffe21e2 HTTP/1.1" 200 None 2026-03-21 09:24:42,628 - lava_callback - DEBUG - Processing node 69be6373448e074b3ffe21e2: baseline-arm64-clabbe 2026-03-21 09:24:42,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:42,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:42,970 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69be6373448e074b3ffe21e2/log.txt.gz 2026-03-21 09:24:43,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:43,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:43,290 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69be6373448e074b3ffe21e2/lava-logs.yaml 2026-03-21 09:24:43,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:43,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:43,421 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69be6373448e074b3ffe21e2/lava_callback.json.gz 2026-03-21 09:24:43,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:43,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6373448e074b3ffe21e2 HTTP/1.1" 200 None 2026-03-21 09:24:43,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:43,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:43,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:43,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6373448e074b3ffe21e2 HTTP/1.1" 200 None 2026-03-21 09:24:43,503 - lava_callback - INFO - Completed processing callback for node 69be6373448e074b3ffe21e2 INFO: 172.20.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33730 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:24:58,388 - lava_callback - INFO - Received callback for node 69be6365448e074b3ffe21bb INFO: 172.20.0.1:33746 - "POST /node/69be6365448e074b3ffe21bb HTTP/1.0" 202 Accepted 2026-03-21 09:24:58,394 - lava_callback - INFO - Starting async job submit for node 69be6365448e074b3ffe21bb 2026-03-21 09:24:58,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:58,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6365448e074b3ffe21bb HTTP/1.1" 200 None 2026-03-21 09:24:58,434 - lava_callback - DEBUG - Processing node 69be6365448e074b3ffe21bb: baseline-arm64-broonie 2026-03-21 09:24:58,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:58,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:58,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6365448e074b3ffe21bb/log.txt.gz 2026-03-21 09:24:59,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:59,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:59,374 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6365448e074b3ffe21bb/lava-logs.yaml 2026-03-21 09:24:59,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:24:59,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:24:59,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6365448e074b3ffe21bb/lava_callback.json.gz 2026-03-21 09:24:59,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:59,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6365448e074b3ffe21bb HTTP/1.1" 200 None 2026-03-21 09:24:59,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:59,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:24:59,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:59,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6365448e074b3ffe21bb HTTP/1.1" 200 None 2026-03-21 09:24:59,622 - lava_callback - INFO - Completed processing callback for node 69be6365448e074b3ffe21bb 2026-03-21 09:24:59,868 - lava_callback - INFO - Received callback for node 69be636b448e074b3ffe21c9 INFO: 172.20.0.1:34066 - "POST /node/69be636b448e074b3ffe21c9 HTTP/1.0" 202 Accepted 2026-03-21 09:24:59,875 - lava_callback - INFO - Starting async job submit for node 69be636b448e074b3ffe21c9 2026-03-21 09:24:59,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:24:59,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be636b448e074b3ffe21c9 HTTP/1.1" 200 None 2026-03-21 09:24:59,940 - lava_callback - DEBUG - Processing node 69be636b448e074b3ffe21c9: baseline-arm64-broonie 2026-03-21 09:25:00,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:00,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:00,352 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636b448e074b3ffe21c9/log.txt.gz 2026-03-21 09:25:00,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:00,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:00,689 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636b448e074b3ffe21c9/lava-logs.yaml 2026-03-21 09:25:00,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:00,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:00,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636b448e074b3ffe21c9/lava_callback.json.gz 2026-03-21 09:25:00,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:00,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be636b448e074b3ffe21c9 HTTP/1.1" 200 None 2026-03-21 09:25:00,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:00,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:25:00,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:00,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be636b448e074b3ffe21c9 HTTP/1.1" 200 None 2026-03-21 09:25:00,939 - lava_callback - INFO - Completed processing callback for node 69be636b448e074b3ffe21c9 2026-03-21 09:25:01,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:01,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:34082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41010 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:25:24,566 - lava_callback - INFO - Received callback for node 69be633b448e074b3ffe2118 INFO: 172.20.0.1:40608 - "POST /node/69be633b448e074b3ffe2118 HTTP/1.0" 202 Accepted 2026-03-21 09:25:24,571 - lava_callback - INFO - Starting async job submit for node 69be633b448e074b3ffe2118 2026-03-21 09:25:24,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:24,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633b448e074b3ffe2118 HTTP/1.1" 200 None 2026-03-21 09:25:24,609 - lava_callback - DEBUG - Processing node 69be633b448e074b3ffe2118: baseline-arm64 2026-03-21 09:25:24,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:25,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:25,113 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633b448e074b3ffe2118/log.txt.gz 2026-03-21 09:25:25,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:25,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:25,540 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633b448e074b3ffe2118/lava-logs.yaml 2026-03-21 09:25:25,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:25,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:25,701 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be633b448e074b3ffe2118/lava_callback.json.gz 2026-03-21 09:25:25,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:25,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be633b448e074b3ffe2118 HTTP/1.1" 200 None 2026-03-21 09:25:25,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:25,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:25:25,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:25,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be633b448e074b3ffe2118 HTTP/1.1" 200 None 2026-03-21 09:25:25,793 - lava_callback - INFO - Completed processing callback for node 69be633b448e074b3ffe2118 INFO: 172.20.0.1:40622 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:25:30,484 - lava_callback - INFO - Received callback for node 69be6362448e074b3ffe2198 INFO: 172.20.0.1:37358 - "POST /node/69be6362448e074b3ffe2198 HTTP/1.0" 202 Accepted 2026-03-21 09:25:30,490 - lava_callback - INFO - Starting async job submit for node 69be6362448e074b3ffe2198 2026-03-21 09:25:30,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:30,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6362448e074b3ffe2198 HTTP/1.1" 200 None 2026-03-21 09:25:30,527 - lava_callback - DEBUG - Processing node 69be6362448e074b3ffe2198: baseline-arm64-broonie 2026-03-21 09:25:30,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:30,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:30,860 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6362448e074b3ffe2198/log.txt.gz 2026-03-21 09:25:31,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:31,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:31,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 09:25:31,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:31,206 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6362448e074b3ffe2198/lava-logs.yaml 2026-03-21 09:25:31,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:31,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:31,347 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6362448e074b3ffe2198/lava_callback.json.gz 2026-03-21 09:25:31,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:31,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6362448e074b3ffe2198 HTTP/1.1" 200 None 2026-03-21 09:25:31,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:31,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:25:31,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:31,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6362448e074b3ffe2198 HTTP/1.1" 200 None 2026-03-21 09:25:31,435 - lava_callback - INFO - Completed processing callback for node 69be6362448e074b3ffe2198 2026-03-21 09:25:31,979 - lava_callback - INFO - Received callback for node 69be636e448e074b3ffe21d3 INFO: 172.20.0.1:37370 - "POST /node/69be636e448e074b3ffe21d3 HTTP/1.0" 202 Accepted 2026-03-21 09:25:31,988 - lava_callback - INFO - Starting async job submit for node 69be636e448e074b3ffe21d3 2026-03-21 09:25:32,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:32,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be636e448e074b3ffe21d3 HTTP/1.1" 200 None 2026-03-21 09:25:32,043 - lava_callback - DEBUG - Processing node 69be636e448e074b3ffe21d3: baseline-arm64-broonie 2026-03-21 09:25:32,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:32,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:32,446 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636e448e074b3ffe21d3/log.txt.gz 2026-03-21 09:25:32,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:32,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:32,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636e448e074b3ffe21d3/lava-logs.yaml 2026-03-21 09:25:32,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:25:32,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:25:32,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be636e448e074b3ffe21d3/lava_callback.json.gz 2026-03-21 09:25:32,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:32,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be636e448e074b3ffe21d3 HTTP/1.1" 200 None 2026-03-21 09:25:33,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:33,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:25:33,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:33,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be636e448e074b3ffe21d3 HTTP/1.1" 200 None 2026-03-21 09:25:33,050 - lava_callback - INFO - Completed processing callback for node 69be636e448e074b3ffe21d3 INFO: 172.20.0.1:37376 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:25:45,386 - lava_callback - INFO - Received callback for node 69bbc106ec3f435d20d52894 INFO: 172.20.0.1:52104 - "POST /node/69bbc106ec3f435d20d52894 HTTP/1.0" 202 Accepted 2026-03-21 09:25:45,392 - lava_callback - INFO - Starting async job submit for node 69bbc106ec3f435d20d52894 2026-03-21 09:25:45,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:45,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc106ec3f435d20d52894 HTTP/1.1" 200 None 2026-03-21 09:25:45,411 - lava_callback - DEBUG - Processing node 69bbc106ec3f435d20d52894: baseline-arm64-baylibre 2026-03-21 09:25:45,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc106ec3f435d20d52894 2026-03-21 09:25:45,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:45,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc106ec3f435d20d52894 HTTP/1.1" 200 None 2026-03-21 09:25:45,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:45,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:25:45,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:25:45,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc106ec3f435d20d52894 HTTP/1.1" 200 None 2026-03-21 09:25:45,475 - lava_callback - ERROR - Error processing callback for node 69bbc106ec3f435d20d52894: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38450 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:26:01,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:26:01,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47634 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:26:30,409 - lava_callback - INFO - Received callback for node 69be6368448e074b3ffe21c1 INFO: 172.20.0.1:57188 - "POST /node/69be6368448e074b3ffe21c1 HTTP/1.0" 202 Accepted 2026-03-21 09:26:30,415 - lava_callback - INFO - Starting async job submit for node 69be6368448e074b3ffe21c1 2026-03-21 09:26:30,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:26:30,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6368448e074b3ffe21c1 HTTP/1.1" 200 None 2026-03-21 09:26:30,453 - lava_callback - DEBUG - Processing node 69be6368448e074b3ffe21c1: baseline-arm64-broonie 2026-03-21 09:26:30,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:26:30,874 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:26:30,876 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6368448e074b3ffe21c1/log.txt.gz 2026-03-21 09:26:31,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:26:31,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:26:31,225 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6368448e074b3ffe21c1/lava-logs.yaml 2026-03-21 09:26:31,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:26:31,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:26:31,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6368448e074b3ffe21c1/lava_callback.json.gz 2026-03-21 09:26:31,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:26:31,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6368448e074b3ffe21c1 HTTP/1.1" 200 None 2026-03-21 09:26:31,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:26:31,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:26:31,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:26:31,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6368448e074b3ffe21c1 HTTP/1.1" 200 None 2026-03-21 09:26:31,474 - lava_callback - INFO - Completed processing callback for node 69be6368448e074b3ffe21c1 INFO: 172.20.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39584 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:27:01,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:27:01,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44738 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:27:49,848 - lava_callback - INFO - Received callback for node 69be647c448e074b3ffe2326 INFO: 172.20.0.1:41262 - "POST /node/69be647c448e074b3ffe2326 HTTP/1.0" 202 Accepted 2026-03-21 09:27:49,854 - lava_callback - INFO - Starting async job submit for node 69be647c448e074b3ffe2326 2026-03-21 09:27:49,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:27:49,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be647c448e074b3ffe2326 HTTP/1.1" 200 None 2026-03-21 09:27:49,875 - lava_callback - DEBUG - Processing node 69be647c448e074b3ffe2326: baseline-nfs-arm64 2026-03-21 09:27:49,877 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:27:49,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:27:50,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:27:50,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be647c448e074b3ffe2326/lava-logs.yaml 2026-03-21 09:27:50,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:27:50,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:27:50,179 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be647c448e074b3ffe2326/lava_callback.json.gz 2026-03-21 09:27:50,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:27:50,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be647c448e074b3ffe2326 HTTP/1.1" 200 None 2026-03-21 09:27:50,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:27:50,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:27:50,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:27:50,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be647c448e074b3ffe2326 HTTP/1.1" 200 None 2026-03-21 09:27:50,239 - lava_callback - INFO - Completed processing callback for node 69be647c448e074b3ffe2326 INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:28:01,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:28:01,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:29:25,938 - lava_callback - INFO - Received callback for node 69be64be448e074b3ffe2330 INFO: 172.20.0.1:48364 - "POST /node/69be64be448e074b3ffe2330 HTTP/1.0" 202 Accepted 2026-03-21 09:29:25,944 - lava_callback - INFO - Starting async job submit for node 69be64be448e074b3ffe2330 2026-03-21 09:29:25,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:29:25,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be64be448e074b3ffe2330 HTTP/1.1" 200 None 2026-03-21 09:29:25,988 - lava_callback - DEBUG - Processing node 69be64be448e074b3ffe2330: baseline-arm64 2026-03-21 09:29:26,121 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:29:26,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:29:26,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:29:26,363 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be64be448e074b3ffe2330/lava-logs.yaml 2026-03-21 09:29:26,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:29:26,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:29:26,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be64be448e074b3ffe2330/lava_callback.json.gz 2026-03-21 09:29:26,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:29:26,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be64be448e074b3ffe2330 HTTP/1.1" 200 None 2026-03-21 09:29:26,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:29:26,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:29:26,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:29:26,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be64be448e074b3ffe2330 HTTP/1.1" 200 None 2026-03-21 09:29:26,650 - lava_callback - INFO - Completed processing callback for node 69be64be448e074b3ffe2330 2026-03-21 09:29:31,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:29:31,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56982 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:30:14,215 - lava_callback - INFO - Received callback for node 69be6347448e074b3ffe214f INFO: 172.20.0.1:56986 - "POST /node/69be6347448e074b3ffe214f HTTP/1.0" 202 Accepted 2026-03-21 09:30:14,222 - lava_callback - INFO - Starting async job submit for node 69be6347448e074b3ffe214f 2026-03-21 09:30:14,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:14,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6347448e074b3ffe214f HTTP/1.1" 200 None 2026-03-21 09:30:14,278 - lava_callback - DEBUG - Processing node 69be6347448e074b3ffe214f: baseline-arm64 2026-03-21 09:30:14,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:30:14,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:30:14,806 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6347448e074b3ffe214f/log.txt.gz 2026-03-21 09:30:15,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56994 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 09:30:15,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:30:15,243 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6347448e074b3ffe214f/lava-logs.yaml 2026-03-21 09:30:15,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:30:15,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:30:15,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6347448e074b3ffe214f/lava_callback.json.gz DEBUG: Setup failed for 69be6347448e074b3ffe214f. Transitting job node result: incomplete -> fail 2026-03-21 09:30:15,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:15,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6347448e074b3ffe214f HTTP/1.1" 200 None 2026-03-21 09:30:15,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:15,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:30:15,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:15,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6347448e074b3ffe214f HTTP/1.1" 200 None 2026-03-21 09:30:15,686 - lava_callback - INFO - Completed processing callback for node 69be6347448e074b3ffe214f INFO: 172.20.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45284 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:30:31,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:31,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35352 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:30:43,007 - lava_callback - INFO - Received callback for node 69be6506448e074b3ffe233d INFO: 172.20.0.1:33574 - "POST /node/69be6506448e074b3ffe233d HTTP/1.0" 202 Accepted 2026-03-21 09:30:43,013 - lava_callback - INFO - Starting async job submit for node 69be6506448e074b3ffe233d 2026-03-21 09:30:43,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:43,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6506448e074b3ffe233d HTTP/1.1" 200 None 2026-03-21 09:30:43,059 - lava_callback - DEBUG - Processing node 69be6506448e074b3ffe233d: baseline-arm64 2026-03-21 09:30:43,193 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:30:43,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:30:43,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:30:43,435 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6506448e074b3ffe233d/lava-logs.yaml 2026-03-21 09:30:43,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:30:43,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:30:43,577 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6506448e074b3ffe233d/lava_callback.json.gz 2026-03-21 09:30:43,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:43,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6506448e074b3ffe233d HTTP/1.1" 200 None 2026-03-21 09:30:43,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:43,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:30:43,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:30:43,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6506448e074b3ffe233d HTTP/1.1" 200 None 2026-03-21 09:30:43,753 - lava_callback - INFO - Completed processing callback for node 69be6506448e074b3ffe233d INFO: 172.20.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50744 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:31:01,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:31:01,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60024 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:31:53,294 - lava_callback - INFO - Received callback for node 69be634a448e074b3ffe2155 INFO: 172.20.0.1:39426 - "POST /node/69be634a448e074b3ffe2155 HTTP/1.0" 202 Accepted 2026-03-21 09:31:53,300 - lava_callback - INFO - Starting async job submit for node 69be634a448e074b3ffe2155 2026-03-21 09:31:53,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:31:53,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634a448e074b3ffe2155 HTTP/1.1" 200 None 2026-03-21 09:31:53,336 - lava_callback - DEBUG - Processing node 69be634a448e074b3ffe2155: baseline-nfs-arm64 2026-03-21 09:31:53,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:31:53,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:31:53,943 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634a448e074b3ffe2155/log.txt.gz 2026-03-21 09:31:54,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:31:54,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:31:54,425 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634a448e074b3ffe2155/lava-logs.yaml 2026-03-21 09:31:54,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:31:54,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:31:54,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634a448e074b3ffe2155/lava_callback.json.gz 2026-03-21 09:31:54,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:31:54,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634a448e074b3ffe2155 HTTP/1.1" 200 None 2026-03-21 09:31:54,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:31:54,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:31:54,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:31:54,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be634a448e074b3ffe2155 HTTP/1.1" 200 None 2026-03-21 09:31:54,694 - lava_callback - INFO - Completed processing callback for node 69be634a448e074b3ffe2155 INFO: 172.20.0.1:39436 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:32:01,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:01,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46630 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:32:41,809 - lava_callback - INFO - Received callback for node 69be6577448e074b3ffe235a INFO: 172.20.0.1:37182 - "POST /node/69be6577448e074b3ffe235a HTTP/1.0" 202 Accepted 2026-03-21 09:32:41,815 - lava_callback - INFO - Starting async job submit for node 69be6577448e074b3ffe235a 2026-03-21 09:32:41,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:41,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6577448e074b3ffe235a HTTP/1.1" 200 None 2026-03-21 09:32:41,852 - lava_callback - DEBUG - Processing node 69be6577448e074b3ffe235a: baseline-arm64 2026-03-21 09:32:42,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:32:42,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:32:42,306 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6577448e074b3ffe235a/log.txt.gz 2026-03-21 09:32:42,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:32:42,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:32:42,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6577448e074b3ffe235a/lava-logs.yaml 2026-03-21 09:32:42,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:32:42,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:32:42,865 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6577448e074b3ffe235a/lava_callback.json.gz 2026-03-21 09:32:42,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:42,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6577448e074b3ffe235a HTTP/1.1" 200 None 2026-03-21 09:32:42,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:42,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:32:42,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:42,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6577448e074b3ffe235a HTTP/1.1" 200 None 2026-03-21 09:32:43,024 - lava_callback - INFO - Completed processing callback for node 69be6577448e074b3ffe235a 2026-03-21 09:32:45,697 - lava_callback - INFO - Received callback for node 69bbc2b4ec3f435d20d528e4 INFO: 172.20.0.1:37194 - "POST /node/69bbc2b4ec3f435d20d528e4 HTTP/1.0" 202 Accepted 2026-03-21 09:32:45,702 - lava_callback - INFO - Starting async job submit for node 69bbc2b4ec3f435d20d528e4 2026-03-21 09:32:45,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:45,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc2b4ec3f435d20d528e4 HTTP/1.1" 200 None 2026-03-21 09:32:45,717 - lava_callback - DEBUG - Processing node 69bbc2b4ec3f435d20d528e4: baseline-arm64-baylibre 2026-03-21 09:32:45,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc2b4ec3f435d20d528e4 2026-03-21 09:32:45,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:45,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc2b4ec3f435d20d528e4 HTTP/1.1" 200 None 2026-03-21 09:32:45,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:45,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:32:45,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:32:45,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc2b4ec3f435d20d528e4 HTTP/1.1" 200 None 2026-03-21 09:32:45,791 - lava_callback - ERROR - Error processing callback for node 69bbc2b4ec3f435d20d528e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45958 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:33:01,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:33:01,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59418 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:34:06,693 - lava_callback - INFO - Received callback for node 69bbc305ec3f435d20d528f0 INFO: 172.20.0.1:59430 - "POST /node/69bbc305ec3f435d20d528f0 HTTP/1.0" 202 Accepted 2026-03-21 09:34:06,699 - lava_callback - INFO - Starting async job submit for node 69bbc305ec3f435d20d528f0 2026-03-21 09:34:06,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:34:06,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc305ec3f435d20d528f0 HTTP/1.1" 200 None 2026-03-21 09:34:06,715 - lava_callback - DEBUG - Processing node 69bbc305ec3f435d20d528f0: baseline-arm64-baylibre 2026-03-21 09:34:06,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc305ec3f435d20d528f0 2026-03-21 09:34:06,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:34:06,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc305ec3f435d20d528f0 HTTP/1.1" 200 None 2026-03-21 09:34:06,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:34:06,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:34:06,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:34:06,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc305ec3f435d20d528f0 HTTP/1.1" 200 None 2026-03-21 09:34:06,773 - lava_callback - ERROR - Error processing callback for node 69bbc305ec3f435d20d528f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53962 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:35:25,815 - lava_callback - INFO - Received callback for node 69bbc355ec3f435d20d528fd INFO: 172.20.0.1:39094 - "POST /node/69bbc355ec3f435d20d528fd HTTP/1.0" 202 Accepted 2026-03-21 09:35:25,820 - lava_callback - INFO - Starting async job submit for node 69bbc355ec3f435d20d528fd 2026-03-21 09:35:25,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:25,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc355ec3f435d20d528fd HTTP/1.1" 200 None 2026-03-21 09:35:25,836 - lava_callback - DEBUG - Processing node 69bbc355ec3f435d20d528fd: baseline-arm64-baylibre 2026-03-21 09:35:25,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc355ec3f435d20d528fd 2026-03-21 09:35:25,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:25,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc355ec3f435d20d528fd HTTP/1.1" 200 None 2026-03-21 09:35:25,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:25,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:35:25,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:25,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc355ec3f435d20d528fd HTTP/1.1" 200 None 2026-03-21 09:35:25,919 - lava_callback - ERROR - Error processing callback for node 69bbc355ec3f435d20d528fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39098 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:35:29,932 - lava_callback - INFO - Received callback for node 69be660a448e074b3ffe237d INFO: 172.20.0.1:45706 - "POST /node/69be660a448e074b3ffe237d HTTP/1.0" 202 Accepted 2026-03-21 09:35:29,938 - lava_callback - INFO - Starting async job submit for node 69be660a448e074b3ffe237d 2026-03-21 09:35:29,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:29,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be660a448e074b3ffe237d HTTP/1.1" 200 None 2026-03-21 09:35:29,974 - lava_callback - DEBUG - Processing node 69be660a448e074b3ffe237d: baseline-arm64 2026-03-21 09:35:30,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:35:30,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:35:30,369 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be660a448e074b3ffe237d/log.txt.gz 2026-03-21 09:35:30,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:35:30,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:35:30,726 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be660a448e074b3ffe237d/lava-logs.yaml 2026-03-21 09:35:30,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:35:30,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:35:30,900 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be660a448e074b3ffe237d/lava_callback.json.gz 2026-03-21 09:35:30,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:30,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be660a448e074b3ffe237d HTTP/1.1" 200 None 2026-03-21 09:35:30,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:30,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:35:30,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:30,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be660a448e074b3ffe237d HTTP/1.1" 200 None 2026-03-21 09:35:30,979 - lava_callback - INFO - Completed processing callback for node 69be660a448e074b3ffe237d 2026-03-21 09:35:31,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:35:31,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51292 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:36:45,870 - lava_callback - INFO - Received callback for node 69bbc3a6ec3f435d20d52904 INFO: 172.20.0.1:39352 - "POST /node/69bbc3a6ec3f435d20d52904 HTTP/1.0" 202 Accepted 2026-03-21 09:36:45,875 - lava_callback - INFO - Starting async job submit for node 69bbc3a6ec3f435d20d52904 2026-03-21 09:36:45,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:36:45,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc3a6ec3f435d20d52904 HTTP/1.1" 200 None 2026-03-21 09:36:45,891 - lava_callback - DEBUG - Processing node 69bbc3a6ec3f435d20d52904: baseline-arm64-baylibre 2026-03-21 09:36:45,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc3a6ec3f435d20d52904 2026-03-21 09:36:45,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:36:45,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc3a6ec3f435d20d52904 HTTP/1.1" 200 None 2026-03-21 09:36:45,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:36:45,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:36:45,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:36:45,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc3a6ec3f435d20d52904 HTTP/1.1" 200 None 2026-03-21 09:36:45,945 - lava_callback - ERROR - Error processing callback for node 69bbc3a6ec3f435d20d52904: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59134 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:37:10,696 - lava_callback - INFO - Received callback for node 69be634e448e074b3ffe2165 INFO: 172.20.0.1:53092 - "POST /node/69be634e448e074b3ffe2165 HTTP/1.0" 202 Accepted 2026-03-21 09:37:10,703 - lava_callback - INFO - Starting async job submit for node 69be634e448e074b3ffe2165 2026-03-21 09:37:10,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:37:10,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634e448e074b3ffe2165 HTTP/1.1" 200 None 2026-03-21 09:37:10,757 - lava_callback - DEBUG - Processing node 69be634e448e074b3ffe2165: baseline-nfs-arm64 2026-03-21 09:37:11,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:37:11,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:37:11,714 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634e448e074b3ffe2165/log.txt.gz 2026-03-21 09:37:12,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:53098 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:37:12,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:37:12,407 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634e448e074b3ffe2165/lava-logs.yaml 2026-03-21 09:37:12,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:37:12,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:37:12,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634e448e074b3ffe2165/lava_callback.json.gz 2026-03-21 09:37:12,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:37:12,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634e448e074b3ffe2165 HTTP/1.1" 200 None 2026-03-21 09:37:12,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:37:12,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:37:12,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:37:12,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be634e448e074b3ffe2165 HTTP/1.1" 200 None 2026-03-21 09:37:12,683 - lava_callback - INFO - Completed processing callback for node 69be634e448e074b3ffe2165 INFO: 172.20.0.1:53108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:37:31,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:37:31,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51426 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:38:05,917 - lava_callback - INFO - Received callback for node 69bbc3f5ec3f435d20d52917 INFO: 172.20.0.1:37924 - "POST /node/69bbc3f5ec3f435d20d52917 HTTP/1.0" 202 Accepted 2026-03-21 09:38:05,923 - lava_callback - INFO - Starting async job submit for node 69bbc3f5ec3f435d20d52917 2026-03-21 09:38:05,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:05,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc3f5ec3f435d20d52917 HTTP/1.1" 200 None 2026-03-21 09:38:05,938 - lava_callback - DEBUG - Processing node 69bbc3f5ec3f435d20d52917: baseline-arm64-baylibre 2026-03-21 09:38:05,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc3f5ec3f435d20d52917 2026-03-21 09:38:05,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:05,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc3f5ec3f435d20d52917 HTTP/1.1" 200 None 2026-03-21 09:38:05,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:05,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:38:05,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:06,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc3f5ec3f435d20d52917 HTTP/1.1" 200 None 2026-03-21 09:38:06,004 - lava_callback - ERROR - Error processing callback for node 69bbc3f5ec3f435d20d52917: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45060 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:38:12,715 - lava_callback - INFO - Received callback for node 69be6278448e074b3ffe20df INFO: 172.20.0.1:45074 - "POST /node/69be6278448e074b3ffe20df HTTP/1.0" 202 Accepted 2026-03-21 09:38:12,761 - lava_callback - INFO - Starting async job submit for node 69be6278448e074b3ffe20df 2026-03-21 09:38:12,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:12,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6278448e074b3ffe20df HTTP/1.1" 200 None 2026-03-21 09:38:12,800 - lava_callback - DEBUG - Processing node 69be6278448e074b3ffe20df: baseline-arm-pengutronix 2026-03-21 09:38:12,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:38:13,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:38:13,071 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6278448e074b3ffe20df/log.txt.gz 2026-03-21 09:38:13,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:38:13,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:38:13,326 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6278448e074b3ffe20df/lava-logs.yaml 2026-03-21 09:38:13,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:38:13,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:38:13,462 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6278448e074b3ffe20df/lava_callback.json.gz DEBUG: Setup failed for 69be6278448e074b3ffe20df. Transitting job node result: incomplete -> fail 2026-03-21 09:38:13,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:13,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6278448e074b3ffe20df HTTP/1.1" 200 None 2026-03-21 09:38:13,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:13,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 09:38:13,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:13,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6278448e074b3ffe20df HTTP/1.1" 200 None 2026-03-21 09:38:13,610 - lava_callback - INFO - Completed processing callback for node 69be6278448e074b3ffe20df INFO: 172.20.0.1:45080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45088 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:38:17,690 - lava_callback - INFO - Received callback for node 69be66ef448e074b3ffe23af INFO: 172.20.0.1:45092 - "POST /node/69be66ef448e074b3ffe23af HTTP/1.0" 202 Accepted 2026-03-21 09:38:17,696 - lava_callback - INFO - Starting async job submit for node 69be66ef448e074b3ffe23af 2026-03-21 09:38:17,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:17,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be66ef448e074b3ffe23af HTTP/1.1" 200 None 2026-03-21 09:38:17,719 - lava_callback - DEBUG - Processing node 69be66ef448e074b3ffe23af: baseline-nfs-arm64 2026-03-21 09:38:17,721 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:38:17,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:38:17,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:38:17,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be66ef448e074b3ffe23af/lava-logs.yaml 2026-03-21 09:38:17,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:38:17,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:38:17,997 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be66ef448e074b3ffe23af/lava_callback.json.gz 2026-03-21 09:38:17,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:18,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be66ef448e074b3ffe23af HTTP/1.1" 200 None 2026-03-21 09:38:18,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:18,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:38:18,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:18,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be66ef448e074b3ffe23af HTTP/1.1" 200 None 2026-03-21 09:38:18,056 - lava_callback - INFO - Completed processing callback for node 69be66ef448e074b3ffe23af INFO: 172.20.0.1:59810 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:38:31,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:38:31,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:39:11,058 - lava_callback - INFO - Received callback for node 69be631d448e074b3ffe20ed INFO: 172.20.0.1:40424 - "POST /node/69be631d448e074b3ffe20ed HTTP/1.0" 202 Accepted 2026-03-21 09:39:11,064 - lava_callback - INFO - Starting async job submit for node 69be631d448e074b3ffe20ed 2026-03-21 09:39:11,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:11,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be631d448e074b3ffe20ed HTTP/1.1" 200 None 2026-03-21 09:39:11,142 - lava_callback - DEBUG - Processing node 69be631d448e074b3ffe20ed: ltp-smoke 2026-03-21 09:39:11,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:39:11,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:39:11,768 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69be631d448e074b3ffe20ed/log.txt.gz 2026-03-21 09:39:12,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:39:12,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:39:12,283 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69be631d448e074b3ffe20ed/lava-logs.yaml 2026-03-21 09:39:12,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:39:12,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:39:12,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69be631d448e074b3ffe20ed/lava_callback.json.gz 2026-03-21 09:39:12,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:12,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be631d448e074b3ffe20ed HTTP/1.1" 200 None 2026-03-21 09:39:12,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:12,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5456448e074b3ffe1633 HTTP/1.1" 200 None 2026-03-21 09:39:12,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:12,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be631d448e074b3ffe20ed HTTP/1.1" 200 None 2026-03-21 09:39:12,780 - lava_callback - INFO - Completed processing callback for node 69be631d448e074b3ffe20ed INFO: 172.20.0.1:40444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40452 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:39:26,116 - lava_callback - INFO - Received callback for node 69bbc444ec3f435d20d52924 INFO: 172.20.0.1:53880 - "POST /node/69bbc444ec3f435d20d52924 HTTP/1.0" 202 Accepted 2026-03-21 09:39:26,123 - lava_callback - INFO - Starting async job submit for node 69bbc444ec3f435d20d52924 2026-03-21 09:39:26,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:26,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc444ec3f435d20d52924 HTTP/1.1" 200 None 2026-03-21 09:39:26,151 - lava_callback - DEBUG - Processing node 69bbc444ec3f435d20d52924: baseline-arm64-baylibre 2026-03-21 09:39:26,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc444ec3f435d20d52924 2026-03-21 09:39:26,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:26,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc444ec3f435d20d52924 HTTP/1.1" 200 None 2026-03-21 09:39:26,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:26,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:39:26,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:26,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc444ec3f435d20d52924 HTTP/1.1" 200 None 2026-03-21 09:39:26,255 - lava_callback - ERROR - Error processing callback for node 69bbc444ec3f435d20d52924: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:39:31,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:39:31,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59618 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:40:41,219 - lava_callback - INFO - Received callback for node 69be6731448e074b3ffe23c2 INFO: 172.20.0.1:42498 - "POST /node/69be6731448e074b3ffe23c2 HTTP/1.0" 202 Accepted 2026-03-21 09:40:41,231 - lava_callback - INFO - Starting async job submit for node 69be6731448e074b3ffe23c2 2026-03-21 09:40:41,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:41,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6731448e074b3ffe23c2 HTTP/1.1" 200 None 2026-03-21 09:40:41,309 - lava_callback - DEBUG - Processing node 69be6731448e074b3ffe23c2: baseline-arm64 2026-03-21 09:40:41,520 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:40:41,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:40:41,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:40:41,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6731448e074b3ffe23c2/lava-logs.yaml 2026-03-21 09:40:41,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:40:41,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:40:41,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6731448e074b3ffe23c2/lava_callback.json.gz 2026-03-21 09:40:42,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:42,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6731448e074b3ffe23c2 HTTP/1.1" 200 None 2026-03-21 09:40:42,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:42,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:40:42,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:42,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6731448e074b3ffe23c2 HTTP/1.1" 200 None 2026-03-21 09:40:42,151 - lava_callback - INFO - Completed processing callback for node 69be6731448e074b3ffe23c2 2026-03-21 09:40:46,183 - lava_callback - INFO - Received callback for node 69bbc495ec3f435d20d52931 INFO: 172.20.0.1:42502 - "POST /node/69bbc495ec3f435d20d52931 HTTP/1.0" 202 Accepted 2026-03-21 09:40:46,188 - lava_callback - INFO - Starting async job submit for node 69bbc495ec3f435d20d52931 2026-03-21 09:40:46,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:46,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc495ec3f435d20d52931 HTTP/1.1" 200 None 2026-03-21 09:40:46,204 - lava_callback - DEBUG - Processing node 69bbc495ec3f435d20d52931: baseline-arm64-baylibre 2026-03-21 09:40:46,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc495ec3f435d20d52931 2026-03-21 09:40:46,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:46,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc495ec3f435d20d52931 HTTP/1.1" 200 None 2026-03-21 09:40:46,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:46,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:40:46,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:40:46,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc495ec3f435d20d52931 HTTP/1.1" 200 None 2026-03-21 09:40:46,268 - lava_callback - ERROR - Error processing callback for node 69bbc495ec3f435d20d52931: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:41:01,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:41:01,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:42:13,364 - lava_callback - INFO - Received callback for node 69bbc4eeec3f435d20d5293d INFO: 172.20.0.1:53942 - "POST /node/69bbc4eeec3f435d20d5293d HTTP/1.0" 202 Accepted 2026-03-21 09:42:13,370 - lava_callback - INFO - Starting async job submit for node 69bbc4eeec3f435d20d5293d 2026-03-21 09:42:13,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:13,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc4eeec3f435d20d5293d HTTP/1.1" 200 None 2026-03-21 09:42:13,386 - lava_callback - DEBUG - Processing node 69bbc4eeec3f435d20d5293d: baseline-arm64-clabbe 2026-03-21 09:42:13,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc4eeec3f435d20d5293d 2026-03-21 09:42:13,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:13,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc4eeec3f435d20d5293d HTTP/1.1" 200 None 2026-03-21 09:42:13,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:13,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:42:13,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:13,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc4eeec3f435d20d5293d HTTP/1.1" 200 None 2026-03-21 09:42:13,447 - lava_callback - ERROR - Error processing callback for node 69bbc4eeec3f435d20d5293d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53956 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:42:25,833 - lava_callback - INFO - Received callback for node 69be634c448e074b3ffe215b INFO: 172.20.0.1:36546 - "POST /node/69be634c448e074b3ffe215b HTTP/1.0" 202 Accepted 2026-03-21 09:42:25,841 - lava_callback - INFO - Starting async job submit for node 69be634c448e074b3ffe215b 2026-03-21 09:42:25,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:25,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634c448e074b3ffe215b HTTP/1.1" 200 None 2026-03-21 09:42:25,903 - lava_callback - DEBUG - Processing node 69be634c448e074b3ffe215b: baseline-nfs-arm64 INFO: 172.20.0.1:36560 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:42:26,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:42:27,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:42:27,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634c448e074b3ffe215b/log.txt.gz 2026-03-21 09:42:27,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:42:28,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:42:28,068 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634c448e074b3ffe215b/lava-logs.yaml 2026-03-21 09:42:28,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:42:28,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:42:28,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be634c448e074b3ffe215b/lava_callback.json.gz DEBUG: Setup failed for 69be634c448e074b3ffe215b. Transitting job node result: incomplete -> fail 2026-03-21 09:42:28,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:28,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be634c448e074b3ffe215b HTTP/1.1" 200 None 2026-03-21 09:42:28,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:28,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:42:28,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:28,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be634c448e074b3ffe215b HTTP/1.1" 200 None 2026-03-21 09:42:28,535 - lava_callback - INFO - Completed processing callback for node 69be634c448e074b3ffe215b 2026-03-21 09:42:31,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:31,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37228 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:42:40,069 - lava_callback - INFO - Received callback for node 69be67c9448e074b3ffe23e0 INFO: 172.20.0.1:48584 - "POST /node/69be67c9448e074b3ffe23e0 HTTP/1.0" 202 Accepted 2026-03-21 09:42:40,076 - lava_callback - INFO - Starting async job submit for node 69be67c9448e074b3ffe23e0 2026-03-21 09:42:40,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:40,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be67c9448e074b3ffe23e0 HTTP/1.1" 200 None 2026-03-21 09:42:40,127 - lava_callback - DEBUG - Processing node 69be67c9448e074b3ffe23e0: baseline-arm64 2026-03-21 09:42:40,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:42:40,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:42:40,411 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be67c9448e074b3ffe23e0/log.txt.gz 2026-03-21 09:42:40,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:42:40,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:42:40,681 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be67c9448e074b3ffe23e0/lava-logs.yaml 2026-03-21 09:42:40,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:42:40,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:42:40,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be67c9448e074b3ffe23e0/lava_callback.json.gz 2026-03-21 09:42:40,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:40,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be67c9448e074b3ffe23e0 HTTP/1.1" 200 None 2026-03-21 09:42:40,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:40,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:42:40,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:42:40,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be67c9448e074b3ffe23e0 HTTP/1.1" 200 None 2026-03-21 09:42:40,983 - lava_callback - INFO - Completed processing callback for node 69be67c9448e074b3ffe23e0 INFO: 172.20.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44260 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:43:01,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:01,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52090 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:43:33,499 - lava_callback - INFO - Received callback for node 69bbc540ec3f435d20d52948 INFO: 172.20.0.1:45320 - "POST /node/69bbc540ec3f435d20d52948 HTTP/1.0" 202 Accepted 2026-03-21 09:43:33,504 - lava_callback - INFO - Starting async job submit for node 69bbc540ec3f435d20d52948 2026-03-21 09:43:33,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:33,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc540ec3f435d20d52948 HTTP/1.1" 200 None 2026-03-21 09:43:33,520 - lava_callback - DEBUG - Processing node 69bbc540ec3f435d20d52948: baseline-arm64-clabbe 2026-03-21 09:43:33,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc540ec3f435d20d52948 2026-03-21 09:43:33,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:33,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc540ec3f435d20d52948 HTTP/1.1" 200 None 2026-03-21 09:43:33,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:33,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:43:33,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:33,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc540ec3f435d20d52948 HTTP/1.1" 200 None 2026-03-21 09:43:33,598 - lava_callback - ERROR - Error processing callback for node 69bbc540ec3f435d20d52948: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:43:40,077 - lava_callback - INFO - Received callback for node 69be6812448e074b3ffe23f2 INFO: 172.20.0.1:39738 - "POST /node/69be6812448e074b3ffe23f2 HTTP/1.0" 202 Accepted 2026-03-21 09:43:40,083 - lava_callback - INFO - Starting async job submit for node 69be6812448e074b3ffe23f2 2026-03-21 09:43:40,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:40,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6812448e074b3ffe23f2 HTTP/1.1" 200 None 2026-03-21 09:43:40,121 - lava_callback - DEBUG - Processing node 69be6812448e074b3ffe23f2: baseline-arm64 2026-03-21 09:43:40,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:43:40,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:43:40,581 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6812448e074b3ffe23f2/log.txt.gz 2026-03-21 09:43:40,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:43:40,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:43:40,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6812448e074b3ffe23f2/lava-logs.yaml 2026-03-21 09:43:40,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:43:41,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:43:41,143 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6812448e074b3ffe23f2/lava_callback.json.gz 2026-03-21 09:43:41,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:41,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6812448e074b3ffe23f2 HTTP/1.1" 200 None 2026-03-21 09:43:41,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:41,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:43:41,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:43:41,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6812448e074b3ffe23f2 HTTP/1.1" 200 None 2026-03-21 09:43:41,228 - lava_callback - INFO - Completed processing callback for node 69be6812448e074b3ffe23f2 INFO: 172.20.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60244 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:44:01,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:44:01,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58584 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:44:53,523 - lava_callback - INFO - Received callback for node 69bbc590ec3f435d20d52953 INFO: 172.20.0.1:48364 - "POST /node/69bbc590ec3f435d20d52953 HTTP/1.0" 202 Accepted 2026-03-21 09:44:53,529 - lava_callback - INFO - Starting async job submit for node 69bbc590ec3f435d20d52953 2026-03-21 09:44:53,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:44:53,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc590ec3f435d20d52953 HTTP/1.1" 200 None 2026-03-21 09:44:53,545 - lava_callback - DEBUG - Processing node 69bbc590ec3f435d20d52953: baseline-arm64-clabbe 2026-03-21 09:44:53,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc590ec3f435d20d52953 2026-03-21 09:44:53,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:44:53,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc590ec3f435d20d52953 HTTP/1.1" 200 None 2026-03-21 09:44:53,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:44:53,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:44:53,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:44:53,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc590ec3f435d20d52953 HTTP/1.1" 200 None 2026-03-21 09:44:53,610 - lava_callback - ERROR - Error processing callback for node 69bbc590ec3f435d20d52953: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54034 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:45:11,064 - lava_callback - INFO - Received callback for node 69be6551448e074b3ffe234e INFO: 172.20.0.1:34226 - "POST /node/69be6551448e074b3ffe234e HTTP/1.0" 202 Accepted 2026-03-21 09:45:11,071 - lava_callback - INFO - Starting async job submit for node 69be6551448e074b3ffe234e 2026-03-21 09:45:11,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:45:11,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6551448e074b3ffe234e HTTP/1.1" 200 None 2026-03-21 09:45:11,137 - lava_callback - DEBUG - Processing node 69be6551448e074b3ffe234e: baseline-arm 2026-03-21 09:45:11,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:45:11,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:45:11,748 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be6551448e074b3ffe234e/log.txt.gz 2026-03-21 09:45:12,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:45:12,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:45:12,288 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be6551448e074b3ffe234e/lava-logs.yaml 2026-03-21 09:45:12,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:45:12,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:45:12,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be6551448e074b3ffe234e/lava_callback.json.gz DEBUG: Setup failed for 69be6551448e074b3ffe234e. Transitting job node result: incomplete -> fail 2026-03-21 09:45:12,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:45:12,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6551448e074b3ffe234e HTTP/1.1" 200 None 2026-03-21 09:45:12,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:45:12,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 09:45:12,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:45:12,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6551448e074b3ffe234e HTTP/1.1" 200 None 2026-03-21 09:45:12,770 - lava_callback - INFO - Completed processing callback for node 69be6551448e074b3ffe234e INFO: 172.20.0.1:34256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39764 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:45:31,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:45:31,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:46:13,613 - lava_callback - INFO - Received callback for node 69bbc5dfec3f435d20d5295f INFO: 172.20.0.1:42776 - "POST /node/69bbc5dfec3f435d20d5295f HTTP/1.0" 202 Accepted 2026-03-21 09:46:13,618 - lava_callback - INFO - Starting async job submit for node 69bbc5dfec3f435d20d5295f 2026-03-21 09:46:13,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:46:13,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc5dfec3f435d20d5295f HTTP/1.1" 200 None 2026-03-21 09:46:13,636 - lava_callback - DEBUG - Processing node 69bbc5dfec3f435d20d5295f: baseline-arm64-clabbe 2026-03-21 09:46:13,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc5dfec3f435d20d5295f 2026-03-21 09:46:13,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:46:13,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc5dfec3f435d20d5295f HTTP/1.1" 200 None 2026-03-21 09:46:13,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:46:13,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:46:13,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:46:13,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc5dfec3f435d20d5295f HTTP/1.1" 200 None 2026-03-21 09:46:13,705 - lava_callback - ERROR - Error processing callback for node 69bbc5dfec3f435d20d5295f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42366 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:47:33,648 - lava_callback - INFO - Received callback for node 69bbc62dec3f435d20d5296d INFO: 172.20.0.1:45046 - "POST /node/69bbc62dec3f435d20d5296d HTTP/1.0" 202 Accepted 2026-03-21 09:47:33,653 - lava_callback - INFO - Starting async job submit for node 69bbc62dec3f435d20d5296d 2026-03-21 09:47:33,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:33,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc62dec3f435d20d5296d HTTP/1.1" 200 None 2026-03-21 09:47:33,669 - lava_callback - DEBUG - Processing node 69bbc62dec3f435d20d5296d: baseline-arm64-clabbe 2026-03-21 09:47:33,669 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc62dec3f435d20d5296d 2026-03-21 09:47:33,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:33,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc62dec3f435d20d5296d HTTP/1.1" 200 None 2026-03-21 09:47:33,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:33,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:47:33,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:33,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc62dec3f435d20d5296d HTTP/1.1" 200 None 2026-03-21 09:47:33,731 - lava_callback - ERROR - Error processing callback for node 69bbc62dec3f435d20d5296d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45062 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:47:42,272 - lava_callback - INFO - Received callback for node 69be6781448e074b3ffe23d6 INFO: 172.20.0.1:59274 - "POST /node/69be6781448e074b3ffe23d6 HTTP/1.0" 202 Accepted 2026-03-21 09:47:42,279 - lava_callback - INFO - Starting async job submit for node 69be6781448e074b3ffe23d6 2026-03-21 09:47:42,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:42,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6781448e074b3ffe23d6 HTTP/1.1" 200 None 2026-03-21 09:47:42,330 - lava_callback - DEBUG - Processing node 69be6781448e074b3ffe23d6: baseline-arm64 2026-03-21 09:47:42,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:47:42,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:47:42,838 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6781448e074b3ffe23d6/log.txt.gz 2026-03-21 09:47:43,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:47:43,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:47:43,279 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6781448e074b3ffe23d6/lava-logs.yaml 2026-03-21 09:47:43,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:47:43,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:47:43,452 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6781448e074b3ffe23d6/lava_callback.json.gz DEBUG: Setup failed for 69be6781448e074b3ffe23d6. Transitting job node result: incomplete -> fail 2026-03-21 09:47:43,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:43,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6781448e074b3ffe23d6 HTTP/1.1" 200 None 2026-03-21 09:47:43,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:43,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:47:43,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:47:43,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6781448e074b3ffe23d6 HTTP/1.1" 200 None 2026-03-21 09:47:43,687 - lava_callback - INFO - Completed processing callback for node 69be6781448e074b3ffe23d6 INFO: 172.20.0.1:59286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:48:01,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:48:01,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:48:53,681 - lava_callback - INFO - Received callback for node 69bbc67bec3f435d20d52978 INFO: 172.20.0.1:32902 - "POST /node/69bbc67bec3f435d20d52978 HTTP/1.0" 202 Accepted 2026-03-21 09:48:53,687 - lava_callback - INFO - Starting async job submit for node 69bbc67bec3f435d20d52978 2026-03-21 09:48:53,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:48:53,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc67bec3f435d20d52978 HTTP/1.1" 200 None 2026-03-21 09:48:53,705 - lava_callback - DEBUG - Processing node 69bbc67bec3f435d20d52978: baseline-arm64-clabbe 2026-03-21 09:48:53,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc67bec3f435d20d52978 2026-03-21 09:48:53,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:48:53,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc67bec3f435d20d52978 HTTP/1.1" 200 None 2026-03-21 09:48:53,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:48:53,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 09:48:53,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:48:53,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc67bec3f435d20d52978 HTTP/1.1" 200 None 2026-03-21 09:48:53,760 - lava_callback - ERROR - Error processing callback for node 69bbc67bec3f435d20d52978: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56450 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:51:26,881 - lava_callback - INFO - Received callback for node 69bbc711ec3f435d20d5298a INFO: 172.20.0.1:56452 - "POST /node/69bbc711ec3f435d20d5298a HTTP/1.0" 202 Accepted 2026-03-21 09:51:26,887 - lava_callback - INFO - Starting async job submit for node 69bbc711ec3f435d20d5298a 2026-03-21 09:51:26,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:51:26,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc711ec3f435d20d5298a HTTP/1.1" 200 None 2026-03-21 09:51:26,902 - lava_callback - DEBUG - Processing node 69bbc711ec3f435d20d5298a: baseline-arm64-baylibre 2026-03-21 09:51:26,902 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc711ec3f435d20d5298a 2026-03-21 09:51:26,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:51:26,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc711ec3f435d20d5298a HTTP/1.1" 200 None 2026-03-21 09:51:26,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:51:26,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:51:26,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:51:26,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc711ec3f435d20d5298a HTTP/1.1" 200 None 2026-03-21 09:51:26,966 - lava_callback - ERROR - Error processing callback for node 69bbc711ec3f435d20d5298a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39460 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:52:46,740 - lava_callback - INFO - Received callback for node 69bbc761ec3f435d20d52990 INFO: 172.20.0.1:39474 - "POST /node/69bbc761ec3f435d20d52990 HTTP/1.0" 202 Accepted 2026-03-21 09:52:46,746 - lava_callback - INFO - Starting async job submit for node 69bbc761ec3f435d20d52990 2026-03-21 09:52:46,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:52:46,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc761ec3f435d20d52990 HTTP/1.1" 200 None 2026-03-21 09:52:46,762 - lava_callback - DEBUG - Processing node 69bbc761ec3f435d20d52990: baseline-arm64-baylibre 2026-03-21 09:52:46,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc761ec3f435d20d52990 2026-03-21 09:52:46,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:52:46,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc761ec3f435d20d52990 HTTP/1.1" 200 None 2026-03-21 09:52:46,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:52:46,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:52:46,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:52:46,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc761ec3f435d20d52990 HTTP/1.1" 200 None 2026-03-21 09:52:46,814 - lava_callback - ERROR - Error processing callback for node 69bbc761ec3f435d20d52990: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46404 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:53:38,150 - lava_callback - INFO - Received callback for node 69be6a89448e074b3ffe2460 INFO: 172.20.0.1:46418 - "POST /node/69be6a89448e074b3ffe2460 HTTP/1.0" 202 Accepted 2026-03-21 09:53:38,158 - lava_callback - INFO - Starting async job submit for node 69be6a89448e074b3ffe2460 2026-03-21 09:53:38,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:53:38,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6a89448e074b3ffe2460 HTTP/1.1" 200 None 2026-03-21 09:53:38,175 - lava_callback - DEBUG - Processing node 69be6a89448e074b3ffe2460: baseline-nfs-arm64 2026-03-21 09:53:38,177 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:53:38,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:53:38,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:53:38,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6a89448e074b3ffe2460/lava-logs.yaml 2026-03-21 09:53:38,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:53:38,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:53:38,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6a89448e074b3ffe2460/lava_callback.json.gz 2026-03-21 09:53:38,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:53:38,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6a89448e074b3ffe2460 HTTP/1.1" 200 None 2026-03-21 09:53:38,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:53:38,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:53:38,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:53:38,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6a89448e074b3ffe2460 HTTP/1.1" 200 None 2026-03-21 09:53:38,541 - lava_callback - INFO - Completed processing callback for node 69be6a89448e074b3ffe2460 INFO: 172.20.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39172 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:54:01,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:54:01,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47134 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:54:13,873 - lava_callback - INFO - Received callback for node 69bbc7b9ec3f435d20d5299d INFO: 172.20.0.1:47150 - "POST /node/69bbc7b9ec3f435d20d5299d HTTP/1.0" 202 Accepted 2026-03-21 09:54:13,879 - lava_callback - INFO - Starting async job submit for node 69bbc7b9ec3f435d20d5299d 2026-03-21 09:54:13,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:54:13,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc7b9ec3f435d20d5299d HTTP/1.1" 200 None 2026-03-21 09:54:13,894 - lava_callback - DEBUG - Processing node 69bbc7b9ec3f435d20d5299d: baseline-arm64-clabbe 2026-03-21 09:54:13,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc7b9ec3f435d20d5299d 2026-03-21 09:54:13,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:54:13,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc7b9ec3f435d20d5299d HTTP/1.1" 200 None 2026-03-21 09:54:13,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:54:13,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:54:13,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:54:13,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc7b9ec3f435d20d5299d HTTP/1.1" 200 None 2026-03-21 09:54:13,950 - lava_callback - ERROR - Error processing callback for node 69bbc7b9ec3f435d20d5299d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57254 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:56:31,389 - lava_callback - INFO - Received callback for node 69be6364448e074b3ffe21b8 INFO: 172.20.0.1:42818 - "POST /node/69be6364448e074b3ffe21b8 HTTP/1.0" 202 Accepted 2026-03-21 09:56:31,394 - lava_callback - INFO - Starting async job submit for node 69be6364448e074b3ffe21b8 2026-03-21 09:56:31,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:31,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6364448e074b3ffe21b8 HTTP/1.1" 200 None 2026-03-21 09:56:31,449 - lava_callback - DEBUG - Processing node 69be6364448e074b3ffe21b8: baseline-arm64-broonie 2026-03-21 09:56:31,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:56:31,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:56:31,670 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6364448e074b3ffe21b8/log.txt.gz 2026-03-21 09:56:31,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:56:31,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:56:31,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6364448e074b3ffe21b8/lava-logs.yaml 2026-03-21 09:56:31,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:56:31,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:56:31,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6364448e074b3ffe21b8/lava_callback.json.gz 2026-03-21 09:56:32,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:32,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6364448e074b3ffe21b8 HTTP/1.1" 200 None 2026-03-21 09:56:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:32,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:56:32,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:32,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6364448e074b3ffe21b8 HTTP/1.1" 200 None 2026-03-21 09:56:32,125 - lava_callback - INFO - Completed processing callback for node 69be6364448e074b3ffe21b8 INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59226 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:56:53,286 - lava_callback - INFO - Received callback for node 69be6b14448e074b3ffe247a INFO: 172.20.0.1:44282 - "POST /node/69be6b14448e074b3ffe247a HTTP/1.0" 202 Accepted 2026-03-21 09:56:53,293 - lava_callback - INFO - Starting async job submit for node 69be6b14448e074b3ffe247a 2026-03-21 09:56:53,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:53,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6b14448e074b3ffe247a HTTP/1.1" 200 None 2026-03-21 09:56:53,339 - lava_callback - DEBUG - Processing node 69be6b14448e074b3ffe247a: baseline-arm64 2026-03-21 09:56:53,535 - lava_callback - WARNING - Failed to upload log 2026-03-21 09:56:53,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:56:53,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:56:53,841 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6b14448e074b3ffe247a/lava-logs.yaml 2026-03-21 09:56:53,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 09:56:53,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 09:56:53,980 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6b14448e074b3ffe247a/lava_callback.json.gz 2026-03-21 09:56:54,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:54,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6b14448e074b3ffe247a HTTP/1.1" 200 None 2026-03-21 09:56:54,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:54,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 09:56:54,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:54,051 - lava_callback - INFO - Received callback for node 69bbc85aec3f435d20d529b1 INFO: 172.20.0.1:44294 - "POST /node/69bbc85aec3f435d20d529b1 HTTP/1.0" 202 Accepted 2026-03-21 09:56:54,057 - lava_callback - INFO - Starting async job submit for node 69bbc85aec3f435d20d529b1 2026-03-21 09:56:54,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:54,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6b14448e074b3ffe247a HTTP/1.1" 200 None 2026-03-21 09:56:54,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc85aec3f435d20d529b1 HTTP/1.1" 200 None 2026-03-21 09:56:54,146 - lava_callback - DEBUG - Processing node 69bbc85aec3f435d20d529b1: baseline-arm64-clabbe 2026-03-21 09:56:54,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc85aec3f435d20d529b1 2026-03-21 09:56:54,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:54,170 - lava_callback - INFO - Completed processing callback for node 69be6b14448e074b3ffe247a 2026-03-21 09:56:54,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc85aec3f435d20d529b1 HTTP/1.1" 200 None 2026-03-21 09:56:54,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:54,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 09:56:54,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:56:54,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc85aec3f435d20d529b1 HTTP/1.1" 200 None 2026-03-21 09:56:54,242 - lava_callback - ERROR - Error processing callback for node 69bbc85aec3f435d20d529b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44302 - "GET / HTTP/1.0" 200 OK 2026-03-21 09:57:01,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 09:57:01,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:00:04,937 - lava_callback - INFO - Received callback for node 69be6bf7448e074b3ffe2499 INFO: 172.20.0.1:47464 - "POST /node/69be6bf7448e074b3ffe2499 HTTP/1.0" 202 Accepted 2026-03-21 10:00:04,944 - lava_callback - INFO - Starting async job submit for node 69be6bf7448e074b3ffe2499 2026-03-21 10:00:04,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:04,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6bf7448e074b3ffe2499 HTTP/1.1" 200 None 2026-03-21 10:00:04,989 - lava_callback - DEBUG - Processing node 69be6bf7448e074b3ffe2499: baseline-arm64 2026-03-21 10:00:05,123 - lava_callback - WARNING - Failed to upload log 2026-03-21 10:00:05,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:00:05,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:00:05,380 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6bf7448e074b3ffe2499/lava-logs.yaml 2026-03-21 10:00:05,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:00:05,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:00:05,535 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6bf7448e074b3ffe2499/lava_callback.json.gz 2026-03-21 10:00:05,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:05,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6bf7448e074b3ffe2499 HTTP/1.1" 200 None 2026-03-21 10:00:05,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:05,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 10:00:05,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:05,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6bf7448e074b3ffe2499 HTTP/1.1" 200 None 2026-03-21 10:00:05,690 - lava_callback - INFO - Completed processing callback for node 69be6bf7448e074b3ffe2499 INFO: 172.20.0.1:47476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53216 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:00:31,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:31,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:00:54,205 - lava_callback - INFO - Received callback for node 69bbc948ec3f435d20d529d2 INFO: 172.20.0.1:38810 - "POST /node/69bbc948ec3f435d20d529d2 HTTP/1.0" 202 Accepted 2026-03-21 10:00:54,211 - lava_callback - INFO - Starting async job submit for node 69bbc948ec3f435d20d529d2 2026-03-21 10:00:54,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:54,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc948ec3f435d20d529d2 HTTP/1.1" 200 None 2026-03-21 10:00:54,227 - lava_callback - DEBUG - Processing node 69bbc948ec3f435d20d529d2: baseline-arm64-clabbe 2026-03-21 10:00:54,228 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc948ec3f435d20d529d2 2026-03-21 10:00:54,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:54,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc948ec3f435d20d529d2 HTTP/1.1" 200 None 2026-03-21 10:00:54,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:54,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 10:00:54,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:00:54,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc948ec3f435d20d529d2 HTTP/1.1" 200 None 2026-03-21 10:00:54,280 - lava_callback - ERROR - Error processing callback for node 69bbc948ec3f435d20d529d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44560 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:02:14,227 - lava_callback - INFO - Received callback for node 69bbc999ec3f435d20d529d8 INFO: 172.20.0.1:44564 - "POST /node/69bbc999ec3f435d20d529d8 HTTP/1.0" 202 Accepted 2026-03-21 10:02:14,233 - lava_callback - INFO - Starting async job submit for node 69bbc999ec3f435d20d529d8 2026-03-21 10:02:14,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:02:14,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc999ec3f435d20d529d8 HTTP/1.1" 200 None 2026-03-21 10:02:14,249 - lava_callback - DEBUG - Processing node 69bbc999ec3f435d20d529d8: baseline-arm64-clabbe 2026-03-21 10:02:14,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbc999ec3f435d20d529d8 2026-03-21 10:02:14,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:02:14,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbc999ec3f435d20d529d8 HTTP/1.1" 200 None 2026-03-21 10:02:14,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:02:14,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 10:02:14,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:02:14,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbc999ec3f435d20d529d8 HTTP/1.1" 200 None 2026-03-21 10:02:14,303 - lava_callback - ERROR - Error processing callback for node 69bbc999ec3f435d20d529d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:04:54,325 - lava_callback - INFO - Received callback for node 69bbca3aec3f435d20d529ee INFO: 172.20.0.1:43126 - "POST /node/69bbca3aec3f435d20d529ee HTTP/1.0" 202 Accepted 2026-03-21 10:04:54,330 - lava_callback - INFO - Starting async job submit for node 69bbca3aec3f435d20d529ee 2026-03-21 10:04:54,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:04:54,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbca3aec3f435d20d529ee HTTP/1.1" 200 None 2026-03-21 10:04:54,348 - lava_callback - DEBUG - Processing node 69bbca3aec3f435d20d529ee: baseline-arm64-clabbe 2026-03-21 10:04:54,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbca3aec3f435d20d529ee 2026-03-21 10:04:54,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:04:54,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbca3aec3f435d20d529ee HTTP/1.1" 200 None 2026-03-21 10:04:54,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:04:54,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 10:04:54,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:04:54,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbca3aec3f435d20d529ee HTTP/1.1" 200 None 2026-03-21 10:04:54,405 - lava_callback - ERROR - Error processing callback for node 69bbca3aec3f435d20d529ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46034 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:07:34,448 - lava_callback - INFO - Received callback for node 69bbcad8ec3f435d20d52a02 INFO: 172.20.0.1:52788 - "POST /node/69bbcad8ec3f435d20d52a02 HTTP/1.0" 202 Accepted 2026-03-21 10:07:34,454 - lava_callback - INFO - Starting async job submit for node 69bbcad8ec3f435d20d52a02 2026-03-21 10:07:34,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:07:34,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbcad8ec3f435d20d52a02 HTTP/1.1" 200 None 2026-03-21 10:07:34,470 - lava_callback - DEBUG - Processing node 69bbcad8ec3f435d20d52a02: baseline-arm64-clabbe 2026-03-21 10:07:34,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbcad8ec3f435d20d52a02 2026-03-21 10:07:34,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:07:34,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbcad8ec3f435d20d52a02 HTTP/1.1" 200 None 2026-03-21 10:07:34,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:07:34,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3c8061727ff69d6f21d98 HTTP/1.1" 200 None 2026-03-21 10:07:34,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:07:34,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbcad8ec3f435d20d52a02 HTTP/1.1" 200 None 2026-03-21 10:07:34,530 - lava_callback - ERROR - Error processing callback for node 69bbcad8ec3f435d20d52a02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:11:32,381 - lava_callback - INFO - Received callback for node 69be6ac9448e074b3ffe246f INFO: 172.20.0.1:41066 - "POST /node/69be6ac9448e074b3ffe246f HTTP/1.0" 202 Accepted 2026-03-21 10:11:32,388 - lava_callback - INFO - Starting async job submit for node 69be6ac9448e074b3ffe246f 2026-03-21 10:11:32,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:32,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6ac9448e074b3ffe246f HTTP/1.1" 200 None 2026-03-21 10:11:32,464 - lava_callback - DEBUG - Processing node 69be6ac9448e074b3ffe246f: ltp-smoke 2026-03-21 10:11:32,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:11:33,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:11:33,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69be6ac9448e074b3ffe246f/log.txt.gz 2026-03-21 10:11:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:11:33,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:11:33,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69be6ac9448e074b3ffe246f/lava-logs.yaml 2026-03-21 10:11:33,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:11:33,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:11:33,770 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69be6ac9448e074b3ffe246f/lava_callback.json.gz 2026-03-21 10:11:33,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6ac9448e074b3ffe246f HTTP/1.1" 200 None 2026-03-21 10:11:33,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:33,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5456448e074b3ffe1633 HTTP/1.1" 200 None 2026-03-21 10:11:33,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:33,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6ac9448e074b3ffe246f HTTP/1.1" 200 None 2026-03-21 10:11:34,093 - lava_callback - INFO - Completed processing callback for node 69be6ac9448e074b3ffe246f INFO: 172.20.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48836 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:11:51,401 - lava_callback - INFO - Received callback for node 69be6d9a448e074b3ffe24d0 INFO: 172.20.0.1:60386 - "POST /node/69be6d9a448e074b3ffe24d0 HTTP/1.0" 202 Accepted 2026-03-21 10:11:51,408 - lava_callback - INFO - Starting async job submit for node 69be6d9a448e074b3ffe24d0 2026-03-21 10:11:51,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:51,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6d9a448e074b3ffe24d0 HTTP/1.1" 200 None 2026-03-21 10:11:51,466 - lava_callback - DEBUG - Processing node 69be6d9a448e074b3ffe24d0: baseline-arm64 2026-03-21 10:11:51,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:11:51,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:11:51,986 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6d9a448e074b3ffe24d0/log.txt.gz 2026-03-21 10:11:52,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:11:52,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:11:52,399 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6d9a448e074b3ffe24d0/lava-logs.yaml 2026-03-21 10:11:52,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:11:52,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:11:52,558 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6d9a448e074b3ffe24d0/lava_callback.json.gz DEBUG: Setup failed for 69be6d9a448e074b3ffe24d0. Transitting job node result: incomplete -> fail 2026-03-21 10:11:52,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:52,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6d9a448e074b3ffe24d0 HTTP/1.1" 200 None 2026-03-21 10:11:52,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:52,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 10:11:52,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:11:52,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6d9a448e074b3ffe24d0 HTTP/1.1" 200 None 2026-03-21 10:11:52,758 - lava_callback - INFO - Completed processing callback for node 69be6d9a448e074b3ffe24d0 INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:12:01,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:12:01,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50440 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:12:13,546 - lava_callback - INFO - Received callback for node 69be6a67448e074b3ffe2455 INFO: 172.20.0.1:50450 - "POST /node/69be6a67448e074b3ffe2455 HTTP/1.0" 202 Accepted 2026-03-21 10:12:13,593 - lava_callback - INFO - Starting async job submit for node 69be6a67448e074b3ffe2455 2026-03-21 10:12:13,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:12:13,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6a67448e074b3ffe2455 HTTP/1.1" 200 None 2026-03-21 10:12:13,635 - lava_callback - DEBUG - Processing node 69be6a67448e074b3ffe2455: baseline-arm-pengutronix 2026-03-21 10:12:13,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:12:13,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:12:13,864 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6a67448e074b3ffe2455/log.txt.gz 2026-03-21 10:12:13,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:12:14,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:12:14,089 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6a67448e074b3ffe2455/lava-logs.yaml 2026-03-21 10:12:14,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:12:14,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:12:14,196 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6a67448e074b3ffe2455/lava_callback.json.gz DEBUG: Setup failed for 69be6a67448e074b3ffe2455. Transitting job node result: incomplete -> fail 2026-03-21 10:12:14,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:12:14,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6a67448e074b3ffe2455 HTTP/1.1" 200 None 2026-03-21 10:12:14,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:12:14,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 10:12:14,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:12:14,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6a67448e074b3ffe2455 HTTP/1.1" 200 None 2026-03-21 10:12:14,367 - lava_callback - INFO - Completed processing callback for node 69be6a67448e074b3ffe2455 INFO: 172.20.0.1:50466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39696 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:12:31,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:12:31,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41044 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:17:08,230 - lava_callback - INFO - Received callback for node 69be6eef448e074b3ffe24f0 INFO: 172.20.0.1:41054 - "POST /node/69be6eef448e074b3ffe24f0 HTTP/1.0" 202 Accepted 2026-03-21 10:17:08,237 - lava_callback - INFO - Starting async job submit for node 69be6eef448e074b3ffe24f0 2026-03-21 10:17:08,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:17:08,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6eef448e074b3ffe24f0 HTTP/1.1" 200 None 2026-03-21 10:17:08,309 - lava_callback - DEBUG - Processing node 69be6eef448e074b3ffe24f0: baseline-arm64 2026-03-21 10:17:08,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:17:08,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:17:08,828 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6eef448e074b3ffe24f0/log.txt.gz 2026-03-21 10:17:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:17:09,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:17:09,268 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6eef448e074b3ffe24f0/lava-logs.yaml 2026-03-21 10:17:09,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:17:09,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:17:09,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6eef448e074b3ffe24f0/lava_callback.json.gz DEBUG: Setup failed for 69be6eef448e074b3ffe24f0. Transitting job node result: incomplete -> fail 2026-03-21 10:17:09,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:17:09,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6eef448e074b3ffe24f0 HTTP/1.1" 200 None 2026-03-21 10:17:09,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:17:09,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 10:17:09,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:17:09,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6eef448e074b3ffe24f0 HTTP/1.1" 200 None 2026-03-21 10:17:09,677 - lava_callback - INFO - Completed processing callback for node 69be6eef448e074b3ffe24f0 INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50846 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:17:31,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:17:31,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40186 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:18:00,607 - lava_callback - INFO - Received callback for node 69be6bb3448e074b3ffe2490 INFO: 172.20.0.1:42794 - "POST /node/69be6bb3448e074b3ffe2490 HTTP/1.0" 202 Accepted 2026-03-21 10:18:00,615 - lava_callback - INFO - Starting async job submit for node 69be6bb3448e074b3ffe2490 2026-03-21 10:18:00,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:18:00,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6bb3448e074b3ffe2490 HTTP/1.1" 200 None 2026-03-21 10:18:00,699 - lava_callback - DEBUG - Processing node 69be6bb3448e074b3ffe2490: baseline-nfs-arm64 2026-03-21 10:18:01,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:18:01,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:18:01,871 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6bb3448e074b3ffe2490/log.txt.gz 2026-03-21 10:18:02,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:18:02,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:18:02,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6bb3448e074b3ffe2490/lava-logs.yaml 2026-03-21 10:18:02,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:18:02,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:18:02,909 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6bb3448e074b3ffe2490/lava_callback.json.gz DEBUG: Setup failed for 69be6bb3448e074b3ffe2490. Transitting job node result: incomplete -> fail 2026-03-21 10:18:02,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:18:03,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6bb3448e074b3ffe2490 HTTP/1.1" 200 None 2026-03-21 10:18:03,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:18:03,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 10:18:03,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:18:03,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6bb3448e074b3ffe2490 HTTP/1.1" 200 None 2026-03-21 10:18:03,192 - lava_callback - INFO - Completed processing callback for node 69be6bb3448e074b3ffe2490 INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33992 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:18:31,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:18:31,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55094 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:20:37,671 - lava_callback - INFO - Received callback for node 69be6cda448e074b3ffe24b4 INFO: 172.20.0.1:55108 - "POST /node/69be6cda448e074b3ffe24b4 HTTP/1.0" 202 Accepted 2026-03-21 10:20:37,677 - lava_callback - INFO - Starting async job submit for node 69be6cda448e074b3ffe24b4 2026-03-21 10:20:37,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:20:37,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6cda448e074b3ffe24b4 HTTP/1.1" 200 None 2026-03-21 10:20:37,754 - lava_callback - DEBUG - Processing node 69be6cda448e074b3ffe24b4: baseline-arm 2026-03-21 10:20:38,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:20:38,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:20:38,499 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69be6cda448e074b3ffe24b4/log.txt.gz 2026-03-21 10:20:38,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:20:39,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:20:39,081 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69be6cda448e074b3ffe24b4/lava-logs.yaml 2026-03-21 10:20:39,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:20:39,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:20:39,265 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69be6cda448e074b3ffe24b4/lava_callback.json.gz DEBUG: Setup failed for 69be6cda448e074b3ffe24b4. Transitting job node result: incomplete -> fail 2026-03-21 10:20:39,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:20:39,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6cda448e074b3ffe24b4 HTTP/1.1" 200 None 2026-03-21 10:20:39,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:20:39,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 10:20:39,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:20:39,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6cda448e074b3ffe24b4 HTTP/1.1" 200 None 2026-03-21 10:20:39,590 - lava_callback - INFO - Completed processing callback for node 69be6cda448e074b3ffe24b4 INFO: 172.20.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:21:01,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:21:01,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44050 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:32:53,992 - lava_callback - INFO - Received callback for node 69be6f3e448e074b3ffe2505 INFO: 172.20.0.1:41486 - "POST /node/69be6f3e448e074b3ffe2505 HTTP/1.0" 202 Accepted 2026-03-21 10:32:54,041 - lava_callback - INFO - Starting async job submit for node 69be6f3e448e074b3ffe2505 2026-03-21 10:32:54,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:32:54,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6f3e448e074b3ffe2505 HTTP/1.1" 200 None 2026-03-21 10:32:54,080 - lava_callback - DEBUG - Processing node 69be6f3e448e074b3ffe2505: baseline-arm-pengutronix 2026-03-21 10:32:54,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:32:54,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:32:54,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6f3e448e074b3ffe2505/log.txt.gz 2026-03-21 10:32:54,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:32:54,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:32:54,620 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6f3e448e074b3ffe2505/lava-logs.yaml 2026-03-21 10:32:54,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:32:54,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:32:54,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69be6f3e448e074b3ffe2505/lava_callback.json.gz DEBUG: Setup failed for 69be6f3e448e074b3ffe2505. Transitting job node result: incomplete -> fail 2026-03-21 10:32:54,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:32:54,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6f3e448e074b3ffe2505 HTTP/1.1" 200 None 2026-03-21 10:32:54,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:32:54,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-21 10:32:54,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:32:54,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6f3e448e074b3ffe2505 HTTP/1.1" 200 None 2026-03-21 10:32:54,952 - lava_callback - INFO - Completed processing callback for node 69be6f3e448e074b3ffe2505 INFO: 172.20.0.1:41498 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:33:01,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:33:01,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:38:01,919 - lava_callback - INFO - Received callback for node 69be7063448e074b3ffe2515 INFO: 172.20.0.1:45522 - "POST /node/69be7063448e074b3ffe2515 HTTP/1.0" 202 Accepted 2026-03-21 10:38:01,927 - lava_callback - INFO - Starting async job submit for node 69be7063448e074b3ffe2515 2026-03-21 10:38:01,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:38:01,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be7063448e074b3ffe2515 HTTP/1.1" 200 None 2026-03-21 10:38:01,988 - lava_callback - DEBUG - Processing node 69be7063448e074b3ffe2515: baseline-nfs-arm64 2026-03-21 10:38:03,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:38:03,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:38:03,220 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be7063448e074b3ffe2515/log.txt.gz 2026-03-21 10:38:03,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:38:04,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:38:04,074 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be7063448e074b3ffe2515/lava-logs.yaml 2026-03-21 10:38:04,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:38:04,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:38:04,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be7063448e074b3ffe2515/lava_callback.json.gz DEBUG: Setup failed for 69be7063448e074b3ffe2515. Transitting job node result: incomplete -> fail 2026-03-21 10:38:04,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:38:04,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be7063448e074b3ffe2515 HTTP/1.1" 200 None 2026-03-21 10:38:04,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:38:04,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 10:38:04,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:38:04,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be7063448e074b3ffe2515 HTTP/1.1" 200 None 2026-03-21 10:38:04,569 - lava_callback - INFO - Completed processing callback for node 69be7063448e074b3ffe2515 INFO: 172.20.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:38:31,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:38:31,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57388 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:41:49,461 - lava_callback - INFO - Received callback for node 69bbd2e6ec3f435d20d52b2b INFO: 172.20.0.1:57396 - "POST /node/69bbd2e6ec3f435d20d52b2b HTTP/1.0" 202 Accepted 2026-03-21 10:41:49,467 - lava_callback - INFO - Starting async job submit for node 69bbd2e6ec3f435d20d52b2b 2026-03-21 10:41:49,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:41:49,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd2e6ec3f435d20d52b2b HTTP/1.1" 200 None 2026-03-21 10:41:49,483 - lava_callback - DEBUG - Processing node 69bbd2e6ec3f435d20d52b2b: baseline-arm-baylibre 2026-03-21 10:41:49,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd2e6ec3f435d20d52b2b 2026-03-21 10:41:49,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:41:49,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd2e6ec3f435d20d52b2b HTTP/1.1" 200 None 2026-03-21 10:41:49,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:41:49,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:41:49,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:41:49,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd2e6ec3f435d20d52b2b HTTP/1.1" 200 None 2026-03-21 10:41:49,591 - lava_callback - ERROR - Error processing callback for node 69bbd2e6ec3f435d20d52b2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43514 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:43:29,462 - lava_callback - INFO - Received callback for node 69bbd34fec3f435d20d52b3d INFO: 172.20.0.1:43528 - "POST /node/69bbd34fec3f435d20d52b3d HTTP/1.0" 202 Accepted 2026-03-21 10:43:29,467 - lava_callback - INFO - Starting async job submit for node 69bbd34fec3f435d20d52b3d 2026-03-21 10:43:29,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:43:29,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd34fec3f435d20d52b3d HTTP/1.1" 200 None 2026-03-21 10:43:29,483 - lava_callback - DEBUG - Processing node 69bbd34fec3f435d20d52b3d: baseline-arm-baylibre 2026-03-21 10:43:29,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd34fec3f435d20d52b3d 2026-03-21 10:43:29,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:43:29,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd34fec3f435d20d52b3d HTTP/1.1" 200 None 2026-03-21 10:43:29,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:43:29,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:43:29,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:43:29,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd34fec3f435d20d52b3d HTTP/1.1" 200 None 2026-03-21 10:43:29,544 - lava_callback - ERROR - Error processing callback for node 69bbd34fec3f435d20d52b3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:44:09,511 - lava_callback - INFO - Received callback for node 69bbd376ec3f435d20d52b4a INFO: 172.20.0.1:50200 - "POST /node/69bbd376ec3f435d20d52b4a HTTP/1.0" 202 Accepted 2026-03-21 10:44:09,516 - lava_callback - INFO - Starting async job submit for node 69bbd376ec3f435d20d52b4a 2026-03-21 10:44:09,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:44:09,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd376ec3f435d20d52b4a HTTP/1.1" 200 None 2026-03-21 10:44:09,533 - lava_callback - DEBUG - Processing node 69bbd376ec3f435d20d52b4a: baseline-arm-baylibre 2026-03-21 10:44:09,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd376ec3f435d20d52b4a 2026-03-21 10:44:09,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:44:09,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd376ec3f435d20d52b4a HTTP/1.1" 200 None 2026-03-21 10:44:09,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:44:09,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:44:09,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:44:09,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd376ec3f435d20d52b4a HTTP/1.1" 200 None 2026-03-21 10:44:09,597 - lava_callback - ERROR - Error processing callback for node 69bbd376ec3f435d20d52b4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:45:34,008 - lava_callback - INFO - Received callback for node 69be6e38448e074b3ffe24e2 INFO: 172.20.0.1:40222 - "POST /node/69be6e38448e074b3ffe24e2 HTTP/1.0" 202 Accepted 2026-03-21 10:45:34,015 - lava_callback - INFO - Starting async job submit for node 69be6e38448e074b3ffe24e2 2026-03-21 10:45:34,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:34,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6e38448e074b3ffe24e2 HTTP/1.1" 200 None 2026-03-21 10:45:34,050 - lava_callback - DEBUG - Processing node 69be6e38448e074b3ffe24e2: baseline-arm64-broonie 2026-03-21 10:45:34,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:45:34,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:45:34,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6e38448e074b3ffe24e2/log.txt.gz 2026-03-21 10:45:34,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:45:34,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:45:34,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6e38448e074b3ffe24e2/lava-logs.yaml 2026-03-21 10:45:34,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:45:34,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:45:34,993 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be6e38448e074b3ffe24e2/lava_callback.json.gz 2026-03-21 10:45:35,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:35,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6e38448e074b3ffe24e2 HTTP/1.1" 200 None 2026-03-21 10:45:35,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:35,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 10:45:35,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:35,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6e38448e074b3ffe24e2 HTTP/1.1" 200 None 2026-03-21 10:45:35,106 - lava_callback - INFO - Completed processing callback for node 69be6e38448e074b3ffe24e2 INFO: 172.20.0.1:40228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38642 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:45:49,678 - lava_callback - INFO - Received callback for node 69bbd3c9ec3f435d20d52b59 INFO: 172.20.0.1:38658 - "POST /node/69bbd3c9ec3f435d20d52b59 HTTP/1.0" 202 Accepted 2026-03-21 10:45:49,683 - lava_callback - INFO - Starting async job submit for node 69bbd3c9ec3f435d20d52b59 2026-03-21 10:45:49,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:49,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd3c9ec3f435d20d52b59 HTTP/1.1" 200 None 2026-03-21 10:45:49,700 - lava_callback - DEBUG - Processing node 69bbd3c9ec3f435d20d52b59: baseline-arm-baylibre 2026-03-21 10:45:49,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd3c9ec3f435d20d52b59 2026-03-21 10:45:49,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd3c9ec3f435d20d52b59 HTTP/1.1" 200 None 2026-03-21 10:45:49,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:49,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:45:49,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:45:49,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd3c9ec3f435d20d52b59 HTTP/1.1" 200 None 2026-03-21 10:45:49,750 - lava_callback - ERROR - Error processing callback for node 69bbd3c9ec3f435d20d52b59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49212 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:46:01,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:46:01,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43038 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:47:49,831 - lava_callback - INFO - Received callback for node 69bbd442ec3f435d20d52b78 INFO: 172.20.0.1:38950 - "POST /node/69bbd442ec3f435d20d52b78 HTTP/1.0" 202 Accepted 2026-03-21 10:47:49,836 - lava_callback - INFO - Starting async job submit for node 69bbd442ec3f435d20d52b78 2026-03-21 10:47:49,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:47:49,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd442ec3f435d20d52b78 HTTP/1.1" 200 None 2026-03-21 10:47:49,852 - lava_callback - DEBUG - Processing node 69bbd442ec3f435d20d52b78: baseline-arm-baylibre 2026-03-21 10:47:49,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd442ec3f435d20d52b78 2026-03-21 10:47:49,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:47:49,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd442ec3f435d20d52b78 HTTP/1.1" 200 None 2026-03-21 10:47:49,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:47:49,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:47:49,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:47:49,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd442ec3f435d20d52b78 HTTP/1.1" 200 None 2026-03-21 10:47:49,921 - lava_callback - ERROR - Error processing callback for node 69bbd442ec3f435d20d52b78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56736 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:48:09,706 - lava_callback - INFO - Received callback for node 69bbd467ec3f435d20d52b87 INFO: 172.20.0.1:60986 - "POST /node/69bbd467ec3f435d20d52b87 HTTP/1.0" 202 Accepted 2026-03-21 10:48:09,711 - lava_callback - INFO - Starting async job submit for node 69bbd467ec3f435d20d52b87 2026-03-21 10:48:09,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:48:09,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd467ec3f435d20d52b87 HTTP/1.1" 200 None 2026-03-21 10:48:09,728 - lava_callback - DEBUG - Processing node 69bbd467ec3f435d20d52b87: baseline-arm-baylibre 2026-03-21 10:48:09,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd467ec3f435d20d52b87 2026-03-21 10:48:09,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:48:09,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd467ec3f435d20d52b87 HTTP/1.1" 200 None 2026-03-21 10:48:09,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:48:09,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:48:09,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:48:09,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd467ec3f435d20d52b87 HTTP/1.1" 200 None 2026-03-21 10:48:09,787 - lava_callback - ERROR - Error processing callback for node 69bbd467ec3f435d20d52b87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50754 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:50:56,392 - lava_callback - INFO - Received callback for node 69bbd507ec3f435d20d52bc0 INFO: 172.20.0.1:44376 - "POST /node/69bbd507ec3f435d20d52bc0 HTTP/1.0" 202 Accepted 2026-03-21 10:50:56,398 - lava_callback - INFO - Starting async job submit for node 69bbd507ec3f435d20d52bc0 2026-03-21 10:50:56,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:50:56,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd507ec3f435d20d52bc0 HTTP/1.1" 200 None 2026-03-21 10:50:56,414 - lava_callback - DEBUG - Processing node 69bbd507ec3f435d20d52bc0: baseline-arm-clabbe 2026-03-21 10:50:56,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd507ec3f435d20d52bc0 2026-03-21 10:50:56,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:50:56,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd507ec3f435d20d52bc0 HTTP/1.1" 200 None 2026-03-21 10:50:56,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:50:56,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:50:56,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:50:56,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd507ec3f435d20d52bc0 HTTP/1.1" 200 None 2026-03-21 10:50:56,472 - lava_callback - ERROR - Error processing callback for node 69bbd507ec3f435d20d52bc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36414 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:51:36,327 - lava_callback - INFO - Received callback for node 69bbd536ec3f435d20d52bce INFO: 172.20.0.1:35474 - "POST /node/69bbd536ec3f435d20d52bce HTTP/1.0" 202 Accepted 2026-03-21 10:51:36,333 - lava_callback - INFO - Starting async job submit for node 69bbd536ec3f435d20d52bce 2026-03-21 10:51:36,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:51:36,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd536ec3f435d20d52bce HTTP/1.1" 200 None 2026-03-21 10:51:36,356 - lava_callback - DEBUG - Processing node 69bbd536ec3f435d20d52bce: baseline-arm-clabbe 2026-03-21 10:51:36,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd536ec3f435d20d52bce 2026-03-21 10:51:36,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:51:36,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd536ec3f435d20d52bce HTTP/1.1" 200 None 2026-03-21 10:51:36,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:51:36,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:51:36,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:51:36,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd536ec3f435d20d52bce HTTP/1.1" 200 None 2026-03-21 10:51:36,442 - lava_callback - ERROR - Error processing callback for node 69bbd536ec3f435d20d52bce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39186 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:53:56,426 - lava_callback - INFO - Received callback for node 69bbd5afec3f435d20d52be3 INFO: 172.20.0.1:41594 - "POST /node/69bbd5afec3f435d20d52be3 HTTP/1.0" 202 Accepted 2026-03-21 10:53:56,432 - lava_callback - INFO - Starting async job submit for node 69bbd5afec3f435d20d52be3 2026-03-21 10:53:56,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:53:56,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd5afec3f435d20d52be3 HTTP/1.1" 200 None 2026-03-21 10:53:56,469 - lava_callback - DEBUG - Processing node 69bbd5afec3f435d20d52be3: baseline-arm-clabbe 2026-03-21 10:53:56,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd5afec3f435d20d52be3 2026-03-21 10:53:56,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:53:56,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd5afec3f435d20d52be3 HTTP/1.1" 200 None 2026-03-21 10:53:56,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:41606 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:53:56,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:53:56,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:53:56,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd5afec3f435d20d52be3 HTTP/1.1" 200 None 2026-03-21 10:53:56,561 - lava_callback - ERROR - Error processing callback for node 69bbd5afec3f435d20d52be3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39518 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:54:56,510 - lava_callback - INFO - Received callback for node 69bbd5eaec3f435d20d52bf5 INFO: 172.20.0.1:49846 - "POST /node/69bbd5eaec3f435d20d52bf5 HTTP/1.0" 202 Accepted 2026-03-21 10:54:56,515 - lava_callback - INFO - Starting async job submit for node 69bbd5eaec3f435d20d52bf5 2026-03-21 10:54:56,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:54:56,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd5eaec3f435d20d52bf5 HTTP/1.1" 200 None 2026-03-21 10:54:56,532 - lava_callback - DEBUG - Processing node 69bbd5eaec3f435d20d52bf5: baseline-arm-clabbe 2026-03-21 10:54:56,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd5eaec3f435d20d52bf5 2026-03-21 10:54:56,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:54:56,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd5eaec3f435d20d52bf5 HTTP/1.1" 200 None 2026-03-21 10:54:56,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:54:56,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:54:56,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:54:56,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd5eaec3f435d20d52bf5 HTTP/1.1" 200 None 2026-03-21 10:54:56,605 - lava_callback - ERROR - Error processing callback for node 69bbd5eaec3f435d20d52bf5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45424 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:55:56,573 - lava_callback - INFO - Received callback for node 69bbd626ec3f435d20d52c08 INFO: 172.20.0.1:45438 - "POST /node/69bbd626ec3f435d20d52c08 HTTP/1.0" 202 Accepted 2026-03-21 10:55:56,578 - lava_callback - INFO - Starting async job submit for node 69bbd626ec3f435d20d52c08 2026-03-21 10:55:56,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:55:56,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd626ec3f435d20d52c08 HTTP/1.1" 200 None 2026-03-21 10:55:56,601 - lava_callback - DEBUG - Processing node 69bbd626ec3f435d20d52c08: baseline-arm-clabbe 2026-03-21 10:55:56,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbd626ec3f435d20d52c08 2026-03-21 10:55:56,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:55:56,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbd626ec3f435d20d52c08 HTTP/1.1" 200 None 2026-03-21 10:55:56,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:55:56,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 10:55:56,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:55:56,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbd626ec3f435d20d52c08 HTTP/1.1" 200 None 2026-03-21 10:55:56,656 - lava_callback - ERROR - Error processing callback for node 69bbd626ec3f435d20d52c08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58852 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:58:02,151 - lava_callback - INFO - Received callback for node 69be7515448e074b3ffe252f INFO: 172.20.0.1:36732 - "POST /node/69be7515448e074b3ffe252f HTTP/1.0" 202 Accepted 2026-03-21 10:58:02,159 - lava_callback - INFO - Starting async job submit for node 69be7515448e074b3ffe252f 2026-03-21 10:58:02,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:58:02,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be7515448e074b3ffe252f HTTP/1.1" 200 None 2026-03-21 10:58:02,217 - lava_callback - DEBUG - Processing node 69be7515448e074b3ffe252f: baseline-nfs-arm64 2026-03-21 10:58:03,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:58:03,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:58:03,403 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be7515448e074b3ffe252f/log.txt.gz 2026-03-21 10:58:03,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:58:04,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:58:04,258 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be7515448e074b3ffe252f/lava-logs.yaml 2026-03-21 10:58:04,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 10:58:04,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 10:58:04,456 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be7515448e074b3ffe252f/lava_callback.json.gz DEBUG: Setup failed for 69be7515448e074b3ffe252f. Transitting job node result: incomplete -> fail 2026-03-21 10:58:04,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:58:04,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be7515448e074b3ffe252f HTTP/1.1" 200 None 2026-03-21 10:58:04,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:58:04,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 10:58:04,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:58:04,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be7515448e074b3ffe252f HTTP/1.1" 200 None 2026-03-21 10:58:04,710 - lava_callback - INFO - Completed processing callback for node 69be7515448e074b3ffe252f INFO: 172.20.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46904 - "GET / HTTP/1.0" 200 OK 2026-03-21 10:58:31,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 10:58:31,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59750 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:17:34,714 - lava_callback - INFO - Received callback for node 69be76e0448e074b3ffe2551 INFO: 172.20.0.1:53128 - "POST /node/69be76e0448e074b3ffe2551 HTTP/1.0" 202 Accepted 2026-03-21 11:17:34,721 - lava_callback - INFO - Starting async job submit for node 69be76e0448e074b3ffe2551 2026-03-21 11:17:34,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:17:34,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be76e0448e074b3ffe2551 HTTP/1.1" 200 None 2026-03-21 11:17:34,761 - lava_callback - DEBUG - Processing node 69be76e0448e074b3ffe2551: baseline-arm64-broonie 2026-03-21 11:17:34,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 11:17:35,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 11:17:35,108 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be76e0448e074b3ffe2551/log.txt.gz 2026-03-21 11:17:35,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 11:17:35,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 11:17:35,475 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be76e0448e074b3ffe2551/lava-logs.yaml 2026-03-21 11:17:35,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 11:17:35,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 11:17:35,617 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be76e0448e074b3ffe2551/lava_callback.json.gz 2026-03-21 11:17:35,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:17:35,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be76e0448e074b3ffe2551 HTTP/1.1" 200 None 2026-03-21 11:17:35,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:17:35,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 11:17:35,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:17:35,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be76e0448e074b3ffe2551 HTTP/1.1" 200 None 2026-03-21 11:17:35,761 - lava_callback - INFO - Completed processing callback for node 69be76e0448e074b3ffe2551 INFO: 172.20.0.1:53130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55340 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:18:01,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:18:01,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49870 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:37:52,506 - lava_callback - INFO - Received callback for node 69bbe008ec3f435d20d52cbc INFO: 172.20.0.1:48950 - "POST /node/69bbe008ec3f435d20d52cbc HTTP/1.0" 202 Accepted 2026-03-21 11:37:52,512 - lava_callback - INFO - Starting async job submit for node 69bbe008ec3f435d20d52cbc 2026-03-21 11:37:52,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:37:52,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe008ec3f435d20d52cbc HTTP/1.1" 200 None 2026-03-21 11:37:52,553 - lava_callback - DEBUG - Processing node 69bbe008ec3f435d20d52cbc: baseline-arm64-baylibre 2026-03-21 11:37:52,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe008ec3f435d20d52cbc 2026-03-21 11:37:52,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:37:52,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe008ec3f435d20d52cbc HTTP/1.1" 200 None 2026-03-21 11:37:52,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:37:52,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:37:52,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:37:52,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe008ec3f435d20d52cbc HTTP/1.1" 200 None 2026-03-21 11:37:52,655 - lava_callback - ERROR - Error processing callback for node 69bbe008ec3f435d20d52cbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58012 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:40:52,677 - lava_callback - INFO - Received callback for node 69bbe0bcec3f435d20d52cc9 INFO: 172.20.0.1:60420 - "POST /node/69bbe0bcec3f435d20d52cc9 HTTP/1.0" 202 Accepted 2026-03-21 11:40:52,683 - lava_callback - INFO - Starting async job submit for node 69bbe0bcec3f435d20d52cc9 2026-03-21 11:40:52,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:40:52,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe0bcec3f435d20d52cc9 HTTP/1.1" 200 None 2026-03-21 11:40:52,701 - lava_callback - DEBUG - Processing node 69bbe0bcec3f435d20d52cc9: baseline-arm64-baylibre 2026-03-21 11:40:52,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe0bcec3f435d20d52cc9 2026-03-21 11:40:52,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:40:52,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe0bcec3f435d20d52cc9 HTTP/1.1" 200 None 2026-03-21 11:40:52,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:40:52,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:40:52,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:40:52,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe0bcec3f435d20d52cc9 HTTP/1.1" 200 None 2026-03-21 11:40:52,773 - lava_callback - ERROR - Error processing callback for node 69bbe0bcec3f435d20d52cc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35770 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:42:12,769 - lava_callback - INFO - Received callback for node 69bbe10cec3f435d20d52cd1 INFO: 172.20.0.1:44654 - "POST /node/69bbe10cec3f435d20d52cd1 HTTP/1.0" 202 Accepted 2026-03-21 11:42:12,775 - lava_callback - INFO - Starting async job submit for node 69bbe10cec3f435d20d52cd1 2026-03-21 11:42:12,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:42:12,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe10cec3f435d20d52cd1 HTTP/1.1" 200 None 2026-03-21 11:42:12,792 - lava_callback - DEBUG - Processing node 69bbe10cec3f435d20d52cd1: baseline-arm64-baylibre 2026-03-21 11:42:12,792 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe10cec3f435d20d52cd1 2026-03-21 11:42:12,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:42:12,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe10cec3f435d20d52cd1 HTTP/1.1" 200 None 2026-03-21 11:42:12,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:42:12,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:42:12,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:42:12,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe10cec3f435d20d52cd1 HTTP/1.1" 200 None 2026-03-21 11:42:12,854 - lava_callback - ERROR - Error processing callback for node 69bbe10cec3f435d20d52cd1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40334 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:43:52,783 - lava_callback - INFO - Received callback for node 69bbe171ec3f435d20d52cdf INFO: 172.20.0.1:32834 - "POST /node/69bbe171ec3f435d20d52cdf HTTP/1.0" 202 Accepted 2026-03-21 11:43:52,788 - lava_callback - INFO - Starting async job submit for node 69bbe171ec3f435d20d52cdf 2026-03-21 11:43:52,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:43:52,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe171ec3f435d20d52cdf HTTP/1.1" 200 None 2026-03-21 11:43:52,804 - lava_callback - DEBUG - Processing node 69bbe171ec3f435d20d52cdf: baseline-arm64-baylibre 2026-03-21 11:43:52,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe171ec3f435d20d52cdf 2026-03-21 11:43:52,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:43:52,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe171ec3f435d20d52cdf HTTP/1.1" 200 None 2026-03-21 11:43:52,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:43:52,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:43:52,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:43:52,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe171ec3f435d20d52cdf HTTP/1.1" 200 None 2026-03-21 11:43:52,865 - lava_callback - ERROR - Error processing callback for node 69bbe171ec3f435d20d52cdf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54138 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:45:12,907 - lava_callback - INFO - Received callback for node 69bbe1bfec3f435d20d52ce7 INFO: 172.20.0.1:55530 - "POST /node/69bbe1bfec3f435d20d52ce7 HTTP/1.0" 202 Accepted 2026-03-21 11:45:12,912 - lava_callback - INFO - Starting async job submit for node 69bbe1bfec3f435d20d52ce7 2026-03-21 11:45:12,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:45:12,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe1bfec3f435d20d52ce7 HTTP/1.1" 200 None 2026-03-21 11:45:12,929 - lava_callback - DEBUG - Processing node 69bbe1bfec3f435d20d52ce7: baseline-arm64-baylibre 2026-03-21 11:45:12,929 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe1bfec3f435d20d52ce7 2026-03-21 11:45:12,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:45:12,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe1bfec3f435d20d52ce7 HTTP/1.1" 200 None 2026-03-21 11:45:12,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:45:12,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:45:12,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:45:13,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe1bfec3f435d20d52ce7 HTTP/1.1" 200 None 2026-03-21 11:45:13,002 - lava_callback - ERROR - Error processing callback for node 69bbe1bfec3f435d20d52ce7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:46:32,961 - lava_callback - INFO - Received callback for node 69bbe211ec3f435d20d52cf3 INFO: 172.20.0.1:54094 - "POST /node/69bbe211ec3f435d20d52cf3 HTTP/1.0" 202 Accepted 2026-03-21 11:46:32,967 - lava_callback - INFO - Starting async job submit for node 69bbe211ec3f435d20d52cf3 2026-03-21 11:46:32,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:46:32,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe211ec3f435d20d52cf3 HTTP/1.1" 200 None 2026-03-21 11:46:32,985 - lava_callback - DEBUG - Processing node 69bbe211ec3f435d20d52cf3: baseline-arm64-baylibre 2026-03-21 11:46:32,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe211ec3f435d20d52cf3 2026-03-21 11:46:32,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:46:33,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe211ec3f435d20d52cf3 HTTP/1.1" 200 None 2026-03-21 11:46:33,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:46:33,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:46:33,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:46:33,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe211ec3f435d20d52cf3 HTTP/1.1" 200 None 2026-03-21 11:46:33,066 - lava_callback - ERROR - Error processing callback for node 69bbe211ec3f435d20d52cf3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41946 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:49:35,943 - lava_callback - INFO - Received callback for node 69be7e61448e074b3ffe25da INFO: 172.20.0.1:55232 - "POST /node/69be7e61448e074b3ffe25da HTTP/1.0" 202 Accepted 2026-03-21 11:49:35,950 - lava_callback - INFO - Starting async job submit for node 69be7e61448e074b3ffe25da 2026-03-21 11:49:35,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:49:35,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be7e61448e074b3ffe25da HTTP/1.1" 200 None 2026-03-21 11:49:35,991 - lava_callback - DEBUG - Processing node 69be7e61448e074b3ffe25da: baseline-arm64-broonie 2026-03-21 11:49:36,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 11:49:36,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 11:49:36,335 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be7e61448e074b3ffe25da/log.txt.gz 2026-03-21 11:49:36,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:49:36,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 11:49:36,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be7e61448e074b3ffe25da/lava-logs.yaml 2026-03-21 11:49:36,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 11:49:36,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 11:49:36,861 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be7e61448e074b3ffe25da/lava_callback.json.gz 2026-03-21 11:49:36,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:49:36,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be7e61448e074b3ffe25da HTTP/1.1" 200 None 2026-03-21 11:49:36,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:49:36,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 11:49:36,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:49:36,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be7e61448e074b3ffe25da HTTP/1.1" 200 None 2026-03-21 11:49:36,971 - lava_callback - INFO - Completed processing callback for node 69be7e61448e074b3ffe25da INFO: 172.20.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56886 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:50:01,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:50:01,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58578 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:50:18,787 - lava_callback - INFO - Received callback for node 69bbe2f2ec3f435d20d52d08 INFO: 172.20.0.1:58582 - "POST /node/69bbe2f2ec3f435d20d52d08 HTTP/1.0" 202 Accepted 2026-03-21 11:50:18,792 - lava_callback - INFO - Starting async job submit for node 69bbe2f2ec3f435d20d52d08 2026-03-21 11:50:18,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:50:18,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe2f2ec3f435d20d52d08 HTTP/1.1" 200 None 2026-03-21 11:50:18,808 - lava_callback - DEBUG - Processing node 69bbe2f2ec3f435d20d52d08: baseline-arm64-clabbe 2026-03-21 11:50:18,808 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe2f2ec3f435d20d52d08 2026-03-21 11:50:18,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:50:18,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe2f2ec3f435d20d52d08 HTTP/1.1" 200 None 2026-03-21 11:50:18,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:50:18,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:50:18,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:50:18,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe2f2ec3f435d20d52d08 HTTP/1.1" 200 None 2026-03-21 11:50:18,884 - lava_callback - ERROR - Error processing callback for node 69bbe2f2ec3f435d20d52d08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39496 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:52:58,823 - lava_callback - INFO - Received callback for node 69bbe396ec3f435d20d52d18 INFO: 172.20.0.1:39498 - "POST /node/69bbe396ec3f435d20d52d18 HTTP/1.0" 202 Accepted 2026-03-21 11:52:58,828 - lava_callback - INFO - Starting async job submit for node 69bbe396ec3f435d20d52d18 2026-03-21 11:52:58,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:52:58,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe396ec3f435d20d52d18 HTTP/1.1" 200 None 2026-03-21 11:52:58,851 - lava_callback - DEBUG - Processing node 69bbe396ec3f435d20d52d18: baseline-arm64-clabbe 2026-03-21 11:52:58,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe396ec3f435d20d52d18 2026-03-21 11:52:58,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:52:58,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe396ec3f435d20d52d18 HTTP/1.1" 200 None 2026-03-21 11:52:58,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:52:58,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:52:58,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:52:58,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe396ec3f435d20d52d18 HTTP/1.1" 200 None 2026-03-21 11:52:58,907 - lava_callback - ERROR - Error processing callback for node 69bbe396ec3f435d20d52d18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40648 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:54:18,907 - lava_callback - INFO - Received callback for node 69bbe3e6ec3f435d20d52d23 INFO: 172.20.0.1:40664 - "POST /node/69bbe3e6ec3f435d20d52d23 HTTP/1.0" 202 Accepted 2026-03-21 11:54:18,913 - lava_callback - INFO - Starting async job submit for node 69bbe3e6ec3f435d20d52d23 2026-03-21 11:54:18,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:54:18,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe3e6ec3f435d20d52d23 HTTP/1.1" 200 None 2026-03-21 11:54:18,933 - lava_callback - DEBUG - Processing node 69bbe3e6ec3f435d20d52d23: baseline-arm64-clabbe 2026-03-21 11:54:18,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe3e6ec3f435d20d52d23 2026-03-21 11:54:18,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:54:18,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe3e6ec3f435d20d52d23 HTTP/1.1" 200 None 2026-03-21 11:54:18,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:54:18,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:54:18,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:54:19,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe3e6ec3f435d20d52d23 HTTP/1.1" 200 None 2026-03-21 11:54:19,012 - lava_callback - ERROR - Error processing callback for node 69bbe3e6ec3f435d20d52d23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43610 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:55:58,947 - lava_callback - INFO - Received callback for node 69bbe44aec3f435d20d52d2e INFO: 172.20.0.1:43622 - "POST /node/69bbe44aec3f435d20d52d2e HTTP/1.0" 202 Accepted 2026-03-21 11:55:58,953 - lava_callback - INFO - Starting async job submit for node 69bbe44aec3f435d20d52d2e 2026-03-21 11:55:58,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:55:58,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe44aec3f435d20d52d2e HTTP/1.1" 200 None 2026-03-21 11:55:58,991 - lava_callback - DEBUG - Processing node 69bbe44aec3f435d20d52d2e: baseline-arm64-clabbe 2026-03-21 11:55:58,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe44aec3f435d20d52d2e 2026-03-21 11:55:58,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:55:59,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe44aec3f435d20d52d2e HTTP/1.1" 200 None 2026-03-21 11:55:59,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:55:59,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:55:59,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:55:59,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe44aec3f435d20d52d2e HTTP/1.1" 200 None 2026-03-21 11:55:59,103 - lava_callback - ERROR - Error processing callback for node 69bbe44aec3f435d20d52d2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:57:19,071 - lava_callback - INFO - Received callback for node 69bbe499ec3f435d20d52d39 INFO: 172.20.0.1:56238 - "POST /node/69bbe499ec3f435d20d52d39 HTTP/1.0" 202 Accepted 2026-03-21 11:57:19,077 - lava_callback - INFO - Starting async job submit for node 69bbe499ec3f435d20d52d39 2026-03-21 11:57:19,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:57:19,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe499ec3f435d20d52d39 HTTP/1.1" 200 None 2026-03-21 11:57:19,102 - lava_callback - DEBUG - Processing node 69bbe499ec3f435d20d52d39: baseline-arm64-clabbe 2026-03-21 11:57:19,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe499ec3f435d20d52d39 2026-03-21 11:57:19,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:57:19,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe499ec3f435d20d52d39 HTTP/1.1" 200 None 2026-03-21 11:57:19,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:57:19,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:57:19,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:57:19,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe499ec3f435d20d52d39 HTTP/1.1" 200 None 2026-03-21 11:57:19,182 - lava_callback - ERROR - Error processing callback for node 69bbe499ec3f435d20d52d39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK 2026-03-21 11:58:39,114 - lava_callback - INFO - Received callback for node 69bbe4e9ec3f435d20d52d43 INFO: 172.20.0.1:40564 - "POST /node/69bbe4e9ec3f435d20d52d43 HTTP/1.0" 202 Accepted 2026-03-21 11:58:39,119 - lava_callback - INFO - Starting async job submit for node 69bbe4e9ec3f435d20d52d43 2026-03-21 11:58:39,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:58:39,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe4e9ec3f435d20d52d43 HTTP/1.1" 200 None 2026-03-21 11:58:39,172 - lava_callback - DEBUG - Processing node 69bbe4e9ec3f435d20d52d43: baseline-arm64-clabbe 2026-03-21 11:58:39,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbe4e9ec3f435d20d52d43 2026-03-21 11:58:39,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:58:39,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbe4e9ec3f435d20d52d43 HTTP/1.1" 200 None 2026-03-21 11:58:39,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:58:39,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 11:58:39,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 11:58:39,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbe4e9ec3f435d20d52d43 HTTP/1.1" 200 None 2026-03-21 11:58:39,234 - lava_callback - ERROR - Error processing callback for node 69bbe4e9ec3f435d20d52d43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:00:57,683 - lava_callback - INFO - Received callback for node 69be6343448e074b3ffe212d INFO: 172.20.0.1:51116 - "POST /node/69be6343448e074b3ffe212d HTTP/1.0" 202 Accepted 2026-03-21 12:00:57,690 - lava_callback - INFO - Starting async job submit for node 69be6343448e074b3ffe212d 2026-03-21 12:00:57,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:00:57,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6343448e074b3ffe212d HTTP/1.1" 200 None 2026-03-21 12:00:57,737 - lava_callback - DEBUG - Processing node 69be6343448e074b3ffe212d: baseline-arm64 2026-03-21 12:00:58,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 12:00:58,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 12:00:58,465 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6343448e074b3ffe212d/log.txt.gz 2026-03-21 12:00:58,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 12:00:58,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 12:00:58,976 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6343448e074b3ffe212d/lava-logs.yaml 2026-03-21 12:00:58,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 12:00:59,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 12:00:59,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69be6343448e074b3ffe212d/lava_callback.json.gz 2026-03-21 12:00:59,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:00:59,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6343448e074b3ffe212d HTTP/1.1" 200 None 2026-03-21 12:00:59,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:00:59,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 12:00:59,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:00:59,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6343448e074b3ffe212d HTTP/1.1" 200 None 2026-03-21 12:00:59,309 - lava_callback - INFO - Completed processing callback for node 69be6343448e074b3ffe212d 2026-03-21 12:01:01,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:01:01,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47752 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:11:49,359 - lava_callback - INFO - Received callback for node 69be5f50448e074b3ffe2053 INFO: 172.20.0.1:47758 - "POST /node/69be5f50448e074b3ffe2053 HTTP/1.0" 202 Accepted 2026-03-21 12:11:49,365 - lava_callback - INFO - Starting async job submit for node 69be5f50448e074b3ffe2053 2026-03-21 12:11:49,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:11:49,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f50448e074b3ffe2053 HTTP/1.1" 200 None 2026-03-21 12:11:49,421 - lava_callback - DEBUG - Processing node 69be5f50448e074b3ffe2053: baseline-x86 2026-03-21 12:11:49,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 12:11:49,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 12:11:49,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f50448e074b3ffe2053/log.txt.gz 2026-03-21 12:11:50,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 12:11:50,234 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 12:11:50,236 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f50448e074b3ffe2053/lava-logs.yaml 2026-03-21 12:11:50,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 12:11:50,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 12:11:50,413 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69be5f50448e074b3ffe2053/lava_callback.json.gz 2026-03-21 12:11:50,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:11:50,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5f50448e074b3ffe2053 HTTP/1.1" 200 None 2026-03-21 12:11:50,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:11:50,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5478448e074b3ffe1656 HTTP/1.1" 200 None 2026-03-21 12:11:50,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:11:50,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be5f50448e074b3ffe2053 HTTP/1.1" 200 None 2026-03-21 12:11:50,499 - lava_callback - INFO - Completed processing callback for node 69be5f50448e074b3ffe2053 INFO: 172.20.0.1:52344 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:12:01,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:12:01,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51752 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:48:36,473 - lava_callback - INFO - Received callback for node 69bbf092ec3f435d20d52da0 INFO: 172.20.0.1:47874 - "POST /node/69bbf092ec3f435d20d52da0 HTTP/1.0" 202 Accepted 2026-03-21 12:48:36,479 - lava_callback - INFO - Starting async job submit for node 69bbf092ec3f435d20d52da0 2026-03-21 12:48:36,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:48:36,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf092ec3f435d20d52da0 HTTP/1.1" 200 None 2026-03-21 12:48:36,499 - lava_callback - DEBUG - Processing node 69bbf092ec3f435d20d52da0: baseline-arm-baylibre 2026-03-21 12:48:36,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf092ec3f435d20d52da0 2026-03-21 12:48:36,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:48:36,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf092ec3f435d20d52da0 HTTP/1.1" 200 None 2026-03-21 12:48:36,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:48:36,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 12:48:36,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:48:36,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf092ec3f435d20d52da0 HTTP/1.1" 200 None 2026-03-21 12:48:36,552 - lava_callback - ERROR - Error processing callback for node 69bbf092ec3f435d20d52da0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57346 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:53:41,425 - lava_callback - INFO - Received callback for node 69bbf1c5ec3f435d20d52e09 INFO: 172.20.0.1:47230 - "POST /node/69bbf1c5ec3f435d20d52e09 HTTP/1.0" 202 Accepted 2026-03-21 12:53:41,431 - lava_callback - INFO - Starting async job submit for node 69bbf1c5ec3f435d20d52e09 2026-03-21 12:53:41,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:53:41,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf1c5ec3f435d20d52e09 HTTP/1.1" 200 None 2026-03-21 12:53:41,483 - lava_callback - DEBUG - Processing node 69bbf1c5ec3f435d20d52e09: baseline-arm-clabbe 2026-03-21 12:53:41,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf1c5ec3f435d20d52e09 2026-03-21 12:53:41,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:53:41,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf1c5ec3f435d20d52e09 HTTP/1.1" 200 None 2026-03-21 12:53:41,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:53:41,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 12:53:41,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:53:41,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf1c5ec3f435d20d52e09 HTTP/1.1" 200 None 2026-03-21 12:53:41,539 - lava_callback - ERROR - Error processing callback for node 69bbf1c5ec3f435d20d52e09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:54:16,845 - lava_callback - INFO - Received callback for node 69bbf1e6ec3f435d20d52e1d INFO: 172.20.0.1:36182 - "POST /node/69bbf1e6ec3f435d20d52e1d HTTP/1.0" 202 Accepted 2026-03-21 12:54:16,850 - lava_callback - INFO - Starting async job submit for node 69bbf1e6ec3f435d20d52e1d 2026-03-21 12:54:16,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:54:16,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf1e6ec3f435d20d52e1d HTTP/1.1" 200 None 2026-03-21 12:54:16,873 - lava_callback - DEBUG - Processing node 69bbf1e6ec3f435d20d52e1d: baseline-arm-baylibre 2026-03-21 12:54:16,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf1e6ec3f435d20d52e1d 2026-03-21 12:54:16,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:54:16,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf1e6ec3f435d20d52e1d HTTP/1.1" 200 None 2026-03-21 12:54:16,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:54:16,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 12:54:16,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:54:16,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf1e6ec3f435d20d52e1d HTTP/1.1" 200 None 2026-03-21 12:54:16,939 - lava_callback - ERROR - Error processing callback for node 69bbf1e6ec3f435d20d52e1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44442 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:55:01,411 - lava_callback - INFO - Received callback for node 69bbf212ec3f435d20d52e31 INFO: 172.20.0.1:60772 - "POST /node/69bbf212ec3f435d20d52e31 HTTP/1.0" 202 Accepted 2026-03-21 12:55:01,417 - lava_callback - INFO - Starting async job submit for node 69bbf212ec3f435d20d52e31 2026-03-21 12:55:01,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:01,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf212ec3f435d20d52e31 HTTP/1.1" 200 None 2026-03-21 12:55:01,439 - lava_callback - DEBUG - Processing node 69bbf212ec3f435d20d52e31: baseline-arm-clabbe 2026-03-21 12:55:01,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf212ec3f435d20d52e31 2026-03-21 12:55:01,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:01,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf212ec3f435d20d52e31 HTTP/1.1" 200 None 2026-03-21 12:55:01,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:01,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 12:55:01,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:01,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf212ec3f435d20d52e31 HTTP/1.1" 200 None 2026-03-21 12:55:01,494 - lava_callback - ERROR - Error processing callback for node 69bbf212ec3f435d20d52e31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59734 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:55:41,405 - lava_callback - INFO - Received callback for node 69bbf239ec3f435d20d52e40 INFO: 172.20.0.1:42306 - "POST /node/69bbf239ec3f435d20d52e40 HTTP/1.0" 202 Accepted 2026-03-21 12:55:41,410 - lava_callback - INFO - Starting async job submit for node 69bbf239ec3f435d20d52e40 2026-03-21 12:55:41,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:41,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf239ec3f435d20d52e40 HTTP/1.1" 200 None 2026-03-21 12:55:41,429 - lava_callback - DEBUG - Processing node 69bbf239ec3f435d20d52e40: baseline-arm-clabbe 2026-03-21 12:55:41,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf239ec3f435d20d52e40 2026-03-21 12:55:41,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:41,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf239ec3f435d20d52e40 HTTP/1.1" 200 None 2026-03-21 12:55:41,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:41,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 12:55:41,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:55:41,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf239ec3f435d20d52e40 HTTP/1.1" 200 None 2026-03-21 12:55:41,493 - lava_callback - ERROR - Error processing callback for node 69bbf239ec3f435d20d52e40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51278 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:56:16,927 - lava_callback - INFO - Received callback for node 69bbf25cec3f435d20d52e53 INFO: 172.20.0.1:51280 - "POST /node/69bbf25cec3f435d20d52e53 HTTP/1.0" 202 Accepted 2026-03-21 12:56:16,932 - lava_callback - INFO - Starting async job submit for node 69bbf25cec3f435d20d52e53 2026-03-21 12:56:16,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:16,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf25cec3f435d20d52e53 HTTP/1.1" 200 None 2026-03-21 12:56:16,947 - lava_callback - DEBUG - Processing node 69bbf25cec3f435d20d52e53: baseline-arm-baylibre 2026-03-21 12:56:16,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf25cec3f435d20d52e53 2026-03-21 12:56:16,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:16,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf25cec3f435d20d52e53 HTTP/1.1" 200 None 2026-03-21 12:56:16,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 12:56:16,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:17,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf25cec3f435d20d52e53 HTTP/1.1" 200 None 2026-03-21 12:56:17,016 - lava_callback - ERROR - Error processing callback for node 69bbf25cec3f435d20d52e53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41060 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:56:41,509 - lava_callback - INFO - Received callback for node 69bbf287ec3f435d20d52e67 INFO: 172.20.0.1:51806 - "POST /node/69bbf287ec3f435d20d52e67 HTTP/1.0" 202 Accepted 2026-03-21 12:56:41,514 - lava_callback - INFO - Starting async job submit for node 69bbf287ec3f435d20d52e67 2026-03-21 12:56:41,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:41,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf287ec3f435d20d52e67 HTTP/1.1" 200 None 2026-03-21 12:56:41,530 - lava_callback - DEBUG - Processing node 69bbf287ec3f435d20d52e67: baseline-arm-clabbe 2026-03-21 12:56:41,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf287ec3f435d20d52e67 2026-03-21 12:56:41,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:41,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf287ec3f435d20d52e67 HTTP/1.1" 200 None 2026-03-21 12:56:41,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:41,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 12:56:41,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:56:41,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf287ec3f435d20d52e67 HTTP/1.1" 200 None 2026-03-21 12:56:41,592 - lava_callback - ERROR - Error processing callback for node 69bbf287ec3f435d20d52e67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49764 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:57:16,838 - lava_callback - INFO - Received callback for node 69bbf2a8ec3f435d20d52e74 INFO: 172.20.0.1:49772 - "POST /node/69bbf2a8ec3f435d20d52e74 HTTP/1.0" 202 Accepted 2026-03-21 12:57:16,844 - lava_callback - INFO - Starting async job submit for node 69bbf2a8ec3f435d20d52e74 2026-03-21 12:57:16,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:16,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf2a8ec3f435d20d52e74 HTTP/1.1" 200 None 2026-03-21 12:57:16,865 - lava_callback - DEBUG - Processing node 69bbf2a8ec3f435d20d52e74: baseline-arm-baylibre 2026-03-21 12:57:16,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf2a8ec3f435d20d52e74 2026-03-21 12:57:16,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:16,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf2a8ec3f435d20d52e74 HTTP/1.1" 200 None 2026-03-21 12:57:16,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 12:57:16,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:16,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf2a8ec3f435d20d52e74 HTTP/1.1" 200 None 2026-03-21 12:57:16,938 - lava_callback - ERROR - Error processing callback for node 69bbf2a8ec3f435d20d52e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54830 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:57:56,917 - lava_callback - INFO - Received callback for node 69bbf2cfec3f435d20d52e82 INFO: 172.20.0.1:54832 - "POST /node/69bbf2cfec3f435d20d52e82 HTTP/1.0" 202 Accepted 2026-03-21 12:57:56,923 - lava_callback - INFO - Starting async job submit for node 69bbf2cfec3f435d20d52e82 2026-03-21 12:57:56,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:56,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf2cfec3f435d20d52e82 HTTP/1.1" 200 None 2026-03-21 12:57:56,944 - lava_callback - DEBUG - Processing node 69bbf2cfec3f435d20d52e82: baseline-arm-baylibre 2026-03-21 12:57:56,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf2cfec3f435d20d52e82 2026-03-21 12:57:56,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:56,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf2cfec3f435d20d52e82 HTTP/1.1" 200 None 2026-03-21 12:57:56,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:56,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 12:57:56,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:57:56,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf2cfec3f435d20d52e82 HTTP/1.1" 200 None 2026-03-21 12:57:57,000 - lava_callback - ERROR - Error processing callback for node 69bbf2cfec3f435d20d52e82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33780 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:58:41,629 - lava_callback - INFO - Received callback for node 69bbf2f9ec3f435d20d52e8f INFO: 172.20.0.1:46956 - "POST /node/69bbf2f9ec3f435d20d52e8f HTTP/1.0" 202 Accepted 2026-03-21 12:58:41,635 - lava_callback - INFO - Starting async job submit for node 69bbf2f9ec3f435d20d52e8f 2026-03-21 12:58:41,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:58:41,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf2f9ec3f435d20d52e8f HTTP/1.1" 200 None 2026-03-21 12:58:41,655 - lava_callback - DEBUG - Processing node 69bbf2f9ec3f435d20d52e8f: baseline-arm-clabbe 2026-03-21 12:58:41,655 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf2f9ec3f435d20d52e8f 2026-03-21 12:58:41,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:58:41,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf2f9ec3f435d20d52e8f HTTP/1.1" 200 None 2026-03-21 12:58:41,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:58:41,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 12:58:41,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:58:41,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf2f9ec3f435d20d52e8f HTTP/1.1" 200 None 2026-03-21 12:58:41,711 - lava_callback - ERROR - Error processing callback for node 69bbf2f9ec3f435d20d52e8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53116 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:59:17,011 - lava_callback - INFO - Received callback for node 69bbf31aec3f435d20d52ea3 INFO: 172.20.0.1:53132 - "POST /node/69bbf31aec3f435d20d52ea3 HTTP/1.0" 202 Accepted 2026-03-21 12:59:17,017 - lava_callback - INFO - Starting async job submit for node 69bbf31aec3f435d20d52ea3 2026-03-21 12:59:17,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:17,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf31aec3f435d20d52ea3 HTTP/1.1" 200 None 2026-03-21 12:59:17,033 - lava_callback - DEBUG - Processing node 69bbf31aec3f435d20d52ea3: baseline-arm-baylibre 2026-03-21 12:59:17,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf31aec3f435d20d52ea3 2026-03-21 12:59:17,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:17,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf31aec3f435d20d52ea3 HTTP/1.1" 200 None 2026-03-21 12:59:17,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:17,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 12:59:17,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:17,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf31aec3f435d20d52ea3 HTTP/1.1" 200 None 2026-03-21 12:59:17,097 - lava_callback - ERROR - Error processing callback for node 69bbf31aec3f435d20d52ea3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45826 - "GET / HTTP/1.0" 200 OK 2026-03-21 12:59:57,264 - lava_callback - INFO - Received callback for node 69bbf342ec3f435d20d52eb9 INFO: 172.20.0.1:45834 - "POST /node/69bbf342ec3f435d20d52eb9 HTTP/1.0" 202 Accepted 2026-03-21 12:59:57,270 - lava_callback - INFO - Starting async job submit for node 69bbf342ec3f435d20d52eb9 2026-03-21 12:59:57,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:57,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf342ec3f435d20d52eb9 HTTP/1.1" 200 None 2026-03-21 12:59:57,289 - lava_callback - DEBUG - Processing node 69bbf342ec3f435d20d52eb9: baseline-arm-baylibre 2026-03-21 12:59:57,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf342ec3f435d20d52eb9 2026-03-21 12:59:57,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:57,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf342ec3f435d20d52eb9 HTTP/1.1" 200 None 2026-03-21 12:59:57,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:57,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 12:59:57,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 12:59:57,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf342ec3f435d20d52eb9 HTTP/1.1" 200 None 2026-03-21 12:59:57,352 - lava_callback - ERROR - Error processing callback for node 69bbf342ec3f435d20d52eb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:00:37,041 - lava_callback - INFO - Received callback for node 69bbf367ec3f435d20d52ec5 INFO: 172.20.0.1:57406 - "POST /node/69bbf367ec3f435d20d52ec5 HTTP/1.0" 202 Accepted 2026-03-21 13:00:37,047 - lava_callback - INFO - Starting async job submit for node 69bbf367ec3f435d20d52ec5 2026-03-21 13:00:37,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:00:37,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf367ec3f435d20d52ec5 HTTP/1.1" 200 None 2026-03-21 13:00:37,064 - lava_callback - DEBUG - Processing node 69bbf367ec3f435d20d52ec5: baseline-arm-baylibre 2026-03-21 13:00:37,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf367ec3f435d20d52ec5 2026-03-21 13:00:37,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:00:37,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf367ec3f435d20d52ec5 HTTP/1.1" 200 None 2026-03-21 13:00:37,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:00:37,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:00:37,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:00:37,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf367ec3f435d20d52ec5 HTTP/1.1" 200 None 2026-03-21 13:00:37,122 - lava_callback - ERROR - Error processing callback for node 69bbf367ec3f435d20d52ec5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:03:01,740 - lava_callback - INFO - Received callback for node 69bbf403ec3f435d20d52f06 INFO: 172.20.0.1:56000 - "POST /node/69bbf403ec3f435d20d52f06 HTTP/1.0" 202 Accepted 2026-03-21 13:03:01,746 - lava_callback - INFO - Starting async job submit for node 69bbf403ec3f435d20d52f06 2026-03-21 13:03:01,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:03:01,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf403ec3f435d20d52f06 HTTP/1.1" 200 None 2026-03-21 13:03:01,768 - lava_callback - DEBUG - Processing node 69bbf403ec3f435d20d52f06: baseline-arm-clabbe 2026-03-21 13:03:01,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf403ec3f435d20d52f06 2026-03-21 13:03:01,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:03:01,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf403ec3f435d20d52f06 HTTP/1.1" 200 None 2026-03-21 13:03:01,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:03:01,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:03:01,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:03:01,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf403ec3f435d20d52f06 HTTP/1.1" 200 None 2026-03-21 13:03:01,849 - lava_callback - ERROR - Error processing callback for node 69bbf403ec3f435d20d52f06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:05:01,737 - lava_callback - INFO - Received callback for node 69bbf478ec3f435d20d52f31 INFO: 172.20.0.1:52146 - "POST /node/69bbf478ec3f435d20d52f31 HTTP/1.0" 202 Accepted 2026-03-21 13:05:01,743 - lava_callback - INFO - Starting async job submit for node 69bbf478ec3f435d20d52f31 2026-03-21 13:05:01,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:01,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf478ec3f435d20d52f31 HTTP/1.1" 200 None 2026-03-21 13:05:01,760 - lava_callback - DEBUG - Processing node 69bbf478ec3f435d20d52f31: baseline-arm-clabbe 2026-03-21 13:05:01,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf478ec3f435d20d52f31 2026-03-21 13:05:01,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:01,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf478ec3f435d20d52f31 HTTP/1.1" 200 None 2026-03-21 13:05:01,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:01,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:05:01,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:01,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf478ec3f435d20d52f31 HTTP/1.1" 200 None 2026-03-21 13:05:01,813 - lava_callback - ERROR - Error processing callback for node 69bbf478ec3f435d20d52f31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47036 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:05:41,829 - lava_callback - INFO - Received callback for node 69bbf49eec3f435d20d52f3f INFO: 172.20.0.1:34874 - "POST /node/69bbf49eec3f435d20d52f3f HTTP/1.0" 202 Accepted 2026-03-21 13:05:41,835 - lava_callback - INFO - Starting async job submit for node 69bbf49eec3f435d20d52f3f 2026-03-21 13:05:41,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:41,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf49eec3f435d20d52f3f HTTP/1.1" 200 None 2026-03-21 13:05:41,851 - lava_callback - DEBUG - Processing node 69bbf49eec3f435d20d52f3f: baseline-arm-clabbe 2026-03-21 13:05:41,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf49eec3f435d20d52f3f 2026-03-21 13:05:41,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:41,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf49eec3f435d20d52f3f HTTP/1.1" 200 None 2026-03-21 13:05:41,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:41,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:05:41,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:05:41,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf49eec3f435d20d52f3f HTTP/1.1" 200 None 2026-03-21 13:05:41,964 - lava_callback - ERROR - Error processing callback for node 69bbf49eec3f435d20d52f3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60424 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:06:17,354 - lava_callback - INFO - Received callback for node 69bbf4bfec3f435d20d52f53 INFO: 172.20.0.1:60430 - "POST /node/69bbf4bfec3f435d20d52f53 HTTP/1.0" 202 Accepted 2026-03-21 13:06:17,359 - lava_callback - INFO - Starting async job submit for node 69bbf4bfec3f435d20d52f53 2026-03-21 13:06:17,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:06:17,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf4bfec3f435d20d52f53 HTTP/1.1" 200 None 2026-03-21 13:06:17,379 - lava_callback - DEBUG - Processing node 69bbf4bfec3f435d20d52f53: baseline-arm-baylibre 2026-03-21 13:06:17,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf4bfec3f435d20d52f53 2026-03-21 13:06:17,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:06:17,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf4bfec3f435d20d52f53 HTTP/1.1" 200 None 2026-03-21 13:06:17,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:06:17,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:06:17,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:06:17,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf4bfec3f435d20d52f53 HTTP/1.1" 200 None 2026-03-21 13:06:17,437 - lava_callback - ERROR - Error processing callback for node 69bbf4bfec3f435d20d52f53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:07:01,919 - lava_callback - INFO - Received callback for node 69bbf4eaec3f435d20d52f62 INFO: 172.20.0.1:40270 - "POST /node/69bbf4eaec3f435d20d52f62 HTTP/1.0" 202 Accepted 2026-03-21 13:07:01,924 - lava_callback - INFO - Starting async job submit for node 69bbf4eaec3f435d20d52f62 2026-03-21 13:07:01,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:01,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf4eaec3f435d20d52f62 HTTP/1.1" 200 None 2026-03-21 13:07:01,945 - lava_callback - DEBUG - Processing node 69bbf4eaec3f435d20d52f62: baseline-arm-clabbe 2026-03-21 13:07:01,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf4eaec3f435d20d52f62 2026-03-21 13:07:01,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:01,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf4eaec3f435d20d52f62 HTTP/1.1" 200 None 2026-03-21 13:07:01,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:02,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:07:02,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:02,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf4eaec3f435d20d52f62 HTTP/1.1" 200 None 2026-03-21 13:07:02,050 - lava_callback - ERROR - Error processing callback for node 69bbf4eaec3f435d20d52f62: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40948 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:07:41,902 - lava_callback - INFO - Received callback for node 69bbf511ec3f435d20d52f70 INFO: 172.20.0.1:51604 - "POST /node/69bbf511ec3f435d20d52f70 HTTP/1.0" 202 Accepted 2026-03-21 13:07:41,907 - lava_callback - INFO - Starting async job submit for node 69bbf511ec3f435d20d52f70 2026-03-21 13:07:41,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:41,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf511ec3f435d20d52f70 HTTP/1.1" 200 None 2026-03-21 13:07:41,924 - lava_callback - DEBUG - Processing node 69bbf511ec3f435d20d52f70: baseline-arm-clabbe 2026-03-21 13:07:41,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf511ec3f435d20d52f70 2026-03-21 13:07:41,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:41,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf511ec3f435d20d52f70 HTTP/1.1" 200 None 2026-03-21 13:07:41,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:41,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:07:41,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:07:41,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf511ec3f435d20d52f70 HTTP/1.1" 200 None 2026-03-21 13:07:41,980 - lava_callback - ERROR - Error processing callback for node 69bbf511ec3f435d20d52f70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51622 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:09:02,001 - lava_callback - INFO - Received callback for node 69bbf55cec3f435d20d52f9e INFO: 172.20.0.1:49998 - "POST /node/69bbf55cec3f435d20d52f9e HTTP/1.0" 202 Accepted 2026-03-21 13:09:02,008 - lava_callback - INFO - Starting async job submit for node 69bbf55cec3f435d20d52f9e 2026-03-21 13:09:02,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:09:02,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf55cec3f435d20d52f9e HTTP/1.1" 200 None 2026-03-21 13:09:02,031 - lava_callback - DEBUG - Processing node 69bbf55cec3f435d20d52f9e: baseline-arm-clabbe 2026-03-21 13:09:02,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf55cec3f435d20d52f9e 2026-03-21 13:09:02,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:09:02,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf55cec3f435d20d52f9e HTTP/1.1" 200 None 2026-03-21 13:09:02,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:09:02,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:09:02,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:09:02,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf55cec3f435d20d52f9e HTTP/1.1" 200 None 2026-03-21 13:09:02,096 - lava_callback - ERROR - Error processing callback for node 69bbf55cec3f435d20d52f9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39158 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:10:02,120 - lava_callback - INFO - Received callback for node 69bbf5a6ec3f435d20d52fb0 INFO: 172.20.0.1:48590 - "POST /node/69bbf5a6ec3f435d20d52fb0 HTTP/1.0" 202 Accepted 2026-03-21 13:10:02,126 - lava_callback - INFO - Starting async job submit for node 69bbf5a6ec3f435d20d52fb0 2026-03-21 13:10:02,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:02,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf5a6ec3f435d20d52fb0 HTTP/1.1" 200 None 2026-03-21 13:10:02,161 - lava_callback - DEBUG - Processing node 69bbf5a6ec3f435d20d52fb0: baseline-arm-clabbe 2026-03-21 13:10:02,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf5a6ec3f435d20d52fb0 2026-03-21 13:10:02,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:02,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf5a6ec3f435d20d52fb0 HTTP/1.1" 200 None 2026-03-21 13:10:02,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:02,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:10:02,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:02,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf5a6ec3f435d20d52fb0 HTTP/1.1" 200 None 2026-03-21 13:10:02,261 - lava_callback - ERROR - Error processing callback for node 69bbf5a6ec3f435d20d52fb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:10:42,024 - lava_callback - INFO - Received callback for node 69bbf5cdec3f435d20d52fbf INFO: 172.20.0.1:49038 - "POST /node/69bbf5cdec3f435d20d52fbf HTTP/1.0" 202 Accepted 2026-03-21 13:10:42,030 - lava_callback - INFO - Starting async job submit for node 69bbf5cdec3f435d20d52fbf 2026-03-21 13:10:42,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:42,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf5cdec3f435d20d52fbf HTTP/1.1" 200 None 2026-03-21 13:10:42,046 - lava_callback - DEBUG - Processing node 69bbf5cdec3f435d20d52fbf: baseline-arm-clabbe 2026-03-21 13:10:42,046 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf5cdec3f435d20d52fbf 2026-03-21 13:10:42,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:42,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf5cdec3f435d20d52fbf HTTP/1.1" 200 None 2026-03-21 13:10:42,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:42,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:10:42,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:10:42,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf5cdec3f435d20d52fbf HTTP/1.1" 200 None 2026-03-21 13:10:42,118 - lava_callback - ERROR - Error processing callback for node 69bbf5cdec3f435d20d52fbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:12:02,228 - lava_callback - INFO - Received callback for node 69bbf619ec3f435d20d52fdf INFO: 172.20.0.1:36718 - "POST /node/69bbf619ec3f435d20d52fdf HTTP/1.0" 202 Accepted 2026-03-21 13:12:02,233 - lava_callback - INFO - Starting async job submit for node 69bbf619ec3f435d20d52fdf 2026-03-21 13:12:02,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:02,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf619ec3f435d20d52fdf HTTP/1.1" 200 None 2026-03-21 13:12:02,263 - lava_callback - DEBUG - Processing node 69bbf619ec3f435d20d52fdf: baseline-arm-clabbe 2026-03-21 13:12:02,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf619ec3f435d20d52fdf 2026-03-21 13:12:02,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:02,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf619ec3f435d20d52fdf HTTP/1.1" 200 None 2026-03-21 13:12:02,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:02,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:12:02,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:02,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf619ec3f435d20d52fdf HTTP/1.1" 200 None 2026-03-21 13:12:02,325 - lava_callback - ERROR - Error processing callback for node 69bbf619ec3f435d20d52fdf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:12:42,164 - lava_callback - INFO - Received callback for node 69bbf640ec3f435d20d52fee INFO: 172.20.0.1:52154 - "POST /node/69bbf640ec3f435d20d52fee HTTP/1.0" 202 Accepted 2026-03-21 13:12:42,169 - lava_callback - INFO - Starting async job submit for node 69bbf640ec3f435d20d52fee 2026-03-21 13:12:42,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:42,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf640ec3f435d20d52fee HTTP/1.1" 200 None 2026-03-21 13:12:42,185 - lava_callback - DEBUG - Processing node 69bbf640ec3f435d20d52fee: baseline-arm-clabbe 2026-03-21 13:12:42,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf640ec3f435d20d52fee 2026-03-21 13:12:42,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:42,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf640ec3f435d20d52fee HTTP/1.1" 200 None 2026-03-21 13:12:42,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:42,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:12:42,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:12:42,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf640ec3f435d20d52fee HTTP/1.1" 200 None 2026-03-21 13:12:42,257 - lava_callback - ERROR - Error processing callback for node 69bbf640ec3f435d20d52fee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:13:22,236 - lava_callback - INFO - Received callback for node 69bbf667ec3f435d20d53000 INFO: 172.20.0.1:59698 - "POST /node/69bbf667ec3f435d20d53000 HTTP/1.0" 202 Accepted 2026-03-21 13:13:22,242 - lava_callback - INFO - Starting async job submit for node 69bbf667ec3f435d20d53000 2026-03-21 13:13:22,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:13:22,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf667ec3f435d20d53000 HTTP/1.1" 200 None 2026-03-21 13:13:22,259 - lava_callback - DEBUG - Processing node 69bbf667ec3f435d20d53000: baseline-arm-clabbe 2026-03-21 13:13:22,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf667ec3f435d20d53000 2026-03-21 13:13:22,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:13:22,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf667ec3f435d20d53000 HTTP/1.1" 200 None 2026-03-21 13:13:22,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:13:22,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:13:22,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:13:22,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf667ec3f435d20d53000 HTTP/1.1" 200 None 2026-03-21 13:13:22,322 - lava_callback - ERROR - Error processing callback for node 69bbf667ec3f435d20d53000: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:14:02,148 - lava_callback - INFO - Received callback for node 69bbf68dec3f435d20d53011 INFO: 172.20.0.1:43430 - "POST /node/69bbf68dec3f435d20d53011 HTTP/1.0" 202 Accepted 2026-03-21 13:14:02,154 - lava_callback - INFO - Starting async job submit for node 69bbf68dec3f435d20d53011 2026-03-21 13:14:02,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:02,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf68dec3f435d20d53011 HTTP/1.1" 200 None 2026-03-21 13:14:02,170 - lava_callback - DEBUG - Processing node 69bbf68dec3f435d20d53011: baseline-arm-clabbe 2026-03-21 13:14:02,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf68dec3f435d20d53011 2026-03-21 13:14:02,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:02,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf68dec3f435d20d53011 HTTP/1.1" 200 None 2026-03-21 13:14:02,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:02,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:14:02,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:02,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf68dec3f435d20d53011 HTTP/1.1" 200 None 2026-03-21 13:14:02,238 - lava_callback - ERROR - Error processing callback for node 69bbf68dec3f435d20d53011: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51468 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:14:42,253 - lava_callback - INFO - Received callback for node 69bbf6b3ec3f435d20d5301e INFO: 172.20.0.1:47214 - "POST /node/69bbf6b3ec3f435d20d5301e HTTP/1.0" 202 Accepted 2026-03-21 13:14:42,258 - lava_callback - INFO - Starting async job submit for node 69bbf6b3ec3f435d20d5301e 2026-03-21 13:14:42,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:42,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf6b3ec3f435d20d5301e HTTP/1.1" 200 None 2026-03-21 13:14:42,291 - lava_callback - DEBUG - Processing node 69bbf6b3ec3f435d20d5301e: baseline-arm-clabbe 2026-03-21 13:14:42,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf6b3ec3f435d20d5301e 2026-03-21 13:14:42,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:42,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf6b3ec3f435d20d5301e HTTP/1.1" 200 None 2026-03-21 13:14:42,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:42,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:14:42,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:14:42,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf6b3ec3f435d20d5301e HTTP/1.1" 200 None 2026-03-21 13:14:42,361 - lava_callback - ERROR - Error processing callback for node 69bbf6b3ec3f435d20d5301e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41800 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:15:22,223 - lava_callback - INFO - Received callback for node 69bbf6d9ec3f435d20d5302f INFO: 172.20.0.1:39802 - "POST /node/69bbf6d9ec3f435d20d5302f HTTP/1.0" 202 Accepted 2026-03-21 13:15:22,229 - lava_callback - INFO - Starting async job submit for node 69bbf6d9ec3f435d20d5302f 2026-03-21 13:15:22,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:15:22,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf6d9ec3f435d20d5302f HTTP/1.1" 200 None 2026-03-21 13:15:22,248 - lava_callback - DEBUG - Processing node 69bbf6d9ec3f435d20d5302f: baseline-arm-clabbe 2026-03-21 13:15:22,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf6d9ec3f435d20d5302f 2026-03-21 13:15:22,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:15:22,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf6d9ec3f435d20d5302f HTTP/1.1" 200 None 2026-03-21 13:15:22,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:15:22,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:15:22,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:15:22,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf6d9ec3f435d20d5302f HTTP/1.1" 200 None 2026-03-21 13:15:22,335 - lava_callback - ERROR - Error processing callback for node 69bbf6d9ec3f435d20d5302f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:16:02,337 - lava_callback - INFO - Received callback for node 69bbf6ffec3f435d20d5303f INFO: 172.20.0.1:46210 - "POST /node/69bbf6ffec3f435d20d5303f HTTP/1.0" 202 Accepted 2026-03-21 13:16:02,342 - lava_callback - INFO - Starting async job submit for node 69bbf6ffec3f435d20d5303f 2026-03-21 13:16:02,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:16:02,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf6ffec3f435d20d5303f HTTP/1.1" 200 None 2026-03-21 13:16:02,359 - lava_callback - DEBUG - Processing node 69bbf6ffec3f435d20d5303f: baseline-arm-clabbe 2026-03-21 13:16:02,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf6ffec3f435d20d5303f 2026-03-21 13:16:02,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:16:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf6ffec3f435d20d5303f HTTP/1.1" 200 None 2026-03-21 13:16:02,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:16:02,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 13:16:02,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:16:02,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf6ffec3f435d20d5303f HTTP/1.1" 200 None 2026-03-21 13:16:02,450 - lava_callback - ERROR - Error processing callback for node 69bbf6ffec3f435d20d5303f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53152 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:23:18,397 - lava_callback - INFO - Received callback for node 69bbf8b8ec3f435d20d53081 INFO: 172.20.0.1:53156 - "POST /node/69bbf8b8ec3f435d20d53081 HTTP/1.0" 202 Accepted 2026-03-21 13:23:18,402 - lava_callback - INFO - Starting async job submit for node 69bbf8b8ec3f435d20d53081 2026-03-21 13:23:18,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:23:18,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf8b8ec3f435d20d53081 HTTP/1.1" 200 None 2026-03-21 13:23:18,418 - lava_callback - DEBUG - Processing node 69bbf8b8ec3f435d20d53081: baseline-arm64-baylibre 2026-03-21 13:23:18,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf8b8ec3f435d20d53081 2026-03-21 13:23:18,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:23:18,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf8b8ec3f435d20d53081 HTTP/1.1" 200 None 2026-03-21 13:23:18,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:23:18,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:23:18,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:23:18,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf8b8ec3f435d20d53081 HTTP/1.1" 200 None 2026-03-21 13:23:18,496 - lava_callback - ERROR - Error processing callback for node 69bbf8b8ec3f435d20d53081: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:24:38,387 - lava_callback - INFO - Received callback for node 69bbf907ec3f435d20d5308f INFO: 172.20.0.1:39562 - "POST /node/69bbf907ec3f435d20d5308f HTTP/1.0" 202 Accepted 2026-03-21 13:24:38,392 - lava_callback - INFO - Starting async job submit for node 69bbf907ec3f435d20d5308f 2026-03-21 13:24:38,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:24:38,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf907ec3f435d20d5308f HTTP/1.1" 200 None 2026-03-21 13:24:38,408 - lava_callback - DEBUG - Processing node 69bbf907ec3f435d20d5308f: baseline-arm64-baylibre 2026-03-21 13:24:38,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf907ec3f435d20d5308f 2026-03-21 13:24:38,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:24:38,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf907ec3f435d20d5308f HTTP/1.1" 200 None 2026-03-21 13:24:38,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:24:38,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:24:38,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:24:38,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf907ec3f435d20d5308f HTTP/1.1" 200 None 2026-03-21 13:24:38,474 - lava_callback - ERROR - Error processing callback for node 69bbf907ec3f435d20d5308f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40772 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:25:58,443 - lava_callback - INFO - Received callback for node 69bbf956ec3f435d20d53096 INFO: 172.20.0.1:40780 - "POST /node/69bbf956ec3f435d20d53096 HTTP/1.0" 202 Accepted 2026-03-21 13:25:58,449 - lava_callback - INFO - Starting async job submit for node 69bbf956ec3f435d20d53096 2026-03-21 13:25:58,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:25:58,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf956ec3f435d20d53096 HTTP/1.1" 200 None 2026-03-21 13:25:58,465 - lava_callback - DEBUG - Processing node 69bbf956ec3f435d20d53096: baseline-arm64-baylibre 2026-03-21 13:25:58,465 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf956ec3f435d20d53096 2026-03-21 13:25:58,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:25:58,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf956ec3f435d20d53096 HTTP/1.1" 200 None 2026-03-21 13:25:58,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:25:58,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:25:58,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:25:58,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf956ec3f435d20d53096 HTTP/1.1" 200 None 2026-03-21 13:25:58,528 - lava_callback - ERROR - Error processing callback for node 69bbf956ec3f435d20d53096: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46628 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:27:18,506 - lava_callback - INFO - Received callback for node 69bbf9a4ec3f435d20d530a2 INFO: 172.20.0.1:46640 - "POST /node/69bbf9a4ec3f435d20d530a2 HTTP/1.0" 202 Accepted 2026-03-21 13:27:18,512 - lava_callback - INFO - Starting async job submit for node 69bbf9a4ec3f435d20d530a2 2026-03-21 13:27:18,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:27:18,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf9a4ec3f435d20d530a2 HTTP/1.1" 200 None 2026-03-21 13:27:18,542 - lava_callback - DEBUG - Processing node 69bbf9a4ec3f435d20d530a2: baseline-arm64-baylibre 2026-03-21 13:27:18,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbf9a4ec3f435d20d530a2 2026-03-21 13:27:18,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:27:18,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbf9a4ec3f435d20d530a2 HTTP/1.1" 200 None 2026-03-21 13:27:18,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:27:18,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:27:18,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:27:18,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbf9a4ec3f435d20d530a2 HTTP/1.1" 200 None 2026-03-21 13:27:18,603 - lava_callback - ERROR - Error processing callback for node 69bbf9a4ec3f435d20d530a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35092 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:30:03,031 - lava_callback - INFO - Received callback for node 69bbfa4aec3f435d20d530bb INFO: 172.20.0.1:52850 - "POST /node/69bbfa4aec3f435d20d530bb HTTP/1.0" 202 Accepted 2026-03-21 13:30:03,039 - lava_callback - INFO - Starting async job submit for node 69bbfa4aec3f435d20d530bb 2026-03-21 13:30:03,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:30:03,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfa4aec3f435d20d530bb HTTP/1.1" 200 None 2026-03-21 13:30:03,055 - lava_callback - DEBUG - Processing node 69bbfa4aec3f435d20d530bb: baseline-arm64-clabbe 2026-03-21 13:30:03,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfa4aec3f435d20d530bb 2026-03-21 13:30:03,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:30:03,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfa4aec3f435d20d530bb HTTP/1.1" 200 None 2026-03-21 13:30:03,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:30:03,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:30:03,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:30:03,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfa4aec3f435d20d530bb HTTP/1.1" 200 None 2026-03-21 13:30:03,112 - lava_callback - ERROR - Error processing callback for node 69bbfa4aec3f435d20d530bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47508 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:31:23,019 - lava_callback - INFO - Received callback for node 69bbfaa5ec3f435d20d530c9 INFO: 172.20.0.1:47350 - "POST /node/69bbfaa5ec3f435d20d530c9 HTTP/1.0" 202 Accepted 2026-03-21 13:31:23,025 - lava_callback - INFO - Starting async job submit for node 69bbfaa5ec3f435d20d530c9 2026-03-21 13:31:23,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:31:23,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfaa5ec3f435d20d530c9 HTTP/1.1" 200 None 2026-03-21 13:31:23,040 - lava_callback - DEBUG - Processing node 69bbfaa5ec3f435d20d530c9: baseline-arm64-clabbe 2026-03-21 13:31:23,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfaa5ec3f435d20d530c9 2026-03-21 13:31:23,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:31:23,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfaa5ec3f435d20d530c9 HTTP/1.1" 200 None 2026-03-21 13:31:23,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:31:23,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:31:23,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:31:23,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfaa5ec3f435d20d530c9 HTTP/1.1" 200 None 2026-03-21 13:31:23,101 - lava_callback - ERROR - Error processing callback for node 69bbfaa5ec3f435d20d530c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45664 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:32:29,204 - lava_callback - INFO - Received callback for node 69be6352448e074b3ffe216e INFO: 172.20.0.1:45680 - "POST /node/69be6352448e074b3ffe216e HTTP/1.0" 202 Accepted 2026-03-21 13:32:29,210 - lava_callback - INFO - Starting async job submit for node 69be6352448e074b3ffe216e 2026-03-21 13:32:29,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:29,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6352448e074b3ffe216e HTTP/1.1" 200 None 2026-03-21 13:32:29,251 - lava_callback - DEBUG - Processing node 69be6352448e074b3ffe216e: baseline-nfs-arm64 2026-03-21 13:32:29,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 13:32:29,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 13:32:29,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6352448e074b3ffe216e/log.txt.gz 2026-03-21 13:32:30,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 13:32:30,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 13:32:30,431 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6352448e074b3ffe216e/lava-logs.yaml 2026-03-21 13:32:30,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 13:32:30,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 13:32:30,639 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be6352448e074b3ffe216e/lava_callback.json.gz 2026-03-21 13:32:30,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:30,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be6352448e074b3ffe216e HTTP/1.1" 200 None 2026-03-21 13:32:30,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:30,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 13:32:30,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:30,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be6352448e074b3ffe216e HTTP/1.1" 200 None 2026-03-21 13:32:30,787 - lava_callback - INFO - Completed processing callback for node 69be6352448e074b3ffe216e 2026-03-21 13:32:31,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:31,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48680 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:32:43,146 - lava_callback - INFO - Received callback for node 69bbfaf4ec3f435d20d530d1 INFO: 172.20.0.1:35086 - "POST /node/69bbfaf4ec3f435d20d530d1 HTTP/1.0" 202 Accepted 2026-03-21 13:32:43,152 - lava_callback - INFO - Starting async job submit for node 69bbfaf4ec3f435d20d530d1 2026-03-21 13:32:43,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:43,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfaf4ec3f435d20d530d1 HTTP/1.1" 200 None 2026-03-21 13:32:43,190 - lava_callback - DEBUG - Processing node 69bbfaf4ec3f435d20d530d1: baseline-arm64-clabbe 2026-03-21 13:32:43,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfaf4ec3f435d20d530d1 2026-03-21 13:32:43,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:43,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfaf4ec3f435d20d530d1 HTTP/1.1" 200 None 2026-03-21 13:32:43,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:43,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:32:43,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:32:43,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfaf4ec3f435d20d530d1 HTTP/1.1" 200 None 2026-03-21 13:32:43,257 - lava_callback - ERROR - Error processing callback for node 69bbfaf4ec3f435d20d530d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50698 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:34:03,127 - lava_callback - INFO - Received callback for node 69bbfb43ec3f435d20d530e2 INFO: 172.20.0.1:42346 - "POST /node/69bbfb43ec3f435d20d530e2 HTTP/1.0" 202 Accepted 2026-03-21 13:34:03,133 - lava_callback - INFO - Starting async job submit for node 69bbfb43ec3f435d20d530e2 2026-03-21 13:34:03,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:34:03,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfb43ec3f435d20d530e2 HTTP/1.1" 200 None 2026-03-21 13:34:03,154 - lava_callback - DEBUG - Processing node 69bbfb43ec3f435d20d530e2: baseline-arm64-clabbe 2026-03-21 13:34:03,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfb43ec3f435d20d530e2 2026-03-21 13:34:03,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:34:03,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfb43ec3f435d20d530e2 HTTP/1.1" 200 None 2026-03-21 13:34:03,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:34:03,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:34:03,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:34:03,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfb43ec3f435d20d530e2 HTTP/1.1" 200 None 2026-03-21 13:34:03,209 - lava_callback - ERROR - Error processing callback for node 69bbfb43ec3f435d20d530e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46440 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:35:18,995 - lava_callback - INFO - Received callback for node 69bbfb8aec3f435d20d530ec INFO: 172.20.0.1:46454 - "POST /node/69bbfb8aec3f435d20d530ec HTTP/1.0" 202 Accepted 2026-03-21 13:35:19,000 - lava_callback - INFO - Starting async job submit for node 69bbfb8aec3f435d20d530ec 2026-03-21 13:35:19,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:35:19,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfb8aec3f435d20d530ec HTTP/1.1" 200 None 2026-03-21 13:35:19,017 - lava_callback - DEBUG - Processing node 69bbfb8aec3f435d20d530ec: baseline-arm64-baylibre 2026-03-21 13:35:19,017 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfb8aec3f435d20d530ec 2026-03-21 13:35:19,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:35:19,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfb8aec3f435d20d530ec HTTP/1.1" 200 None 2026-03-21 13:35:19,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:35:19,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:35:19,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:35:19,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfb8aec3f435d20d530ec HTTP/1.1" 200 None 2026-03-21 13:35:19,081 - lava_callback - ERROR - Error processing callback for node 69bbfb8aec3f435d20d530ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39210 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:36:43,553 - lava_callback - INFO - Received callback for node 69bbfbe3ec3f435d20d530fb INFO: 172.20.0.1:33596 - "POST /node/69bbfbe3ec3f435d20d530fb HTTP/1.0" 202 Accepted 2026-03-21 13:36:43,559 - lava_callback - INFO - Starting async job submit for node 69bbfbe3ec3f435d20d530fb 2026-03-21 13:36:43,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:36:43,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfbe3ec3f435d20d530fb HTTP/1.1" 200 None 2026-03-21 13:36:43,575 - lava_callback - DEBUG - Processing node 69bbfbe3ec3f435d20d530fb: baseline-arm64-clabbe 2026-03-21 13:36:43,575 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfbe3ec3f435d20d530fb 2026-03-21 13:36:43,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:36:43,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfbe3ec3f435d20d530fb HTTP/1.1" 200 None 2026-03-21 13:36:43,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:36:43,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:36:43,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:36:43,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfbe3ec3f435d20d530fb HTTP/1.1" 200 None 2026-03-21 13:36:43,635 - lava_callback - ERROR - Error processing callback for node 69bbfbe3ec3f435d20d530fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49966 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:37:59,126 - lava_callback - INFO - Received callback for node 69bbfc2aec3f435d20d53109 INFO: 172.20.0.1:49976 - "POST /node/69bbfc2aec3f435d20d53109 HTTP/1.0" 202 Accepted 2026-03-21 13:37:59,132 - lava_callback - INFO - Starting async job submit for node 69bbfc2aec3f435d20d53109 2026-03-21 13:37:59,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:37:59,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfc2aec3f435d20d53109 HTTP/1.1" 200 None 2026-03-21 13:37:59,158 - lava_callback - DEBUG - Processing node 69bbfc2aec3f435d20d53109: baseline-arm64-baylibre 2026-03-21 13:37:59,159 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfc2aec3f435d20d53109 2026-03-21 13:37:59,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:37:59,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfc2aec3f435d20d53109 HTTP/1.1" 200 None 2026-03-21 13:37:59,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:37:59,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:37:59,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:37:59,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfc2aec3f435d20d53109 HTTP/1.1" 200 None 2026-03-21 13:37:59,209 - lava_callback - ERROR - Error processing callback for node 69bbfc2aec3f435d20d53109: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:39:23,395 - lava_callback - INFO - Received callback for node 69bbfc80ec3f435d20d53112 INFO: 172.20.0.1:52020 - "POST /node/69bbfc80ec3f435d20d53112 HTTP/1.0" 202 Accepted 2026-03-21 13:39:23,401 - lava_callback - INFO - Starting async job submit for node 69bbfc80ec3f435d20d53112 2026-03-21 13:39:23,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:39:23,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfc80ec3f435d20d53112 HTTP/1.1" 200 None 2026-03-21 13:39:23,421 - lava_callback - DEBUG - Processing node 69bbfc80ec3f435d20d53112: baseline-arm64-clabbe 2026-03-21 13:39:23,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfc80ec3f435d20d53112 2026-03-21 13:39:23,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:39:23,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfc80ec3f435d20d53112 HTTP/1.1" 200 None 2026-03-21 13:39:23,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:39:23,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 13:39:23,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:39:23,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfc80ec3f435d20d53112 HTTP/1.1" 200 None 2026-03-21 13:39:23,483 - lava_callback - ERROR - Error processing callback for node 69bbfc80ec3f435d20d53112: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33374 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:40:39,273 - lava_callback - INFO - Received callback for node 69bbfcc8ec3f435d20d53121 INFO: 172.20.0.1:33388 - "POST /node/69bbfcc8ec3f435d20d53121 HTTP/1.0" 202 Accepted 2026-03-21 13:40:39,279 - lava_callback - INFO - Starting async job submit for node 69bbfcc8ec3f435d20d53121 2026-03-21 13:40:39,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:40:39,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfcc8ec3f435d20d53121 HTTP/1.1" 200 None 2026-03-21 13:40:39,296 - lava_callback - DEBUG - Processing node 69bbfcc8ec3f435d20d53121: baseline-arm64-baylibre 2026-03-21 13:40:39,296 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfcc8ec3f435d20d53121 2026-03-21 13:40:39,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:40:39,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfcc8ec3f435d20d53121 HTTP/1.1" 200 None 2026-03-21 13:40:39,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:40:39,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:40:39,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:40:39,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfcc8ec3f435d20d53121 HTTP/1.1" 200 None 2026-03-21 13:40:39,348 - lava_callback - ERROR - Error processing callback for node 69bbfcc8ec3f435d20d53121: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58334 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:41:59,264 - lava_callback - INFO - Received callback for node 69bbfd15ec3f435d20d53126 INFO: 172.20.0.1:58340 - "POST /node/69bbfd15ec3f435d20d53126 HTTP/1.0" 202 Accepted 2026-03-21 13:41:59,270 - lava_callback - INFO - Starting async job submit for node 69bbfd15ec3f435d20d53126 2026-03-21 13:41:59,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:41:59,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfd15ec3f435d20d53126 HTTP/1.1" 200 None 2026-03-21 13:41:59,286 - lava_callback - DEBUG - Processing node 69bbfd15ec3f435d20d53126: baseline-arm64-baylibre 2026-03-21 13:41:59,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfd15ec3f435d20d53126 2026-03-21 13:41:59,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:41:59,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfd15ec3f435d20d53126 HTTP/1.1" 200 None 2026-03-21 13:41:59,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:41:59,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:41:59,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:41:59,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfd15ec3f435d20d53126 HTTP/1.1" 200 None 2026-03-21 13:41:59,342 - lava_callback - ERROR - Error processing callback for node 69bbfd15ec3f435d20d53126: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47046 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:43:19,410 - lava_callback - INFO - Received callback for node 69bbfd67ec3f435d20d53137 INFO: 172.20.0.1:47062 - "POST /node/69bbfd67ec3f435d20d53137 HTTP/1.0" 202 Accepted 2026-03-21 13:43:19,415 - lava_callback - INFO - Starting async job submit for node 69bbfd67ec3f435d20d53137 2026-03-21 13:43:19,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:43:19,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfd67ec3f435d20d53137 HTTP/1.1" 200 None 2026-03-21 13:43:19,430 - lava_callback - DEBUG - Processing node 69bbfd67ec3f435d20d53137: baseline-arm64-baylibre 2026-03-21 13:43:19,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfd67ec3f435d20d53137 2026-03-21 13:43:19,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:43:19,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfd67ec3f435d20d53137 HTTP/1.1" 200 None 2026-03-21 13:43:19,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:43:19,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:43:19,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:43:19,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfd67ec3f435d20d53137 HTTP/1.1" 200 None 2026-03-21 13:43:19,502 - lava_callback - ERROR - Error processing callback for node 69bbfd67ec3f435d20d53137: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33350 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:44:39,433 - lava_callback - INFO - Received callback for node 69bbfdb2ec3f435d20d53146 INFO: 172.20.0.1:33362 - "POST /node/69bbfdb2ec3f435d20d53146 HTTP/1.0" 202 Accepted 2026-03-21 13:44:39,439 - lava_callback - INFO - Starting async job submit for node 69bbfdb2ec3f435d20d53146 2026-03-21 13:44:39,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:44:39,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfdb2ec3f435d20d53146 HTTP/1.1" 200 None 2026-03-21 13:44:39,456 - lava_callback - DEBUG - Processing node 69bbfdb2ec3f435d20d53146: baseline-arm64-baylibre 2026-03-21 13:44:39,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfdb2ec3f435d20d53146 2026-03-21 13:44:39,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:44:39,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfdb2ec3f435d20d53146 HTTP/1.1" 200 None 2026-03-21 13:44:39,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:44:39,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:44:39,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:44:39,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfdb2ec3f435d20d53146 HTTP/1.1" 200 None 2026-03-21 13:44:39,518 - lava_callback - ERROR - Error processing callback for node 69bbfdb2ec3f435d20d53146: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52200 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:45:39,450 - lava_callback - INFO - Received callback for node 69bbfe00ec3f435d20d5314e INFO: 172.20.0.1:52208 - "POST /node/69bbfe00ec3f435d20d5314e HTTP/1.0" 202 Accepted 2026-03-21 13:45:39,455 - lava_callback - INFO - Starting async job submit for node 69bbfe00ec3f435d20d5314e 2026-03-21 13:45:39,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:45:39,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfe00ec3f435d20d5314e HTTP/1.1" 200 None 2026-03-21 13:45:39,472 - lava_callback - DEBUG - Processing node 69bbfe00ec3f435d20d5314e: baseline-arm64-baylibre 2026-03-21 13:45:39,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfe00ec3f435d20d5314e 2026-03-21 13:45:39,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:45:39,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfe00ec3f435d20d5314e HTTP/1.1" 200 None 2026-03-21 13:45:39,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:45:39,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:45:39,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:45:39,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfe00ec3f435d20d5314e HTTP/1.1" 200 None 2026-03-21 13:45:39,563 - lava_callback - ERROR - Error processing callback for node 69bbfe00ec3f435d20d5314e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43494 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:46:59,636 - lava_callback - INFO - Received callback for node 69bbfe4cec3f435d20d5315d INFO: 172.20.0.1:43504 - "POST /node/69bbfe4cec3f435d20d5315d HTTP/1.0" 202 Accepted 2026-03-21 13:46:59,642 - lava_callback - INFO - Starting async job submit for node 69bbfe4cec3f435d20d5315d 2026-03-21 13:46:59,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:46:59,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfe4cec3f435d20d5315d HTTP/1.1" 200 None 2026-03-21 13:46:59,657 - lava_callback - DEBUG - Processing node 69bbfe4cec3f435d20d5315d: baseline-arm64-baylibre 2026-03-21 13:46:59,657 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfe4cec3f435d20d5315d 2026-03-21 13:46:59,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:46:59,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfe4cec3f435d20d5315d HTTP/1.1" 200 None 2026-03-21 13:46:59,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:46:59,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:46:59,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:46:59,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfe4cec3f435d20d5315d HTTP/1.1" 200 None 2026-03-21 13:46:59,727 - lava_callback - ERROR - Error processing callback for node 69bbfe4cec3f435d20d5315d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35254 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:48:19,738 - lava_callback - INFO - Received callback for node 69bbfea1ec3f435d20d5316e INFO: 172.20.0.1:39772 - "POST /node/69bbfea1ec3f435d20d5316e HTTP/1.0" 202 Accepted 2026-03-21 13:48:19,744 - lava_callback - INFO - Starting async job submit for node 69bbfea1ec3f435d20d5316e 2026-03-21 13:48:19,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:48:19,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfea1ec3f435d20d5316e HTTP/1.1" 200 None 2026-03-21 13:48:19,760 - lava_callback - DEBUG - Processing node 69bbfea1ec3f435d20d5316e: baseline-arm64-baylibre 2026-03-21 13:48:19,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfea1ec3f435d20d5316e 2026-03-21 13:48:19,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:48:19,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfea1ec3f435d20d5316e HTTP/1.1" 200 None 2026-03-21 13:48:19,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:48:19,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:48:19,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:48:19,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfea1ec3f435d20d5316e HTTP/1.1" 200 None 2026-03-21 13:48:19,835 - lava_callback - ERROR - Error processing callback for node 69bbfea1ec3f435d20d5316e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60038 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:49:39,667 - lava_callback - INFO - Received callback for node 69bbfef1ec3f435d20d53174 INFO: 172.20.0.1:60048 - "POST /node/69bbfef1ec3f435d20d53174 HTTP/1.0" 202 Accepted 2026-03-21 13:49:39,673 - lava_callback - INFO - Starting async job submit for node 69bbfef1ec3f435d20d53174 2026-03-21 13:49:39,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:49:39,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfef1ec3f435d20d53174 HTTP/1.1" 200 None 2026-03-21 13:49:39,690 - lava_callback - DEBUG - Processing node 69bbfef1ec3f435d20d53174: baseline-arm64-baylibre 2026-03-21 13:49:39,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbfef1ec3f435d20d53174 2026-03-21 13:49:39,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:49:39,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbfef1ec3f435d20d53174 HTTP/1.1" 200 None 2026-03-21 13:49:39,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:49:39,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:49:39,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:49:39,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbfef1ec3f435d20d53174 HTTP/1.1" 200 None 2026-03-21 13:49:39,773 - lava_callback - ERROR - Error processing callback for node 69bbfef1ec3f435d20d53174: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:51:19,749 - lava_callback - INFO - Received callback for node 69bbff43ec3f435d20d53184 INFO: 172.20.0.1:58368 - "POST /node/69bbff43ec3f435d20d53184 HTTP/1.0" 202 Accepted 2026-03-21 13:51:19,754 - lava_callback - INFO - Starting async job submit for node 69bbff43ec3f435d20d53184 2026-03-21 13:51:19,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:51:19,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbff43ec3f435d20d53184 HTTP/1.1" 200 None 2026-03-21 13:51:19,783 - lava_callback - DEBUG - Processing node 69bbff43ec3f435d20d53184: baseline-arm64-baylibre 2026-03-21 13:51:19,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbff43ec3f435d20d53184 2026-03-21 13:51:19,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:51:19,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbff43ec3f435d20d53184 HTTP/1.1" 200 None 2026-03-21 13:51:19,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:51:19,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:51:19,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:51:19,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbff43ec3f435d20d53184 HTTP/1.1" 200 None 2026-03-21 13:51:19,859 - lava_callback - ERROR - Error processing callback for node 69bbff43ec3f435d20d53184: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:51:59,914 - lava_callback - INFO - Received callback for node 69be9fe5448e074b3ffe29d3 INFO: 172.20.0.1:39004 - "POST /node/69be9fe5448e074b3ffe29d3 HTTP/1.0" 202 Accepted 2026-03-21 13:51:59,920 - lava_callback - INFO - Starting async job submit for node 69be9fe5448e074b3ffe29d3 2026-03-21 13:51:59,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:51:59,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be9fe5448e074b3ffe29d3 HTTP/1.1" 200 None 2026-03-21 13:51:59,962 - lava_callback - DEBUG - Processing node 69be9fe5448e074b3ffe29d3: baseline-nfs-arm64 2026-03-21 13:52:00,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 13:52:00,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 13:52:00,651 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be9fe5448e074b3ffe29d3/log.txt.gz 2026-03-21 13:52:00,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 13:52:01,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 13:52:01,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be9fe5448e074b3ffe29d3/lava-logs.yaml 2026-03-21 13:52:01,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 13:52:01,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 13:52:01,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be9fe5448e074b3ffe29d3/lava_callback.json.gz 2026-03-21 13:52:01,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:01,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be9fe5448e074b3ffe29d3 HTTP/1.1" 200 None 2026-03-21 13:52:01,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:01,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 13:52:01,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:01,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be9fe5448e074b3ffe29d3 HTTP/1.1" 200 None 2026-03-21 13:52:01,690 - lava_callback - INFO - Completed processing callback for node 69be9fe5448e074b3ffe29d3 2026-03-21 13:52:01,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:01,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36148 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:52:39,823 - lava_callback - INFO - Received callback for node 69bbff95ec3f435d20d53194 INFO: 172.20.0.1:39364 - "POST /node/69bbff95ec3f435d20d53194 HTTP/1.0" 202 Accepted 2026-03-21 13:52:39,828 - lava_callback - INFO - Starting async job submit for node 69bbff95ec3f435d20d53194 2026-03-21 13:52:39,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:39,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbff95ec3f435d20d53194 HTTP/1.1" 200 None 2026-03-21 13:52:39,844 - lava_callback - DEBUG - Processing node 69bbff95ec3f435d20d53194: baseline-arm64-baylibre 2026-03-21 13:52:39,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbff95ec3f435d20d53194 2026-03-21 13:52:39,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:39,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbff95ec3f435d20d53194 HTTP/1.1" 200 None 2026-03-21 13:52:39,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:39,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:52:39,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:52:39,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbff95ec3f435d20d53194 HTTP/1.1" 200 None 2026-03-21 13:52:39,906 - lava_callback - ERROR - Error processing callback for node 69bbff95ec3f435d20d53194: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59686 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:54:07,322 - lava_callback - INFO - Received callback for node 69bbffebec3f435d20d531a0 INFO: 172.20.0.1:59698 - "POST /node/69bbffebec3f435d20d531a0 HTTP/1.0" 202 Accepted 2026-03-21 13:54:07,328 - lava_callback - INFO - Starting async job submit for node 69bbffebec3f435d20d531a0 2026-03-21 13:54:07,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:54:07,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbffebec3f435d20d531a0 HTTP/1.1" 200 None 2026-03-21 13:54:07,366 - lava_callback - DEBUG - Processing node 69bbffebec3f435d20d531a0: baseline-arm64-clabbe 2026-03-21 13:54:07,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbffebec3f435d20d531a0 2026-03-21 13:54:07,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:54:07,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbffebec3f435d20d531a0 HTTP/1.1" 200 None 2026-03-21 13:54:07,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:54:07,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:54:07,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:54:07,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbffebec3f435d20d531a0 HTTP/1.1" 200 None 2026-03-21 13:54:07,458 - lava_callback - ERROR - Error processing callback for node 69bbffebec3f435d20d531a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53174 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:55:24,137 - lava_callback - INFO - Received callback for node 69bc003cec3f435d20d531a7 INFO: 172.20.0.1:50666 - "POST /node/69bc003cec3f435d20d531a7 HTTP/1.0" 202 Accepted 2026-03-21 13:55:24,142 - lava_callback - INFO - Starting async job submit for node 69bc003cec3f435d20d531a7 2026-03-21 13:55:24,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:55:24,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc003cec3f435d20d531a7 HTTP/1.1" 200 None 2026-03-21 13:55:24,158 - lava_callback - DEBUG - Processing node 69bc003cec3f435d20d531a7: baseline-arm64-clabbe 2026-03-21 13:55:24,159 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc003cec3f435d20d531a7 2026-03-21 13:55:24,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:55:24,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc003cec3f435d20d531a7 HTTP/1.1" 200 None 2026-03-21 13:55:24,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:55:24,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:55:24,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:55:24,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc003cec3f435d20d531a7 HTTP/1.1" 200 None 2026-03-21 13:55:24,209 - lava_callback - ERROR - Error processing callback for node 69bc003cec3f435d20d531a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51356 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:56:44,149 - lava_callback - INFO - Received callback for node 69bc008dec3f435d20d531b7 INFO: 172.20.0.1:38026 - "POST /node/69bc008dec3f435d20d531b7 HTTP/1.0" 202 Accepted 2026-03-21 13:56:44,154 - lava_callback - INFO - Starting async job submit for node 69bc008dec3f435d20d531b7 2026-03-21 13:56:44,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:56:44,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc008dec3f435d20d531b7 HTTP/1.1" 200 None 2026-03-21 13:56:44,183 - lava_callback - DEBUG - Processing node 69bc008dec3f435d20d531b7: baseline-arm64-clabbe 2026-03-21 13:56:44,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc008dec3f435d20d531b7 2026-03-21 13:56:44,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:56:44,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc008dec3f435d20d531b7 HTTP/1.1" 200 None 2026-03-21 13:56:44,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:56:44,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:56:44,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:56:44,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc008dec3f435d20d531b7 HTTP/1.1" 200 None 2026-03-21 13:56:44,242 - lava_callback - ERROR - Error processing callback for node 69bc008dec3f435d20d531b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36368 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:58:04,195 - lava_callback - INFO - Received callback for node 69bc00ddec3f435d20d531c6 INFO: 172.20.0.1:57276 - "POST /node/69bc00ddec3f435d20d531c6 HTTP/1.0" 202 Accepted 2026-03-21 13:58:04,200 - lava_callback - INFO - Starting async job submit for node 69bc00ddec3f435d20d531c6 2026-03-21 13:58:04,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:58:04,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc00ddec3f435d20d531c6 HTTP/1.1" 200 None 2026-03-21 13:58:04,218 - lava_callback - DEBUG - Processing node 69bc00ddec3f435d20d531c6: baseline-arm64-clabbe 2026-03-21 13:58:04,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc00ddec3f435d20d531c6 2026-03-21 13:58:04,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:58:04,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc00ddec3f435d20d531c6 HTTP/1.1" 200 None 2026-03-21 13:58:04,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:58:04,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:58:04,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:58:04,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc00ddec3f435d20d531c6 HTTP/1.1" 200 None 2026-03-21 13:58:04,282 - lava_callback - ERROR - Error processing callback for node 69bc00ddec3f435d20d531c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47372 - "GET / HTTP/1.0" 200 OK 2026-03-21 13:59:24,198 - lava_callback - INFO - Received callback for node 69bc012eec3f435d20d531cf INFO: 172.20.0.1:38540 - "POST /node/69bc012eec3f435d20d531cf HTTP/1.0" 202 Accepted 2026-03-21 13:59:24,204 - lava_callback - INFO - Starting async job submit for node 69bc012eec3f435d20d531cf 2026-03-21 13:59:24,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:59:24,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc012eec3f435d20d531cf HTTP/1.1" 200 None 2026-03-21 13:59:24,219 - lava_callback - DEBUG - Processing node 69bc012eec3f435d20d531cf: baseline-arm64-clabbe 2026-03-21 13:59:24,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc012eec3f435d20d531cf 2026-03-21 13:59:24,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:59:24,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc012eec3f435d20d531cf HTTP/1.1" 200 None 2026-03-21 13:59:24,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:59:24,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 13:59:24,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 13:59:24,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc012eec3f435d20d531cf HTTP/1.1" 200 None 2026-03-21 13:59:24,279 - lava_callback - ERROR - Error processing callback for node 69bc012eec3f435d20d531cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39470 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:00:44,322 - lava_callback - INFO - Received callback for node 69bc0189ec3f435d20d531dd INFO: 172.20.0.1:37120 - "POST /node/69bc0189ec3f435d20d531dd HTTP/1.0" 202 Accepted 2026-03-21 14:00:44,328 - lava_callback - INFO - Starting async job submit for node 69bc0189ec3f435d20d531dd 2026-03-21 14:00:44,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:00:44,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0189ec3f435d20d531dd HTTP/1.1" 200 None 2026-03-21 14:00:44,344 - lava_callback - DEBUG - Processing node 69bc0189ec3f435d20d531dd: baseline-arm64-clabbe 2026-03-21 14:00:44,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0189ec3f435d20d531dd 2026-03-21 14:00:44,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:00:44,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0189ec3f435d20d531dd HTTP/1.1" 200 None 2026-03-21 14:00:44,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:00:44,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:00:44,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:00:44,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0189ec3f435d20d531dd HTTP/1.1" 200 None 2026-03-21 14:00:44,405 - lava_callback - ERROR - Error processing callback for node 69bc0189ec3f435d20d531dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42186 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:02:24,310 - lava_callback - INFO - Received callback for node 69bc01dcec3f435d20d531ea INFO: 172.20.0.1:38340 - "POST /node/69bc01dcec3f435d20d531ea HTTP/1.0" 202 Accepted 2026-03-21 14:02:24,315 - lava_callback - INFO - Starting async job submit for node 69bc01dcec3f435d20d531ea 2026-03-21 14:02:24,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:02:24,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc01dcec3f435d20d531ea HTTP/1.1" 200 None 2026-03-21 14:02:24,331 - lava_callback - DEBUG - Processing node 69bc01dcec3f435d20d531ea: baseline-arm64-clabbe 2026-03-21 14:02:24,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc01dcec3f435d20d531ea 2026-03-21 14:02:24,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:02:24,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc01dcec3f435d20d531ea HTTP/1.1" 200 None 2026-03-21 14:02:24,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:02:24,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:02:24,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:02:24,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc01dcec3f435d20d531ea HTTP/1.1" 200 None 2026-03-21 14:02:24,394 - lava_callback - ERROR - Error processing callback for node 69bc01dcec3f435d20d531ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34014 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:03:44,446 - lava_callback - INFO - Received callback for node 69bc022cec3f435d20d531fa INFO: 172.20.0.1:55306 - "POST /node/69bc022cec3f435d20d531fa HTTP/1.0" 202 Accepted 2026-03-21 14:03:44,454 - lava_callback - INFO - Starting async job submit for node 69bc022cec3f435d20d531fa 2026-03-21 14:03:44,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:03:44,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc022cec3f435d20d531fa HTTP/1.1" 200 None 2026-03-21 14:03:44,472 - lava_callback - DEBUG - Processing node 69bc022cec3f435d20d531fa: baseline-arm64-clabbe 2026-03-21 14:03:44,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc022cec3f435d20d531fa 2026-03-21 14:03:44,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:03:44,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc022cec3f435d20d531fa HTTP/1.1" 200 None 2026-03-21 14:03:44,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:03:44,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:03:44,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:03:44,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc022cec3f435d20d531fa HTTP/1.1" 200 None 2026-03-21 14:03:44,527 - lava_callback - ERROR - Error processing callback for node 69bc022cec3f435d20d531fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:05:04,474 - lava_callback - INFO - Received callback for node 69bc027dec3f435d20d53208 INFO: 172.20.0.1:45312 - "POST /node/69bc027dec3f435d20d53208 HTTP/1.0" 202 Accepted 2026-03-21 14:05:04,480 - lava_callback - INFO - Starting async job submit for node 69bc027dec3f435d20d53208 2026-03-21 14:05:04,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:05:04,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc027dec3f435d20d53208 HTTP/1.1" 200 None 2026-03-21 14:05:04,496 - lava_callback - DEBUG - Processing node 69bc027dec3f435d20d53208: baseline-arm64-clabbe 2026-03-21 14:05:04,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc027dec3f435d20d53208 2026-03-21 14:05:04,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:05:04,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc027dec3f435d20d53208 HTTP/1.1" 200 None 2026-03-21 14:05:04,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:05:04,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:05:04,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:05:04,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc027dec3f435d20d53208 HTTP/1.1" 200 None 2026-03-21 14:05:04,565 - lava_callback - ERROR - Error processing callback for node 69bc027dec3f435d20d53208: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37644 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:07:44,561 - lava_callback - INFO - Received callback for node 69bc031bec3f435d20d53218 INFO: 172.20.0.1:56162 - "POST /node/69bc031bec3f435d20d53218 HTTP/1.0" 202 Accepted 2026-03-21 14:07:44,567 - lava_callback - INFO - Starting async job submit for node 69bc031bec3f435d20d53218 2026-03-21 14:07:44,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:07:44,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc031bec3f435d20d53218 HTTP/1.1" 200 None 2026-03-21 14:07:44,593 - lava_callback - DEBUG - Processing node 69bc031bec3f435d20d53218: baseline-arm64-clabbe 2026-03-21 14:07:44,593 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc031bec3f435d20d53218 2026-03-21 14:07:44,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:07:44,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc031bec3f435d20d53218 HTTP/1.1" 200 None 2026-03-21 14:07:44,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:07:44,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:07:44,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:07:44,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc031bec3f435d20d53218 HTTP/1.1" 200 None 2026-03-21 14:07:44,652 - lava_callback - ERROR - Error processing callback for node 69bc031bec3f435d20d53218: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:09:04,643 - lava_callback - INFO - Received callback for node 69bc036eec3f435d20d53222 INFO: 172.20.0.1:50556 - "POST /node/69bc036eec3f435d20d53222 HTTP/1.0" 202 Accepted 2026-03-21 14:09:04,648 - lava_callback - INFO - Starting async job submit for node 69bc036eec3f435d20d53222 2026-03-21 14:09:04,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:09:04,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc036eec3f435d20d53222 HTTP/1.1" 200 None 2026-03-21 14:09:04,665 - lava_callback - DEBUG - Processing node 69bc036eec3f435d20d53222: baseline-arm64-clabbe 2026-03-21 14:09:04,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc036eec3f435d20d53222 2026-03-21 14:09:04,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:09:04,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc036eec3f435d20d53222 HTTP/1.1" 200 None 2026-03-21 14:09:04,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:09:04,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:09:04,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:09:04,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc036eec3f435d20d53222 HTTP/1.1" 200 None 2026-03-21 14:09:04,722 - lava_callback - ERROR - Error processing callback for node 69bc036eec3f435d20d53222: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41774 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:10:00,867 - lava_callback - INFO - Received callback for node 69bc03b7ec3f435d20d53228 INFO: 172.20.0.1:46108 - "POST /node/69bc03b7ec3f435d20d53228 HTTP/1.0" 202 Accepted 2026-03-21 14:10:00,872 - lava_callback - INFO - Starting async job submit for node 69bc03b7ec3f435d20d53228 2026-03-21 14:10:00,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:10:00,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc03b7ec3f435d20d53228 HTTP/1.1" 200 None 2026-03-21 14:10:00,890 - lava_callback - DEBUG - Processing node 69bc03b7ec3f435d20d53228: baseline-arm64-baylibre 2026-03-21 14:10:00,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc03b7ec3f435d20d53228 2026-03-21 14:10:00,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:10:00,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc03b7ec3f435d20d53228 HTTP/1.1" 200 None 2026-03-21 14:10:00,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:10:00,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:10:00,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:10:00,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc03b7ec3f435d20d53228 HTTP/1.1" 200 None 2026-03-21 14:10:00,948 - lava_callback - ERROR - Error processing callback for node 69bc03b7ec3f435d20d53228: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43156 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:11:44,721 - lava_callback - INFO - Received callback for node 69bc040dec3f435d20d53234 INFO: 172.20.0.1:50800 - "POST /node/69bc040dec3f435d20d53234 HTTP/1.0" 202 Accepted 2026-03-21 14:11:44,727 - lava_callback - INFO - Starting async job submit for node 69bc040dec3f435d20d53234 2026-03-21 14:11:44,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:11:44,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc040dec3f435d20d53234 HTTP/1.1" 200 None 2026-03-21 14:11:44,744 - lava_callback - DEBUG - Processing node 69bc040dec3f435d20d53234: baseline-arm64-clabbe 2026-03-21 14:11:44,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc040dec3f435d20d53234 2026-03-21 14:11:44,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:11:44,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc040dec3f435d20d53234 HTTP/1.1" 200 None 2026-03-21 14:11:44,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:11:44,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:11:44,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:11:44,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc040dec3f435d20d53234 HTTP/1.1" 200 None 2026-03-21 14:11:44,804 - lava_callback - ERROR - Error processing callback for node 69bc040dec3f435d20d53234: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33828 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:12:40,982 - lava_callback - INFO - Received callback for node 69bc0457ec3f435d20d5323c INFO: 172.20.0.1:56220 - "POST /node/69bc0457ec3f435d20d5323c HTTP/1.0" 202 Accepted 2026-03-21 14:12:40,987 - lava_callback - INFO - Starting async job submit for node 69bc0457ec3f435d20d5323c 2026-03-21 14:12:40,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:12:41,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0457ec3f435d20d5323c HTTP/1.1" 200 None 2026-03-21 14:12:41,004 - lava_callback - DEBUG - Processing node 69bc0457ec3f435d20d5323c: baseline-arm64-baylibre 2026-03-21 14:12:41,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0457ec3f435d20d5323c 2026-03-21 14:12:41,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:12:41,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0457ec3f435d20d5323c HTTP/1.1" 200 None 2026-03-21 14:12:41,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:12:41,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:12:41,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:12:41,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0457ec3f435d20d5323c HTTP/1.1" 200 None 2026-03-21 14:12:41,110 - lava_callback - ERROR - Error processing callback for node 69bc0457ec3f435d20d5323c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33378 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:14:24,829 - lava_callback - INFO - Received callback for node 69bc04adec3f435d20d53248 INFO: 172.20.0.1:58470 - "POST /node/69bc04adec3f435d20d53248 HTTP/1.0" 202 Accepted 2026-03-21 14:14:24,834 - lava_callback - INFO - Starting async job submit for node 69bc04adec3f435d20d53248 2026-03-21 14:14:24,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:14:24,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc04adec3f435d20d53248 HTTP/1.1" 200 None 2026-03-21 14:14:24,851 - lava_callback - DEBUG - Processing node 69bc04adec3f435d20d53248: baseline-arm64-clabbe 2026-03-21 14:14:24,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc04adec3f435d20d53248 2026-03-21 14:14:24,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:14:24,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc04adec3f435d20d53248 HTTP/1.1" 200 None 2026-03-21 14:14:24,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:14:24,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:14:24,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:14:24,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc04adec3f435d20d53248 HTTP/1.1" 200 None 2026-03-21 14:14:24,911 - lava_callback - ERROR - Error processing callback for node 69bc04adec3f435d20d53248: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44868 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:15:44,926 - lava_callback - INFO - Received callback for node 69bc04fdec3f435d20d53255 INFO: 172.20.0.1:35844 - "POST /node/69bc04fdec3f435d20d53255 HTTP/1.0" 202 Accepted 2026-03-21 14:15:44,932 - lava_callback - INFO - Starting async job submit for node 69bc04fdec3f435d20d53255 2026-03-21 14:15:44,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:15:44,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc04fdec3f435d20d53255 HTTP/1.1" 200 None 2026-03-21 14:15:44,948 - lava_callback - DEBUG - Processing node 69bc04fdec3f435d20d53255: baseline-arm64-clabbe 2026-03-21 14:15:44,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc04fdec3f435d20d53255 2026-03-21 14:15:44,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:15:44,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc04fdec3f435d20d53255 HTTP/1.1" 200 None 2026-03-21 14:15:44,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:15:44,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:15:44,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:15:44,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc04fdec3f435d20d53255 HTTP/1.1" 200 None 2026-03-21 14:15:45,003 - lava_callback - ERROR - Error processing callback for node 69bc04fdec3f435d20d53255: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57240 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:17:04,966 - lava_callback - INFO - Received callback for node 69bc054dec3f435d20d53263 INFO: 172.20.0.1:43360 - "POST /node/69bc054dec3f435d20d53263 HTTP/1.0" 202 Accepted 2026-03-21 14:17:04,972 - lava_callback - INFO - Starting async job submit for node 69bc054dec3f435d20d53263 2026-03-21 14:17:04,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:17:04,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc054dec3f435d20d53263 HTTP/1.1" 200 None 2026-03-21 14:17:04,988 - lava_callback - DEBUG - Processing node 69bc054dec3f435d20d53263: baseline-arm64-clabbe 2026-03-21 14:17:04,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc054dec3f435d20d53263 2026-03-21 14:17:04,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:17:05,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc054dec3f435d20d53263 HTTP/1.1" 200 None 2026-03-21 14:17:05,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:17:05,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:17:05,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:17:05,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc054dec3f435d20d53263 HTTP/1.1" 200 None 2026-03-21 14:17:05,054 - lava_callback - ERROR - Error processing callback for node 69bc054dec3f435d20d53263: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54088 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:18:25,042 - lava_callback - INFO - Received callback for node 69bc059dec3f435d20d53269 INFO: 172.20.0.1:52034 - "POST /node/69bc059dec3f435d20d53269 HTTP/1.0" 202 Accepted 2026-03-21 14:18:25,048 - lava_callback - INFO - Starting async job submit for node 69bc059dec3f435d20d53269 2026-03-21 14:18:25,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:18:25,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc059dec3f435d20d53269 HTTP/1.1" 200 None 2026-03-21 14:18:25,064 - lava_callback - DEBUG - Processing node 69bc059dec3f435d20d53269: baseline-arm64-clabbe 2026-03-21 14:18:25,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc059dec3f435d20d53269 2026-03-21 14:18:25,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:18:25,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc059dec3f435d20d53269 HTTP/1.1" 200 None 2026-03-21 14:18:25,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:18:25,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:18:25,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:18:25,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc059dec3f435d20d53269 HTTP/1.1" 200 None 2026-03-21 14:18:25,130 - lava_callback - ERROR - Error processing callback for node 69bc059dec3f435d20d53269: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41664 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:19:45,032 - lava_callback - INFO - Received callback for node 69bc05ecec3f435d20d5327f INFO: 172.20.0.1:51258 - "POST /node/69bc05ecec3f435d20d5327f HTTP/1.0" 202 Accepted 2026-03-21 14:19:45,038 - lava_callback - INFO - Starting async job submit for node 69bc05ecec3f435d20d5327f 2026-03-21 14:19:45,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:45,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc05ecec3f435d20d5327f HTTP/1.1" 200 None 2026-03-21 14:19:45,053 - lava_callback - DEBUG - Processing node 69bc05ecec3f435d20d5327f: baseline-arm64-clabbe 2026-03-21 14:19:45,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc05ecec3f435d20d5327f 2026-03-21 14:19:45,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:45,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc05ecec3f435d20d5327f HTTP/1.1" 200 None 2026-03-21 14:19:45,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:45,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:19:45,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:45,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc05ecec3f435d20d5327f HTTP/1.1" 200 None 2026-03-21 14:19:45,116 - lava_callback - ERROR - Error processing callback for node 69bc05ecec3f435d20d5327f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:19:48,263 - lava_callback - INFO - Received callback for node 69bea5a3448e074b3ffe2ac1 INFO: 172.20.0.1:51264 - "POST /node/69bea5a3448e074b3ffe2ac1 HTTP/1.0" 202 Accepted 2026-03-21 14:19:48,269 - lava_callback - INFO - Starting async job submit for node 69bea5a3448e074b3ffe2ac1 2026-03-21 14:19:48,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:48,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bea5a3448e074b3ffe2ac1 HTTP/1.1" 200 None 2026-03-21 14:19:48,310 - lava_callback - DEBUG - Processing node 69bea5a3448e074b3ffe2ac1: baseline-nfs-arm64 2026-03-21 14:19:48,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 14:19:48,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 14:19:48,845 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bea5a3448e074b3ffe2ac1/log.txt.gz 2026-03-21 14:19:49,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 14:19:49,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 14:19:49,293 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bea5a3448e074b3ffe2ac1/lava-logs.yaml 2026-03-21 14:19:49,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 14:19:49,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 14:19:49,439 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bea5a3448e074b3ffe2ac1/lava_callback.json.gz 2026-03-21 14:19:49,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:49,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bea5a3448e074b3ffe2ac1 HTTP/1.1" 200 None 2026-03-21 14:19:49,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:49,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 14:19:49,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:19:49,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bea5a3448e074b3ffe2ac1 HTTP/1.1" 200 None 2026-03-21 14:19:49,612 - lava_callback - INFO - Completed processing callback for node 69bea5a3448e074b3ffe2ac1 INFO: 172.20.0.1:47900 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:20:01,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:20:01,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60566 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:21:05,099 - lava_callback - INFO - Received callback for node 69bc063bec3f435d20d53286 INFO: 172.20.0.1:59806 - "POST /node/69bc063bec3f435d20d53286 HTTP/1.0" 202 Accepted 2026-03-21 14:21:05,104 - lava_callback - INFO - Starting async job submit for node 69bc063bec3f435d20d53286 2026-03-21 14:21:05,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:21:05,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc063bec3f435d20d53286 HTTP/1.1" 200 None 2026-03-21 14:21:05,120 - lava_callback - DEBUG - Processing node 69bc063bec3f435d20d53286: baseline-arm64-clabbe 2026-03-21 14:21:05,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc063bec3f435d20d53286 2026-03-21 14:21:05,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:21:05,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc063bec3f435d20d53286 HTTP/1.1" 200 None 2026-03-21 14:21:05,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:21:05,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 14:21:05,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:21:05,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc063bec3f435d20d53286 HTTP/1.1" 200 None 2026-03-21 14:21:05,174 - lava_callback - ERROR - Error processing callback for node 69bc063bec3f435d20d53286: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43050 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:34:54,211 - lava_callback - INFO - Received callback for node 69bea9f6448e074b3ffe2b62 INFO: 172.20.0.1:40956 - "POST /node/69bea9f6448e074b3ffe2b62 HTTP/1.0" 202 Accepted 2026-03-21 14:34:54,218 - lava_callback - INFO - Starting async job submit for node 69bea9f6448e074b3ffe2b62 2026-03-21 14:34:54,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:34:54,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bea9f6448e074b3ffe2b62 HTTP/1.1" 200 None 2026-03-21 14:34:54,280 - lava_callback - DEBUG - Processing node 69bea9f6448e074b3ffe2b62: baseline-nfs-arm64 2026-03-21 14:34:54,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 14:34:54,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 14:34:54,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bea9f6448e074b3ffe2b62/log.txt.gz 2026-03-21 14:34:55,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 14:34:55,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 14:34:55,466 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bea9f6448e074b3ffe2b62/lava-logs.yaml 2026-03-21 14:34:55,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 14:34:55,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 14:34:55,648 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bea9f6448e074b3ffe2b62/lava_callback.json.gz 2026-03-21 14:34:55,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:34:55,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bea9f6448e074b3ffe2b62 HTTP/1.1" 200 None 2026-03-21 14:34:55,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:34:55,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-21 14:34:55,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:34:55,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bea9f6448e074b3ffe2b62 HTTP/1.1" 200 None 2026-03-21 14:34:55,804 - lava_callback - INFO - Completed processing callback for node 69bea9f6448e074b3ffe2b62 INFO: 172.20.0.1:40970 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:35:01,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:35:01,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55494 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:41:43,189 - lava_callback - INFO - Received callback for node 69bc0b20ec3f435d20d532b7 INFO: 172.20.0.1:54870 - "POST /node/69bc0b20ec3f435d20d532b7 HTTP/1.0" 202 Accepted 2026-03-21 14:41:43,195 - lava_callback - INFO - Starting async job submit for node 69bc0b20ec3f435d20d532b7 2026-03-21 14:41:43,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:41:43,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0b20ec3f435d20d532b7 HTTP/1.1" 200 None 2026-03-21 14:41:43,211 - lava_callback - DEBUG - Processing node 69bc0b20ec3f435d20d532b7: baseline-arm-baylibre 2026-03-21 14:41:43,211 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0b20ec3f435d20d532b7 2026-03-21 14:41:43,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:41:43,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0b20ec3f435d20d532b7 HTTP/1.1" 200 None 2026-03-21 14:41:43,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:41:43,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:41:43,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:41:43,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0b20ec3f435d20d532b7 HTTP/1.1" 200 None 2026-03-21 14:41:43,269 - lava_callback - ERROR - Error processing callback for node 69bc0b20ec3f435d20d532b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45504 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:42:22,535 - lava_callback - INFO - Received callback for node 69bc0b4aec3f435d20d532cd INFO: 172.20.0.1:49930 - "POST /node/69bc0b4aec3f435d20d532cd HTTP/1.0" 202 Accepted 2026-03-21 14:42:22,540 - lava_callback - INFO - Starting async job submit for node 69bc0b4aec3f435d20d532cd 2026-03-21 14:42:22,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:42:22,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0b4aec3f435d20d532cd HTTP/1.1" 200 None 2026-03-21 14:42:22,556 - lava_callback - DEBUG - Processing node 69bc0b4aec3f435d20d532cd: baseline-arm-baylibre 2026-03-21 14:42:22,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0b4aec3f435d20d532cd 2026-03-21 14:42:22,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:42:22,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0b4aec3f435d20d532cd HTTP/1.1" 200 None 2026-03-21 14:42:22,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:42:22,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:42:22,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:42:22,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0b4aec3f435d20d532cd HTTP/1.1" 200 None 2026-03-21 14:42:22,617 - lava_callback - ERROR - Error processing callback for node 69bc0b4aec3f435d20d532cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52912 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:44:22,659 - lava_callback - INFO - Received callback for node 69bc0bb3ec3f435d20d532eb INFO: 172.20.0.1:46568 - "POST /node/69bc0bb3ec3f435d20d532eb HTTP/1.0" 202 Accepted 2026-03-21 14:44:22,665 - lava_callback - INFO - Starting async job submit for node 69bc0bb3ec3f435d20d532eb 2026-03-21 14:44:22,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:44:22,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0bb3ec3f435d20d532eb HTTP/1.1" 200 None 2026-03-21 14:44:22,681 - lava_callback - DEBUG - Processing node 69bc0bb3ec3f435d20d532eb: baseline-arm-baylibre 2026-03-21 14:44:22,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0bb3ec3f435d20d532eb 2026-03-21 14:44:22,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:44:22,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0bb3ec3f435d20d532eb HTTP/1.1" 200 None 2026-03-21 14:44:22,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:44:22,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:44:22,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:44:22,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0bb3ec3f435d20d532eb HTTP/1.1" 200 None 2026-03-21 14:44:22,735 - lava_callback - ERROR - Error processing callback for node 69bc0bb3ec3f435d20d532eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60384 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:45:02,677 - lava_callback - INFO - Received callback for node 69bc0bdcec3f435d20d532fb INFO: 172.20.0.1:52244 - "POST /node/69bc0bdcec3f435d20d532fb HTTP/1.0" 202 Accepted 2026-03-21 14:45:02,683 - lava_callback - INFO - Starting async job submit for node 69bc0bdcec3f435d20d532fb 2026-03-21 14:45:02,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:02,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0bdcec3f435d20d532fb HTTP/1.1" 200 None 2026-03-21 14:45:02,699 - lava_callback - DEBUG - Processing node 69bc0bdcec3f435d20d532fb: baseline-arm-baylibre 2026-03-21 14:45:02,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0bdcec3f435d20d532fb 2026-03-21 14:45:02,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:02,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0bdcec3f435d20d532fb HTTP/1.1" 200 None 2026-03-21 14:45:02,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:02,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:45:02,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:02,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0bdcec3f435d20d532fb HTTP/1.1" 200 None 2026-03-21 14:45:02,768 - lava_callback - ERROR - Error processing callback for node 69bc0bdcec3f435d20d532fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44210 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:45:42,721 - lava_callback - INFO - Received callback for node 69bc0c02ec3f435d20d5330b INFO: 172.20.0.1:58132 - "POST /node/69bc0c02ec3f435d20d5330b HTTP/1.0" 202 Accepted 2026-03-21 14:45:42,727 - lava_callback - INFO - Starting async job submit for node 69bc0c02ec3f435d20d5330b 2026-03-21 14:45:42,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:42,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c02ec3f435d20d5330b HTTP/1.1" 200 None 2026-03-21 14:45:42,752 - lava_callback - DEBUG - Processing node 69bc0c02ec3f435d20d5330b: baseline-arm-baylibre 2026-03-21 14:45:42,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0c02ec3f435d20d5330b 2026-03-21 14:45:42,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:42,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c02ec3f435d20d5330b HTTP/1.1" 200 None 2026-03-21 14:45:42,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:42,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:45:42,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:45:42,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0c02ec3f435d20d5330b HTTP/1.1" 200 None 2026-03-21 14:45:42,813 - lava_callback - ERROR - Error processing callback for node 69bc0c02ec3f435d20d5330b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35298 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:46:22,702 - lava_callback - INFO - Received callback for node 69bc0c29ec3f435d20d53337 INFO: 172.20.0.1:40062 - "POST /node/69bc0c29ec3f435d20d53337 HTTP/1.0" 202 Accepted 2026-03-21 14:46:22,708 - lava_callback - INFO - Starting async job submit for node 69bc0c29ec3f435d20d53337 2026-03-21 14:46:22,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:22,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c29ec3f435d20d53337 HTTP/1.1" 200 None 2026-03-21 14:46:22,723 - lava_callback - DEBUG - Processing node 69bc0c29ec3f435d20d53337: baseline-arm-baylibre 2026-03-21 14:46:22,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0c29ec3f435d20d53337 2026-03-21 14:46:22,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:22,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c29ec3f435d20d53337 HTTP/1.1" 200 None 2026-03-21 14:46:22,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:22,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:46:22,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:22,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0c29ec3f435d20d53337 HTTP/1.1" 200 None 2026-03-21 14:46:22,793 - lava_callback - ERROR - Error processing callback for node 69bc0c29ec3f435d20d53337: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:46:42,744 - lava_callback - INFO - Received callback for node 69bc0c4dec3f435d20d53354 INFO: 172.20.0.1:40494 - "POST /node/69bc0c4dec3f435d20d53354 HTTP/1.0" 202 Accepted 2026-03-21 14:46:42,749 - lava_callback - INFO - Starting async job submit for node 69bc0c4dec3f435d20d53354 2026-03-21 14:46:42,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:42,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c4dec3f435d20d53354 HTTP/1.1" 200 None 2026-03-21 14:46:42,773 - lava_callback - DEBUG - Processing node 69bc0c4dec3f435d20d53354: baseline-arm-baylibre 2026-03-21 14:46:42,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0c4dec3f435d20d53354 2026-03-21 14:46:42,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:42,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c4dec3f435d20d53354 HTTP/1.1" 200 None 2026-03-21 14:46:42,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:42,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:46:42,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:46:42,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0c4dec3f435d20d53354 HTTP/1.1" 200 None 2026-03-21 14:46:42,834 - lava_callback - ERROR - Error processing callback for node 69bc0c4dec3f435d20d53354: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37436 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:47:22,807 - lava_callback - INFO - Received callback for node 69bc0c74ec3f435d20d53366 INFO: 172.20.0.1:47534 - "POST /node/69bc0c74ec3f435d20d53366 HTTP/1.0" 202 Accepted 2026-03-21 14:47:22,812 - lava_callback - INFO - Starting async job submit for node 69bc0c74ec3f435d20d53366 2026-03-21 14:47:22,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:47:22,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c74ec3f435d20d53366 HTTP/1.1" 200 None 2026-03-21 14:47:22,828 - lava_callback - DEBUG - Processing node 69bc0c74ec3f435d20d53366: baseline-arm-baylibre 2026-03-21 14:47:22,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0c74ec3f435d20d53366 2026-03-21 14:47:22,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:47:22,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0c74ec3f435d20d53366 HTTP/1.1" 200 None 2026-03-21 14:47:22,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:47:22,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:47:22,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:47:22,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0c74ec3f435d20d53366 HTTP/1.1" 200 None 2026-03-21 14:47:22,885 - lava_callback - ERROR - Error processing callback for node 69bc0c74ec3f435d20d53366: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:49:26,281 - lava_callback - INFO - Received callback for node 69bc0ce5ec3f435d20d5338c INFO: 172.20.0.1:41102 - "POST /node/69bc0ce5ec3f435d20d5338c HTTP/1.0" 202 Accepted 2026-03-21 14:49:26,286 - lava_callback - INFO - Starting async job submit for node 69bc0ce5ec3f435d20d5338c 2026-03-21 14:49:26,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:49:26,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0ce5ec3f435d20d5338c HTTP/1.1" 200 None 2026-03-21 14:49:26,302 - lava_callback - DEBUG - Processing node 69bc0ce5ec3f435d20d5338c: baseline-arm-clabbe 2026-03-21 14:49:26,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0ce5ec3f435d20d5338c 2026-03-21 14:49:26,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:49:26,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0ce5ec3f435d20d5338c HTTP/1.1" 200 None 2026-03-21 14:49:26,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:49:26,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:49:26,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:49:26,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0ce5ec3f435d20d5338c HTTP/1.1" 200 None 2026-03-21 14:49:26,373 - lava_callback - ERROR - Error processing callback for node 69bc0ce5ec3f435d20d5338c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42262 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:50:06,298 - lava_callback - INFO - Received callback for node 69bc0d0cec3f435d20d533a1 INFO: 172.20.0.1:57840 - "POST /node/69bc0d0cec3f435d20d533a1 HTTP/1.0" 202 Accepted 2026-03-21 14:50:06,304 - lava_callback - INFO - Starting async job submit for node 69bc0d0cec3f435d20d533a1 2026-03-21 14:50:06,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:50:06,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0d0cec3f435d20d533a1 HTTP/1.1" 200 None 2026-03-21 14:50:06,319 - lava_callback - DEBUG - Processing node 69bc0d0cec3f435d20d533a1: baseline-arm-clabbe 2026-03-21 14:50:06,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0d0cec3f435d20d533a1 2026-03-21 14:50:06,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:50:06,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0d0cec3f435d20d533a1 HTTP/1.1" 200 None 2026-03-21 14:50:06,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:50:06,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:50:06,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:50:06,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0d0cec3f435d20d533a1 HTTP/1.1" 200 None 2026-03-21 14:50:06,393 - lava_callback - ERROR - Error processing callback for node 69bc0d0cec3f435d20d533a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34010 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:51:06,295 - lava_callback - INFO - Received callback for node 69bc0d57ec3f435d20d533ba INFO: 172.20.0.1:36810 - "POST /node/69bc0d57ec3f435d20d533ba HTTP/1.0" 202 Accepted 2026-03-21 14:51:06,300 - lava_callback - INFO - Starting async job submit for node 69bc0d57ec3f435d20d533ba 2026-03-21 14:51:06,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:06,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0d57ec3f435d20d533ba HTTP/1.1" 200 None 2026-03-21 14:51:06,339 - lava_callback - DEBUG - Processing node 69bc0d57ec3f435d20d533ba: baseline-arm-clabbe 2026-03-21 14:51:06,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0d57ec3f435d20d533ba 2026-03-21 14:51:06,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:06,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0d57ec3f435d20d533ba HTTP/1.1" 200 None 2026-03-21 14:51:06,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:06,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:51:06,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:06,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0d57ec3f435d20d533ba HTTP/1.1" 200 None 2026-03-21 14:51:06,409 - lava_callback - ERROR - Error processing callback for node 69bc0d57ec3f435d20d533ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40922 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:51:46,346 - lava_callback - INFO - Received callback for node 69bc0d7fec3f435d20d533c7 INFO: 172.20.0.1:49222 - "POST /node/69bc0d7fec3f435d20d533c7 HTTP/1.0" 202 Accepted 2026-03-21 14:51:46,352 - lava_callback - INFO - Starting async job submit for node 69bc0d7fec3f435d20d533c7 2026-03-21 14:51:46,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:46,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0d7fec3f435d20d533c7 HTTP/1.1" 200 None 2026-03-21 14:51:46,374 - lava_callback - DEBUG - Processing node 69bc0d7fec3f435d20d533c7: baseline-arm-clabbe 2026-03-21 14:51:46,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0d7fec3f435d20d533c7 2026-03-21 14:51:46,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:46,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0d7fec3f435d20d533c7 HTTP/1.1" 200 None 2026-03-21 14:51:46,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:46,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:51:46,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:51:46,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0d7fec3f435d20d533c7 HTTP/1.1" 200 None 2026-03-21 14:51:46,431 - lava_callback - ERROR - Error processing callback for node 69bc0d7fec3f435d20d533c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59286 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:54:23,452 - lava_callback - INFO - Received callback for node 69bc0e0fec3f435d20d533f5 INFO: 172.20.0.1:44220 - "POST /node/69bc0e0fec3f435d20d533f5 HTTP/1.0" 202 Accepted 2026-03-21 14:54:23,457 - lava_callback - INFO - Starting async job submit for node 69bc0e0fec3f435d20d533f5 2026-03-21 14:54:23,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:54:23,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e0fec3f435d20d533f5 HTTP/1.1" 200 None 2026-03-21 14:54:23,473 - lava_callback - DEBUG - Processing node 69bc0e0fec3f435d20d533f5: baseline-arm-baylibre 2026-03-21 14:54:23,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0e0fec3f435d20d533f5 2026-03-21 14:54:23,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:54:23,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e0fec3f435d20d533f5 HTTP/1.1" 200 None 2026-03-21 14:54:23,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:54:23,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 14:54:23,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:54:23,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0e0fec3f435d20d533f5 HTTP/1.1" 200 None 2026-03-21 14:54:23,531 - lava_callback - ERROR - Error processing callback for node 69bc0e0fec3f435d20d533f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56914 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:55:03,144 - lava_callback - INFO - Received callback for node 69bc0e36ec3f435d20d53401 INFO: 172.20.0.1:47694 - "POST /node/69bc0e36ec3f435d20d53401 HTTP/1.0" 202 Accepted 2026-03-21 14:55:03,149 - lava_callback - INFO - Starting async job submit for node 69bc0e36ec3f435d20d53401 2026-03-21 14:55:03,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:03,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e36ec3f435d20d53401 HTTP/1.1" 200 None 2026-03-21 14:55:03,166 - lava_callback - DEBUG - Processing node 69bc0e36ec3f435d20d53401: baseline-arm-baylibre 2026-03-21 14:55:03,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0e36ec3f435d20d53401 2026-03-21 14:55:03,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:03,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e36ec3f435d20d53401 HTTP/1.1" 200 None 2026-03-21 14:55:03,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:03,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 14:55:03,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:03,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0e36ec3f435d20d53401 HTTP/1.1" 200 None 2026-03-21 14:55:03,222 - lava_callback - ERROR - Error processing callback for node 69bc0e36ec3f435d20d53401: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37120 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:55:43,182 - lava_callback - INFO - Received callback for node 69bc0e5cec3f435d20d5340d INFO: 172.20.0.1:33432 - "POST /node/69bc0e5cec3f435d20d5340d HTTP/1.0" 202 Accepted 2026-03-21 14:55:43,187 - lava_callback - INFO - Starting async job submit for node 69bc0e5cec3f435d20d5340d 2026-03-21 14:55:43,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:43,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e5cec3f435d20d5340d HTTP/1.1" 200 None 2026-03-21 14:55:43,203 - lava_callback - DEBUG - Processing node 69bc0e5cec3f435d20d5340d: baseline-arm-baylibre 2026-03-21 14:55:43,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0e5cec3f435d20d5340d 2026-03-21 14:55:43,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:43,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e5cec3f435d20d5340d HTTP/1.1" 200 None 2026-03-21 14:55:43,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:43,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 14:55:43,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:55:43,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0e5cec3f435d20d5340d HTTP/1.1" 200 None 2026-03-21 14:55:43,264 - lava_callback - ERROR - Error processing callback for node 69bc0e5cec3f435d20d5340d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50480 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:56:23,189 - lava_callback - INFO - Received callback for node 69bc0e83ec3f435d20d53424 INFO: 172.20.0.1:59788 - "POST /node/69bc0e83ec3f435d20d53424 HTTP/1.0" 202 Accepted 2026-03-21 14:56:23,194 - lava_callback - INFO - Starting async job submit for node 69bc0e83ec3f435d20d53424 2026-03-21 14:56:23,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:23,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e83ec3f435d20d53424 HTTP/1.1" 200 None 2026-03-21 14:56:23,210 - lava_callback - DEBUG - Processing node 69bc0e83ec3f435d20d53424: baseline-arm-baylibre 2026-03-21 14:56:23,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0e83ec3f435d20d53424 2026-03-21 14:56:23,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:23,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0e83ec3f435d20d53424 HTTP/1.1" 200 None 2026-03-21 14:56:23,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:23,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 14:56:23,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:23,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0e83ec3f435d20d53424 HTTP/1.1" 200 None 2026-03-21 14:56:23,276 - lava_callback - ERROR - Error processing callback for node 69bc0e83ec3f435d20d53424: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40772 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:56:46,606 - lava_callback - INFO - Received callback for node 69bc0eabec3f435d20d53435 INFO: 172.20.0.1:40786 - "POST /node/69bc0eabec3f435d20d53435 HTTP/1.0" 202 Accepted 2026-03-21 14:56:46,612 - lava_callback - INFO - Starting async job submit for node 69bc0eabec3f435d20d53435 2026-03-21 14:56:46,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:46,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0eabec3f435d20d53435 HTTP/1.1" 200 None 2026-03-21 14:56:46,628 - lava_callback - DEBUG - Processing node 69bc0eabec3f435d20d53435: baseline-arm-clabbe 2026-03-21 14:56:46,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0eabec3f435d20d53435 2026-03-21 14:56:46,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:46,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0eabec3f435d20d53435 HTTP/1.1" 200 None 2026-03-21 14:56:46,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:46,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 14:56:46,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:56:46,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0eabec3f435d20d53435 HTTP/1.1" 200 None 2026-03-21 14:56:46,697 - lava_callback - ERROR - Error processing callback for node 69bc0eabec3f435d20d53435: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49174 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:57:23,348 - lava_callback - INFO - Received callback for node 69bc0ed0ec3f435d20d53463 INFO: 172.20.0.1:45550 - "POST /node/69bc0ed0ec3f435d20d53463 HTTP/1.0" 202 Accepted 2026-03-21 14:57:23,353 - lava_callback - INFO - Starting async job submit for node 69bc0ed0ec3f435d20d53463 2026-03-21 14:57:23,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:57:23,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0ed0ec3f435d20d53463 HTTP/1.1" 200 None 2026-03-21 14:57:23,387 - lava_callback - DEBUG - Processing node 69bc0ed0ec3f435d20d53463: baseline-arm-baylibre 2026-03-21 14:57:23,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0ed0ec3f435d20d53463 2026-03-21 14:57:23,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:57:23,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0ed0ec3f435d20d53463 HTTP/1.1" 200 None 2026-03-21 14:57:23,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:57:23,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 14:57:23,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:57:23,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0ed0ec3f435d20d53463 HTTP/1.1" 200 None 2026-03-21 14:57:23,443 - lava_callback - ERROR - Error processing callback for node 69bc0ed0ec3f435d20d53463: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33532 - "GET / HTTP/1.0" 200 OK 2026-03-21 14:59:26,711 - lava_callback - INFO - Received callback for node 69bc0f41ec3f435d20d53484 INFO: 172.20.0.1:33540 - "POST /node/69bc0f41ec3f435d20d53484 HTTP/1.0" 202 Accepted 2026-03-21 14:59:26,717 - lava_callback - INFO - Starting async job submit for node 69bc0f41ec3f435d20d53484 2026-03-21 14:59:26,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:59:26,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0f41ec3f435d20d53484 HTTP/1.1" 200 None 2026-03-21 14:59:26,732 - lava_callback - DEBUG - Processing node 69bc0f41ec3f435d20d53484: baseline-arm-clabbe 2026-03-21 14:59:26,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0f41ec3f435d20d53484 2026-03-21 14:59:26,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:59:26,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0f41ec3f435d20d53484 HTTP/1.1" 200 None 2026-03-21 14:59:26,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:59:26,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 14:59:26,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 14:59:26,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0f41ec3f435d20d53484 HTTP/1.1" 200 None 2026-03-21 14:59:26,793 - lava_callback - ERROR - Error processing callback for node 69bc0f41ec3f435d20d53484: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:00:06,854 - lava_callback - INFO - Received callback for node 69bc0f68ec3f435d20d53493 INFO: 172.20.0.1:59406 - "POST /node/69bc0f68ec3f435d20d53493 HTTP/1.0" 202 Accepted 2026-03-21 15:00:06,860 - lava_callback - INFO - Starting async job submit for node 69bc0f68ec3f435d20d53493 2026-03-21 15:00:06,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:06,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0f68ec3f435d20d53493 HTTP/1.1" 200 None 2026-03-21 15:00:06,876 - lava_callback - DEBUG - Processing node 69bc0f68ec3f435d20d53493: baseline-arm-clabbe 2026-03-21 15:00:06,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0f68ec3f435d20d53493 2026-03-21 15:00:06,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:06,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0f68ec3f435d20d53493 HTTP/1.1" 200 None 2026-03-21 15:00:06,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:06,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 15:00:06,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:06,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0f68ec3f435d20d53493 HTTP/1.1" 200 None 2026-03-21 15:00:06,932 - lava_callback - ERROR - Error processing callback for node 69bc0f68ec3f435d20d53493: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38616 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:00:46,804 - lava_callback - INFO - Received callback for node 69bc0f8fec3f435d20d5349c INFO: 172.20.0.1:38628 - "POST /node/69bc0f8fec3f435d20d5349c HTTP/1.0" 202 Accepted 2026-03-21 15:00:46,810 - lava_callback - INFO - Starting async job submit for node 69bc0f8fec3f435d20d5349c 2026-03-21 15:00:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:46,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0f8fec3f435d20d5349c HTTP/1.1" 200 None 2026-03-21 15:00:46,825 - lava_callback - DEBUG - Processing node 69bc0f8fec3f435d20d5349c: baseline-arm-clabbe 2026-03-21 15:00:46,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0f8fec3f435d20d5349c 2026-03-21 15:00:46,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:46,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0f8fec3f435d20d5349c HTTP/1.1" 200 None 2026-03-21 15:00:46,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:46,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 15:00:46,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:00:46,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0f8fec3f435d20d5349c HTTP/1.1" 200 None 2026-03-21 15:00:46,881 - lava_callback - ERROR - Error processing callback for node 69bc0f8fec3f435d20d5349c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37666 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:01:26,755 - lava_callback - INFO - Received callback for node 69bc0fb6ec3f435d20d534a6 INFO: 172.20.0.1:37678 - "POST /node/69bc0fb6ec3f435d20d534a6 HTTP/1.0" 202 Accepted 2026-03-21 15:01:26,761 - lava_callback - INFO - Starting async job submit for node 69bc0fb6ec3f435d20d534a6 2026-03-21 15:01:26,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:01:26,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0fb6ec3f435d20d534a6 HTTP/1.1" 200 None 2026-03-21 15:01:26,782 - lava_callback - DEBUG - Processing node 69bc0fb6ec3f435d20d534a6: baseline-arm-clabbe 2026-03-21 15:01:26,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0fb6ec3f435d20d534a6 2026-03-21 15:01:26,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:01:26,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0fb6ec3f435d20d534a6 HTTP/1.1" 200 None 2026-03-21 15:01:26,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:01:26,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 15:01:26,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:01:26,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0fb6ec3f435d20d534a6 HTTP/1.1" 200 None 2026-03-21 15:01:26,848 - lava_callback - ERROR - Error processing callback for node 69bc0fb6ec3f435d20d534a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60396 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:02:06,841 - lava_callback - INFO - Received callback for node 69bc0fdbec3f435d20d534b4 INFO: 172.20.0.1:60410 - "POST /node/69bc0fdbec3f435d20d534b4 HTTP/1.0" 202 Accepted 2026-03-21 15:02:06,847 - lava_callback - INFO - Starting async job submit for node 69bc0fdbec3f435d20d534b4 2026-03-21 15:02:06,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:06,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0fdbec3f435d20d534b4 HTTP/1.1" 200 None 2026-03-21 15:02:06,863 - lava_callback - DEBUG - Processing node 69bc0fdbec3f435d20d534b4: baseline-arm-clabbe 2026-03-21 15:02:06,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc0fdbec3f435d20d534b4 2026-03-21 15:02:06,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:06,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc0fdbec3f435d20d534b4 HTTP/1.1" 200 None 2026-03-21 15:02:06,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:06,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-21 15:02:06,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:06,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc0fdbec3f435d20d534b4 HTTP/1.1" 200 None 2026-03-21 15:02:06,922 - lava_callback - ERROR - Error processing callback for node 69bc0fdbec3f435d20d534b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:02:46,783 - lava_callback - INFO - Received callback for node 69bc1001ec3f435d20d534c0 INFO: 172.20.0.1:55926 - "POST /node/69bc1001ec3f435d20d534c0 HTTP/1.0" 202 Accepted 2026-03-21 15:02:46,789 - lava_callback - INFO - Starting async job submit for node 69bc1001ec3f435d20d534c0 2026-03-21 15:02:46,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:46,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1001ec3f435d20d534c0 HTTP/1.1" 200 None 2026-03-21 15:02:46,804 - lava_callback - DEBUG - Processing node 69bc1001ec3f435d20d534c0: baseline-arm-clabbe 2026-03-21 15:02:46,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1001ec3f435d20d534c0 2026-03-21 15:02:46,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:46,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1001ec3f435d20d534c0 HTTP/1.1" 200 None 2026-03-21 15:02:46,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:46,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 15:02:46,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:02:46,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1001ec3f435d20d534c0 HTTP/1.1" 200 None 2026-03-21 15:02:46,860 - lava_callback - ERROR - Error processing callback for node 69bc1001ec3f435d20d534c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53738 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:03:06,862 - lava_callback - INFO - Received callback for node 69bc1027ec3f435d20d534ca INFO: 172.20.0.1:53742 - "POST /node/69bc1027ec3f435d20d534ca HTTP/1.0" 202 Accepted 2026-03-21 15:03:06,867 - lava_callback - INFO - Starting async job submit for node 69bc1027ec3f435d20d534ca 2026-03-21 15:03:06,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:06,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1027ec3f435d20d534ca HTTP/1.1" 200 None 2026-03-21 15:03:06,889 - lava_callback - DEBUG - Processing node 69bc1027ec3f435d20d534ca: baseline-arm-clabbe 2026-03-21 15:03:06,889 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1027ec3f435d20d534ca 2026-03-21 15:03:06,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:06,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1027ec3f435d20d534ca HTTP/1.1" 200 None 2026-03-21 15:03:06,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:06,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 15:03:06,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:06,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1027ec3f435d20d534ca HTTP/1.1" 200 None 2026-03-21 15:03:06,957 - lava_callback - ERROR - Error processing callback for node 69bc1027ec3f435d20d534ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:03:46,887 - lava_callback - INFO - Received callback for node 69bc104dec3f435d20d534d4 INFO: 172.20.0.1:48196 - "POST /node/69bc104dec3f435d20d534d4 HTTP/1.0" 202 Accepted 2026-03-21 15:03:46,892 - lava_callback - INFO - Starting async job submit for node 69bc104dec3f435d20d534d4 2026-03-21 15:03:46,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:46,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc104dec3f435d20d534d4 HTTP/1.1" 200 None 2026-03-21 15:03:46,908 - lava_callback - DEBUG - Processing node 69bc104dec3f435d20d534d4: baseline-arm-clabbe 2026-03-21 15:03:46,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc104dec3f435d20d534d4 2026-03-21 15:03:46,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:46,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc104dec3f435d20d534d4 HTTP/1.1" 200 None 2026-03-21 15:03:46,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:46,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 15:03:46,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:03:46,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc104dec3f435d20d534d4 HTTP/1.1" 200 None 2026-03-21 15:03:46,972 - lava_callback - ERROR - Error processing callback for node 69bc104dec3f435d20d534d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48838 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:04:27,031 - lava_callback - INFO - Received callback for node 69bc1073ec3f435d20d534e4 INFO: 172.20.0.1:48852 - "POST /node/69bc1073ec3f435d20d534e4 HTTP/1.0" 202 Accepted 2026-03-21 15:04:27,036 - lava_callback - INFO - Starting async job submit for node 69bc1073ec3f435d20d534e4 2026-03-21 15:04:27,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:04:27,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1073ec3f435d20d534e4 HTTP/1.1" 200 None 2026-03-21 15:04:27,052 - lava_callback - DEBUG - Processing node 69bc1073ec3f435d20d534e4: baseline-arm-clabbe 2026-03-21 15:04:27,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1073ec3f435d20d534e4 2026-03-21 15:04:27,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:04:27,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1073ec3f435d20d534e4 HTTP/1.1" 200 None 2026-03-21 15:04:27,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:04:27,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 15:04:27,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:04:27,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1073ec3f435d20d534e4 HTTP/1.1" 200 None 2026-03-21 15:04:27,117 - lava_callback - ERROR - Error processing callback for node 69bc1073ec3f435d20d534e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:05:06,969 - lava_callback - INFO - Received callback for node 69bc1098ec3f435d20d534ec INFO: 172.20.0.1:38638 - "POST /node/69bc1098ec3f435d20d534ec HTTP/1.0" 202 Accepted 2026-03-21 15:05:06,975 - lava_callback - INFO - Starting async job submit for node 69bc1098ec3f435d20d534ec 2026-03-21 15:05:06,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:05:06,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1098ec3f435d20d534ec HTTP/1.1" 200 None 2026-03-21 15:05:06,990 - lava_callback - DEBUG - Processing node 69bc1098ec3f435d20d534ec: baseline-arm-clabbe 2026-03-21 15:05:06,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1098ec3f435d20d534ec 2026-03-21 15:05:06,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:05:07,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1098ec3f435d20d534ec HTTP/1.1" 200 None 2026-03-21 15:05:07,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:05:07,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e350ec88e58bad0d0dd HTTP/1.1" 200 None 2026-03-21 15:05:07,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:05:07,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1098ec3f435d20d534ec HTTP/1.1" 200 None 2026-03-21 15:05:07,053 - lava_callback - ERROR - Error processing callback for node 69bc1098ec3f435d20d534ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35662 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:13:24,249 - lava_callback - INFO - Received callback for node 69bc1291ec3f435d20d53539 INFO: 172.20.0.1:58088 - "POST /node/69bc1291ec3f435d20d53539 HTTP/1.0" 202 Accepted 2026-03-21 15:13:24,254 - lava_callback - INFO - Starting async job submit for node 69bc1291ec3f435d20d53539 2026-03-21 15:13:24,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:13:24,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1291ec3f435d20d53539 HTTP/1.1" 200 None 2026-03-21 15:13:24,270 - lava_callback - DEBUG - Processing node 69bc1291ec3f435d20d53539: baseline-arm64-baylibre 2026-03-21 15:13:24,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1291ec3f435d20d53539 2026-03-21 15:13:24,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:13:24,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1291ec3f435d20d53539 HTTP/1.1" 200 None 2026-03-21 15:13:24,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:13:24,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:13:24,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:13:24,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1291ec3f435d20d53539 HTTP/1.1" 200 None 2026-03-21 15:13:24,326 - lava_callback - ERROR - Error processing callback for node 69bc1291ec3f435d20d53539: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:15:04,277 - lava_callback - INFO - Received callback for node 69bc12e4ec3f435d20d5353e INFO: 172.20.0.1:58734 - "POST /node/69bc12e4ec3f435d20d5353e HTTP/1.0" 202 Accepted 2026-03-21 15:15:04,283 - lava_callback - INFO - Starting async job submit for node 69bc12e4ec3f435d20d5353e 2026-03-21 15:15:04,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:15:04,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc12e4ec3f435d20d5353e HTTP/1.1" 200 None 2026-03-21 15:15:04,318 - lava_callback - DEBUG - Processing node 69bc12e4ec3f435d20d5353e: baseline-arm64-baylibre 2026-03-21 15:15:04,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc12e4ec3f435d20d5353e 2026-03-21 15:15:04,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:15:04,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc12e4ec3f435d20d5353e HTTP/1.1" 200 None 2026-03-21 15:15:04,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:15:04,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:15:04,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:15:04,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc12e4ec3f435d20d5353e HTTP/1.1" 200 None 2026-03-21 15:15:04,379 - lava_callback - ERROR - Error processing callback for node 69bc12e4ec3f435d20d5353e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34222 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:16:24,336 - lava_callback - INFO - Received callback for node 69bc1334ec3f435d20d53549 INFO: 172.20.0.1:47062 - "POST /node/69bc1334ec3f435d20d53549 HTTP/1.0" 202 Accepted 2026-03-21 15:16:24,342 - lava_callback - INFO - Starting async job submit for node 69bc1334ec3f435d20d53549 2026-03-21 15:16:24,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:16:24,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1334ec3f435d20d53549 HTTP/1.1" 200 None 2026-03-21 15:16:24,358 - lava_callback - DEBUG - Processing node 69bc1334ec3f435d20d53549: baseline-arm64-baylibre 2026-03-21 15:16:24,358 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1334ec3f435d20d53549 2026-03-21 15:16:24,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:16:24,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1334ec3f435d20d53549 HTTP/1.1" 200 None 2026-03-21 15:16:24,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:16:24,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:16:24,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:16:24,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1334ec3f435d20d53549 HTTP/1.1" 200 None 2026-03-21 15:16:24,429 - lava_callback - ERROR - Error processing callback for node 69bc1334ec3f435d20d53549: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34018 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:17:44,430 - lava_callback - INFO - Received callback for node 69bc1384ec3f435d20d53557 INFO: 172.20.0.1:53832 - "POST /node/69bc1384ec3f435d20d53557 HTTP/1.0" 202 Accepted 2026-03-21 15:17:44,436 - lava_callback - INFO - Starting async job submit for node 69bc1384ec3f435d20d53557 2026-03-21 15:17:44,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:17:44,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1384ec3f435d20d53557 HTTP/1.1" 200 None 2026-03-21 15:17:44,451 - lava_callback - DEBUG - Processing node 69bc1384ec3f435d20d53557: baseline-arm64-baylibre 2026-03-21 15:17:44,452 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1384ec3f435d20d53557 2026-03-21 15:17:44,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:17:44,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1384ec3f435d20d53557 HTTP/1.1" 200 None 2026-03-21 15:17:44,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:17:44,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:17:44,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:17:44,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1384ec3f435d20d53557 HTTP/1.1" 200 None 2026-03-21 15:17:44,519 - lava_callback - ERROR - Error processing callback for node 69bc1384ec3f435d20d53557: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35730 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:20:27,623 - lava_callback - INFO - Received callback for node 69bc1428ec3f435d20d5356c INFO: 172.20.0.1:35736 - "POST /node/69bc1428ec3f435d20d5356c HTTP/1.0" 202 Accepted 2026-03-21 15:20:27,628 - lava_callback - INFO - Starting async job submit for node 69bc1428ec3f435d20d5356c 2026-03-21 15:20:27,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:20:27,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1428ec3f435d20d5356c HTTP/1.1" 200 None 2026-03-21 15:20:27,649 - lava_callback - DEBUG - Processing node 69bc1428ec3f435d20d5356c: baseline-arm64-clabbe 2026-03-21 15:20:27,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1428ec3f435d20d5356c 2026-03-21 15:20:27,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:20:27,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1428ec3f435d20d5356c HTTP/1.1" 200 None 2026-03-21 15:20:27,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:20:27,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:20:27,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:20:27,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1428ec3f435d20d5356c HTTP/1.1" 200 None 2026-03-21 15:20:27,703 - lava_callback - ERROR - Error processing callback for node 69bc1428ec3f435d20d5356c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49350 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:21:47,739 - lava_callback - INFO - Received callback for node 69bc1477ec3f435d20d53578 INFO: 172.20.0.1:49354 - "POST /node/69bc1477ec3f435d20d53578 HTTP/1.0" 202 Accepted 2026-03-21 15:21:47,745 - lava_callback - INFO - Starting async job submit for node 69bc1477ec3f435d20d53578 2026-03-21 15:21:47,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:21:47,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1477ec3f435d20d53578 HTTP/1.1" 200 None 2026-03-21 15:21:47,761 - lava_callback - DEBUG - Processing node 69bc1477ec3f435d20d53578: baseline-arm64-clabbe 2026-03-21 15:21:47,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1477ec3f435d20d53578 2026-03-21 15:21:47,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:21:47,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1477ec3f435d20d53578 HTTP/1.1" 200 None 2026-03-21 15:21:47,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:21:47,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:21:47,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:21:47,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1477ec3f435d20d53578 HTTP/1.1" 200 None 2026-03-21 15:21:47,812 - lava_callback - ERROR - Error processing callback for node 69bc1477ec3f435d20d53578: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:23:07,726 - lava_callback - INFO - Received callback for node 69bc14c7ec3f435d20d53584 INFO: 172.20.0.1:58590 - "POST /node/69bc14c7ec3f435d20d53584 HTTP/1.0" 202 Accepted 2026-03-21 15:23:07,731 - lava_callback - INFO - Starting async job submit for node 69bc14c7ec3f435d20d53584 2026-03-21 15:23:07,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:23:07,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc14c7ec3f435d20d53584 HTTP/1.1" 200 None 2026-03-21 15:23:07,748 - lava_callback - DEBUG - Processing node 69bc14c7ec3f435d20d53584: baseline-arm64-clabbe 2026-03-21 15:23:07,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc14c7ec3f435d20d53584 2026-03-21 15:23:07,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:23:07,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc14c7ec3f435d20d53584 HTTP/1.1" 200 None 2026-03-21 15:23:07,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:23:07,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:23:07,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:23:07,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc14c7ec3f435d20d53584 HTTP/1.1" 200 None 2026-03-21 15:23:07,816 - lava_callback - ERROR - Error processing callback for node 69bc14c7ec3f435d20d53584: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:24:27,729 - lava_callback - INFO - Received callback for node 69bc1519ec3f435d20d5358a INFO: 172.20.0.1:58570 - "POST /node/69bc1519ec3f435d20d5358a HTTP/1.0" 202 Accepted 2026-03-21 15:24:27,738 - lava_callback - INFO - Starting async job submit for node 69bc1519ec3f435d20d5358a 2026-03-21 15:24:27,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:24:27,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1519ec3f435d20d5358a HTTP/1.1" 200 None 2026-03-21 15:24:27,755 - lava_callback - DEBUG - Processing node 69bc1519ec3f435d20d5358a: baseline-arm64-clabbe 2026-03-21 15:24:27,755 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1519ec3f435d20d5358a 2026-03-21 15:24:27,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:24:27,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1519ec3f435d20d5358a HTTP/1.1" 200 None 2026-03-21 15:24:27,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:24:27,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:24:27,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:24:27,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1519ec3f435d20d5358a HTTP/1.1" 200 None 2026-03-21 15:24:27,816 - lava_callback - ERROR - Error processing callback for node 69bc1519ec3f435d20d5358a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36806 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:25:47,828 - lava_callback - INFO - Received callback for node 69bc1569ec3f435d20d53595 INFO: 172.20.0.1:36818 - "POST /node/69bc1569ec3f435d20d53595 HTTP/1.0" 202 Accepted 2026-03-21 15:25:47,834 - lava_callback - INFO - Starting async job submit for node 69bc1569ec3f435d20d53595 2026-03-21 15:25:47,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:25:47,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1569ec3f435d20d53595 HTTP/1.1" 200 None 2026-03-21 15:25:47,850 - lava_callback - DEBUG - Processing node 69bc1569ec3f435d20d53595: baseline-arm64-clabbe 2026-03-21 15:25:47,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1569ec3f435d20d53595 2026-03-21 15:25:47,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:25:47,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1569ec3f435d20d53595 HTTP/1.1" 200 None 2026-03-21 15:25:47,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:25:47,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:25:47,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:25:47,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1569ec3f435d20d53595 HTTP/1.1" 200 None 2026-03-21 15:25:47,906 - lava_callback - ERROR - Error processing callback for node 69bc1569ec3f435d20d53595: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49646 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:27:07,858 - lava_callback - INFO - Received callback for node 69bc15bbec3f435d20d535a1 INFO: 172.20.0.1:49652 - "POST /node/69bc15bbec3f435d20d535a1 HTTP/1.0" 202 Accepted 2026-03-21 15:27:07,864 - lava_callback - INFO - Starting async job submit for node 69bc15bbec3f435d20d535a1 2026-03-21 15:27:07,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:27:07,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc15bbec3f435d20d535a1 HTTP/1.1" 200 None 2026-03-21 15:27:07,880 - lava_callback - DEBUG - Processing node 69bc15bbec3f435d20d535a1: baseline-arm64-clabbe 2026-03-21 15:27:07,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc15bbec3f435d20d535a1 2026-03-21 15:27:07,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:27:07,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc15bbec3f435d20d535a1 HTTP/1.1" 200 None 2026-03-21 15:27:07,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:27:07,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-21 15:27:07,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:27:07,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc15bbec3f435d20d535a1 HTTP/1.1" 200 None 2026-03-21 15:27:07,945 - lava_callback - ERROR - Error processing callback for node 69bc15bbec3f435d20d535a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58320 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:29:25,069 - lava_callback - INFO - Received callback for node 69bc1645ec3f435d20d535ac INFO: 172.20.0.1:46926 - "POST /node/69bc1645ec3f435d20d535ac HTTP/1.0" 202 Accepted 2026-03-21 15:29:25,075 - lava_callback - INFO - Starting async job submit for node 69bc1645ec3f435d20d535ac 2026-03-21 15:29:25,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:29:25,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1645ec3f435d20d535ac HTTP/1.1" 200 None 2026-03-21 15:29:25,090 - lava_callback - DEBUG - Processing node 69bc1645ec3f435d20d535ac: baseline-arm64-baylibre 2026-03-21 15:29:25,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1645ec3f435d20d535ac 2026-03-21 15:29:25,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:29:25,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1645ec3f435d20d535ac HTTP/1.1" 200 None 2026-03-21 15:29:25,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:29:25,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:29:25,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:29:25,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1645ec3f435d20d535ac HTTP/1.1" 200 None 2026-03-21 15:29:25,150 - lava_callback - ERROR - Error processing callback for node 69bc1645ec3f435d20d535ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55266 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:30:45,113 - lava_callback - INFO - Received callback for node 69bc1695ec3f435d20d535b9 INFO: 172.20.0.1:37710 - "POST /node/69bc1695ec3f435d20d535b9 HTTP/1.0" 202 Accepted 2026-03-21 15:30:45,118 - lava_callback - INFO - Starting async job submit for node 69bc1695ec3f435d20d535b9 2026-03-21 15:30:45,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:30:45,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1695ec3f435d20d535b9 HTTP/1.1" 200 None 2026-03-21 15:30:45,134 - lava_callback - DEBUG - Processing node 69bc1695ec3f435d20d535b9: baseline-arm64-baylibre 2026-03-21 15:30:45,135 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1695ec3f435d20d535b9 2026-03-21 15:30:45,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:30:45,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1695ec3f435d20d535b9 HTTP/1.1" 200 None 2026-03-21 15:30:45,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:30:45,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:30:45,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:30:45,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1695ec3f435d20d535b9 HTTP/1.1" 200 None 2026-03-21 15:30:45,201 - lava_callback - ERROR - Error processing callback for node 69bc1695ec3f435d20d535b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53706 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:33:25,316 - lava_callback - INFO - Received callback for node 69bc1732ec3f435d20d535c4 INFO: 172.20.0.1:44278 - "POST /node/69bc1732ec3f435d20d535c4 HTTP/1.0" 202 Accepted 2026-03-21 15:33:25,322 - lava_callback - INFO - Starting async job submit for node 69bc1732ec3f435d20d535c4 2026-03-21 15:33:25,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:33:25,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1732ec3f435d20d535c4 HTTP/1.1" 200 None 2026-03-21 15:33:25,339 - lava_callback - DEBUG - Processing node 69bc1732ec3f435d20d535c4: baseline-arm64-baylibre 2026-03-21 15:33:25,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1732ec3f435d20d535c4 2026-03-21 15:33:25,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:33:25,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1732ec3f435d20d535c4 HTTP/1.1" 200 None 2026-03-21 15:33:25,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:33:25,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:33:25,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:33:25,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1732ec3f435d20d535c4 HTTP/1.1" 200 None 2026-03-21 15:33:25,407 - lava_callback - ERROR - Error processing callback for node 69bc1732ec3f435d20d535c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35364 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33958 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:35:25,512 - lava_callback - INFO - Received callback for node 69bc17abec3f435d20d535d0 INFO: 172.20.0.1:49726 - "POST /node/69bc17abec3f435d20d535d0 HTTP/1.0" 202 Accepted 2026-03-21 15:35:25,518 - lava_callback - INFO - Starting async job submit for node 69bc17abec3f435d20d535d0 2026-03-21 15:35:25,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:35:25,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc17abec3f435d20d535d0 HTTP/1.1" 200 None 2026-03-21 15:35:25,534 - lava_callback - DEBUG - Processing node 69bc17abec3f435d20d535d0: baseline-arm64-baylibre 2026-03-21 15:35:25,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc17abec3f435d20d535d0 2026-03-21 15:35:25,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:35:25,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc17abec3f435d20d535d0 HTTP/1.1" 200 None 2026-03-21 15:35:25,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:35:25,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:35:25,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:35:25,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc17abec3f435d20d535d0 HTTP/1.1" 200 None 2026-03-21 15:35:25,589 - lava_callback - ERROR - Error processing callback for node 69bc17abec3f435d20d535d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44620 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:36:45,621 - lava_callback - INFO - Received callback for node 69bc17f8ec3f435d20d535df INFO: 172.20.0.1:45146 - "POST /node/69bc17f8ec3f435d20d535df HTTP/1.0" 202 Accepted 2026-03-21 15:36:45,626 - lava_callback - INFO - Starting async job submit for node 69bc17f8ec3f435d20d535df 2026-03-21 15:36:45,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:36:45,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc17f8ec3f435d20d535df HTTP/1.1" 200 None 2026-03-21 15:36:45,642 - lava_callback - DEBUG - Processing node 69bc17f8ec3f435d20d535df: baseline-arm64-baylibre 2026-03-21 15:36:45,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc17f8ec3f435d20d535df 2026-03-21 15:36:45,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:36:45,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc17f8ec3f435d20d535df HTTP/1.1" 200 None 2026-03-21 15:36:45,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:36:45,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:36:45,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:36:45,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc17f8ec3f435d20d535df HTTP/1.1" 200 None 2026-03-21 15:36:45,748 - lava_callback - ERROR - Error processing callback for node 69bc17f8ec3f435d20d535df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38922 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:39:25,976 - lava_callback - INFO - Received callback for node 69bc189eec3f435d20d535f1 INFO: 172.20.0.1:41520 - "POST /node/69bc189eec3f435d20d535f1 HTTP/1.0" 202 Accepted 2026-03-21 15:39:25,982 - lava_callback - INFO - Starting async job submit for node 69bc189eec3f435d20d535f1 2026-03-21 15:39:25,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:39:26,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc189eec3f435d20d535f1 HTTP/1.1" 200 None 2026-03-21 15:39:26,004 - lava_callback - DEBUG - Processing node 69bc189eec3f435d20d535f1: baseline-arm64-baylibre 2026-03-21 15:39:26,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc189eec3f435d20d535f1 2026-03-21 15:39:26,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:39:26,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc189eec3f435d20d535f1 HTTP/1.1" 200 None 2026-03-21 15:39:26,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:39:26,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:39:26,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:39:26,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc189eec3f435d20d535f1 HTTP/1.1" 200 None 2026-03-21 15:39:26,070 - lava_callback - ERROR - Error processing callback for node 69bc189eec3f435d20d535f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46848 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:40:45,703 - lava_callback - INFO - Received callback for node 69bc18f1ec3f435d20d53600 INFO: 172.20.0.1:53030 - "POST /node/69bc18f1ec3f435d20d53600 HTTP/1.0" 202 Accepted 2026-03-21 15:40:45,709 - lava_callback - INFO - Starting async job submit for node 69bc18f1ec3f435d20d53600 2026-03-21 15:40:45,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:40:45,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc18f1ec3f435d20d53600 HTTP/1.1" 200 None 2026-03-21 15:40:45,726 - lava_callback - DEBUG - Processing node 69bc18f1ec3f435d20d53600: baseline-arm64-baylibre 2026-03-21 15:40:45,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc18f1ec3f435d20d53600 2026-03-21 15:40:45,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:40:45,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc18f1ec3f435d20d53600 HTTP/1.1" 200 None 2026-03-21 15:40:45,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:40:45,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:40:45,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:40:45,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc18f1ec3f435d20d53600 HTTP/1.1" 200 None 2026-03-21 15:40:45,790 - lava_callback - ERROR - Error processing callback for node 69bc18f1ec3f435d20d53600: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53616 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:43:28,722 - lava_callback - INFO - Received callback for node 69bc1994ec3f435d20d5360f INFO: 172.20.0.1:53620 - "POST /node/69bc1994ec3f435d20d5360f HTTP/1.0" 202 Accepted 2026-03-21 15:43:28,728 - lava_callback - INFO - Starting async job submit for node 69bc1994ec3f435d20d5360f 2026-03-21 15:43:28,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:43:28,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1994ec3f435d20d5360f HTTP/1.1" 200 None 2026-03-21 15:43:28,744 - lava_callback - DEBUG - Processing node 69bc1994ec3f435d20d5360f: baseline-arm64-clabbe 2026-03-21 15:43:28,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1994ec3f435d20d5360f 2026-03-21 15:43:28,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:43:28,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1994ec3f435d20d5360f HTTP/1.1" 200 None 2026-03-21 15:43:28,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:43:28,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:43:28,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:43:28,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1994ec3f435d20d5360f HTTP/1.1" 200 None 2026-03-21 15:43:28,796 - lava_callback - ERROR - Error processing callback for node 69bc1994ec3f435d20d5360f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52894 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:44:48,766 - lava_callback - INFO - Received callback for node 69bc19e0ec3f435d20d5361f INFO: 172.20.0.1:52908 - "POST /node/69bc19e0ec3f435d20d5361f HTTP/1.0" 202 Accepted 2026-03-21 15:44:48,772 - lava_callback - INFO - Starting async job submit for node 69bc19e0ec3f435d20d5361f 2026-03-21 15:44:48,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:44:48,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc19e0ec3f435d20d5361f HTTP/1.1" 200 None 2026-03-21 15:44:48,788 - lava_callback - DEBUG - Processing node 69bc19e0ec3f435d20d5361f: baseline-arm64-clabbe 2026-03-21 15:44:48,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc19e0ec3f435d20d5361f 2026-03-21 15:44:48,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:44:48,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc19e0ec3f435d20d5361f HTTP/1.1" 200 None 2026-03-21 15:44:48,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:44:48,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:44:48,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:44:48,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc19e0ec3f435d20d5361f HTTP/1.1" 200 None 2026-03-21 15:44:48,852 - lava_callback - ERROR - Error processing callback for node 69bc19e0ec3f435d20d5361f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:46:08,827 - lava_callback - INFO - Received callback for node 69bc1a2fec3f435d20d53631 INFO: 172.20.0.1:53882 - "POST /node/69bc1a2fec3f435d20d53631 HTTP/1.0" 202 Accepted 2026-03-21 15:46:08,832 - lava_callback - INFO - Starting async job submit for node 69bc1a2fec3f435d20d53631 2026-03-21 15:46:08,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:46:08,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1a2fec3f435d20d53631 HTTP/1.1" 200 None 2026-03-21 15:46:08,849 - lava_callback - DEBUG - Processing node 69bc1a2fec3f435d20d53631: baseline-arm64-clabbe 2026-03-21 15:46:08,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1a2fec3f435d20d53631 2026-03-21 15:46:08,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:46:08,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1a2fec3f435d20d53631 HTTP/1.1" 200 None 2026-03-21 15:46:08,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:46:08,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:46:08,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:46:08,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1a2fec3f435d20d53631 HTTP/1.1" 200 None 2026-03-21 15:46:08,906 - lava_callback - ERROR - Error processing callback for node 69bc1a2fec3f435d20d53631: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:47:28,849 - lava_callback - INFO - Received callback for node 69bc1a85ec3f435d20d53639 INFO: 172.20.0.1:53226 - "POST /node/69bc1a85ec3f435d20d53639 HTTP/1.0" 202 Accepted 2026-03-21 15:47:28,855 - lava_callback - INFO - Starting async job submit for node 69bc1a85ec3f435d20d53639 2026-03-21 15:47:28,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:47:28,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1a85ec3f435d20d53639 HTTP/1.1" 200 None 2026-03-21 15:47:28,874 - lava_callback - DEBUG - Processing node 69bc1a85ec3f435d20d53639: baseline-arm64-clabbe 2026-03-21 15:47:28,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1a85ec3f435d20d53639 2026-03-21 15:47:28,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:47:28,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1a85ec3f435d20d53639 HTTP/1.1" 200 None 2026-03-21 15:47:28,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:47:28,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:47:28,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:47:28,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1a85ec3f435d20d53639 HTTP/1.1" 200 None 2026-03-21 15:47:28,964 - lava_callback - ERROR - Error processing callback for node 69bc1a85ec3f435d20d53639: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:48:48,912 - lava_callback - INFO - Received callback for node 69bc1ad5ec3f435d20d53648 INFO: 172.20.0.1:53896 - "POST /node/69bc1ad5ec3f435d20d53648 HTTP/1.0" 202 Accepted 2026-03-21 15:48:48,918 - lava_callback - INFO - Starting async job submit for node 69bc1ad5ec3f435d20d53648 2026-03-21 15:48:48,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:48:48,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1ad5ec3f435d20d53648 HTTP/1.1" 200 None 2026-03-21 15:48:48,934 - lava_callback - DEBUG - Processing node 69bc1ad5ec3f435d20d53648: baseline-arm64-clabbe 2026-03-21 15:48:48,934 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1ad5ec3f435d20d53648 2026-03-21 15:48:48,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:48:48,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1ad5ec3f435d20d53648 HTTP/1.1" 200 None 2026-03-21 15:48:48,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:48:48,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:48:48,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:48:48,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1ad5ec3f435d20d53648 HTTP/1.1" 200 None 2026-03-21 15:48:48,996 - lava_callback - ERROR - Error processing callback for node 69bc1ad5ec3f435d20d53648: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45056 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:50:08,925 - lava_callback - INFO - Received callback for node 69bc1b26ec3f435d20d53658 INFO: 172.20.0.1:45070 - "POST /node/69bc1b26ec3f435d20d53658 HTTP/1.0" 202 Accepted 2026-03-21 15:50:08,931 - lava_callback - INFO - Starting async job submit for node 69bc1b26ec3f435d20d53658 2026-03-21 15:50:08,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:50:08,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1b26ec3f435d20d53658 HTTP/1.1" 200 None 2026-03-21 15:50:08,953 - lava_callback - DEBUG - Processing node 69bc1b26ec3f435d20d53658: baseline-arm64-clabbe 2026-03-21 15:50:08,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1b26ec3f435d20d53658 2026-03-21 15:50:08,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:50:08,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1b26ec3f435d20d53658 HTTP/1.1" 200 None 2026-03-21 15:50:08,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:50:08,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b90e430ec88e58bad0d0ec HTTP/1.1" 200 None 2026-03-21 15:50:08,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:50:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1b26ec3f435d20d53658 HTTP/1.1" 200 None 2026-03-21 15:50:09,008 - lava_callback - ERROR - Error processing callback for node 69bc1b26ec3f435d20d53658: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55788 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:55:46,502 - lava_callback - INFO - Received callback for node 69bc1c73ec3f435d20d53676 INFO: 172.20.0.1:35670 - "POST /node/69bc1c73ec3f435d20d53676 HTTP/1.0" 202 Accepted 2026-03-21 15:55:46,507 - lava_callback - INFO - Starting async job submit for node 69bc1c73ec3f435d20d53676 2026-03-21 15:55:46,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:35684 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:55:46,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1c73ec3f435d20d53676 HTTP/1.1" 200 None 2026-03-21 15:55:46,523 - lava_callback - DEBUG - Processing node 69bc1c73ec3f435d20d53676: baseline-arm-baylibre 2026-03-21 15:55:46,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1c73ec3f435d20d53676 2026-03-21 15:55:46,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:55:46,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1c73ec3f435d20d53676 HTTP/1.1" 200 None 2026-03-21 15:55:46,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:55:46,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 15:55:46,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:55:46,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1c73ec3f435d20d53676 HTTP/1.1" 200 None 2026-03-21 15:55:46,579 - lava_callback - ERROR - Error processing callback for node 69bc1c73ec3f435d20d53676: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50178 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:56:26,532 - lava_callback - INFO - Received callback for node 69bc1c9bec3f435d20d5368b INFO: 172.20.0.1:50188 - "POST /node/69bc1c9bec3f435d20d5368b HTTP/1.0" 202 Accepted 2026-03-21 15:56:26,538 - lava_callback - INFO - Starting async job submit for node 69bc1c9bec3f435d20d5368b 2026-03-21 15:56:26,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:56:26,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1c9bec3f435d20d5368b HTTP/1.1" 200 None 2026-03-21 15:56:26,554 - lava_callback - DEBUG - Processing node 69bc1c9bec3f435d20d5368b: baseline-arm-baylibre 2026-03-21 15:56:26,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1c9bec3f435d20d5368b 2026-03-21 15:56:26,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:56:26,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1c9bec3f435d20d5368b HTTP/1.1" 200 None 2026-03-21 15:56:26,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:56:26,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 15:56:26,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:56:26,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1c9bec3f435d20d5368b HTTP/1.1" 200 None 2026-03-21 15:56:26,606 - lava_callback - ERROR - Error processing callback for node 69bc1c9bec3f435d20d5368b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57918 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:57:06,592 - lava_callback - INFO - Received callback for node 69bc1cc2ec3f435d20d53697 INFO: 172.20.0.1:57928 - "POST /node/69bc1cc2ec3f435d20d53697 HTTP/1.0" 202 Accepted 2026-03-21 15:57:06,597 - lava_callback - INFO - Starting async job submit for node 69bc1cc2ec3f435d20d53697 2026-03-21 15:57:06,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:06,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1cc2ec3f435d20d53697 HTTP/1.1" 200 None 2026-03-21 15:57:06,615 - lava_callback - DEBUG - Processing node 69bc1cc2ec3f435d20d53697: baseline-arm-baylibre 2026-03-21 15:57:06,615 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1cc2ec3f435d20d53697 2026-03-21 15:57:06,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:06,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1cc2ec3f435d20d53697 HTTP/1.1" 200 None 2026-03-21 15:57:06,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:06,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 15:57:06,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:06,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1cc2ec3f435d20d53697 HTTP/1.1" 200 None 2026-03-21 15:57:06,678 - lava_callback - ERROR - Error processing callback for node 69bc1cc2ec3f435d20d53697: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39286 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:57:46,591 - lava_callback - INFO - Received callback for node 69bc1ce9ec3f435d20d536a1 INFO: 172.20.0.1:39294 - "POST /node/69bc1ce9ec3f435d20d536a1 HTTP/1.0" 202 Accepted 2026-03-21 15:57:46,596 - lava_callback - INFO - Starting async job submit for node 69bc1ce9ec3f435d20d536a1 2026-03-21 15:57:46,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:46,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1ce9ec3f435d20d536a1 HTTP/1.1" 200 None 2026-03-21 15:57:46,612 - lava_callback - DEBUG - Processing node 69bc1ce9ec3f435d20d536a1: baseline-arm-baylibre 2026-03-21 15:57:46,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1ce9ec3f435d20d536a1 2026-03-21 15:57:46,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:46,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1ce9ec3f435d20d536a1 HTTP/1.1" 200 None 2026-03-21 15:57:46,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:46,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 15:57:46,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:57:46,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1ce9ec3f435d20d536a1 HTTP/1.1" 200 None 2026-03-21 15:57:46,693 - lava_callback - ERROR - Error processing callback for node 69bc1ce9ec3f435d20d536a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48406 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:58:27,041 - lava_callback - INFO - Received callback for node 69bc1d10ec3f435d20d536ab INFO: 172.20.0.1:48408 - "POST /node/69bc1d10ec3f435d20d536ab HTTP/1.0" 202 Accepted 2026-03-21 15:58:27,046 - lava_callback - INFO - Starting async job submit for node 69bc1d10ec3f435d20d536ab 2026-03-21 15:58:27,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:58:27,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d10ec3f435d20d536ab HTTP/1.1" 200 None 2026-03-21 15:58:27,062 - lava_callback - DEBUG - Processing node 69bc1d10ec3f435d20d536ab: baseline-arm-baylibre 2026-03-21 15:58:27,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1d10ec3f435d20d536ab 2026-03-21 15:58:27,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:58:27,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d10ec3f435d20d536ab HTTP/1.1" 200 None 2026-03-21 15:58:27,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:58:27,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 15:58:27,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:58:27,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1d10ec3f435d20d536ab HTTP/1.1" 200 None 2026-03-21 15:58:27,125 - lava_callback - ERROR - Error processing callback for node 69bc1d10ec3f435d20d536ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58072 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:59:06,660 - lava_callback - INFO - Received callback for node 69bc1d37ec3f435d20d536b5 INFO: 172.20.0.1:58084 - "POST /node/69bc1d37ec3f435d20d536b5 HTTP/1.0" 202 Accepted 2026-03-21 15:59:06,665 - lava_callback - INFO - Starting async job submit for node 69bc1d37ec3f435d20d536b5 2026-03-21 15:59:06,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:06,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d37ec3f435d20d536b5 HTTP/1.1" 200 None 2026-03-21 15:59:06,686 - lava_callback - DEBUG - Processing node 69bc1d37ec3f435d20d536b5: baseline-arm-baylibre 2026-03-21 15:59:06,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1d37ec3f435d20d536b5 2026-03-21 15:59:06,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:06,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d37ec3f435d20d536b5 HTTP/1.1" 200 None 2026-03-21 15:59:06,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:06,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 15:59:06,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:06,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1d37ec3f435d20d536b5 HTTP/1.1" 200 None 2026-03-21 15:59:06,743 - lava_callback - ERROR - Error processing callback for node 69bc1d37ec3f435d20d536b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK 2026-03-21 15:59:49,365 - lava_callback - INFO - Received callback for node 69bc1d61ec3f435d20d536bf INFO: 172.20.0.1:39924 - "POST /node/69bc1d61ec3f435d20d536bf HTTP/1.0" 202 Accepted 2026-03-21 15:59:49,370 - lava_callback - INFO - Starting async job submit for node 69bc1d61ec3f435d20d536bf 2026-03-21 15:59:49,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:49,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d61ec3f435d20d536bf HTTP/1.1" 200 None 2026-03-21 15:59:49,391 - lava_callback - DEBUG - Processing node 69bc1d61ec3f435d20d536bf: baseline-arm-clabbe 2026-03-21 15:59:49,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1d61ec3f435d20d536bf 2026-03-21 15:59:49,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:49,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d61ec3f435d20d536bf HTTP/1.1" 200 None 2026-03-21 15:59:49,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:49,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 15:59:49,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 15:59:49,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1d61ec3f435d20d536bf HTTP/1.1" 200 None 2026-03-21 15:59:49,449 - lava_callback - ERROR - Error processing callback for node 69bc1d61ec3f435d20d536bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57090 - "GET / HTTP/1.0" 200 OK 2026-03-21 16:00:29,347 - lava_callback - INFO - Received callback for node 69bc1d88ec3f435d20d536c9 INFO: 172.20.0.1:57096 - "POST /node/69bc1d88ec3f435d20d536c9 HTTP/1.0" 202 Accepted 2026-03-21 16:00:29,353 - lava_callback - INFO - Starting async job submit for node 69bc1d88ec3f435d20d536c9 2026-03-21 16:00:29,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:00:29,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d88ec3f435d20d536c9 HTTP/1.1" 200 None 2026-03-21 16:00:29,369 - lava_callback - DEBUG - Processing node 69bc1d88ec3f435d20d536c9: baseline-arm-clabbe 2026-03-21 16:00:29,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1d88ec3f435d20d536c9 2026-03-21 16:00:29,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:00:29,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1d88ec3f435d20d536c9 HTTP/1.1" 200 None 2026-03-21 16:00:29,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:00:29,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 16:00:29,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:00:29,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1d88ec3f435d20d536c9 HTTP/1.1" 200 None 2026-03-21 16:00:29,441 - lava_callback - ERROR - Error processing callback for node 69bc1d88ec3f435d20d536c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK 2026-03-21 16:01:09,392 - lava_callback - INFO - Received callback for node 69bc1dafec3f435d20d536d3 INFO: 172.20.0.1:38834 - "POST /node/69bc1dafec3f435d20d536d3 HTTP/1.0" 202 Accepted 2026-03-21 16:01:09,398 - lava_callback - INFO - Starting async job submit for node 69bc1dafec3f435d20d536d3 2026-03-21 16:01:09,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:09,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1dafec3f435d20d536d3 HTTP/1.1" 200 None 2026-03-21 16:01:09,433 - lava_callback - DEBUG - Processing node 69bc1dafec3f435d20d536d3: baseline-arm-clabbe 2026-03-21 16:01:09,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1dafec3f435d20d536d3 2026-03-21 16:01:09,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:09,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1dafec3f435d20d536d3 HTTP/1.1" 200 None 2026-03-21 16:01:09,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:09,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 16:01:09,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:09,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1dafec3f435d20d536d3 HTTP/1.1" 200 None 2026-03-21 16:01:09,498 - lava_callback - ERROR - Error processing callback for node 69bc1dafec3f435d20d536d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41132 - "GET / HTTP/1.0" 200 OK 2026-03-21 16:01:49,427 - lava_callback - INFO - Received callback for node 69bc1dd7ec3f435d20d536df INFO: 172.20.0.1:41148 - "POST /node/69bc1dd7ec3f435d20d536df HTTP/1.0" 202 Accepted 2026-03-21 16:01:49,432 - lava_callback - INFO - Starting async job submit for node 69bc1dd7ec3f435d20d536df 2026-03-21 16:01:49,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:49,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1dd7ec3f435d20d536df HTTP/1.1" 200 None 2026-03-21 16:01:49,450 - lava_callback - DEBUG - Processing node 69bc1dd7ec3f435d20d536df: baseline-arm-clabbe 2026-03-21 16:01:49,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1dd7ec3f435d20d536df 2026-03-21 16:01:49,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:49,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1dd7ec3f435d20d536df HTTP/1.1" 200 None 2026-03-21 16:01:49,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:49,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 16:01:49,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:01:49,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1dd7ec3f435d20d536df HTTP/1.1" 200 None 2026-03-21 16:01:49,533 - lava_callback - ERROR - Error processing callback for node 69bc1dd7ec3f435d20d536df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52050 - "GET / HTTP/1.0" 200 OK 2026-03-21 16:02:09,481 - lava_callback - INFO - Received callback for node 69bc1dfeec3f435d20d536e9 INFO: 172.20.0.1:52064 - "POST /node/69bc1dfeec3f435d20d536e9 HTTP/1.0" 202 Accepted 2026-03-21 16:02:09,486 - lava_callback - INFO - Starting async job submit for node 69bc1dfeec3f435d20d536e9 2026-03-21 16:02:09,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:02:09,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1dfeec3f435d20d536e9 HTTP/1.1" 200 None 2026-03-21 16:02:09,509 - lava_callback - DEBUG - Processing node 69bc1dfeec3f435d20d536e9: baseline-arm-clabbe 2026-03-21 16:02:09,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bc1dfeec3f435d20d536e9 2026-03-21 16:02:09,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:02:09,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc1dfeec3f435d20d536e9 HTTP/1.1" 200 None 2026-03-21 16:02:09,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:02:09,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9747a151254d948eee9f0 HTTP/1.1" 200 None 2026-03-21 16:02:09,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 16:02:09,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bc1dfeec3f435d20d536e9 HTTP/1.1" 200 None 2026-03-21 16:02:09,562 - lava_callback - ERROR - Error processing callback for node 69bc1dfeec3f435d20d536e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]