INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:45038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43584 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:31:25,271 - lava_callback - INFO - Received callback for node 69ae14618ef4424e487b2f5c 2026-03-09 00:31:25,278 - lava_callback - INFO - Starting async job submit for node 69ae14618ef4424e487b2f5c INFO: 172.21.0.1:40074 - "POST /node/69ae14618ef4424e487b2f5c HTTP/1.0" 202 Accepted 2026-03-09 00:31:25,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:31:25,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae14618ef4424e487b2f5c HTTP/1.1" 200 None 2026-03-09 00:31:25,364 - lava_callback - DEBUG - Processing node 69ae14618ef4424e487b2f5c: baseline-arm 2026-03-09 00:31:25,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:31:25,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:31:25,800 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae14618ef4424e487b2f5c/log.txt.gz 2026-03-09 00:31:25,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:31:26,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:31:26,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae14618ef4424e487b2f5c/lava-logs.yaml 2026-03-09 00:31:26,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:31:26,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:31:26,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae14618ef4424e487b2f5c/lava_callback.json.gz 2026-03-09 00:31:26,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:31:26,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae14618ef4424e487b2f5c HTTP/1.1" 200 None 2026-03-09 00:31:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:31:26,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae116d8ef4424e487b2ea8 HTTP/1.1" 200 None 2026-03-09 00:31:26,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:31:26,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae14618ef4424e487b2f5c HTTP/1.1" 200 None 2026-03-09 00:31:26,411 - lava_callback - INFO - Completed processing callback for node 69ae14618ef4424e487b2f5c INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35340 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:31:56,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:31:56,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36054 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:39:30,808 - lava_callback - INFO - Received callback for node 69ab73baefa461f55e59d30c INFO: 172.21.0.1:45740 - "POST /node/69ab73baefa461f55e59d30c HTTP/1.0" 202 Accepted 2026-03-09 00:39:30,813 - lava_callback - INFO - Starting async job submit for node 69ab73baefa461f55e59d30c 2026-03-09 00:39:30,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:39:30,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab73baefa461f55e59d30c HTTP/1.1" 200 None 2026-03-09 00:39:30,830 - lava_callback - DEBUG - Processing node 69ab73baefa461f55e59d30c: baseline-arm-baylibre 2026-03-09 00:39:30,830 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab73baefa461f55e59d30c 2026-03-09 00:39:30,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:39:30,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab73baefa461f55e59d30c HTTP/1.1" 200 None 2026-03-09 00:39:30,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:39:30,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 00:39:30,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:39:30,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab73baefa461f55e59d30c HTTP/1.1" 200 None 2026-03-09 00:39:30,882 - lava_callback - ERROR - Error processing callback for node 69ab73baefa461f55e59d30c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35360 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:41:10,843 - lava_callback - INFO - Received callback for node 69ab741defa461f55e59d331 INFO: 172.21.0.1:46146 - "POST /node/69ab741defa461f55e59d331 HTTP/1.0" 202 Accepted 2026-03-09 00:41:10,852 - lava_callback - INFO - Starting async job submit for node 69ab741defa461f55e59d331 2026-03-09 00:41:10,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:10,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab741defa461f55e59d331 HTTP/1.1" 200 None 2026-03-09 00:41:10,890 - lava_callback - DEBUG - Processing node 69ab741defa461f55e59d331: baseline-arm-baylibre 2026-03-09 00:41:10,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab741defa461f55e59d331 2026-03-09 00:41:10,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:10,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab741defa461f55e59d331 HTTP/1.1" 200 None 2026-03-09 00:41:10,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:11,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 00:41:11,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:11,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab741defa461f55e59d331 HTTP/1.1" 200 None 2026-03-09 00:41:11,065 - lava_callback - ERROR - Error processing callback for node 69ab741defa461f55e59d331: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60938 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:41:50,830 - lava_callback - INFO - Received callback for node 69ab7441efa461f55e59d346 INFO: 172.21.0.1:41128 - "POST /node/69ab7441efa461f55e59d346 HTTP/1.0" 202 Accepted 2026-03-09 00:41:50,839 - lava_callback - INFO - Starting async job submit for node 69ab7441efa461f55e59d346 2026-03-09 00:41:50,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:50,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7441efa461f55e59d346 HTTP/1.1" 200 None 2026-03-09 00:41:50,868 - lava_callback - DEBUG - Processing node 69ab7441efa461f55e59d346: baseline-arm-baylibre 2026-03-09 00:41:50,868 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7441efa461f55e59d346 2026-03-09 00:41:50,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:50,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7441efa461f55e59d346 HTTP/1.1" 200 None 2026-03-09 00:41:50,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:50,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 00:41:50,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:41:50,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7441efa461f55e59d346 HTTP/1.1" 200 None 2026-03-09 00:41:50,957 - lava_callback - ERROR - Error processing callback for node 69ab7441efa461f55e59d346: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44356 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:42:30,968 - lava_callback - INFO - Received callback for node 69ab7465efa461f55e59d359 INFO: 172.21.0.1:49228 - "POST /node/69ab7465efa461f55e59d359 HTTP/1.0" 202 Accepted 2026-03-09 00:42:30,974 - lava_callback - INFO - Starting async job submit for node 69ab7465efa461f55e59d359 2026-03-09 00:42:30,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:30,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7465efa461f55e59d359 HTTP/1.1" 200 None 2026-03-09 00:42:30,990 - lava_callback - DEBUG - Processing node 69ab7465efa461f55e59d359: baseline-arm-baylibre 2026-03-09 00:42:30,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7465efa461f55e59d359 2026-03-09 00:42:30,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:31,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7465efa461f55e59d359 HTTP/1.1" 200 None 2026-03-09 00:42:31,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:31,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 00:42:31,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:31,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7465efa461f55e59d359 HTTP/1.1" 200 None 2026-03-09 00:42:31,055 - lava_callback - ERROR - Error processing callback for node 69ab7465efa461f55e59d359: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:42:35,120 - lava_callback - INFO - Received callback for node 69ae17638ef4424e487b3421 INFO: 172.21.0.1:49232 - "POST /node/69ae17638ef4424e487b3421 HTTP/1.0" 202 Accepted 2026-03-09 00:42:35,126 - lava_callback - INFO - Starting async job submit for node 69ae17638ef4424e487b3421 2026-03-09 00:42:35,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:35,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17638ef4424e487b3421 HTTP/1.1" 200 None 2026-03-09 00:42:35,144 - lava_callback - DEBUG - Processing node 69ae17638ef4424e487b3421: baseline-arm 2026-03-09 00:42:35,150 - lava_callback - WARNING - Failed to upload log 2026-03-09 00:42:35,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:35,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:35,266 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17638ef4424e487b3421/lava-logs.yaml 2026-03-09 00:42:35,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:35,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:35,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17638ef4424e487b3421/lava_callback.json.gz 2026-03-09 00:42:35,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:35,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17638ef4424e487b3421 HTTP/1.1" 200 None 2026-03-09 00:42:35,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:35,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:42:35,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:35,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17638ef4424e487b3421 HTTP/1.1" 200 None 2026-03-09 00:42:35,445 - lava_callback - INFO - Completed processing callback for node 69ae17638ef4424e487b3421 INFO: 172.21.0.1:49248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33958 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:42:50,844 - lava_callback - INFO - Received callback for node 69ab7489efa461f55e59d365 INFO: 172.21.0.1:37962 - "POST /node/69ab7489efa461f55e59d365 HTTP/1.0" 202 Accepted 2026-03-09 00:42:50,850 - lava_callback - INFO - Starting async job submit for node 69ab7489efa461f55e59d365 2026-03-09 00:42:50,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:50,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7489efa461f55e59d365 HTTP/1.1" 200 None 2026-03-09 00:42:50,866 - lava_callback - DEBUG - Processing node 69ab7489efa461f55e59d365: baseline-arm-baylibre 2026-03-09 00:42:50,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7489efa461f55e59d365 2026-03-09 00:42:50,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:50,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7489efa461f55e59d365 HTTP/1.1" 200 None 2026-03-09 00:42:50,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:50,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 00:42:50,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:50,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7489efa461f55e59d365 HTTP/1.1" 200 None 2026-03-09 00:42:50,949 - lava_callback - ERROR - Error processing callback for node 69ab7489efa461f55e59d365: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:42:52,015 - lava_callback - INFO - Received callback for node 69ae17648ef4424e487b3426 INFO: 172.21.0.1:37966 - "POST /node/69ae17648ef4424e487b3426 HTTP/1.0" 202 Accepted 2026-03-09 00:42:52,062 - lava_callback - INFO - Starting async job submit for node 69ae17648ef4424e487b3426 2026-03-09 00:42:52,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:52,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17648ef4424e487b3426 HTTP/1.1" 200 None 2026-03-09 00:42:52,097 - lava_callback - DEBUG - Processing node 69ae17648ef4424e487b3426: baseline-arm 2026-03-09 00:42:52,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:52,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:52,369 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17648ef4424e487b3426/log.txt.gz 2026-03-09 00:42:52,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:52,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:52,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17648ef4424e487b3426/lava-logs.yaml 2026-03-09 00:42:52,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:52,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:52,720 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17648ef4424e487b3426/lava_callback.json.gz 2026-03-09 00:42:52,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:52,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17648ef4424e487b3426 HTTP/1.1" 200 None 2026-03-09 00:42:52,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:52,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:42:52,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:52,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17648ef4424e487b3426 HTTP/1.1" 200 None 2026-03-09 00:42:52,799 - lava_callback - INFO - Completed processing callback for node 69ae17648ef4424e487b3426 2026-03-09 00:42:53,390 - lava_callback - INFO - Received callback for node 69ae17678ef4424e487b3430 INFO: 172.21.0.1:37970 - "POST /node/69ae17678ef4424e487b3430 HTTP/1.0" 202 Accepted 2026-03-09 00:42:53,438 - lava_callback - INFO - Starting async job submit for node 69ae17678ef4424e487b3430 2026-03-09 00:42:53,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:53,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17678ef4424e487b3430 HTTP/1.1" 200 None 2026-03-09 00:42:53,471 - lava_callback - DEBUG - Processing node 69ae17678ef4424e487b3430: baseline-arm 2026-03-09 00:42:53,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:53,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:53,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17678ef4424e487b3430/log.txt.gz 2026-03-09 00:42:53,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:53,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:53,966 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17678ef4424e487b3430/lava-logs.yaml 2026-03-09 00:42:53,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:42:54,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:42:54,107 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17678ef4424e487b3430/lava_callback.json.gz 2026-03-09 00:42:54,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:54,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17678ef4424e487b3430 HTTP/1.1" 200 None 2026-03-09 00:42:54,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:54,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:42:54,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:54,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17678ef4424e487b3430 HTTP/1.1" 200 None 2026-03-09 00:42:54,184 - lava_callback - INFO - Completed processing callback for node 69ae17678ef4424e487b3430 2026-03-09 00:42:56,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:42:56,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33732 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:43:26,549 - lava_callback - INFO - Received callback for node 69ae17668ef4424e487b342c INFO: 172.21.0.1:33742 - "POST /node/69ae17668ef4424e487b342c HTTP/1.0" 202 Accepted 2026-03-09 00:43:26,555 - lava_callback - INFO - Starting async job submit for node 69ae17668ef4424e487b342c 2026-03-09 00:43:26,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:26,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17668ef4424e487b342c HTTP/1.1" 200 None 2026-03-09 00:43:26,592 - lava_callback - DEBUG - Processing node 69ae17668ef4424e487b342c: baseline-arm 2026-03-09 00:43:26,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:26,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:26,994 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17668ef4424e487b342c/log.txt.gz 2026-03-09 00:43:27,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:27,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:27,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17668ef4424e487b342c/lava-logs.yaml 2026-03-09 00:43:27,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:27,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:27,528 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17668ef4424e487b342c/lava_callback.json.gz 2026-03-09 00:43:27,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:27,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17668ef4424e487b342c HTTP/1.1" 200 None 2026-03-09 00:43:27,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:27,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:43:27,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:27,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17668ef4424e487b342c HTTP/1.1" 200 None 2026-03-09 00:43:27,638 - lava_callback - INFO - Completed processing callback for node 69ae17668ef4424e487b342c 2026-03-09 00:43:30,907 - lava_callback - INFO - Received callback for node 69ab74acefa461f55e59d37a INFO: 172.21.0.1:55458 - "POST /node/69ab74acefa461f55e59d37a HTTP/1.0" 202 Accepted 2026-03-09 00:43:30,912 - lava_callback - INFO - Starting async job submit for node 69ab74acefa461f55e59d37a 2026-03-09 00:43:30,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:30,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab74acefa461f55e59d37a HTTP/1.1" 200 None 2026-03-09 00:43:30,932 - lava_callback - DEBUG - Processing node 69ab74acefa461f55e59d37a: baseline-arm-baylibre 2026-03-09 00:43:30,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab74acefa461f55e59d37a 2026-03-09 00:43:30,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:30,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab74acefa461f55e59d37a HTTP/1.1" 200 None 2026-03-09 00:43:30,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:30,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 00:43:30,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:30,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab74acefa461f55e59d37a HTTP/1.1" 200 None 2026-03-09 00:43:30,986 - lava_callback - ERROR - Error processing callback for node 69ab74acefa461f55e59d37a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:43:38,603 - lava_callback - INFO - Received callback for node 69ae17858ef4424e487b34cf INFO: 172.21.0.1:55472 - "POST /node/69ae17858ef4424e487b34cf HTTP/1.0" 202 Accepted 2026-03-09 00:43:38,609 - lava_callback - INFO - Starting async job submit for node 69ae17858ef4424e487b34cf 2026-03-09 00:43:38,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:38,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17858ef4424e487b34cf HTTP/1.1" 200 None 2026-03-09 00:43:38,642 - lava_callback - DEBUG - Processing node 69ae17858ef4424e487b34cf: baseline-arm-pengutronix 2026-03-09 00:43:38,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:38,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:38,931 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17858ef4424e487b34cf/log.txt.gz 2026-03-09 00:43:39,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:39,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:39,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17858ef4424e487b34cf/lava-logs.yaml 2026-03-09 00:43:39,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:39,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:39,354 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17858ef4424e487b34cf/lava_callback.json.gz 2026-03-09 00:43:39,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:39,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17858ef4424e487b34cf HTTP/1.1" 200 None 2026-03-09 00:43:39,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:39,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:43:39,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:39,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17858ef4424e487b34cf HTTP/1.1" 200 None 2026-03-09 00:43:39,438 - lava_callback - INFO - Completed processing callback for node 69ae17858ef4424e487b34cf INFO: 172.21.0.1:35950 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:43:46,038 - lava_callback - INFO - Received callback for node 69ae17638ef4424e487b341e INFO: 172.21.0.1:35962 - "POST /node/69ae17638ef4424e487b341e HTTP/1.0" 202 Accepted 2026-03-09 00:43:46,044 - lava_callback - INFO - Starting async job submit for node 69ae17638ef4424e487b341e 2026-03-09 00:43:46,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:46,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17638ef4424e487b341e HTTP/1.1" 200 None 2026-03-09 00:43:46,080 - lava_callback - DEBUG - Processing node 69ae17638ef4424e487b341e: baseline-arm 2026-03-09 00:43:46,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:46,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:46,428 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17638ef4424e487b341e/log.txt.gz INFO: 172.21.0.1:35970 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:43:46,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:46,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:46,784 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17638ef4424e487b341e/lava-logs.yaml 2026-03-09 00:43:46,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:46,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:46,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17638ef4424e487b341e/lava_callback.json.gz 2026-03-09 00:43:46,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:46,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17638ef4424e487b341e HTTP/1.1" 200 None 2026-03-09 00:43:46,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:47,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:43:47,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:47,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17638ef4424e487b341e HTTP/1.1" 200 None 2026-03-09 00:43:47,047 - lava_callback - INFO - Completed processing callback for node 69ae17638ef4424e487b341e 2026-03-09 00:43:48,687 - lava_callback - INFO - Received callback for node 69ae17898ef4424e487b34d8 INFO: 172.21.0.1:35972 - "POST /node/69ae17898ef4424e487b34d8 HTTP/1.0" 202 Accepted 2026-03-09 00:43:48,733 - lava_callback - INFO - Starting async job submit for node 69ae17898ef4424e487b34d8 2026-03-09 00:43:48,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:48,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17898ef4424e487b34d8 HTTP/1.1" 200 None 2026-03-09 00:43:48,779 - lava_callback - DEBUG - Processing node 69ae17898ef4424e487b34d8: baseline-arm-pengutronix 2026-03-09 00:43:48,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:49,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:49,160 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17898ef4424e487b34d8/log.txt.gz 2026-03-09 00:43:49,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:49,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:49,490 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17898ef4424e487b34d8/lava-logs.yaml 2026-03-09 00:43:49,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:49,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:49,610 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17898ef4424e487b34d8/lava_callback.json.gz 2026-03-09 00:43:49,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:49,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17898ef4424e487b34d8 HTTP/1.1" 200 None 2026-03-09 00:43:49,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:49,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:43:49,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:49,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17898ef4424e487b34d8 HTTP/1.1" 200 None 2026-03-09 00:43:49,694 - lava_callback - INFO - Completed processing callback for node 69ae17898ef4424e487b34d8 2026-03-09 00:43:55,657 - lava_callback - INFO - Received callback for node 69ae17868ef4424e487b34d2 INFO: 172.21.0.1:55400 - "POST /node/69ae17868ef4424e487b34d2 HTTP/1.0" 202 Accepted 2026-03-09 00:43:55,663 - lava_callback - INFO - Starting async job submit for node 69ae17868ef4424e487b34d2 2026-03-09 00:43:55,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:55,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17868ef4424e487b34d2 HTTP/1.1" 200 None 2026-03-09 00:43:55,696 - lava_callback - DEBUG - Processing node 69ae17868ef4424e487b34d2: baseline-arm-pengutronix 2026-03-09 00:43:55,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:55,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:55,995 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17868ef4424e487b34d2/log.txt.gz 2026-03-09 00:43:56,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:56,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:56,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17868ef4424e487b34d2/lava-logs.yaml 2026-03-09 00:43:56,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:43:56,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:43:56,415 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17868ef4424e487b34d2/lava_callback.json.gz 2026-03-09 00:43:56,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:56,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17868ef4424e487b34d2 HTTP/1.1" 200 None 2026-03-09 00:43:56,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:56,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:56,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:43:56,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:43:56,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-09 00:43:56,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17868ef4424e487b34d2 HTTP/1.1" 200 None 2026-03-09 00:43:56,507 - lava_callback - INFO - Completed processing callback for node 69ae17868ef4424e487b34d2 INFO: 172.21.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50980 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:44:22,134 - lava_callback - INFO - Received callback for node 69ae177a8ef4424e487b34a2 INFO: 172.21.0.1:33904 - "POST /node/69ae177a8ef4424e487b34a2 HTTP/1.0" 202 Accepted 2026-03-09 00:44:22,140 - lava_callback - INFO - Starting async job submit for node 69ae177a8ef4424e487b34a2 2026-03-09 00:44:22,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:22,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177a8ef4424e487b34a2 HTTP/1.1" 200 None 2026-03-09 00:44:22,177 - lava_callback - DEBUG - Processing node 69ae177a8ef4424e487b34a2: baseline-arm-clabbe 2026-03-09 00:44:22,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:44:22,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:44:22,521 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177a8ef4424e487b34a2/log.txt.gz 2026-03-09 00:44:22,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:44:22,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:44:22,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177a8ef4424e487b34a2/lava-logs.yaml 2026-03-09 00:44:22,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:44:22,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:44:22,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177a8ef4424e487b34a2/lava_callback.json.gz 2026-03-09 00:44:23,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:23,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177a8ef4424e487b34a2 HTTP/1.1" 200 None 2026-03-09 00:44:23,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:23,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:44:23,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:23,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae177a8ef4424e487b34a2 HTTP/1.1" 200 None 2026-03-09 00:44:23,088 - lava_callback - INFO - Completed processing callback for node 69ae177a8ef4424e487b34a2 2026-03-09 00:44:26,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:26,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33914 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:44:31,734 - lava_callback - INFO - Received callback for node 69ae17828ef4424e487b34c4 INFO: 172.21.0.1:42992 - "POST /node/69ae17828ef4424e487b34c4 HTTP/1.0" 202 Accepted 2026-03-09 00:44:31,740 - lava_callback - INFO - Starting async job submit for node 69ae17828ef4424e487b34c4 2026-03-09 00:44:31,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:31,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17828ef4424e487b34c4 HTTP/1.1" 200 None 2026-03-09 00:44:31,778 - lava_callback - DEBUG - Processing node 69ae17828ef4424e487b34c4: baseline-arm-kontron 2026-03-09 00:44:31,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:44:32,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:44:32,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69ae17828ef4424e487b34c4/log.txt.gz 2026-03-09 00:44:32,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:44:32,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:44:32,358 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69ae17828ef4424e487b34c4/lava-logs.yaml 2026-03-09 00:44:32,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:44:32,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:44:32,503 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69ae17828ef4424e487b34c4/lava_callback.json.gz 2026-03-09 00:44:32,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:32,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17828ef4424e487b34c4 HTTP/1.1" 200 None 2026-03-09 00:44:32,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:32,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:44:32,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:44:32,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17828ef4424e487b34c4 HTTP/1.1" 200 None 2026-03-09 00:44:32,604 - lava_callback - INFO - Completed processing callback for node 69ae17828ef4424e487b34c4 INFO: 172.21.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34108 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:44:56,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47034 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:44:56,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 00:45:00,330 - lava_callback - INFO - Received callback for node 69ae179e8ef4424e487b3532 INFO: 172.21.0.1:46694 - "POST /node/69ae179e8ef4424e487b3532 HTTP/1.0" 202 Accepted 2026-03-09 00:45:00,337 - lava_callback - INFO - Starting async job submit for node 69ae179e8ef4424e487b3532 2026-03-09 00:45:00,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:00,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae179e8ef4424e487b3532 HTTP/1.1" 200 None 2026-03-09 00:45:00,396 - lava_callback - DEBUG - Processing node 69ae179e8ef4424e487b3532: ltp-dio 2026-03-09 00:45:00,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:45:01,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:45:01,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69ae179e8ef4424e487b3532/log.txt.gz 2026-03-09 00:45:01,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:45:01,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:45:01,863 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69ae179e8ef4424e487b3532/lava-logs.yaml 2026-03-09 00:45:01,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:45:02,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:45:02,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69ae179e8ef4424e487b3532/lava_callback.json.gz 2026-03-09 00:45:02,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:02,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae179e8ef4424e487b3532 HTTP/1.1" 200 None 2026-03-09 00:45:02,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:02,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:45:02,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:02,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae179e8ef4424e487b3532 HTTP/1.1" 200 None 2026-03-09 00:45:02,174 - lava_callback - INFO - Completed processing callback for node 69ae179e8ef4424e487b3532 INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42476 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:45:26,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:26,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 00:45:30,464 - lava_callback - INFO - Received callback for node 69ae181e8ef4424e487b3620 INFO: 172.21.0.1:45390 - "POST /node/69ae181e8ef4424e487b3620 HTTP/1.0" 202 Accepted 2026-03-09 00:45:30,470 - lava_callback - INFO - Starting async job submit for node 69ae181e8ef4424e487b3620 2026-03-09 00:45:30,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:30,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae181e8ef4424e487b3620 HTTP/1.1" 200 None 2026-03-09 00:45:30,487 - lava_callback - DEBUG - Processing node 69ae181e8ef4424e487b3620: baseline-arm 2026-03-09 00:45:30,492 - lava_callback - WARNING - Failed to upload log 2026-03-09 00:45:30,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:45:30,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:45:30,605 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae181e8ef4424e487b3620/lava-logs.yaml 2026-03-09 00:45:30,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:45:30,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:45:30,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae181e8ef4424e487b3620/lava_callback.json.gz 2026-03-09 00:45:30,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:30,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae181e8ef4424e487b3620 HTTP/1.1" 200 None 2026-03-09 00:45:30,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:30,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:45:30,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:30,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae181e8ef4424e487b3620 HTTP/1.1" 200 None 2026-03-09 00:45:30,837 - lava_callback - INFO - Completed processing callback for node 69ae181e8ef4424e487b3620 INFO: 172.21.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55922 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:45:56,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:45:56,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59080 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:46:09,180 - lava_callback - INFO - Received callback for node 69ae177b8ef4424e487b34a8 INFO: 172.21.0.1:59092 - "POST /node/69ae177b8ef4424e487b34a8 HTTP/1.0" 202 Accepted 2026-03-09 00:46:09,186 - lava_callback - INFO - Starting async job submit for node 69ae177b8ef4424e487b34a8 2026-03-09 00:46:09,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:46:09,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177b8ef4424e487b34a8 HTTP/1.1" 200 None 2026-03-09 00:46:09,225 - lava_callback - DEBUG - Processing node 69ae177b8ef4424e487b34a8: baseline-arm-clabbe 2026-03-09 00:46:09,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:46:09,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:46:09,558 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177b8ef4424e487b34a8/log.txt.gz 2026-03-09 00:46:09,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:46:09,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:46:09,858 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177b8ef4424e487b34a8/lava-logs.yaml 2026-03-09 00:46:09,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:46:10,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:46:10,058 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177b8ef4424e487b34a8/lava_callback.json.gz 2026-03-09 00:46:10,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:46:10,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177b8ef4424e487b34a8 HTTP/1.1" 200 None 2026-03-09 00:46:10,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:46:10,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:46:10,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:46:10,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae177b8ef4424e487b34a8 HTTP/1.1" 200 None 2026-03-09 00:46:10,138 - lava_callback - INFO - Completed processing callback for node 69ae177b8ef4424e487b34a8 INFO: 172.21.0.1:47686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59996 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:46:26,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:46:26,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49500 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:47:44,264 - lava_callback - INFO - Received callback for node 69ae177f8ef4424e487b34b6 INFO: 172.21.0.1:49510 - "POST /node/69ae177f8ef4424e487b34b6 HTTP/1.0" 202 Accepted 2026-03-09 00:47:44,270 - lava_callback - INFO - Starting async job submit for node 69ae177f8ef4424e487b34b6 2026-03-09 00:47:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:47:44,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177f8ef4424e487b34b6 HTTP/1.1" 200 None 2026-03-09 00:47:44,307 - lava_callback - DEBUG - Processing node 69ae177f8ef4424e487b34b6: baseline-arm-clabbe 2026-03-09 00:47:44,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:47:44,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:47:44,712 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177f8ef4424e487b34b6/log.txt.gz 2026-03-09 00:47:44,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:47:45,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:47:45,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177f8ef4424e487b34b6/lava-logs.yaml 2026-03-09 00:47:45,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:47:45,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:47:45,232 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae177f8ef4424e487b34b6/lava_callback.json.gz 2026-03-09 00:47:45,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:47:45,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177f8ef4424e487b34b6 HTTP/1.1" 200 None 2026-03-09 00:47:45,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:47:45,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:47:45,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:47:45,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae177f8ef4424e487b34b6 HTTP/1.1" 200 None 2026-03-09 00:47:45,320 - lava_callback - INFO - Completed processing callback for node 69ae177f8ef4424e487b34b6 INFO: 172.21.0.1:49516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42922 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:47:56,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:47:56,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:49:26,197 - lava_callback - INFO - Received callback for node 69ae14638ef4424e487b2f5f INFO: 172.21.0.1:48814 - "POST /node/69ae14638ef4424e487b2f5f HTTP/1.0" 202 Accepted 2026-03-09 00:49:26,212 - lava_callback - INFO - Starting async job submit for node 69ae14638ef4424e487b2f5f 2026-03-09 00:49:26,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:26,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae14638ef4424e487b2f5f HTTP/1.1" 200 None 2026-03-09 00:49:26,379 - lava_callback - DEBUG - Processing node 69ae14638ef4424e487b2f5f: ltp-smoke INFO: 172.21.0.1:48822 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:49:27,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:49:27,374 - lava_callback - INFO - Received callback for node 69ae17808ef4424e487b34ba 2026-03-09 00:49:27,385 - lava_callback - INFO - Starting async job submit for node 69ae17808ef4424e487b34ba INFO: 172.21.0.1:48836 - "POST /node/69ae17808ef4424e487b34ba HTTP/1.0" 202 Accepted 2026-03-09 00:49:27,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:27,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17808ef4424e487b34ba HTTP/1.1" 200 None 2026-03-09 00:49:27,460 - lava_callback - DEBUG - Processing node 69ae17808ef4424e487b34ba: baseline-arm-clabbe 2026-03-09 00:49:27,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:49:27,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae14638ef4424e487b2f5f/log.txt.gz 2026-03-09 00:49:28,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:49:28,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:49:28,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:49:28,511 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae17808ef4424e487b34ba/log.txt.gz 2026-03-09 00:49:28,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:49:29,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:49:29,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae14638ef4424e487b2f5f/lava-logs.yaml 2026-03-09 00:49:29,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:49:29,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:49:29,305 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae17808ef4424e487b34ba/lava-logs.yaml 2026-03-09 00:49:29,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:49:29,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:49:29,748 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae14638ef4424e487b2f5f/lava_callback.json.gz 2026-03-09 00:49:29,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:29,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae14638ef4424e487b2f5f HTTP/1.1" 200 None 2026-03-09 00:49:29,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:30,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae116d8ef4424e487b2ea8 HTTP/1.1" 200 None 2026-03-09 00:49:30,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:30,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:49:30,040 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69ae17808ef4424e487b34ba/lava_callback.json.gz 2026-03-09 00:49:30,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:30,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae14638ef4424e487b2f5f HTTP/1.1" 200 None 2026-03-09 00:49:30,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17808ef4424e487b34ba HTTP/1.1" 200 None 2026-03-09 00:49:30,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:30,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:49:30,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:30,344 - lava_callback - INFO - Completed processing callback for node 69ae14638ef4424e487b2f5f 2026-03-09 00:49:30,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17808ef4424e487b34ba HTTP/1.1" 200 None 2026-03-09 00:49:30,389 - lava_callback - INFO - Completed processing callback for node 69ae17808ef4424e487b34ba 2026-03-09 00:49:31,413 - lava_callback - INFO - Received callback for node 69ab760befa461f55e59d406 INFO: 172.21.0.1:60150 - "POST /node/69ab760befa461f55e59d406 HTTP/1.0" 202 Accepted 2026-03-09 00:49:31,422 - lava_callback - INFO - Starting async job submit for node 69ab760befa461f55e59d406 2026-03-09 00:49:31,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:31,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760befa461f55e59d406 HTTP/1.1" 200 None 2026-03-09 00:49:31,456 - lava_callback - DEBUG - Processing node 69ab760befa461f55e59d406: baseline-arm-baylibre 2026-03-09 00:49:31,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab760befa461f55e59d406 2026-03-09 00:49:31,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:31,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760befa461f55e59d406 HTTP/1.1" 200 None 2026-03-09 00:49:31,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:31,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:31,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:31,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab760befa461f55e59d406 HTTP/1.1" 200 None 2026-03-09 00:49:31,549 - lava_callback - ERROR - Error processing callback for node 69ab760befa461f55e59d406: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:49:32,060 - lava_callback - INFO - Received callback for node 69ab760aefa461f55e59d401 INFO: 172.21.0.1:60166 - "POST /node/69ab760aefa461f55e59d401 HTTP/1.0" 202 Accepted 2026-03-09 00:49:32,069 - lava_callback - INFO - Starting async job submit for node 69ab760aefa461f55e59d401 2026-03-09 00:49:32,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760aefa461f55e59d401 HTTP/1.1" 200 None 2026-03-09 00:49:32,099 - lava_callback - DEBUG - Processing node 69ab760aefa461f55e59d401: baseline-arm-baylibre 2026-03-09 00:49:32,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab760aefa461f55e59d401 2026-03-09 00:49:32,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760aefa461f55e59d401 HTTP/1.1" 200 None 2026-03-09 00:49:32,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:32,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab760aefa461f55e59d401 HTTP/1.1" 200 None 2026-03-09 00:49:32,195 - lava_callback - ERROR - Error processing callback for node 69ab760aefa461f55e59d401: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:49:32,727 - lava_callback - INFO - Received callback for node 69ab760cefa461f55e59d40c INFO: 172.21.0.1:60170 - "POST /node/69ab760cefa461f55e59d40c HTTP/1.0" 202 Accepted 2026-03-09 00:49:32,750 - lava_callback - INFO - Starting async job submit for node 69ab760cefa461f55e59d40c 2026-03-09 00:49:32,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760cefa461f55e59d40c HTTP/1.1" 200 None 2026-03-09 00:49:32,784 - lava_callback - DEBUG - Processing node 69ab760cefa461f55e59d40c: baseline-arm-baylibre 2026-03-09 00:49:32,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab760cefa461f55e59d40c 2026-03-09 00:49:32,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760cefa461f55e59d40c HTTP/1.1" 200 None 2026-03-09 00:49:32,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:32,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:32,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab760cefa461f55e59d40c HTTP/1.1" 200 None 2026-03-09 00:49:32,882 - lava_callback - ERROR - Error processing callback for node 69ab760cefa461f55e59d40c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:49:33,403 - lava_callback - INFO - Received callback for node 69ab760fefa461f55e59d417 INFO: 172.21.0.1:60180 - "POST /node/69ab760fefa461f55e59d417 HTTP/1.0" 202 Accepted 2026-03-09 00:49:33,412 - lava_callback - INFO - Starting async job submit for node 69ab760fefa461f55e59d417 2026-03-09 00:49:33,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:33,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760fefa461f55e59d417 HTTP/1.1" 200 None 2026-03-09 00:49:33,440 - lava_callback - DEBUG - Processing node 69ab760fefa461f55e59d417: baseline-arm-baylibre 2026-03-09 00:49:33,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab760fefa461f55e59d417 2026-03-09 00:49:33,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:33,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab760fefa461f55e59d417 HTTP/1.1" 200 None 2026-03-09 00:49:33,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:33,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:33,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:33,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab760fefa461f55e59d417 HTTP/1.1" 200 None 2026-03-09 00:49:33,556 - lava_callback - ERROR - Error processing callback for node 69ab760fefa461f55e59d417: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:49:34,051 - lava_callback - INFO - Received callback for node 69ab7610efa461f55e59d41a INFO: 172.21.0.1:60182 - "POST /node/69ab7610efa461f55e59d41a HTTP/1.0" 202 Accepted 2026-03-09 00:49:34,060 - lava_callback - INFO - Starting async job submit for node 69ab7610efa461f55e59d41a 2026-03-09 00:49:34,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:34,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7610efa461f55e59d41a HTTP/1.1" 200 None 2026-03-09 00:49:34,091 - lava_callback - DEBUG - Processing node 69ab7610efa461f55e59d41a: baseline-arm-baylibre 2026-03-09 00:49:34,091 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7610efa461f55e59d41a 2026-03-09 00:49:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:34,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7610efa461f55e59d41a HTTP/1.1" 200 None 2026-03-09 00:49:34,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:34,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:34,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:34,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7610efa461f55e59d41a HTTP/1.1" 200 None 2026-03-09 00:49:34,197 - lava_callback - ERROR - Error processing callback for node 69ab7610efa461f55e59d41a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:49:35,808 - lava_callback - INFO - Received callback for node 69ab761defa461f55e59d43f INFO: 172.21.0.1:60196 - "POST /node/69ab761defa461f55e59d43f HTTP/1.0" 202 Accepted 2026-03-09 00:49:35,818 - lava_callback - INFO - Starting async job submit for node 69ab761defa461f55e59d43f 2026-03-09 00:49:35,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:35,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab761defa461f55e59d43f HTTP/1.1" 200 None 2026-03-09 00:49:35,846 - lava_callback - DEBUG - Processing node 69ab761defa461f55e59d43f: baseline-arm-clabbe 2026-03-09 00:49:35,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab761defa461f55e59d43f 2026-03-09 00:49:35,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:35,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab761defa461f55e59d43f HTTP/1.1" 200 None 2026-03-09 00:49:35,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:35,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:35,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:35,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab761defa461f55e59d43f HTTP/1.1" 200 None 2026-03-09 00:49:35,940 - lava_callback - ERROR - Error processing callback for node 69ab761defa461f55e59d43f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 00:49:36,479 - lava_callback - INFO - Received callback for node 69ab761defa461f55e59d442 INFO: 172.21.0.1:60200 - "POST /node/69ab761defa461f55e59d442 HTTP/1.0" 202 Accepted 2026-03-09 00:49:36,490 - lava_callback - INFO - Starting async job submit for node 69ab761defa461f55e59d442 2026-03-09 00:49:36,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:36,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab761defa461f55e59d442 HTTP/1.1" 200 None 2026-03-09 00:49:36,518 - lava_callback - DEBUG - Processing node 69ab761defa461f55e59d442: baseline-arm-clabbe 2026-03-09 00:49:36,519 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab761defa461f55e59d442 2026-03-09 00:49:36,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:60216 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:49:36,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab761defa461f55e59d442 HTTP/1.1" 200 None 2026-03-09 00:49:36,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:36,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:36,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:36,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab761defa461f55e59d442 HTTP/1.1" 200 None 2026-03-09 00:49:36,618 - lava_callback - ERROR - Error processing callback for node 69ab761defa461f55e59d442: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:49:55,824 - lava_callback - INFO - Received callback for node 69ab7621efa461f55e59d44e INFO: 172.21.0.1:34020 - "POST /node/69ab7621efa461f55e59d44e HTTP/1.0" 202 Accepted 2026-03-09 00:49:55,835 - lava_callback - INFO - Starting async job submit for node 69ab7621efa461f55e59d44e 2026-03-09 00:49:55,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7621efa461f55e59d44e HTTP/1.1" 200 None 2026-03-09 00:49:55,863 - lava_callback - DEBUG - Processing node 69ab7621efa461f55e59d44e: baseline-arm-clabbe 2026-03-09 00:49:55,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7621efa461f55e59d44e 2026-03-09 00:49:55,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:55,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7621efa461f55e59d44e HTTP/1.1" 200 None 2026-03-09 00:49:55,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:55,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 00:49:55,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:55,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7621efa461f55e59d44e HTTP/1.1" 200 None 2026-03-09 00:49:55,964 - lava_callback - ERROR - Error processing callback for node 69ab7621efa461f55e59d44e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34034 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:49:56,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:49:56,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40272 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:51:54,678 - lava_callback - INFO - Received callback for node 69ae198e8ef4424e487b382e INFO: 172.21.0.1:47598 - "POST /node/69ae198e8ef4424e487b382e HTTP/1.0" 202 Accepted 2026-03-09 00:51:54,684 - lava_callback - INFO - Starting async job submit for node 69ae198e8ef4424e487b382e 2026-03-09 00:51:54,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:51:54,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae198e8ef4424e487b382e HTTP/1.1" 200 None 2026-03-09 00:51:54,724 - lava_callback - DEBUG - Processing node 69ae198e8ef4424e487b382e: baseline-arm 2026-03-09 00:51:54,729 - lava_callback - WARNING - Failed to upload log 2026-03-09 00:51:54,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:51:54,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:51:54,899 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae198e8ef4424e487b382e/lava-logs.yaml 2026-03-09 00:51:54,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:51:55,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:51:55,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae198e8ef4424e487b382e/lava_callback.json.gz 2026-03-09 00:51:55,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:51:55,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae198e8ef4424e487b382e HTTP/1.1" 200 None 2026-03-09 00:51:55,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:51:55,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 00:51:55,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:51:55,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae198e8ef4424e487b382e HTTP/1.1" 200 None 2026-03-09 00:51:55,137 - lava_callback - INFO - Completed processing callback for node 69ae198e8ef4424e487b382e INFO: 172.21.0.1:47612 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:51:56,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:51:56,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47932 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:55:16,606 - lava_callback - INFO - Received callback for node 69ae1a578ef4424e487b395a INFO: 172.21.0.1:47940 - "POST /node/69ae1a578ef4424e487b395a HTTP/1.0" 202 Accepted 2026-03-09 00:55:16,616 - lava_callback - INFO - Starting async job submit for node 69ae1a578ef4424e487b395a 2026-03-09 00:55:16,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:16,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a578ef4424e487b395a HTTP/1.1" 200 None 2026-03-09 00:55:16,675 - lava_callback - DEBUG - Processing node 69ae1a578ef4424e487b395a: baseline-x86 2026-03-09 00:55:16,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:55:17,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:55:17,326 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a578ef4424e487b395a/log.txt.gz 2026-03-09 00:55:17,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:55:18,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:55:18,095 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a578ef4424e487b395a/lava-logs.yaml 2026-03-09 00:55:18,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:55:18,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:55:18,612 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a578ef4424e487b395a/lava_callback.json.gz 2026-03-09 00:55:18,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:18,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a578ef4424e487b395a HTTP/1.1" 200 None 2026-03-09 00:55:18,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:18,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 00:55:18,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:18,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a578ef4424e487b395a HTTP/1.1" 200 None 2026-03-09 00:55:18,837 - lava_callback - INFO - Completed processing callback for node 69ae1a578ef4424e487b395a INFO: 172.21.0.1:49414 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:55:26,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:26,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 00:55:28,309 - lava_callback - INFO - Received callback for node 69ae1a598ef4424e487b3963 INFO: 172.21.0.1:49426 - "POST /node/69ae1a598ef4424e487b3963 HTTP/1.0" 202 Accepted 2026-03-09 00:55:28,319 - lava_callback - INFO - Starting async job submit for node 69ae1a598ef4424e487b3963 2026-03-09 00:55:28,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:28,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a598ef4424e487b3963 HTTP/1.1" 200 None 2026-03-09 00:55:28,400 - lava_callback - DEBUG - Processing node 69ae1a598ef4424e487b3963: baseline-x86 2026-03-09 00:55:28,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:55:29,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:55:29,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a598ef4424e487b3963/log.txt.gz 2026-03-09 00:55:29,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:55:30,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:55:30,092 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a598ef4424e487b3963/lava-logs.yaml 2026-03-09 00:55:30,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:55:30,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:55:30,482 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a598ef4424e487b3963/lava_callback.json.gz 2026-03-09 00:55:30,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:30,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a598ef4424e487b3963 HTTP/1.1" 200 None 2026-03-09 00:55:30,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:30,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 00:55:30,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:30,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a598ef4424e487b3963 HTTP/1.1" 200 None 2026-03-09 00:55:30,634 - lava_callback - INFO - Completed processing callback for node 69ae1a598ef4424e487b3963 INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55904 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:55:56,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:55:56,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 00:56:01,101 - lava_callback - INFO - Received callback for node 69ae1a588ef4424e487b395d INFO: 172.21.0.1:35220 - "POST /node/69ae1a588ef4424e487b395d HTTP/1.0" 202 Accepted 2026-03-09 00:56:01,110 - lava_callback - INFO - Starting async job submit for node 69ae1a588ef4424e487b395d 2026-03-09 00:56:01,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:01,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a588ef4424e487b395d HTTP/1.1" 200 None 2026-03-09 00:56:01,185 - lava_callback - DEBUG - Processing node 69ae1a588ef4424e487b395d: baseline-x86 2026-03-09 00:56:01,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:56:02,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:56:02,245 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a588ef4424e487b395d/log.txt.gz 2026-03-09 00:56:02,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:56:03,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:56:03,453 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a588ef4424e487b395d/lava-logs.yaml 2026-03-09 00:56:03,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:56:04,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:56:04,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a588ef4424e487b395d/lava_callback.json.gz 2026-03-09 00:56:04,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:04,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a588ef4424e487b395d HTTP/1.1" 200 None 2026-03-09 00:56:04,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:04,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 00:56:04,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:04,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a588ef4424e487b395d HTTP/1.1" 200 None 2026-03-09 00:56:04,204 - lava_callback - INFO - Completed processing callback for node 69ae1a588ef4424e487b395d INFO: 172.21.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60198 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:56:26,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:26,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 00:56:30,642 - lava_callback - INFO - Received callback for node 69ae1a598ef4424e487b3960 INFO: 172.21.0.1:51398 - "POST /node/69ae1a598ef4424e487b3960 HTTP/1.0" 202 Accepted 2026-03-09 00:56:30,655 - lava_callback - INFO - Starting async job submit for node 69ae1a598ef4424e487b3960 2026-03-09 00:56:30,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:30,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a598ef4424e487b3960 HTTP/1.1" 200 None 2026-03-09 00:56:30,724 - lava_callback - DEBUG - Processing node 69ae1a598ef4424e487b3960: baseline-x86 2026-03-09 00:56:31,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:56:31,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:56:31,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a598ef4424e487b3960/log.txt.gz 2026-03-09 00:56:31,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:56:32,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:56:32,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a598ef4424e487b3960/lava-logs.yaml 2026-03-09 00:56:32,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 00:56:33,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 00:56:33,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a598ef4424e487b3960/lava_callback.json.gz 2026-03-09 00:56:33,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:33,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a598ef4424e487b3960 HTTP/1.1" 200 None 2026-03-09 00:56:33,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:33,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 00:56:33,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:33,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a598ef4424e487b3960 HTTP/1.1" 200 None 2026-03-09 00:56:33,480 - lava_callback - INFO - Completed processing callback for node 69ae1a598ef4424e487b3960 INFO: 172.21.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44296 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:56:56,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:56:56,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38628 - "GET / HTTP/1.0" 200 OK 2026-03-09 00:58:51,813 - lava_callback - INFO - Received callback for node 69ab783cefa461f55e59d6e1 INFO: 172.21.0.1:60428 - "POST /node/69ab783cefa461f55e59d6e1 HTTP/1.0" 202 Accepted 2026-03-09 00:58:51,818 - lava_callback - INFO - Starting async job submit for node 69ab783cefa461f55e59d6e1 2026-03-09 00:58:51,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:58:51,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab783cefa461f55e59d6e1 HTTP/1.1" 200 None 2026-03-09 00:58:51,835 - lava_callback - DEBUG - Processing node 69ab783cefa461f55e59d6e1: baseline-arm-baylibre 2026-03-09 00:58:51,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab783cefa461f55e59d6e1 2026-03-09 00:58:51,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:58:51,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab783cefa461f55e59d6e1 HTTP/1.1" 200 None 2026-03-09 00:58:51,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:58:51,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385666346680f7a28dbcc HTTP/1.1" 200 None 2026-03-09 00:58:51,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 00:58:51,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab783cefa461f55e59d6e1 HTTP/1.1" 200 None 2026-03-09 00:58:51,915 - lava_callback - ERROR - Error processing callback for node 69ab783cefa461f55e59d6e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56018 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:00:11,061 - lava_callback - INFO - Received callback for node 69ae17658ef4424e487b3429 INFO: 172.21.0.1:48222 - "POST /node/69ae17658ef4424e487b3429 HTTP/1.0" 202 Accepted 2026-03-09 01:00:11,068 - lava_callback - INFO - Starting async job submit for node 69ae17658ef4424e487b3429 2026-03-09 01:00:11,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:11,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17658ef4424e487b3429 HTTP/1.1" 200 None 2026-03-09 01:00:11,234 - lava_callback - DEBUG - Processing node 69ae17658ef4424e487b3429: baseline-arm 2026-03-09 01:00:11,850 - lava_callback - INFO - Received callback for node 69ab7886efa461f55e59d713 INFO: 172.21.0.1:48226 - "POST /node/69ab7886efa461f55e59d713 HTTP/1.0" 202 Accepted 2026-03-09 01:00:11,868 - lava_callback - INFO - Starting async job submit for node 69ab7886efa461f55e59d713 2026-03-09 01:00:11,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:11,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7886efa461f55e59d713 HTTP/1.1" 200 None 2026-03-09 01:00:11,951 - lava_callback - DEBUG - Processing node 69ab7886efa461f55e59d713: baseline-arm-baylibre 2026-03-09 01:00:11,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7886efa461f55e59d713 2026-03-09 01:00:11,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:12,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7886efa461f55e59d713 HTTP/1.1" 200 None 2026-03-09 01:00:12,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:12,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385666346680f7a28dbcc HTTP/1.1" 200 None 2026-03-09 01:00:12,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:12,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7886efa461f55e59d713 HTTP/1.1" 200 None 2026-03-09 01:00:12,242 - lava_callback - ERROR - Error processing callback for node 69ab7886efa461f55e59d713: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:00:12,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:00:12,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:00:12,762 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17658ef4424e487b3429/log.txt.gz 2026-03-09 01:00:13,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:00:13,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:00:13,635 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17658ef4424e487b3429/lava-logs.yaml 2026-03-09 01:00:13,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:00:14,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:00:14,179 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae17658ef4424e487b3429/lava_callback.json.gz DEBUG: Setup failed for 69ae17658ef4424e487b3429. Transitting job node result: incomplete -> fail 2026-03-09 01:00:14,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:14,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17658ef4424e487b3429 HTTP/1.1" 200 None 2026-03-09 01:00:14,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:14,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 01:00:14,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:14,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17658ef4424e487b3429 HTTP/1.1" 200 None 2026-03-09 01:00:14,518 - lava_callback - INFO - Completed processing callback for node 69ae17658ef4424e487b3429 INFO: 172.21.0.1:48228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53776 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:00:26,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:00:26,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58818 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:01:11,830 - lava_callback - INFO - Received callback for node 69ab78d1efa461f55e59d737 INFO: 172.21.0.1:50442 - "POST /node/69ab78d1efa461f55e59d737 HTTP/1.0" 202 Accepted 2026-03-09 01:01:11,836 - lava_callback - INFO - Starting async job submit for node 69ab78d1efa461f55e59d737 2026-03-09 01:01:11,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:01:11,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab78d1efa461f55e59d737 HTTP/1.1" 200 None 2026-03-09 01:01:11,852 - lava_callback - DEBUG - Processing node 69ab78d1efa461f55e59d737: baseline-arm-baylibre 2026-03-09 01:01:11,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab78d1efa461f55e59d737 2026-03-09 01:01:11,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:01:11,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab78d1efa461f55e59d737 HTTP/1.1" 200 None 2026-03-09 01:01:11,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:01:11,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385666346680f7a28dbcc HTTP/1.1" 200 None 2026-03-09 01:01:11,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:01:11,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab78d1efa461f55e59d737 HTTP/1.1" 200 None 2026-03-09 01:01:11,909 - lava_callback - ERROR - Error processing callback for node 69ab78d1efa461f55e59d737: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58890 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:02:04,025 - lava_callback - INFO - Received callback for node 69ae1c008ef4424e487b3bcf INFO: 172.21.0.1:37842 - "POST /node/69ae1c008ef4424e487b3bcf HTTP/1.0" 202 Accepted 2026-03-09 01:02:04,037 - lava_callback - INFO - Starting async job submit for node 69ae1c008ef4424e487b3bcf 2026-03-09 01:02:04,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:04,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1c008ef4424e487b3bcf HTTP/1.1" 200 None 2026-03-09 01:02:04,071 - lava_callback - DEBUG - Processing node 69ae1c008ef4424e487b3bcf: baseline-arm 2026-03-09 01:02:04,080 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:02:04,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:02:04,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:02:04,622 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae1c008ef4424e487b3bcf/lava-logs.yaml 2026-03-09 01:02:04,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:02:05,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:02:05,109 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae1c008ef4424e487b3bcf/lava_callback.json.gz 2026-03-09 01:02:05,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:05,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1c008ef4424e487b3bcf HTTP/1.1" 200 None 2026-03-09 01:02:05,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:05,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 01:02:05,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:05,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1c008ef4424e487b3bcf HTTP/1.1" 200 None 2026-03-09 01:02:05,224 - lava_callback - INFO - Completed processing callback for node 69ae1c008ef4424e487b3bcf INFO: 172.21.0.1:37850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:02:26,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:26,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:02:31,961 - lava_callback - INFO - Received callback for node 69ab791defa461f55e59d773 INFO: 172.21.0.1:43430 - "POST /node/69ab791defa461f55e59d773 HTTP/1.0" 202 Accepted 2026-03-09 01:02:31,970 - lava_callback - INFO - Starting async job submit for node 69ab791defa461f55e59d773 2026-03-09 01:02:31,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:32,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab791defa461f55e59d773 HTTP/1.1" 200 None 2026-03-09 01:02:32,001 - lava_callback - DEBUG - Processing node 69ab791defa461f55e59d773: baseline-arm-baylibre 2026-03-09 01:02:32,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab791defa461f55e59d773 2026-03-09 01:02:32,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:32,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab791defa461f55e59d773 HTTP/1.1" 200 None 2026-03-09 01:02:32,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:32,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385666346680f7a28dbcc HTTP/1.1" 200 None 2026-03-09 01:02:32,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:02:32,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab791defa461f55e59d773 HTTP/1.1" 200 None 2026-03-09 01:02:32,129 - lava_callback - ERROR - Error processing callback for node 69ab791defa461f55e59d773: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40010 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:03:52,065 - lava_callback - INFO - Received callback for node 69ab796aefa461f55e59d7a8 INFO: 172.21.0.1:41638 - "POST /node/69ab796aefa461f55e59d7a8 HTTP/1.0" 202 Accepted 2026-03-09 01:03:52,073 - lava_callback - INFO - Starting async job submit for node 69ab796aefa461f55e59d7a8 2026-03-09 01:03:52,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:03:52,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab796aefa461f55e59d7a8 HTTP/1.1" 200 None 2026-03-09 01:03:52,091 - lava_callback - DEBUG - Processing node 69ab796aefa461f55e59d7a8: baseline-arm-baylibre 2026-03-09 01:03:52,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab796aefa461f55e59d7a8 2026-03-09 01:03:52,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:03:52,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab796aefa461f55e59d7a8 HTTP/1.1" 200 None 2026-03-09 01:03:52,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:03:52,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385666346680f7a28dbcc HTTP/1.1" 200 None 2026-03-09 01:03:52,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:03:52,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab796aefa461f55e59d7a8 HTTP/1.1" 200 None 2026-03-09 01:03:52,148 - lava_callback - ERROR - Error processing callback for node 69ab796aefa461f55e59d7a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58518 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:05:32,280 - lava_callback - INFO - Received callback for node 69ab79daefa461f55e59d7f5 INFO: 172.21.0.1:50382 - "POST /node/69ab79daefa461f55e59d7f5 HTTP/1.0" 202 Accepted 2026-03-09 01:05:32,286 - lava_callback - INFO - Starting async job submit for node 69ab79daefa461f55e59d7f5 2026-03-09 01:05:32,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:05:32,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab79daefa461f55e59d7f5 HTTP/1.1" 200 None 2026-03-09 01:05:32,322 - lava_callback - DEBUG - Processing node 69ab79daefa461f55e59d7f5: baseline-arm-baylibre 2026-03-09 01:05:32,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab79daefa461f55e59d7f5 2026-03-09 01:05:32,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:05:32,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab79daefa461f55e59d7f5 HTTP/1.1" 200 None 2026-03-09 01:05:32,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:05:32,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385666346680f7a28dbcc HTTP/1.1" 200 None 2026-03-09 01:05:32,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:05:32,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab79daefa461f55e59d7f5 HTTP/1.1" 200 None 2026-03-09 01:05:32,388 - lava_callback - ERROR - Error processing callback for node 69ab79daefa461f55e59d7f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46218 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:09:12,354 - lava_callback - INFO - Received callback for node 69ab7ab1efa461f55e59d85e INFO: 172.21.0.1:53188 - "POST /node/69ab7ab1efa461f55e59d85e HTTP/1.0" 202 Accepted 2026-03-09 01:09:12,360 - lava_callback - INFO - Starting async job submit for node 69ab7ab1efa461f55e59d85e 2026-03-09 01:09:12,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:09:12,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7ab1efa461f55e59d85e HTTP/1.1" 200 None 2026-03-09 01:09:12,406 - lava_callback - DEBUG - Processing node 69ab7ab1efa461f55e59d85e: baseline-arm64-baylibre 2026-03-09 01:09:12,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7ab1efa461f55e59d85e 2026-03-09 01:09:12,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:09:12,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7ab1efa461f55e59d85e HTTP/1.1" 200 None 2026-03-09 01:09:12,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:09:12,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-09 01:09:12,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:09:12,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7ab1efa461f55e59d85e HTTP/1.1" 200 None 2026-03-09 01:09:12,536 - lava_callback - ERROR - Error processing callback for node 69ab7ab1efa461f55e59d85e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:10:36,834 - lava_callback - INFO - Received callback for node 69ab7b07efa461f55e59d887 INFO: 172.21.0.1:53580 - "POST /node/69ab7b07efa461f55e59d887 HTTP/1.0" 202 Accepted 2026-03-09 01:10:36,840 - lava_callback - INFO - Starting async job submit for node 69ab7b07efa461f55e59d887 2026-03-09 01:10:36,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:10:36,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7b07efa461f55e59d887 HTTP/1.1" 200 None 2026-03-09 01:10:36,856 - lava_callback - DEBUG - Processing node 69ab7b07efa461f55e59d887: baseline-arm64-clabbe 2026-03-09 01:10:36,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7b07efa461f55e59d887 2026-03-09 01:10:36,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:10:36,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7b07efa461f55e59d887 HTTP/1.1" 200 None 2026-03-09 01:10:36,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:10:36,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-09 01:10:36,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:10:36,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7b07efa461f55e59d887 HTTP/1.1" 200 None 2026-03-09 01:10:36,943 - lava_callback - ERROR - Error processing callback for node 69ab7b07efa461f55e59d887: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:11:56,844 - lava_callback - INFO - Received callback for node 69ab7b4befa461f55e59d8a1 INFO: 172.21.0.1:45824 - "POST /node/69ab7b4befa461f55e59d8a1 HTTP/1.0" 202 Accepted 2026-03-09 01:11:56,852 - lava_callback - INFO - Starting async job submit for node 69ab7b4befa461f55e59d8a1 2026-03-09 01:11:56,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:11:56,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7b4befa461f55e59d8a1 HTTP/1.1" 200 None 2026-03-09 01:11:56,879 - lava_callback - DEBUG - Processing node 69ab7b4befa461f55e59d8a1: baseline-arm-clabbe 2026-03-09 01:11:56,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7b4befa461f55e59d8a1 2026-03-09 01:11:56,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:11:56,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7b4befa461f55e59d8a1 HTTP/1.1" 200 None 2026-03-09 01:11:56,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:11:56,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-09 01:11:56,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:11:57,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7b4befa461f55e59d8a1 HTTP/1.1" 200 None 2026-03-09 01:11:57,025 - lava_callback - ERROR - Error processing callback for node 69ab7b4befa461f55e59d8a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46394 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:12:36,835 - lava_callback - INFO - Received callback for node 69ab7b72efa461f55e59d8b9 INFO: 172.21.0.1:46408 - "POST /node/69ab7b72efa461f55e59d8b9 HTTP/1.0" 202 Accepted 2026-03-09 01:12:36,842 - lava_callback - INFO - Starting async job submit for node 69ab7b72efa461f55e59d8b9 2026-03-09 01:12:36,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:36,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7b72efa461f55e59d8b9 HTTP/1.1" 200 None 2026-03-09 01:12:36,871 - lava_callback - DEBUG - Processing node 69ab7b72efa461f55e59d8b9: baseline-arm-clabbe 2026-03-09 01:12:36,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7b72efa461f55e59d8b9 2026-03-09 01:12:36,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:36,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7b72efa461f55e59d8b9 HTTP/1.1" 200 None 2026-03-09 01:12:36,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:36,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-09 01:12:36,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:36,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7b72efa461f55e59d8b9 HTTP/1.1" 200 None 2026-03-09 01:12:36,924 - lava_callback - ERROR - Error processing callback for node 69ab7b72efa461f55e59d8b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:12:38,469 - lava_callback - INFO - Received callback for node 69ae17878ef4424e487b34d5 INFO: 172.21.0.1:46410 - "POST /node/69ae17878ef4424e487b34d5 HTTP/1.0" 202 Accepted 2026-03-09 01:12:38,518 - lava_callback - INFO - Starting async job submit for node 69ae17878ef4424e487b34d5 2026-03-09 01:12:38,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:38,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17878ef4424e487b34d5 HTTP/1.1" 200 None 2026-03-09 01:12:38,572 - lava_callback - DEBUG - Processing node 69ae17878ef4424e487b34d5: baseline-arm-pengutronix 2026-03-09 01:12:38,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:12:39,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:12:39,046 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17878ef4424e487b34d5/log.txt.gz 2026-03-09 01:12:39,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:12:39,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:12:39,343 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17878ef4424e487b34d5/lava-logs.yaml 2026-03-09 01:12:39,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:12:39,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:12:39,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae17878ef4424e487b34d5/lava_callback.json.gz DEBUG: Setup failed for 69ae17878ef4424e487b34d5. Transitting job node result: incomplete -> fail 2026-03-09 01:12:39,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:39,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17878ef4424e487b34d5 HTTP/1.1" 200 None 2026-03-09 01:12:39,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:39,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 01:12:39,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:39,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17878ef4424e487b34d5 HTTP/1.1" 200 None 2026-03-09 01:12:39,671 - lava_callback - INFO - Completed processing callback for node 69ae17878ef4424e487b34d5 INFO: 172.21.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50652 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:12:56,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:12:56,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53744 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:14:19,018 - lava_callback - INFO - Received callback for node 69ae1ed88ef4424e487b3dfa INFO: 172.21.0.1:53760 - "POST /node/69ae1ed88ef4424e487b3dfa HTTP/1.0" 202 Accepted 2026-03-09 01:14:19,026 - lava_callback - INFO - Starting async job submit for node 69ae1ed88ef4424e487b3dfa 2026-03-09 01:14:19,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:19,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ed88ef4424e487b3dfa HTTP/1.1" 200 None 2026-03-09 01:14:19,057 - lava_callback - DEBUG - Processing node 69ae1ed88ef4424e487b3dfa: baseline-arm64 2026-03-09 01:14:19,063 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:14:19,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:14:19,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:14:19,218 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ed88ef4424e487b3dfa/lava-logs.yaml 2026-03-09 01:14:19,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:14:19,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:14:19,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ed88ef4424e487b3dfa/lava_callback.json.gz 2026-03-09 01:14:19,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:19,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ed88ef4424e487b3dfa HTTP/1.1" 200 None 2026-03-09 01:14:19,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:19,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:14:19,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:19,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ed88ef4424e487b3dfa HTTP/1.1" 200 None 2026-03-09 01:14:19,649 - lava_callback - INFO - Completed processing callback for node 69ae1ed88ef4424e487b3dfa INFO: 172.21.0.1:38194 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:14:26,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:26,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35880 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:14:51,778 - lava_callback - INFO - Received callback for node 69ae1eec8ef4424e487b3e5d INFO: 172.21.0.1:48680 - "POST /node/69ae1eec8ef4424e487b3e5d HTTP/1.0" 202 Accepted 2026-03-09 01:14:51,785 - lava_callback - INFO - Starting async job submit for node 69ae1eec8ef4424e487b3e5d 2026-03-09 01:14:51,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:51,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eec8ef4424e487b3e5d HTTP/1.1" 200 None 2026-03-09 01:14:51,829 - lava_callback - DEBUG - Processing node 69ae1eec8ef4424e487b3e5d: baseline-arm64 2026-03-09 01:14:52,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:14:52,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:14:52,469 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eec8ef4424e487b3e5d/log.txt.gz 2026-03-09 01:14:52,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:14:52,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:14:52,858 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eec8ef4424e487b3e5d/lava-logs.yaml 2026-03-09 01:14:52,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:14:53,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:14:53,211 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eec8ef4424e487b3e5d/lava_callback.json.gz 2026-03-09 01:14:53,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:53,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eec8ef4424e487b3e5d HTTP/1.1" 200 None 2026-03-09 01:14:53,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:53,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:14:53,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:53,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1eec8ef4424e487b3e5d HTTP/1.1" 200 None 2026-03-09 01:14:53,312 - lava_callback - INFO - Completed processing callback for node 69ae1eec8ef4424e487b3e5d 2026-03-09 01:14:55,942 - lava_callback - INFO - Received callback for node 69ae1ef88ef4424e487b3e7c INFO: 172.21.0.1:48696 - "POST /node/69ae1ef88ef4424e487b3e7c HTTP/1.0" 202 Accepted 2026-03-09 01:14:55,951 - lava_callback - INFO - Starting async job submit for node 69ae1ef88ef4424e487b3e7c 2026-03-09 01:14:55,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:55,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef88ef4424e487b3e7c HTTP/1.1" 200 None 2026-03-09 01:14:55,970 - lava_callback - DEBUG - Processing node 69ae1ef88ef4424e487b3e7c: baseline-nfs-arm64 2026-03-09 01:14:55,971 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:14:55,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:14:56,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:14:56,271 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef88ef4424e487b3e7c/lava-logs.yaml 2026-03-09 01:14:56,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:48712 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:14:56,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:14:56,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef88ef4424e487b3e7c/lava_callback.json.gz 2026-03-09 01:14:56,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:56,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef88ef4424e487b3e7c HTTP/1.1" 200 None 2026-03-09 01:14:56,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:56,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:14:56,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:56,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef88ef4424e487b3e7c HTTP/1.1" 200 None 2026-03-09 01:14:56,914 - lava_callback - INFO - Completed processing callback for node 69ae1ef88ef4424e487b3e7c 2026-03-09 01:14:56,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:14:57,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52036 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:15:06,754 - lava_callback - INFO - Received callback for node 69ae1ee58ef4424e487b3e4a INFO: 172.21.0.1:52038 - "POST /node/69ae1ee58ef4424e487b3e4a HTTP/1.0" 202 Accepted 2026-03-09 01:15:06,765 - lava_callback - INFO - Starting async job submit for node 69ae1ee58ef4424e487b3e4a 2026-03-09 01:15:06,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:06,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee58ef4424e487b3e4a HTTP/1.1" 200 None 2026-03-09 01:15:06,839 - lava_callback - DEBUG - Processing node 69ae1ee58ef4424e487b3e4a: baseline-arm64 2026-03-09 01:15:07,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:07,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:07,773 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee58ef4424e487b3e4a/log.txt.gz 2026-03-09 01:15:07,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:08,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:08,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee58ef4424e487b3e4a/lava-logs.yaml 2026-03-09 01:15:08,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:09,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:09,019 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee58ef4424e487b3e4a/lava_callback.json.gz 2026-03-09 01:15:09,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:09,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee58ef4424e487b3e4a HTTP/1.1" 200 None 2026-03-09 01:15:09,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:09,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:09,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ee58ef4424e487b3e4a HTTP/1.1" 200 None 2026-03-09 01:15:09,349 - lava_callback - INFO - Completed processing callback for node 69ae1ee58ef4424e487b3e4a INFO: 172.21.0.1:42216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:15:18,756 - lava_callback - INFO - Received callback for node 69ae1eea8ef4424e487b3e56 INFO: 172.21.0.1:42242 - "POST /node/69ae1eea8ef4424e487b3e56 HTTP/1.0" 202 Accepted 2026-03-09 01:15:18,769 - lava_callback - INFO - Starting async job submit for node 69ae1eea8ef4424e487b3e56 2026-03-09 01:15:18,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:18,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eea8ef4424e487b3e56 HTTP/1.1" 200 None 2026-03-09 01:15:18,825 - lava_callback - DEBUG - Processing node 69ae1eea8ef4424e487b3e56: baseline-arm64 2026-03-09 01:15:19,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:19,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:19,612 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eea8ef4424e487b3e56/log.txt.gz 2026-03-09 01:15:19,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:20,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:20,259 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eea8ef4424e487b3e56/lava-logs.yaml 2026-03-09 01:15:20,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:20,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:20,477 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eea8ef4424e487b3e56/lava_callback.json.gz 2026-03-09 01:15:20,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:20,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eea8ef4424e487b3e56 HTTP/1.1" 200 None 2026-03-09 01:15:20,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:20,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:20,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:20,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1eea8ef4424e487b3e56 HTTP/1.1" 200 None 2026-03-09 01:15:20,706 - lava_callback - INFO - Completed processing callback for node 69ae1eea8ef4424e487b3e56 2026-03-09 01:15:23,877 - lava_callback - INFO - Received callback for node 69ae1edc8ef4424e487b3e06 INFO: 172.21.0.1:48298 - "POST /node/69ae1edc8ef4424e487b3e06 HTTP/1.0" 202 Accepted 2026-03-09 01:15:23,910 - lava_callback - INFO - Starting async job submit for node 69ae1edc8ef4424e487b3e06 2026-03-09 01:15:24,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:24,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edc8ef4424e487b3e06 HTTP/1.1" 200 None 2026-03-09 01:15:24,034 - lava_callback - DEBUG - Processing node 69ae1edc8ef4424e487b3e06: baseline-arm64 2026-03-09 01:15:24,263 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:15:24,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:24,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:24,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edc8ef4424e487b3e06/lava-logs.yaml 2026-03-09 01:15:24,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:24,829 - lava_callback - INFO - Received callback for node 69ae1ef18ef4424e487b3e67 INFO: 172.21.0.1:48304 - "POST /node/69ae1ef18ef4424e487b3e67 HTTP/1.0" 202 Accepted 2026-03-09 01:15:24,844 - lava_callback - INFO - Starting async job submit for node 69ae1ef18ef4424e487b3e67 2026-03-09 01:15:24,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:24,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef18ef4424e487b3e67 HTTP/1.1" 200 None 2026-03-09 01:15:24,908 - lava_callback - DEBUG - Processing node 69ae1ef18ef4424e487b3e67: baseline-arm64 2026-03-09 01:15:25,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:25,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:25,434 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edc8ef4424e487b3e06/lava_callback.json.gz 2026-03-09 01:15:25,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:25,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edc8ef4424e487b3e06 HTTP/1.1" 200 None 2026-03-09 01:15:25,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:25,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:25,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:25,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1edc8ef4424e487b3e06 HTTP/1.1" 200 None 2026-03-09 01:15:25,700 - lava_callback - INFO - Completed processing callback for node 69ae1edc8ef4424e487b3e06 2026-03-09 01:15:25,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:25,711 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ef18ef4424e487b3e67/log.txt.gz 2026-03-09 01:15:25,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:26,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:26,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ef18ef4424e487b3e67/lava-logs.yaml 2026-03-09 01:15:26,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:48312 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:15:26,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:26,595 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ef18ef4424e487b3e67/lava_callback.json.gz 2026-03-09 01:15:26,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:26,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef18ef4424e487b3e67 HTTP/1.1" 200 None 2026-03-09 01:15:26,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:26,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:26,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:26,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef18ef4424e487b3e67 HTTP/1.1" 200 None 2026-03-09 01:15:26,764 - lava_callback - INFO - Completed processing callback for node 69ae1ef18ef4424e487b3e67 2026-03-09 01:15:27,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:27,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-09 01:15:27,860 - lava_callback - INFO - Received callback for node 69ae1eec8ef4424e487b3e5b INFO: 172.21.0.1:48316 - "POST /node/69ae1eec8ef4424e487b3e5b HTTP/1.0" 202 Accepted 2026-03-09 01:15:27,872 - lava_callback - INFO - Starting async job submit for node 69ae1eec8ef4424e487b3e5b 2026-03-09 01:15:27,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:27,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eec8ef4424e487b3e5b HTTP/1.1" 200 None 2026-03-09 01:15:27,940 - lava_callback - DEBUG - Processing node 69ae1eec8ef4424e487b3e5b: baseline-arm64 2026-03-09 01:15:28,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:28,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:28,763 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eec8ef4424e487b3e5b/log.txt.gz 2026-03-09 01:15:28,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:29,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:29,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eec8ef4424e487b3e5b/lava-logs.yaml 2026-03-09 01:15:29,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:29,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:29,597 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eec8ef4424e487b3e5b/lava_callback.json.gz 2026-03-09 01:15:29,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:29,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eec8ef4424e487b3e5b HTTP/1.1" 200 None 2026-03-09 01:15:29,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:29,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:29,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:29,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1eec8ef4424e487b3e5b HTTP/1.1" 200 None 2026-03-09 01:15:29,736 - lava_callback - INFO - Completed processing callback for node 69ae1eec8ef4424e487b3e5b 2026-03-09 01:15:34,584 - lava_callback - INFO - Received callback for node 69ae1eda8ef4424e487b3e00 INFO: 172.21.0.1:50362 - "POST /node/69ae1eda8ef4424e487b3e00 HTTP/1.0" 202 Accepted 2026-03-09 01:15:34,594 - lava_callback - INFO - Starting async job submit for node 69ae1eda8ef4424e487b3e00 2026-03-09 01:15:34,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:34,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eda8ef4424e487b3e00 HTTP/1.1" 200 None 2026-03-09 01:15:34,659 - lava_callback - DEBUG - Processing node 69ae1eda8ef4424e487b3e00: baseline-arm64 2026-03-09 01:15:35,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:35,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:35,580 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eda8ef4424e487b3e00/log.txt.gz 2026-03-09 01:15:35,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:36,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:36,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eda8ef4424e487b3e00/lava-logs.yaml 2026-03-09 01:15:36,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:36,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:36,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eda8ef4424e487b3e00/lava_callback.json.gz 2026-03-09 01:15:36,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50374 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:15:36,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eda8ef4424e487b3e00 HTTP/1.1" 200 None 2026-03-09 01:15:36,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:36,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:36,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:36,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1eda8ef4424e487b3e00 HTTP/1.1" 200 None 2026-03-09 01:15:36,570 - lava_callback - INFO - Completed processing callback for node 69ae1eda8ef4424e487b3e00 2026-03-09 01:15:38,386 - lava_callback - INFO - Received callback for node 69ae1ed98ef4424e487b3dfd INFO: 172.21.0.1:50388 - "POST /node/69ae1ed98ef4424e487b3dfd HTTP/1.0" 202 Accepted 2026-03-09 01:15:38,396 - lava_callback - INFO - Starting async job submit for node 69ae1ed98ef4424e487b3dfd 2026-03-09 01:15:38,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:38,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ed98ef4424e487b3dfd HTTP/1.1" 200 None 2026-03-09 01:15:38,471 - lava_callback - DEBUG - Processing node 69ae1ed98ef4424e487b3dfd: baseline-arm64 2026-03-09 01:15:38,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:39,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:39,448 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ed98ef4424e487b3dfd/log.txt.gz 2026-03-09 01:15:39,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:40,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:40,231 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ed98ef4424e487b3dfd/lava-logs.yaml 2026-03-09 01:15:40,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:40,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:40,429 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ed98ef4424e487b3dfd/lava_callback.json.gz 2026-03-09 01:15:40,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:40,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ed98ef4424e487b3dfd HTTP/1.1" 200 None 2026-03-09 01:15:40,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:40,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:40,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:40,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ed98ef4424e487b3dfd HTTP/1.1" 200 None 2026-03-09 01:15:40,526 - lava_callback - INFO - Completed processing callback for node 69ae1ed98ef4424e487b3dfd INFO: 172.21.0.1:37674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37686 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:15:53,585 - lava_callback - INFO - Received callback for node 69ae1edd8ef4424e487b3e09 INFO: 172.21.0.1:47224 - "POST /node/69ae1edd8ef4424e487b3e09 HTTP/1.0" 202 Accepted 2026-03-09 01:15:53,592 - lava_callback - INFO - Starting async job submit for node 69ae1edd8ef4424e487b3e09 2026-03-09 01:15:53,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:53,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edd8ef4424e487b3e09 HTTP/1.1" 200 None 2026-03-09 01:15:53,671 - lava_callback - DEBUG - Processing node 69ae1edd8ef4424e487b3e09: baseline-arm64 2026-03-09 01:15:53,919 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:15:54,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:54,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:54,285 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edd8ef4424e487b3e09/lava-logs.yaml 2026-03-09 01:15:54,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:15:54,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:15:54,464 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edd8ef4424e487b3e09/lava_callback.json.gz 2026-03-09 01:15:54,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:54,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edd8ef4424e487b3e09 HTTP/1.1" 200 None 2026-03-09 01:15:54,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:54,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:15:54,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:54,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1edd8ef4424e487b3e09 HTTP/1.1" 200 None 2026-03-09 01:15:54,736 - lava_callback - INFO - Completed processing callback for node 69ae1edd8ef4424e487b3e09 INFO: 172.21.0.1:47240 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:15:57,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:15:57,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50408 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:16:21,084 - lava_callback - INFO - Received callback for node 69ae1efe8ef4424e487b3eb0 INFO: 172.21.0.1:40856 - "POST /node/69ae1efe8ef4424e487b3eb0 HTTP/1.0" 202 Accepted 2026-03-09 01:16:21,091 - lava_callback - INFO - Starting async job submit for node 69ae1efe8ef4424e487b3eb0 2026-03-09 01:16:21,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:21,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1efe8ef4424e487b3eb0 HTTP/1.1" 200 None 2026-03-09 01:16:21,132 - lava_callback - DEBUG - Processing node 69ae1efe8ef4424e487b3eb0: baseline-nfs-arm64 2026-03-09 01:16:21,379 - lava_callback - INFO - Received callback for node 69ae1f288ef4424e487b3f33 INFO: 172.21.0.1:40866 - "POST /node/69ae1f288ef4424e487b3f33 HTTP/1.0" 202 Accepted 2026-03-09 01:16:21,402 - lava_callback - INFO - Starting async job submit for node 69ae1f288ef4424e487b3f33 2026-03-09 01:16:21,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:21,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f288ef4424e487b3f33 HTTP/1.1" 200 None 2026-03-09 01:16:21,554 - lava_callback - DEBUG - Processing node 69ae1f288ef4424e487b3f33: baseline-arm64-pengutronix 2026-03-09 01:16:21,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:22,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:22,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:22,442 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f288ef4424e487b3f33/log.txt.gz 2026-03-09 01:16:22,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:22,490 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efe8ef4424e487b3eb0/log.txt.gz 2026-03-09 01:16:22,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:22,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:23,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:23,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f288ef4424e487b3f33/lava-logs.yaml 2026-03-09 01:16:23,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:23,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:23,330 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efe8ef4424e487b3eb0/lava-logs.yaml 2026-03-09 01:16:23,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:23,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:23,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f288ef4424e487b3f33/lava_callback.json.gz 2026-03-09 01:16:23,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:23,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:23,891 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efe8ef4424e487b3eb0/lava_callback.json.gz 2026-03-09 01:16:23,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f288ef4424e487b3f33 HTTP/1.1" 200 None 2026-03-09 01:16:23,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:23,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:23,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:16:23,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:24,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1efe8ef4424e487b3eb0 HTTP/1.1" 200 None 2026-03-09 01:16:24,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:24,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f288ef4424e487b3f33 HTTP/1.1" 200 None 2026-03-09 01:16:24,037 - lava_callback - INFO - Completed processing callback for node 69ae1f288ef4424e487b3f33 2026-03-09 01:16:24,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:16:24,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:24,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1efe8ef4424e487b3eb0 HTTP/1.1" 200 None 2026-03-09 01:16:24,150 - lava_callback - INFO - Completed processing callback for node 69ae1efe8ef4424e487b3eb0 INFO: 172.21.0.1:40878 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:16:27,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:27,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:16:27,875 - lava_callback - INFO - Received callback for node 69ae1f008ef4424e487b3eb3 INFO: 172.21.0.1:40890 - "POST /node/69ae1f008ef4424e487b3eb3 HTTP/1.0" 202 Accepted 2026-03-09 01:16:27,887 - lava_callback - INFO - Starting async job submit for node 69ae1f008ef4424e487b3eb3 2026-03-09 01:16:27,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f008ef4424e487b3eb3 HTTP/1.1" 200 None 2026-03-09 01:16:27,955 - lava_callback - DEBUG - Processing node 69ae1f008ef4424e487b3eb3: baseline-nfs-arm64 2026-03-09 01:16:28,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:28,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:28,594 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1f008ef4424e487b3eb3/log.txt.gz 2026-03-09 01:16:28,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:29,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:29,038 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1f008ef4424e487b3eb3/lava-logs.yaml 2026-03-09 01:16:29,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:29,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:29,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1f008ef4424e487b3eb3/lava_callback.json.gz 2026-03-09 01:16:29,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:29,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f008ef4424e487b3eb3 HTTP/1.1" 200 None 2026-03-09 01:16:29,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:29,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:16:29,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:29,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f008ef4424e487b3eb3 HTTP/1.1" 200 None 2026-03-09 01:16:29,285 - lava_callback - INFO - Completed processing callback for node 69ae1f008ef4424e487b3eb3 2026-03-09 01:16:31,432 - lava_callback - INFO - Received callback for node 69ae1f2a8ef4424e487b3f36 INFO: 172.21.0.1:46962 - "POST /node/69ae1f2a8ef4424e487b3f36 HTTP/1.0" 202 Accepted 2026-03-09 01:16:31,438 - lava_callback - INFO - Starting async job submit for node 69ae1f2a8ef4424e487b3f36 2026-03-09 01:16:31,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:31,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f2a8ef4424e487b3f36 HTTP/1.1" 200 None 2026-03-09 01:16:31,501 - lava_callback - DEBUG - Processing node 69ae1f2a8ef4424e487b3f36: baseline-arm64-pengutronix 2026-03-09 01:16:31,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:31,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:31,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f2a8ef4424e487b3f36/log.txt.gz 2026-03-09 01:16:31,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:32,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:32,156 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f2a8ef4424e487b3f36/lava-logs.yaml 2026-03-09 01:16:32,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:32,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:32,292 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f2a8ef4424e487b3f36/lava_callback.json.gz 2026-03-09 01:16:32,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:32,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f2a8ef4424e487b3f36 HTTP/1.1" 200 None 2026-03-09 01:16:32,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:32,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:16:32,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:32,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f2a8ef4424e487b3f36 HTTP/1.1" 200 None 2026-03-09 01:16:32,376 - lava_callback - INFO - Completed processing callback for node 69ae1f2a8ef4424e487b3f36 INFO: 172.21.0.1:46968 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:16:41,292 - lava_callback - INFO - Received callback for node 69ae1f2b8ef4424e487b3f39 INFO: 172.21.0.1:47078 - "POST /node/69ae1f2b8ef4424e487b3f39 HTTP/1.0" 202 Accepted 2026-03-09 01:16:41,299 - lava_callback - INFO - Starting async job submit for node 69ae1f2b8ef4424e487b3f39 2026-03-09 01:16:41,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:41,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f2b8ef4424e487b3f39 HTTP/1.1" 200 None 2026-03-09 01:16:41,334 - lava_callback - DEBUG - Processing node 69ae1f2b8ef4424e487b3f39: baseline-arm64-pengutronix 2026-03-09 01:16:41,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:41,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:41,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f2b8ef4424e487b3f39/log.txt.gz 2026-03-09 01:16:41,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:16:42,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:42,077 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f2b8ef4424e487b3f39/lava-logs.yaml 2026-03-09 01:16:42,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:47082 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:16:42,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:16:42,260 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69ae1f2b8ef4424e487b3f39/lava_callback.json.gz 2026-03-09 01:16:42,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:42,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f2b8ef4424e487b3f39 HTTP/1.1" 200 None 2026-03-09 01:16:42,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:42,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:16:42,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:42,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f2b8ef4424e487b3f39 HTTP/1.1" 200 None 2026-03-09 01:16:42,364 - lava_callback - INFO - Completed processing callback for node 69ae1f2b8ef4424e487b3f39 INFO: 172.21.0.1:47084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50384 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:16:57,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:16:57,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48528 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 01:17:16,052 - lava_callback - INFO - Received callback for node 69ae1f248ef4424e487b3f24 INFO: 172.21.0.1:48536 - "POST /node/69ae1f248ef4424e487b3f24 HTTP/1.0" 202 Accepted 2026-03-09 01:17:16,058 - lava_callback - INFO - Starting async job submit for node 69ae1f248ef4424e487b3f24 2026-03-09 01:17:16,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:17:16,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f248ef4424e487b3f24 HTTP/1.1" 200 None 2026-03-09 01:17:16,098 - lava_callback - DEBUG - Processing node 69ae1f248ef4424e487b3f24: baseline-arm64-clabbe 2026-03-09 01:17:16,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:17:16,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:17:16,470 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69ae1f248ef4424e487b3f24/log.txt.gz INFO: 172.21.0.1:48542 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:17:16,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:17:16,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:17:16,808 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69ae1f248ef4424e487b3f24/lava-logs.yaml 2026-03-09 01:17:16,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:17:16,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:17:16,968 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69ae1f248ef4424e487b3f24/lava_callback.json.gz 2026-03-09 01:17:16,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:17:17,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f248ef4424e487b3f24 HTTP/1.1" 200 None 2026-03-09 01:17:17,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:17:17,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:17:17,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:17:17,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f248ef4424e487b3f24 HTTP/1.1" 200 None 2026-03-09 01:17:17,082 - lava_callback - INFO - Completed processing callback for node 69ae1f248ef4424e487b3f24 INFO: 172.21.0.1:60756 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:17:27,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:17:27,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47966 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:18:01,513 - lava_callback - INFO - Received callback for node 69ae1fa18ef4424e487b3fcd INFO: 172.21.0.1:59770 - "POST /node/69ae1fa18ef4424e487b3fcd HTTP/1.0" 202 Accepted 2026-03-09 01:18:01,519 - lava_callback - INFO - Starting async job submit for node 69ae1fa18ef4424e487b3fcd 2026-03-09 01:18:01,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:18:01,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1fa18ef4424e487b3fcd HTTP/1.1" 200 None 2026-03-09 01:18:01,537 - lava_callback - DEBUG - Processing node 69ae1fa18ef4424e487b3fcd: baseline-nfs-arm64 2026-03-09 01:18:01,538 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:18:01,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:18:01,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:18:01,706 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1fa18ef4424e487b3fcd/lava-logs.yaml 2026-03-09 01:18:01,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:18:01,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:18:01,861 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1fa18ef4424e487b3fcd/lava_callback.json.gz 2026-03-09 01:18:01,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:18:01,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1fa18ef4424e487b3fcd HTTP/1.1" 200 None 2026-03-09 01:18:01,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:18:01,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:18:01,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:18:01,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1fa18ef4424e487b3fcd HTTP/1.1" 200 None 2026-03-09 01:18:01,944 - lava_callback - INFO - Completed processing callback for node 69ae1fa18ef4424e487b3fcd INFO: 172.21.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49394 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:18:27,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:18:27,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45710 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:19:12,949 - lava_callback - INFO - Received callback for node 69ab7d02efa461f55e59d9db INFO: 172.21.0.1:43828 - "POST /node/69ab7d02efa461f55e59d9db HTTP/1.0" 202 Accepted 2026-03-09 01:19:12,955 - lava_callback - INFO - Starting async job submit for node 69ab7d02efa461f55e59d9db 2026-03-09 01:19:12,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:12,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d02efa461f55e59d9db HTTP/1.1" 200 None 2026-03-09 01:19:12,973 - lava_callback - DEBUG - Processing node 69ab7d02efa461f55e59d9db: baseline-arm64-baylibre 2026-03-09 01:19:12,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d02efa461f55e59d9db 2026-03-09 01:19:12,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:12,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d02efa461f55e59d9db HTTP/1.1" 200 None 2026-03-09 01:19:12,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:13,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:13,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:13,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d02efa461f55e59d9db HTTP/1.1" 200 None 2026-03-09 01:19:13,031 - lava_callback - ERROR - Error processing callback for node 69ab7d02efa461f55e59d9db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:19:13,610 - lava_callback - INFO - Received callback for node 69ab7d04efa461f55e59d9de INFO: 172.21.0.1:43834 - "POST /node/69ab7d04efa461f55e59d9de HTTP/1.0" 202 Accepted 2026-03-09 01:19:13,617 - lava_callback - INFO - Starting async job submit for node 69ab7d04efa461f55e59d9de 2026-03-09 01:19:13,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:13,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d04efa461f55e59d9de HTTP/1.1" 200 None 2026-03-09 01:19:13,634 - lava_callback - DEBUG - Processing node 69ab7d04efa461f55e59d9de: baseline-arm64-baylibre 2026-03-09 01:19:13,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d04efa461f55e59d9de 2026-03-09 01:19:13,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:13,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d04efa461f55e59d9de HTTP/1.1" 200 None 2026-03-09 01:19:13,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:13,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:13,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:13,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d04efa461f55e59d9de HTTP/1.1" 200 None 2026-03-09 01:19:13,702 - lava_callback - ERROR - Error processing callback for node 69ab7d04efa461f55e59d9de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:19:14,292 - lava_callback - INFO - Received callback for node 69ab7d05efa461f55e59d9e1 INFO: 172.21.0.1:43840 - "POST /node/69ab7d05efa461f55e59d9e1 HTTP/1.0" 202 Accepted 2026-03-09 01:19:14,298 - lava_callback - INFO - Starting async job submit for node 69ab7d05efa461f55e59d9e1 2026-03-09 01:19:14,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:14,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d05efa461f55e59d9e1 HTTP/1.1" 200 None 2026-03-09 01:19:14,316 - lava_callback - DEBUG - Processing node 69ab7d05efa461f55e59d9e1: baseline-arm64-baylibre 2026-03-09 01:19:14,316 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d05efa461f55e59d9e1 2026-03-09 01:19:14,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:14,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d05efa461f55e59d9e1 HTTP/1.1" 200 None 2026-03-09 01:19:14,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:14,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:14,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:14,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d05efa461f55e59d9e1 HTTP/1.1" 200 None 2026-03-09 01:19:14,377 - lava_callback - ERROR - Error processing callback for node 69ab7d05efa461f55e59d9e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:19:14,954 - lava_callback - INFO - Received callback for node 69ab7d07efa461f55e59d9e8 INFO: 172.21.0.1:43842 - "POST /node/69ab7d07efa461f55e59d9e8 HTTP/1.0" 202 Accepted 2026-03-09 01:19:14,959 - lava_callback - INFO - Starting async job submit for node 69ab7d07efa461f55e59d9e8 2026-03-09 01:19:14,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:14,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d07efa461f55e59d9e8 HTTP/1.1" 200 None 2026-03-09 01:19:14,975 - lava_callback - DEBUG - Processing node 69ab7d07efa461f55e59d9e8: baseline-arm64-baylibre 2026-03-09 01:19:14,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d07efa461f55e59d9e8 2026-03-09 01:19:14,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:14,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d07efa461f55e59d9e8 HTTP/1.1" 200 None 2026-03-09 01:19:14,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:15,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:15,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:15,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d07efa461f55e59d9e8 HTTP/1.1" 200 None 2026-03-09 01:19:15,030 - lava_callback - ERROR - Error processing callback for node 69ab7d07efa461f55e59d9e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43848 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 01:19:15,622 - lava_callback - INFO - Received callback for node 69ab7d09efa461f55e59d9eb INFO: 172.21.0.1:43864 - "POST /node/69ab7d09efa461f55e59d9eb HTTP/1.0" 202 Accepted 2026-03-09 01:19:15,628 - lava_callback - INFO - Starting async job submit for node 69ab7d09efa461f55e59d9eb 2026-03-09 01:19:15,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:15,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d09efa461f55e59d9eb HTTP/1.1" 200 None 2026-03-09 01:19:15,646 - lava_callback - DEBUG - Processing node 69ab7d09efa461f55e59d9eb: baseline-arm64-baylibre 2026-03-09 01:19:15,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d09efa461f55e59d9eb 2026-03-09 01:19:15,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:15,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d09efa461f55e59d9eb HTTP/1.1" 200 None 2026-03-09 01:19:15,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:15,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:15,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:15,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d09efa461f55e59d9eb HTTP/1.1" 200 None 2026-03-09 01:19:15,704 - lava_callback - ERROR - Error processing callback for node 69ab7d09efa461f55e59d9eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:19:16,285 - lava_callback - INFO - Received callback for node 69ab7d0aefa461f55e59d9ee INFO: 172.21.0.1:43868 - "POST /node/69ab7d0aefa461f55e59d9ee HTTP/1.0" 202 Accepted 2026-03-09 01:19:16,291 - lava_callback - INFO - Starting async job submit for node 69ab7d0aefa461f55e59d9ee 2026-03-09 01:19:16,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:16,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d0aefa461f55e59d9ee HTTP/1.1" 200 None 2026-03-09 01:19:16,311 - lava_callback - DEBUG - Processing node 69ab7d0aefa461f55e59d9ee: baseline-arm64-baylibre 2026-03-09 01:19:16,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d0aefa461f55e59d9ee 2026-03-09 01:19:16,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:16,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d0aefa461f55e59d9ee HTTP/1.1" 200 None 2026-03-09 01:19:16,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:16,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:16,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:16,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d0aefa461f55e59d9ee HTTP/1.1" 200 None 2026-03-09 01:19:16,375 - lava_callback - ERROR - Error processing callback for node 69ab7d0aefa461f55e59d9ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:19:21,781 - lava_callback - INFO - Received callback for node 69ae1fed8ef4424e487b3ffe INFO: 172.21.0.1:53144 - "POST /node/69ae1fed8ef4424e487b3ffe HTTP/1.0" 202 Accepted 2026-03-09 01:19:21,788 - lava_callback - INFO - Starting async job submit for node 69ae1fed8ef4424e487b3ffe 2026-03-09 01:19:21,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:21,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1fed8ef4424e487b3ffe HTTP/1.1" 200 None 2026-03-09 01:19:21,835 - lava_callback - DEBUG - Processing node 69ae1fed8ef4424e487b3ffe: baseline-arm64 2026-03-09 01:19:21,984 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:19:22,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:19:22,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:19:22,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1fed8ef4424e487b3ffe/lava-logs.yaml 2026-03-09 01:19:22,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:19:22,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:19:22,557 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1fed8ef4424e487b3ffe/lava_callback.json.gz 2026-03-09 01:19:22,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:22,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1fed8ef4424e487b3ffe HTTP/1.1" 200 None 2026-03-09 01:19:22,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:22,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:19:22,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:22,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1fed8ef4424e487b3ffe HTTP/1.1" 200 None 2026-03-09 01:19:22,730 - lava_callback - INFO - Completed processing callback for node 69ae1fed8ef4424e487b3ffe INFO: 172.21.0.1:53146 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:19:27,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:27,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60690 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:19:37,180 - lava_callback - INFO - Received callback for node 69ab7d1cefa461f55e59da38 INFO: 172.21.0.1:60704 - "POST /node/69ab7d1cefa461f55e59da38 HTTP/1.0" 202 Accepted 2026-03-09 01:19:37,186 - lava_callback - INFO - Starting async job submit for node 69ab7d1cefa461f55e59da38 2026-03-09 01:19:37,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d1cefa461f55e59da38 HTTP/1.1" 200 None 2026-03-09 01:19:37,203 - lava_callback - DEBUG - Processing node 69ab7d1cefa461f55e59da38: baseline-arm64-clabbe 2026-03-09 01:19:37,203 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d1cefa461f55e59da38 2026-03-09 01:19:37,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d1cefa461f55e59da38 HTTP/1.1" 200 None 2026-03-09 01:19:37,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:37,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d1cefa461f55e59da38 HTTP/1.1" 200 None 2026-03-09 01:19:37,264 - lava_callback - ERROR - Error processing callback for node 69ab7d1cefa461f55e59da38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:19:37,853 - lava_callback - INFO - Received callback for node 69ab7d22efa461f55e59da47 INFO: 172.21.0.1:60718 - "POST /node/69ab7d22efa461f55e59da47 HTTP/1.0" 202 Accepted 2026-03-09 01:19:37,859 - lava_callback - INFO - Starting async job submit for node 69ab7d22efa461f55e59da47 2026-03-09 01:19:37,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d22efa461f55e59da47 HTTP/1.1" 200 None 2026-03-09 01:19:37,875 - lava_callback - DEBUG - Processing node 69ab7d22efa461f55e59da47: baseline-arm64-clabbe 2026-03-09 01:19:37,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7d22efa461f55e59da47 2026-03-09 01:19:37,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7d22efa461f55e59da47 HTTP/1.1" 200 None 2026-03-09 01:19:37,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 01:19:37,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:19:37,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7d22efa461f55e59da47 HTTP/1.1" 200 None 2026-03-09 01:19:37,929 - lava_callback - ERROR - Error processing callback for node 69ab7d22efa461f55e59da47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:20:10,575 - lava_callback - INFO - Received callback for node 69ae1edf8ef4424e487b3e3d INFO: 172.21.0.1:37648 - "POST /node/69ae1edf8ef4424e487b3e3d HTTP/1.0" 202 Accepted 2026-03-09 01:20:10,581 - lava_callback - INFO - Starting async job submit for node 69ae1edf8ef4424e487b3e3d 2026-03-09 01:20:10,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:10,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edf8ef4424e487b3e3d HTTP/1.1" 200 None 2026-03-09 01:20:10,619 - lava_callback - DEBUG - Processing node 69ae1edf8ef4424e487b3e3d: baseline-arm64 2026-03-09 01:20:10,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:20:10,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:20:10,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edf8ef4424e487b3e3d/log.txt.gz 2026-03-09 01:20:11,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:20:11,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:20:11,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edf8ef4424e487b3e3d/lava-logs.yaml 2026-03-09 01:20:11,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:20:11,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:20:11,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edf8ef4424e487b3e3d/lava_callback.json.gz 2026-03-09 01:20:11,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:11,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edf8ef4424e487b3e3d HTTP/1.1" 200 None 2026-03-09 01:20:11,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:11,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:20:11,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:11,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1edf8ef4424e487b3e3d HTTP/1.1" 200 None 2026-03-09 01:20:11,646 - lava_callback - INFO - Completed processing callback for node 69ae1edf8ef4424e487b3e3d INFO: 172.21.0.1:37660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52784 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:20:27,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:27,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:20:35,015 - lava_callback - INFO - Received callback for node 69ae1ef28ef4424e487b3e6d INFO: 172.21.0.1:50802 - "POST /node/69ae1ef28ef4424e487b3e6d HTTP/1.0" 202 Accepted 2026-03-09 01:20:35,023 - lava_callback - INFO - Starting async job submit for node 69ae1ef28ef4424e487b3e6d 2026-03-09 01:20:35,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:35,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef28ef4424e487b3e6d HTTP/1.1" 200 None 2026-03-09 01:20:35,063 - lava_callback - DEBUG - Processing node 69ae1ef28ef4424e487b3e6d: baseline-nfs-arm64 2026-03-09 01:20:35,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:20:35,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:20:35,675 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef28ef4424e487b3e6d/log.txt.gz 2026-03-09 01:20:35,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:20:36,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:20:36,106 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef28ef4424e487b3e6d/lava-logs.yaml 2026-03-09 01:20:36,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:20:36,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:20:36,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef28ef4424e487b3e6d/lava_callback.json.gz 2026-03-09 01:20:36,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:36,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef28ef4424e487b3e6d HTTP/1.1" 200 None 2026-03-09 01:20:36,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:36,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:20:36,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:36,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef28ef4424e487b3e6d HTTP/1.1" 200 None 2026-03-09 01:20:36,377 - lava_callback - INFO - Completed processing callback for node 69ae1ef28ef4424e487b3e6d INFO: 172.21.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49400 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:20:57,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:20:57,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57432 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:21:42,178 - lava_callback - INFO - Received callback for node 69ae20428ef4424e487b4028 INFO: 172.21.0.1:38558 - "POST /node/69ae20428ef4424e487b4028 HTTP/1.0" 202 Accepted 2026-03-09 01:21:42,185 - lava_callback - INFO - Starting async job submit for node 69ae20428ef4424e487b4028 2026-03-09 01:21:42,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:38562 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:21:42,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae20428ef4424e487b4028 HTTP/1.1" 200 None 2026-03-09 01:21:42,230 - lava_callback - DEBUG - Processing node 69ae20428ef4424e487b4028: baseline-arm64 2026-03-09 01:21:42,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:21:42,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:21:42,591 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae20428ef4424e487b4028/log.txt.gz 2026-03-09 01:21:42,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:21:42,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:21:42,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae20428ef4424e487b4028/lava-logs.yaml 2026-03-09 01:21:42,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:21:43,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:21:43,153 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae20428ef4424e487b4028/lava_callback.json.gz 2026-03-09 01:21:43,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:21:43,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae20428ef4424e487b4028 HTTP/1.1" 200 None 2026-03-09 01:21:43,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:21:43,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:21:43,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:21:43,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae20428ef4424e487b4028 HTTP/1.1" 200 None 2026-03-09 01:21:43,383 - lava_callback - INFO - Completed processing callback for node 69ae20428ef4424e487b4028 INFO: 172.21.0.1:38566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:21:57,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:21:57,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44370 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:22:20,248 - lava_callback - INFO - Received callback for node 69ae1eed8ef4424e487b3e61 INFO: 172.21.0.1:44442 - "POST /node/69ae1eed8ef4424e487b3e61 HTTP/1.0" 202 Accepted 2026-03-09 01:22:20,255 - lava_callback - INFO - Starting async job submit for node 69ae1eed8ef4424e487b3e61 2026-03-09 01:22:20,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:22:20,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eed8ef4424e487b3e61 HTTP/1.1" 200 None 2026-03-09 01:22:20,308 - lava_callback - DEBUG - Processing node 69ae1eed8ef4424e487b3e61: baseline-arm64 2026-03-09 01:22:20,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:22:20,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:22:20,845 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eed8ef4424e487b3e61/log.txt.gz 2026-03-09 01:22:21,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:22:21,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:22:21,316 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eed8ef4424e487b3e61/lava-logs.yaml 2026-03-09 01:22:21,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:22:21,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:22:21,514 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1eed8ef4424e487b3e61/lava_callback.json.gz DEBUG: Setup failed for 69ae1eed8ef4424e487b3e61. Transitting job node result: incomplete -> fail 2026-03-09 01:22:21,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:22:21,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eed8ef4424e487b3e61 HTTP/1.1" 200 None 2026-03-09 01:22:21,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:22:21,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:22:21,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:22:21,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1eed8ef4424e487b3e61 HTTP/1.1" 200 None 2026-03-09 01:22:21,711 - lava_callback - INFO - Completed processing callback for node 69ae1eed8ef4424e487b3e61 INFO: 172.21.0.1:44444 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:22:27,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:22:27,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47084 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:23:16,654 - lava_callback - INFO - Received callback for node 69ae20f38ef4424e487b4095 INFO: 172.21.0.1:47088 - "POST /node/69ae20f38ef4424e487b4095 HTTP/1.0" 202 Accepted 2026-03-09 01:23:16,659 - lava_callback - INFO - Starting async job submit for node 69ae20f38ef4424e487b4095 2026-03-09 01:23:16,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:23:16,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae20f38ef4424e487b4095 HTTP/1.1" 200 None 2026-03-09 01:23:16,677 - lava_callback - DEBUG - Processing node 69ae20f38ef4424e487b4095: baseline-nfs-arm64 2026-03-09 01:23:16,678 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:23:16,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:23:16,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:23:16,832 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae20f38ef4424e487b4095/lava-logs.yaml 2026-03-09 01:23:16,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:23:16,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:23:16,970 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae20f38ef4424e487b4095/lava_callback.json.gz 2026-03-09 01:23:16,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:23:16,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae20f38ef4424e487b4095 HTTP/1.1" 200 None 2026-03-09 01:23:16,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:23:17,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:23:17,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:23:17,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae20f38ef4424e487b4095 HTTP/1.1" 200 None 2026-03-09 01:23:17,048 - lava_callback - INFO - Completed processing callback for node 69ae20f38ef4424e487b4095 INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:23:27,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:23:27,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45754 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:26:11,586 - lava_callback - INFO - Received callback for node 69ae1ce78ef4424e487b3cce INFO: 172.21.0.1:40902 - "POST /node/69ae1ce78ef4424e487b3cce HTTP/1.0" 202 Accepted 2026-03-09 01:26:11,593 - lava_callback - INFO - Starting async job submit for node 69ae1ce78ef4424e487b3cce 2026-03-09 01:26:11,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:26:11,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ce78ef4424e487b3cce HTTP/1.1" 200 None 2026-03-09 01:26:11,664 - lava_callback - DEBUG - Processing node 69ae1ce78ef4424e487b3cce: baseline-arm 2026-03-09 01:26:12,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:26:12,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:26:12,364 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae1ce78ef4424e487b3cce/log.txt.gz 2026-03-09 01:26:12,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:26:12,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:26:12,944 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae1ce78ef4424e487b3cce/lava-logs.yaml 2026-03-09 01:26:12,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:26:13,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:26:13,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae1ce78ef4424e487b3cce/lava_callback.json.gz DEBUG: Setup failed for 69ae1ce78ef4424e487b3cce. Transitting job node result: incomplete -> fail 2026-03-09 01:26:13,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:26:13,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ce78ef4424e487b3cce HTTP/1.1" 200 None 2026-03-09 01:26:13,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:26:13,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 01:26:13,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:26:13,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ce78ef4424e487b3cce HTTP/1.1" 200 None 2026-03-09 01:26:13,433 - lava_callback - INFO - Completed processing callback for node 69ae1ce78ef4424e487b3cce INFO: 172.21.0.1:40904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:26:27,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:26:27,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55914 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:29:45,507 - lava_callback - INFO - Received callback for node 69ae1ee68ef4424e487b3e4d INFO: 172.21.0.1:55922 - "POST /node/69ae1ee68ef4424e487b3e4d HTTP/1.0" 202 Accepted 2026-03-09 01:29:45,513 - lava_callback - INFO - Starting async job submit for node 69ae1ee68ef4424e487b3e4d 2026-03-09 01:29:45,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:45,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee68ef4424e487b3e4d HTTP/1.1" 200 None 2026-03-09 01:29:45,572 - lava_callback - DEBUG - Processing node 69ae1ee68ef4424e487b3e4d: baseline-arm64 2026-03-09 01:29:45,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:29:45,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:29:45,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee68ef4424e487b3e4d/log.txt.gz 2026-03-09 01:29:46,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:29:46,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:29:46,326 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee68ef4424e487b3e4d/lava-logs.yaml 2026-03-09 01:29:46,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:29:46,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:29:46,483 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee68ef4424e487b3e4d/lava_callback.json.gz 2026-03-09 01:29:46,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:55932 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:29:46,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee68ef4424e487b3e4d HTTP/1.1" 200 None 2026-03-09 01:29:46,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:46,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:29:46,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:46,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ee68ef4424e487b3e4d HTTP/1.1" 200 None 2026-03-09 01:29:46,590 - lava_callback - INFO - Completed processing callback for node 69ae1ee68ef4424e487b3e4d 2026-03-09 01:29:53,606 - lava_callback - INFO - Received callback for node 69ab7f80efa461f55e59dc7b INFO: 172.21.0.1:52624 - "POST /node/69ab7f80efa461f55e59dc7b HTTP/1.0" 202 Accepted 2026-03-09 01:29:53,611 - lava_callback - INFO - Starting async job submit for node 69ab7f80efa461f55e59dc7b 2026-03-09 01:29:53,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:53,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7f80efa461f55e59dc7b HTTP/1.1" 200 None 2026-03-09 01:29:53,627 - lava_callback - DEBUG - Processing node 69ab7f80efa461f55e59dc7b: baseline-arm64-baylibre 2026-03-09 01:29:53,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab7f80efa461f55e59dc7b 2026-03-09 01:29:53,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:53,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab7f80efa461f55e59dc7b HTTP/1.1" 200 None 2026-03-09 01:29:53,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:53,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385776346680f7a28dbe0 HTTP/1.1" 200 None 2026-03-09 01:29:53,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:53,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab7f80efa461f55e59dc7b HTTP/1.1" 200 None 2026-03-09 01:29:53,687 - lava_callback - ERROR - Error processing callback for node 69ab7f80efa461f55e59dc7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52640 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:29:57,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:29:57,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:30:16,711 - lava_callback - INFO - Received callback for node 69ae1ee98ef4424e487b3e53 INFO: 172.21.0.1:35402 - "POST /node/69ae1ee98ef4424e487b3e53 HTTP/1.0" 202 Accepted 2026-03-09 01:30:16,717 - lava_callback - INFO - Starting async job submit for node 69ae1ee98ef4424e487b3e53 2026-03-09 01:30:16,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:16,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee98ef4424e487b3e53 HTTP/1.1" 200 None 2026-03-09 01:30:16,753 - lava_callback - DEBUG - Processing node 69ae1ee98ef4424e487b3e53: baseline-arm64 2026-03-09 01:30:17,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:30:17,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:30:17,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee98ef4424e487b3e53/log.txt.gz 2026-03-09 01:30:17,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:30:17,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:30:17,535 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee98ef4424e487b3e53/lava-logs.yaml 2026-03-09 01:30:17,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:30:17,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:30:17,668 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee98ef4424e487b3e53/lava_callback.json.gz 2026-03-09 01:30:17,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:17,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee98ef4424e487b3e53 HTTP/1.1" 200 None 2026-03-09 01:30:17,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:17,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:30:17,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:17,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ee98ef4424e487b3e53 HTTP/1.1" 200 None 2026-03-09 01:30:17,766 - lava_callback - INFO - Completed processing callback for node 69ae1ee98ef4424e487b3e53 INFO: 172.21.0.1:58408 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:30:27,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:27,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53864 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:30:38,924 - lava_callback - INFO - Received callback for node 69ae1edb8ef4424e487b3e03 INFO: 172.21.0.1:53870 - "POST /node/69ae1edb8ef4424e487b3e03 HTTP/1.0" 202 Accepted 2026-03-09 01:30:38,930 - lava_callback - INFO - Starting async job submit for node 69ae1edb8ef4424e487b3e03 2026-03-09 01:30:38,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:38,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edb8ef4424e487b3e03 HTTP/1.1" 200 None 2026-03-09 01:30:38,967 - lava_callback - DEBUG - Processing node 69ae1edb8ef4424e487b3e03: baseline-arm64 2026-03-09 01:30:39,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:30:39,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:30:39,401 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edb8ef4424e487b3e03/log.txt.gz 2026-03-09 01:30:39,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:30:39,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:30:39,735 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edb8ef4424e487b3e03/lava-logs.yaml 2026-03-09 01:30:39,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:30:39,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:30:39,916 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1edb8ef4424e487b3e03/lava_callback.json.gz 2026-03-09 01:30:39,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:39,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1edb8ef4424e487b3e03 HTTP/1.1" 200 None 2026-03-09 01:30:39,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:39,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:30:39,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:40,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1edb8ef4424e487b3e03 HTTP/1.1" 200 None 2026-03-09 01:30:40,014 - lava_callback - INFO - Completed processing callback for node 69ae1edb8ef4424e487b3e03 INFO: 172.21.0.1:52692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47772 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:30:57,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:30:57,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40958 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:31:28,036 - lava_callback - INFO - Received callback for node 69ae1ef98ef4424e487b3e80 INFO: 172.21.0.1:40960 - "POST /node/69ae1ef98ef4424e487b3e80 HTTP/1.0" 202 Accepted 2026-03-09 01:31:28,042 - lava_callback - INFO - Starting async job submit for node 69ae1ef98ef4424e487b3e80 2026-03-09 01:31:28,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:28,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef98ef4424e487b3e80 HTTP/1.1" 200 None 2026-03-09 01:31:28,092 - lava_callback - DEBUG - Processing node 69ae1ef98ef4424e487b3e80: baseline-nfs-arm64 2026-03-09 01:31:28,141 - lava_callback - INFO - Received callback for node 69ae1efc8ef4424e487b3ead INFO: 172.21.0.1:40962 - "POST /node/69ae1efc8ef4424e487b3ead HTTP/1.0" 202 Accepted 2026-03-09 01:31:28,168 - lava_callback - INFO - Starting async job submit for node 69ae1efc8ef4424e487b3ead 2026-03-09 01:31:28,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:28,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1efc8ef4424e487b3ead HTTP/1.1" 200 None 2026-03-09 01:31:28,330 - lava_callback - DEBUG - Processing node 69ae1efc8ef4424e487b3ead: baseline-nfs-arm64 2026-03-09 01:31:28,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:31:28,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:31:28,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:31:28,933 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef98ef4424e487b3e80/log.txt.gz 2026-03-09 01:31:28,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:31:29,007 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efc8ef4424e487b3ead/log.txt.gz 2026-03-09 01:31:29,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:31:29,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:31:29,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:31:29,644 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef98ef4424e487b3e80/lava-logs.yaml 2026-03-09 01:31:29,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:31:29,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:31:29,664 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efc8ef4424e487b3ead/lava-logs.yaml 2026-03-09 01:31:29,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:31:29,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:31:29,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef98ef4424e487b3e80/lava_callback.json.gz 2026-03-09 01:31:29,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:29,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:31:29,870 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efc8ef4424e487b3ead/lava_callback.json.gz 2026-03-09 01:31:29,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:29,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef98ef4424e487b3e80 HTTP/1.1" 200 None 2026-03-09 01:31:29,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:29,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1efc8ef4424e487b3ead HTTP/1.1" 200 None 2026-03-09 01:31:29,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:29,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:31:29,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:29,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:31:29,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:29,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef98ef4424e487b3e80 HTTP/1.1" 200 None 2026-03-09 01:31:29,982 - lava_callback - INFO - Completed processing callback for node 69ae1ef98ef4424e487b3e80 2026-03-09 01:31:30,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1efc8ef4424e487b3ead HTTP/1.1" 200 None 2026-03-09 01:31:30,037 - lava_callback - INFO - Completed processing callback for node 69ae1efc8ef4424e487b3ead INFO: 172.21.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48746 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:31:57,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:31:57,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:32:26,441 - lava_callback - INFO - Received callback for node 69ae1eef8ef4424e487b3e64 INFO: 172.21.0.1:50238 - "POST /node/69ae1eef8ef4424e487b3e64 HTTP/1.0" 202 Accepted 2026-03-09 01:32:26,453 - lava_callback - INFO - Starting async job submit for node 69ae1eef8ef4424e487b3e64 2026-03-09 01:32:26,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:26,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eef8ef4424e487b3e64 HTTP/1.1" 200 None 2026-03-09 01:32:26,509 - lava_callback - DEBUG - Processing node 69ae1eef8ef4424e487b3e64: baseline-nfs-arm64 INFO: 172.21.0.1:50252 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:32:26,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:32:27,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:32:27,099 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1eef8ef4424e487b3e64/log.txt.gz 2026-03-09 01:32:27,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:32:27,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:32:27,538 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1eef8ef4424e487b3e64/lava-logs.yaml 2026-03-09 01:32:27,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:32:27,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:32:27,771 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1eef8ef4424e487b3e64/lava_callback.json.gz 2026-03-09 01:32:27,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:27,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1eef8ef4424e487b3e64 HTTP/1.1" 200 None 2026-03-09 01:32:27,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:27,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:32:27,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:27,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1eef8ef4424e487b3e64 HTTP/1.1" 200 None 2026-03-09 01:32:27,872 - lava_callback - INFO - Completed processing callback for node 69ae1eef8ef4424e487b3e64 INFO: 172.21.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52044 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:32:57,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:57,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:32:59,025 - lava_callback - INFO - Received callback for node 69ae1ec58ef4424e487b3de6 INFO: 172.21.0.1:52060 - "POST /node/69ae1ec58ef4424e487b3de6 HTTP/1.0" 202 Accepted 2026-03-09 01:32:59,077 - lava_callback - INFO - Starting async job submit for node 69ae1ec58ef4424e487b3de6 2026-03-09 01:32:59,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:59,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ec58ef4424e487b3de6 HTTP/1.1" 200 None 2026-03-09 01:32:59,118 - lava_callback - DEBUG - Processing node 69ae1ec58ef4424e487b3de6: baseline-arm-pengutronix 2026-03-09 01:32:59,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:32:59,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:32:59,383 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae1ec58ef4424e487b3de6/log.txt.gz 2026-03-09 01:32:59,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:32:59,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:32:59,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae1ec58ef4424e487b3de6/lava-logs.yaml 2026-03-09 01:32:59,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:32:59,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:32:59,812 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae1ec58ef4424e487b3de6/lava_callback.json.gz DEBUG: Setup failed for 69ae1ec58ef4424e487b3de6. Transitting job node result: incomplete -> fail 2026-03-09 01:32:59,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:59,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ec58ef4424e487b3de6 HTTP/1.1" 200 None 2026-03-09 01:32:59,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:59,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 01:32:59,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:32:59,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ec58ef4424e487b3de6 HTTP/1.1" 200 None 2026-03-09 01:32:59,987 - lava_callback - INFO - Completed processing callback for node 69ae1ec58ef4424e487b3de6 2026-03-09 01:33:05,752 - lava_callback - INFO - Received callback for node 69ae231f8ef4424e487b41a7 INFO: 172.21.0.1:33412 - "POST /node/69ae231f8ef4424e487b41a7 HTTP/1.0" 202 Accepted 2026-03-09 01:33:05,759 - lava_callback - INFO - Starting async job submit for node 69ae231f8ef4424e487b41a7 2026-03-09 01:33:05,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:33:05,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae231f8ef4424e487b41a7 HTTP/1.1" 200 None 2026-03-09 01:33:05,805 - lava_callback - DEBUG - Processing node 69ae231f8ef4424e487b41a7: baseline-arm64 2026-03-09 01:33:05,942 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:33:06,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:33:06,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:33:06,182 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae231f8ef4424e487b41a7/lava-logs.yaml 2026-03-09 01:33:06,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:33:06,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:33:06,317 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae231f8ef4424e487b41a7/lava_callback.json.gz 2026-03-09 01:33:06,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:33:06,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae231f8ef4424e487b41a7 HTTP/1.1" 200 None 2026-03-09 01:33:06,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:33:06,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:33:06,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:33:06,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae231f8ef4424e487b41a7 HTTP/1.1" 200 None 2026-03-09 01:33:06,492 - lava_callback - INFO - Completed processing callback for node 69ae231f8ef4424e487b41a7 INFO: 172.21.0.1:33424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37338 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:33:27,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:33:27,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51496 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:34:10,407 - lava_callback - INFO - Received callback for node 69ae1ee28ef4424e487b3e44 INFO: 172.21.0.1:40090 - "POST /node/69ae1ee28ef4424e487b3e44 HTTP/1.0" 202 Accepted 2026-03-09 01:34:10,414 - lava_callback - INFO - Starting async job submit for node 69ae1ee28ef4424e487b3e44 2026-03-09 01:34:10,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:10,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee28ef4424e487b3e44 HTTP/1.1" 200 None 2026-03-09 01:34:10,554 - lava_callback - DEBUG - Processing node 69ae1ee28ef4424e487b3e44: baseline-arm64 2026-03-09 01:34:10,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:34:11,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:34:11,198 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee28ef4424e487b3e44/log.txt.gz 2026-03-09 01:34:11,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:34:11,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:34:11,819 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee28ef4424e487b3e44/lava-logs.yaml 2026-03-09 01:34:11,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:34:12,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:34:12,042 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee28ef4424e487b3e44/lava_callback.json.gz 2026-03-09 01:34:12,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:12,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee28ef4424e487b3e44 HTTP/1.1" 200 None 2026-03-09 01:34:12,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:12,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:34:12,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:12,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ee28ef4424e487b3e44 HTTP/1.1" 200 None 2026-03-09 01:34:12,131 - lava_callback - INFO - Completed processing callback for node 69ae1ee28ef4424e487b3e44 INFO: 172.21.0.1:40100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:34:27,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:27,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:34:33,837 - lava_callback - INFO - Received callback for node 69ae1ef68ef4424e487b3e75 INFO: 172.21.0.1:34368 - "POST /node/69ae1ef68ef4424e487b3e75 HTTP/1.0" 202 Accepted 2026-03-09 01:34:33,844 - lava_callback - INFO - Starting async job submit for node 69ae1ef68ef4424e487b3e75 2026-03-09 01:34:33,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:33,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef68ef4424e487b3e75 HTTP/1.1" 200 None 2026-03-09 01:34:33,884 - lava_callback - DEBUG - Processing node 69ae1ef68ef4424e487b3e75: baseline-nfs-arm64 2026-03-09 01:34:34,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:34:34,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:34:34,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef68ef4424e487b3e75/log.txt.gz 2026-03-09 01:34:34,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:34:35,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:34:35,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef68ef4424e487b3e75/lava-logs.yaml 2026-03-09 01:34:35,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:34:35,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:34:35,255 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef68ef4424e487b3e75/lava_callback.json.gz 2026-03-09 01:34:35,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:35,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef68ef4424e487b3e75 HTTP/1.1" 200 None 2026-03-09 01:34:35,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:35,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:34:35,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:35,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef68ef4424e487b3e75 HTTP/1.1" 200 None 2026-03-09 01:34:35,361 - lava_callback - INFO - Completed processing callback for node 69ae1ef68ef4424e487b3e75 INFO: 172.21.0.1:34374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:34:57,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:34:57,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56964 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:35:13,817 - lava_callback - INFO - Received callback for node 69ab80cbefa461f55e59dd1f INFO: 172.21.0.1:41166 - "POST /node/69ab80cbefa461f55e59dd1f HTTP/1.0" 202 Accepted 2026-03-09 01:35:13,825 - lava_callback - INFO - Starting async job submit for node 69ab80cbefa461f55e59dd1f 2026-03-09 01:35:13,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:35:13,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab80cbefa461f55e59dd1f HTTP/1.1" 200 None 2026-03-09 01:35:13,847 - lava_callback - DEBUG - Processing node 69ab80cbefa461f55e59dd1f: baseline-arm64-baylibre 2026-03-09 01:35:13,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab80cbefa461f55e59dd1f 2026-03-09 01:35:13,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:35:13,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab80cbefa461f55e59dd1f HTTP/1.1" 200 None 2026-03-09 01:35:13,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:35:13,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385776346680f7a28dbe0 HTTP/1.1" 200 None 2026-03-09 01:35:13,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:35:13,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab80cbefa461f55e59dd1f HTTP/1.1" 200 None 2026-03-09 01:35:13,943 - lava_callback - ERROR - Error processing callback for node 69ab80cbefa461f55e59dd1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38102 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:36:51,039 - lava_callback - INFO - Received callback for node 69ae1ee38ef4424e487b3e47 INFO: 172.21.0.1:60670 - "POST /node/69ae1ee38ef4424e487b3e47 HTTP/1.0" 202 Accepted 2026-03-09 01:36:51,045 - lava_callback - INFO - Starting async job submit for node 69ae1ee38ef4424e487b3e47 2026-03-09 01:36:51,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:36:51,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee38ef4424e487b3e47 HTTP/1.1" 200 None 2026-03-09 01:36:51,082 - lava_callback - DEBUG - Processing node 69ae1ee38ef4424e487b3e47: baseline-arm64 2026-03-09 01:36:51,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:36:51,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:36:51,762 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee38ef4424e487b3e47/log.txt.gz 2026-03-09 01:36:52,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:36:52,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:36:52,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee38ef4424e487b3e47/lava-logs.yaml 2026-03-09 01:36:52,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:36:52,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:36:52,434 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee38ef4424e487b3e47/lava_callback.json.gz 2026-03-09 01:36:52,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:36:52,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee38ef4424e487b3e47 HTTP/1.1" 200 None 2026-03-09 01:36:52,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:36:52,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:36:52,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:36:52,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ee38ef4424e487b3e47 HTTP/1.1" 200 None 2026-03-09 01:36:52,600 - lava_callback - INFO - Completed processing callback for node 69ae1ee38ef4424e487b3e47 INFO: 172.21.0.1:60678 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:36:57,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:36:57,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:38:22,791 - lava_callback - INFO - Received callback for node 69ae1ef78ef4424e487b3e79 INFO: 172.21.0.1:47594 - "POST /node/69ae1ef78ef4424e487b3e79 HTTP/1.0" 202 Accepted 2026-03-09 01:38:22,798 - lava_callback - INFO - Starting async job submit for node 69ae1ef78ef4424e487b3e79 2026-03-09 01:38:22,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:38:22,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef78ef4424e487b3e79 HTTP/1.1" 200 None 2026-03-09 01:38:22,843 - lava_callback - DEBUG - Processing node 69ae1ef78ef4424e487b3e79: baseline-nfs-arm64 2026-03-09 01:38:23,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:38:23,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:38:23,569 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef78ef4424e487b3e79/log.txt.gz 2026-03-09 01:38:23,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:38:24,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:38:24,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef78ef4424e487b3e79/lava-logs.yaml 2026-03-09 01:38:24,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:38:24,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:38:24,283 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef78ef4424e487b3e79/lava_callback.json.gz 2026-03-09 01:38:24,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:38:24,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef78ef4424e487b3e79 HTTP/1.1" 200 None 2026-03-09 01:38:24,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:38:24,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:38:24,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:38:24,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef78ef4424e487b3e79 HTTP/1.1" 200 None 2026-03-09 01:38:24,384 - lava_callback - INFO - Completed processing callback for node 69ae1ef78ef4424e487b3e79 INFO: 172.21.0.1:47604 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:38:27,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:38:27,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35542 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:39:13,948 - lava_callback - INFO - Received callback for node 69ae24628ef4424e487b425e INFO: 172.21.0.1:54534 - "POST /node/69ae24628ef4424e487b425e HTTP/1.0" 202 Accepted 2026-03-09 01:39:13,955 - lava_callback - INFO - Starting async job submit for node 69ae24628ef4424e487b425e 2026-03-09 01:39:13,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:14,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae24628ef4424e487b425e HTTP/1.1" 200 None 2026-03-09 01:39:14,005 - lava_callback - DEBUG - Processing node 69ae24628ef4424e487b425e: baseline-arm64 2026-03-09 01:39:14,203 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:39:14,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:39:14,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:39:14,497 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae24628ef4424e487b425e/lava-logs.yaml 2026-03-09 01:39:14,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:39:14,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:39:14,636 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae24628ef4424e487b425e/lava_callback.json.gz 2026-03-09 01:39:14,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:14,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae24628ef4424e487b425e HTTP/1.1" 200 None 2026-03-09 01:39:14,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:14,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:39:14,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:14,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae24628ef4424e487b425e HTTP/1.1" 200 None 2026-03-09 01:39:14,808 - lava_callback - INFO - Completed processing callback for node 69ae24628ef4424e487b425e INFO: 172.21.0.1:54542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51250 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:39:27,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:27,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:39:34,069 - lava_callback - INFO - Received callback for node 69ab81c2efa461f55e59dd7f INFO: 172.21.0.1:52096 - "POST /node/69ab81c2efa461f55e59dd7f HTTP/1.0" 202 Accepted 2026-03-09 01:39:34,074 - lava_callback - INFO - Starting async job submit for node 69ab81c2efa461f55e59dd7f 2026-03-09 01:39:34,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:34,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab81c2efa461f55e59dd7f HTTP/1.1" 200 None 2026-03-09 01:39:34,099 - lava_callback - DEBUG - Processing node 69ab81c2efa461f55e59dd7f: baseline-arm-baylibre 2026-03-09 01:39:34,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab81c2efa461f55e59dd7f 2026-03-09 01:39:34,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:34,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab81c2efa461f55e59dd7f HTTP/1.1" 200 None 2026-03-09 01:39:34,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:34,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0e532aaccb4c112eb60 HTTP/1.1" 200 None 2026-03-09 01:39:34,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:34,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab81c2efa461f55e59dd7f HTTP/1.1" 200 None 2026-03-09 01:39:34,152 - lava_callback - ERROR - Error processing callback for node 69ab81c2efa461f55e59dd7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57164 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:39:54,055 - lava_callback - INFO - Received callback for node 69ab81e6efa461f55e59dd97 INFO: 172.21.0.1:35958 - "POST /node/69ab81e6efa461f55e59dd97 HTTP/1.0" 202 Accepted 2026-03-09 01:39:54,060 - lava_callback - INFO - Starting async job submit for node 69ab81e6efa461f55e59dd97 2026-03-09 01:39:54,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:54,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab81e6efa461f55e59dd97 HTTP/1.1" 200 None 2026-03-09 01:39:54,079 - lava_callback - DEBUG - Processing node 69ab81e6efa461f55e59dd97: baseline-arm-baylibre 2026-03-09 01:39:54,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab81e6efa461f55e59dd97 2026-03-09 01:39:54,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:54,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab81e6efa461f55e59dd97 HTTP/1.1" 200 None 2026-03-09 01:39:54,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:54,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0e532aaccb4c112eb60 HTTP/1.1" 200 None 2026-03-09 01:39:54,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:39:54,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab81e6efa461f55e59dd97 HTTP/1.1" 200 None 2026-03-09 01:39:54,141 - lava_callback - ERROR - Error processing callback for node 69ab81e6efa461f55e59dd97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60090 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:40:54,161 - lava_callback - INFO - Received callback for node 69ab8213efa461f55e59ddb0 INFO: 172.21.0.1:59868 - "POST /node/69ab8213efa461f55e59ddb0 HTTP/1.0" 202 Accepted 2026-03-09 01:40:54,167 - lava_callback - INFO - Starting async job submit for node 69ab8213efa461f55e59ddb0 2026-03-09 01:40:54,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:40:54,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8213efa461f55e59ddb0 HTTP/1.1" 200 None 2026-03-09 01:40:54,183 - lava_callback - DEBUG - Processing node 69ab8213efa461f55e59ddb0: baseline-arm64-baylibre 2026-03-09 01:40:54,184 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8213efa461f55e59ddb0 2026-03-09 01:40:54,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:40:54,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8213efa461f55e59ddb0 HTTP/1.1" 200 None 2026-03-09 01:40:54,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:40:54,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 01:40:54,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:40:54,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8213efa461f55e59ddb0 HTTP/1.1" 200 None 2026-03-09 01:40:54,252 - lava_callback - ERROR - Error processing callback for node 69ab8213efa461f55e59ddb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59882 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:41:02,375 - lava_callback - INFO - Received callback for node 69ae25098ef4424e487b42ad INFO: 172.21.0.1:55018 - "POST /node/69ae25098ef4424e487b42ad HTTP/1.0" 202 Accepted 2026-03-09 01:41:02,381 - lava_callback - INFO - Starting async job submit for node 69ae25098ef4424e487b42ad 2026-03-09 01:41:02,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25098ef4424e487b42ad HTTP/1.1" 200 None 2026-03-09 01:41:02,399 - lava_callback - DEBUG - Processing node 69ae25098ef4424e487b42ad: baseline-nfs-arm64 2026-03-09 01:41:02,400 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:41:02,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:41:02,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:41:02,528 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae25098ef4424e487b42ad/lava-logs.yaml 2026-03-09 01:41:02,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:41:02,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:41:02,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae25098ef4424e487b42ad/lava_callback.json.gz 2026-03-09 01:41:02,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:02,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25098ef4424e487b42ad HTTP/1.1" 200 None 2026-03-09 01:41:02,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:02,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:41:02,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:02,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae25098ef4424e487b42ad HTTP/1.1" 200 None 2026-03-09 01:41:02,728 - lava_callback - INFO - Completed processing callback for node 69ae25098ef4424e487b42ad INFO: 172.21.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:41:27,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:27,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:41:34,447 - lava_callback - INFO - Received callback for node 69ae1ee78ef4424e487b3e50 INFO: 172.21.0.1:34224 - "POST /node/69ae1ee78ef4424e487b3e50 HTTP/1.0" 202 Accepted 2026-03-09 01:41:34,453 - lava_callback - INFO - Starting async job submit for node 69ae1ee78ef4424e487b3e50 2026-03-09 01:41:34,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:34,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee78ef4424e487b3e50 HTTP/1.1" 200 None 2026-03-09 01:41:34,489 - lava_callback - DEBUG - Processing node 69ae1ee78ef4424e487b3e50: baseline-arm64 2026-03-09 01:41:34,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:41:35,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:41:35,064 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee78ef4424e487b3e50/log.txt.gz 2026-03-09 01:41:35,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:41:35,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:41:35,482 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee78ef4424e487b3e50/lava-logs.yaml 2026-03-09 01:41:35,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:41:35,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:41:35,676 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee78ef4424e487b3e50/lava_callback.json.gz 2026-03-09 01:41:35,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:35,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee78ef4424e487b3e50 HTTP/1.1" 200 None 2026-03-09 01:41:35,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:35,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:41:35,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:35,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ee78ef4424e487b3e50 HTTP/1.1" 200 None 2026-03-09 01:41:35,771 - lava_callback - INFO - Completed processing callback for node 69ae1ee78ef4424e487b3e50 INFO: 172.21.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43026 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:41:57,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:41:57,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:43:14,271 - lava_callback - INFO - Received callback for node 69ab82a2efa461f55e59ddff INFO: 172.21.0.1:60830 - "POST /node/69ab82a2efa461f55e59ddff HTTP/1.0" 202 Accepted 2026-03-09 01:43:14,277 - lava_callback - INFO - Starting async job submit for node 69ab82a2efa461f55e59ddff 2026-03-09 01:43:14,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:14,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab82a2efa461f55e59ddff HTTP/1.1" 200 None 2026-03-09 01:43:14,292 - lava_callback - DEBUG - Processing node 69ab82a2efa461f55e59ddff: baseline-arm-baylibre 2026-03-09 01:43:14,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab82a2efa461f55e59ddff 2026-03-09 01:43:14,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:14,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab82a2efa461f55e59ddff HTTP/1.1" 200 None 2026-03-09 01:43:14,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:14,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0e532aaccb4c112eb60 HTTP/1.1" 200 None 2026-03-09 01:43:14,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:14,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab82a2efa461f55e59ddff HTTP/1.1" 200 None 2026-03-09 01:43:14,350 - lava_callback - ERROR - Error processing callback for node 69ab82a2efa461f55e59ddff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:43:42,487 - lava_callback - INFO - Received callback for node 69ae1efb8ef4424e487b3e88 INFO: 172.21.0.1:36844 - "POST /node/69ae1efb8ef4424e487b3e88 HTTP/1.0" 202 Accepted 2026-03-09 01:43:42,494 - lava_callback - INFO - Starting async job submit for node 69ae1efb8ef4424e487b3e88 2026-03-09 01:43:42,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:42,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1efb8ef4424e487b3e88 HTTP/1.1" 200 None 2026-03-09 01:43:42,532 - lava_callback - DEBUG - Processing node 69ae1efb8ef4424e487b3e88: baseline-nfs-arm64 INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:43:42,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:43:43,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:43:43,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efb8ef4424e487b3e88/log.txt.gz 2026-03-09 01:43:43,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:43:43,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:43:43,663 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efb8ef4424e487b3e88/lava-logs.yaml 2026-03-09 01:43:43,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:43:43,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:43:43,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1efb8ef4424e487b3e88/lava_callback.json.gz 2026-03-09 01:43:43,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:43,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1efb8ef4424e487b3e88 HTTP/1.1" 200 None 2026-03-09 01:43:43,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:43,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:43:43,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:43,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1efb8ef4424e487b3e88 HTTP/1.1" 200 None 2026-03-09 01:43:43,936 - lava_callback - INFO - Completed processing callback for node 69ae1efb8ef4424e487b3e88 INFO: 172.21.0.1:36856 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:43:54,283 - lava_callback - INFO - Received callback for node 69ab82cfefa461f55e59de13 INFO: 172.21.0.1:40564 - "POST /node/69ab82cfefa461f55e59de13 HTTP/1.0" 202 Accepted 2026-03-09 01:43:54,289 - lava_callback - INFO - Starting async job submit for node 69ab82cfefa461f55e59de13 2026-03-09 01:43:54,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:54,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab82cfefa461f55e59de13 HTTP/1.1" 200 None 2026-03-09 01:43:54,305 - lava_callback - DEBUG - Processing node 69ab82cfefa461f55e59de13: baseline-arm64-baylibre 2026-03-09 01:43:54,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab82cfefa461f55e59de13 2026-03-09 01:43:54,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:54,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab82cfefa461f55e59de13 HTTP/1.1" 200 None 2026-03-09 01:43:54,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:54,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 01:43:54,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:54,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab82cfefa461f55e59de13 HTTP/1.1" 200 None 2026-03-09 01:43:54,372 - lava_callback - ERROR - Error processing callback for node 69ab82cfefa461f55e59de13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:43:57,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:43:57,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 01:44:05,956 - lava_callback - INFO - Received callback for node 69ae25a68ef4424e487b4310 INFO: 172.21.0.1:50240 - "POST /node/69ae25a68ef4424e487b4310 HTTP/1.0" 202 Accepted 2026-03-09 01:44:05,964 - lava_callback - INFO - Starting async job submit for node 69ae25a68ef4424e487b4310 2026-03-09 01:44:05,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:06,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25a68ef4424e487b4310 HTTP/1.1" 200 None 2026-03-09 01:44:06,010 - lava_callback - DEBUG - Processing node 69ae25a68ef4424e487b4310: baseline-arm64 2026-03-09 01:44:06,160 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:44:06,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:44:06,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:44:06,394 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae25a68ef4424e487b4310/lava-logs.yaml 2026-03-09 01:44:06,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:44:06,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:44:06,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae25a68ef4424e487b4310/lava_callback.json.gz 2026-03-09 01:44:06,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:06,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25a68ef4424e487b4310 HTTP/1.1" 200 None 2026-03-09 01:44:06,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:06,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:44:06,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:06,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae25a68ef4424e487b4310 HTTP/1.1" 200 None 2026-03-09 01:44:06,701 - lava_callback - INFO - Completed processing callback for node 69ae25a68ef4424e487b4310 INFO: 172.21.0.1:45364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34246 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:44:27,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:27,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:44:54,335 - lava_callback - INFO - Received callback for node 69ab8313efa461f55e59de33 INFO: 172.21.0.1:36050 - "POST /node/69ab8313efa461f55e59de33 HTTP/1.0" 202 Accepted 2026-03-09 01:44:54,341 - lava_callback - INFO - Starting async job submit for node 69ab8313efa461f55e59de33 2026-03-09 01:44:54,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:54,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8313efa461f55e59de33 HTTP/1.1" 200 None 2026-03-09 01:44:54,356 - lava_callback - DEBUG - Processing node 69ab8313efa461f55e59de33: baseline-arm-baylibre 2026-03-09 01:44:54,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8313efa461f55e59de33 2026-03-09 01:44:54,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:54,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8313efa461f55e59de33 HTTP/1.1" 200 None 2026-03-09 01:44:54,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:54,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0e532aaccb4c112eb60 HTTP/1.1" 200 None 2026-03-09 01:44:54,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:44:54,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8313efa461f55e59de33 HTTP/1.1" 200 None 2026-03-09 01:44:54,410 - lava_callback - ERROR - Error processing callback for node 69ab8313efa461f55e59de33: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34654 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:45:34,379 - lava_callback - INFO - Received callback for node 69ab8337efa461f55e59de42 INFO: 172.21.0.1:41752 - "POST /node/69ab8337efa461f55e59de42 HTTP/1.0" 202 Accepted 2026-03-09 01:45:34,385 - lava_callback - INFO - Starting async job submit for node 69ab8337efa461f55e59de42 2026-03-09 01:45:34,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:45:34,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8337efa461f55e59de42 HTTP/1.1" 200 None 2026-03-09 01:45:34,418 - lava_callback - DEBUG - Processing node 69ab8337efa461f55e59de42: baseline-arm-baylibre 2026-03-09 01:45:34,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8337efa461f55e59de42 2026-03-09 01:45:34,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:45:34,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8337efa461f55e59de42 HTTP/1.1" 200 None 2026-03-09 01:45:34,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:45:34,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0e532aaccb4c112eb60 HTTP/1.1" 200 None 2026-03-09 01:45:34,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:45:34,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8337efa461f55e59de42 HTTP/1.1" 200 None 2026-03-09 01:45:34,475 - lava_callback - ERROR - Error processing callback for node 69ab8337efa461f55e59de42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55130 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:46:14,458 - lava_callback - INFO - Received callback for node 69ab8363efa461f55e59de50 INFO: 172.21.0.1:59838 - "POST /node/69ab8363efa461f55e59de50 HTTP/1.0" 202 Accepted 2026-03-09 01:46:14,464 - lava_callback - INFO - Starting async job submit for node 69ab8363efa461f55e59de50 2026-03-09 01:46:14,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:14,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8363efa461f55e59de50 HTTP/1.1" 200 None 2026-03-09 01:46:14,480 - lava_callback - DEBUG - Processing node 69ab8363efa461f55e59de50: baseline-arm64-baylibre 2026-03-09 01:46:14,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8363efa461f55e59de50 2026-03-09 01:46:14,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:14,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8363efa461f55e59de50 HTTP/1.1" 200 None 2026-03-09 01:46:14,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:14,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 01:46:14,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:14,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8363efa461f55e59de50 HTTP/1.1" 200 None 2026-03-09 01:46:14,541 - lava_callback - ERROR - Error processing callback for node 69ab8363efa461f55e59de50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59854 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:46:23,186 - lava_callback - INFO - Received callback for node 69ae26448ef4424e487b436c INFO: 172.21.0.1:55728 - "POST /node/69ae26448ef4424e487b436c HTTP/1.0" 202 Accepted 2026-03-09 01:46:23,193 - lava_callback - INFO - Starting async job submit for node 69ae26448ef4424e487b436c 2026-03-09 01:46:23,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:23,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae26448ef4424e487b436c HTTP/1.1" 200 None 2026-03-09 01:46:23,239 - lava_callback - DEBUG - Processing node 69ae26448ef4424e487b436c: baseline-arm64 2026-03-09 01:46:23,382 - lava_callback - WARNING - Failed to upload log 2026-03-09 01:46:23,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:23,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:23,653 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae26448ef4424e487b436c/lava-logs.yaml 2026-03-09 01:46:23,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:23,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:23,776 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae26448ef4424e487b436c/lava_callback.json.gz 2026-03-09 01:46:23,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:23,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae26448ef4424e487b436c HTTP/1.1" 200 None 2026-03-09 01:46:23,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:23,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:46:23,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:23,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae26448ef4424e487b436c HTTP/1.1" 200 None 2026-03-09 01:46:23,940 - lava_callback - INFO - Completed processing callback for node 69ae26448ef4424e487b436c INFO: 172.21.0.1:55732 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:46:27,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:27,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:46:40,029 - lava_callback - INFO - Received callback for node 69ae25f78ef4424e487b4354 INFO: 172.21.0.1:44476 - "POST /node/69ae25f78ef4424e487b4354 HTTP/1.0" 202 Accepted 2026-03-09 01:46:40,035 - lava_callback - INFO - Starting async job submit for node 69ae25f78ef4424e487b4354 2026-03-09 01:46:40,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:40,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25f78ef4424e487b4354 HTTP/1.1" 200 None 2026-03-09 01:46:40,074 - lava_callback - DEBUG - Processing node 69ae25f78ef4424e487b4354: baseline-arm64 2026-03-09 01:46:40,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:40,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:40,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae25f78ef4424e487b4354/log.txt.gz 2026-03-09 01:46:40,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:41,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:41,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae25f78ef4424e487b4354/lava-logs.yaml 2026-03-09 01:46:41,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:41,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:41,367 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae25f78ef4424e487b4354/lava_callback.json.gz 2026-03-09 01:46:41,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:41,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25f78ef4424e487b4354 HTTP/1.1" 200 None 2026-03-09 01:46:41,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:41,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:46:41,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:41,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae25f78ef4424e487b4354 HTTP/1.1" 200 None 2026-03-09 01:46:41,529 - lava_callback - INFO - Completed processing callback for node 69ae25f78ef4424e487b4354 INFO: 172.21.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44508 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:46:47,681 - lava_callback - INFO - Received callback for node 69ae17738ef4424e487b344d INFO: 172.21.0.1:44510 - "POST /node/69ae17738ef4424e487b344d HTTP/1.0" 202 Accepted 2026-03-09 01:46:47,687 - lava_callback - INFO - Starting async job submit for node 69ae17738ef4424e487b344d 2026-03-09 01:46:47,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:47,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17738ef4424e487b344d HTTP/1.1" 200 None 2026-03-09 01:46:47,733 - lava_callback - DEBUG - Processing node 69ae17738ef4424e487b344d: baseline-arm-broonie 2026-03-09 01:46:47,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:48,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:48,186 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17738ef4424e487b344d/log.txt.gz 2026-03-09 01:46:48,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:48,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:48,513 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17738ef4424e487b344d/lava-logs.yaml 2026-03-09 01:46:48,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:46:48,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:46:48,688 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17738ef4424e487b344d/lava_callback.json.gz 2026-03-09 01:46:48,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:48,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17738ef4424e487b344d HTTP/1.1" 200 None 2026-03-09 01:46:48,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:48,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 01:46:48,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:48,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17738ef4424e487b344d HTTP/1.1" 200 None 2026-03-09 01:46:48,781 - lava_callback - INFO - Completed processing callback for node 69ae17738ef4424e487b344d INFO: 172.21.0.1:36528 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:46:57,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:46:57,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34530 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:47:06,689 - lava_callback - INFO - Received callback for node 69ae24b68ef4424e487b427f INFO: 172.21.0.1:34546 - "POST /node/69ae24b68ef4424e487b427f HTTP/1.0" 202 Accepted 2026-03-09 01:47:06,696 - lava_callback - INFO - Starting async job submit for node 69ae24b68ef4424e487b427f 2026-03-09 01:47:06,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:47:06,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae24b68ef4424e487b427f HTTP/1.1" 200 None 2026-03-09 01:47:06,748 - lava_callback - DEBUG - Processing node 69ae24b68ef4424e487b427f: baseline-arm64 2026-03-09 01:47:07,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:47:07,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:47:07,263 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae24b68ef4424e487b427f/log.txt.gz 2026-03-09 01:47:07,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:47:07,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:47:07,680 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae24b68ef4424e487b427f/lava-logs.yaml 2026-03-09 01:47:07,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:47:07,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:47:07,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae24b68ef4424e487b427f/lava_callback.json.gz DEBUG: Setup failed for 69ae24b68ef4424e487b427f. Transitting job node result: incomplete -> fail 2026-03-09 01:47:07,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:47:07,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae24b68ef4424e487b427f HTTP/1.1" 200 None 2026-03-09 01:47:07,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:47:07,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:47:07,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:47:07,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae24b68ef4424e487b427f HTTP/1.1" 200 None 2026-03-09 01:47:08,053 - lava_callback - INFO - Completed processing callback for node 69ae24b68ef4424e487b427f INFO: 172.21.0.1:49302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32976 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:47:27,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:47:27,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43898 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:48:54,625 - lava_callback - INFO - Received callback for node 69ab83f6efa461f55e59de80 INFO: 172.21.0.1:35236 - "POST /node/69ab83f6efa461f55e59de80 HTTP/1.0" 202 Accepted 2026-03-09 01:48:54,630 - lava_callback - INFO - Starting async job submit for node 69ab83f6efa461f55e59de80 2026-03-09 01:48:54,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:48:54,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab83f6efa461f55e59de80 HTTP/1.1" 200 None 2026-03-09 01:48:54,649 - lava_callback - DEBUG - Processing node 69ab83f6efa461f55e59de80: baseline-arm-baylibre 2026-03-09 01:48:54,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab83f6efa461f55e59de80 2026-03-09 01:48:54,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:48:54,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab83f6efa461f55e59de80 HTTP/1.1" 200 None 2026-03-09 01:48:54,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:48:54,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0e532aaccb4c112eb60 HTTP/1.1" 200 None 2026-03-09 01:48:54,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:48:54,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab83f6efa461f55e59de80 HTTP/1.1" 200 None 2026-03-09 01:48:54,703 - lava_callback - ERROR - Error processing callback for node 69ab83f6efa461f55e59de80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57622 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:49:34,583 - lava_callback - INFO - Received callback for node 69ab8423efa461f55e59dea2 INFO: 172.21.0.1:48942 - "POST /node/69ab8423efa461f55e59dea2 HTTP/1.0" 202 Accepted 2026-03-09 01:49:34,588 - lava_callback - INFO - Starting async job submit for node 69ab8423efa461f55e59dea2 2026-03-09 01:49:34,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:34,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8423efa461f55e59dea2 HTTP/1.1" 200 None 2026-03-09 01:49:34,606 - lava_callback - DEBUG - Processing node 69ab8423efa461f55e59dea2: baseline-arm64-baylibre 2026-03-09 01:49:34,606 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8423efa461f55e59dea2 2026-03-09 01:49:34,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:34,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8423efa461f55e59dea2 HTTP/1.1" 200 None 2026-03-09 01:49:34,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:34,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 01:49:34,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:34,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8423efa461f55e59dea2 HTTP/1.1" 200 None 2026-03-09 01:49:34,677 - lava_callback - ERROR - Error processing callback for node 69ab8423efa461f55e59dea2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 01:49:35,447 - lava_callback - INFO - Received callback for node 69ae1ee18ef4424e487b3e41 INFO: 172.21.0.1:48954 - "POST /node/69ae1ee18ef4424e487b3e41 HTTP/1.0" 202 Accepted 2026-03-09 01:49:35,454 - lava_callback - INFO - Starting async job submit for node 69ae1ee18ef4424e487b3e41 2026-03-09 01:49:35,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:35,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee18ef4424e487b3e41 HTTP/1.1" 200 None 2026-03-09 01:49:35,492 - lava_callback - DEBUG - Processing node 69ae1ee18ef4424e487b3e41: baseline-arm64 2026-03-09 01:49:35,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:49:36,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:49:36,023 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee18ef4424e487b3e41/log.txt.gz 2026-03-09 01:49:36,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:49:36,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:49:36,463 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee18ef4424e487b3e41/lava-logs.yaml 2026-03-09 01:49:36,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:48962 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:49:36,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:49:36,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ee18ef4424e487b3e41/lava_callback.json.gz 2026-03-09 01:49:36,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:36,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ee18ef4424e487b3e41 HTTP/1.1" 200 None 2026-03-09 01:49:36,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:36,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:49:36,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:36,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ee18ef4424e487b3e41 HTTP/1.1" 200 None 2026-03-09 01:49:36,707 - lava_callback - INFO - Completed processing callback for node 69ae1ee18ef4424e487b3e41 INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51400 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:49:57,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:49:57,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57356 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:50:54,764 - lava_callback - INFO - Received callback for node 69ab8472efa461f55e59dec4 INFO: 172.21.0.1:45560 - "POST /node/69ab8472efa461f55e59dec4 HTTP/1.0" 202 Accepted 2026-03-09 01:50:54,769 - lava_callback - INFO - Starting async job submit for node 69ab8472efa461f55e59dec4 2026-03-09 01:50:54,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:50:54,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8472efa461f55e59dec4 HTTP/1.1" 200 None 2026-03-09 01:50:54,785 - lava_callback - DEBUG - Processing node 69ab8472efa461f55e59dec4: baseline-arm64-baylibre 2026-03-09 01:50:54,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8472efa461f55e59dec4 2026-03-09 01:50:54,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:50:54,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8472efa461f55e59dec4 HTTP/1.1" 200 None 2026-03-09 01:50:54,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:50:54,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 01:50:54,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:50:54,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8472efa461f55e59dec4 HTTP/1.1" 200 None 2026-03-09 01:50:54,855 - lava_callback - ERROR - Error processing callback for node 69ab8472efa461f55e59dec4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59302 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:53:22,511 - lava_callback - INFO - Received callback for node 69ae27918ef4424e487b43e7 INFO: 172.21.0.1:56442 - "POST /node/69ae27918ef4424e487b43e7 HTTP/1.0" 202 Accepted 2026-03-09 01:53:22,518 - lava_callback - INFO - Starting async job submit for node 69ae27918ef4424e487b43e7 2026-03-09 01:53:22,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:53:22,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae27918ef4424e487b43e7 HTTP/1.1" 200 None 2026-03-09 01:53:22,566 - lava_callback - DEBUG - Processing node 69ae27918ef4424e487b43e7: baseline-arm64 2026-03-09 01:53:22,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:53:23,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:53:23,156 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae27918ef4424e487b43e7/log.txt.gz 2026-03-09 01:53:23,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:53:23,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:53:23,673 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae27918ef4424e487b43e7/lava-logs.yaml 2026-03-09 01:53:23,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:53:23,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:53:23,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae27918ef4424e487b43e7/lava_callback.json.gz 2026-03-09 01:53:23,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:53:23,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae27918ef4424e487b43e7 HTTP/1.1" 200 None 2026-03-09 01:53:23,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:53:23,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:53:23,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:53:23,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae27918ef4424e487b43e7 HTTP/1.1" 200 None 2026-03-09 01:53:24,031 - lava_callback - INFO - Completed processing callback for node 69ae27918ef4424e487b43e7 INFO: 172.21.0.1:56444 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:53:27,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:53:27,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:54:55,029 - lava_callback - INFO - Received callback for node 69ab855befa461f55e59df17 INFO: 172.21.0.1:59566 - "POST /node/69ab855befa461f55e59df17 HTTP/1.0" 202 Accepted 2026-03-09 01:54:55,038 - lava_callback - INFO - Starting async job submit for node 69ab855befa461f55e59df17 2026-03-09 01:54:55,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:54:55,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab855befa461f55e59df17 HTTP/1.1" 200 None 2026-03-09 01:54:55,055 - lava_callback - DEBUG - Processing node 69ab855befa461f55e59df17: baseline-arm64-baylibre 2026-03-09 01:54:55,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab855befa461f55e59df17 2026-03-09 01:54:55,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:54:55,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab855befa461f55e59df17 HTTP/1.1" 200 None 2026-03-09 01:54:55,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:54:55,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 01:54:55,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:54:55,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab855befa461f55e59df17 HTTP/1.1" 200 None 2026-03-09 01:54:55,116 - lava_callback - ERROR - Error processing callback for node 69ab855befa461f55e59df17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52872 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:57:38,653 - lava_callback - INFO - Received callback for node 69ae28d98ef4424e487b4452 INFO: 172.21.0.1:52878 - "POST /node/69ae28d98ef4424e487b4452 HTTP/1.0" 202 Accepted 2026-03-09 01:57:38,659 - lava_callback - INFO - Starting async job submit for node 69ae28d98ef4424e487b4452 2026-03-09 01:57:38,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:57:38,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae28d98ef4424e487b4452 HTTP/1.1" 200 None 2026-03-09 01:57:38,696 - lava_callback - DEBUG - Processing node 69ae28d98ef4424e487b4452: baseline-arm64 2026-03-09 01:57:39,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:57:39,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:57:39,324 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae28d98ef4424e487b4452/log.txt.gz 2026-03-09 01:57:39,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:57:39,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:57:39,796 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae28d98ef4424e487b4452/lava-logs.yaml 2026-03-09 01:57:39,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:57:39,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:57:39,972 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae28d98ef4424e487b4452/lava_callback.json.gz 2026-03-09 01:57:39,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:57:40,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae28d98ef4424e487b4452 HTTP/1.1" 200 None 2026-03-09 01:57:40,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:57:40,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:57:40,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:57:40,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae28d98ef4424e487b4452 HTTP/1.1" 200 None 2026-03-09 01:57:40,108 - lava_callback - INFO - Completed processing callback for node 69ae28d98ef4424e487b4452 INFO: 172.21.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:57:57,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:57:57,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56544 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:58:15,067 - lava_callback - INFO - Received callback for node 69ab8635efa461f55e59df75 INFO: 172.21.0.1:53548 - "POST /node/69ab8635efa461f55e59df75 HTTP/1.0" 202 Accepted 2026-03-09 01:58:15,073 - lava_callback - INFO - Starting async job submit for node 69ab8635efa461f55e59df75 2026-03-09 01:58:15,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:58:15,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8635efa461f55e59df75 HTTP/1.1" 200 None 2026-03-09 01:58:15,088 - lava_callback - DEBUG - Processing node 69ab8635efa461f55e59df75: baseline-arm64-baylibre 2026-03-09 01:58:15,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8635efa461f55e59df75 2026-03-09 01:58:15,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:58:15,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8635efa461f55e59df75 HTTP/1.1" 200 None 2026-03-09 01:58:15,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:58:15,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385776346680f7a28dbe0 HTTP/1.1" 200 None 2026-03-09 01:58:15,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:58:15,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8635efa461f55e59df75 HTTP/1.1" 200 None 2026-03-09 01:58:15,139 - lava_callback - ERROR - Error processing callback for node 69ab8635efa461f55e59df75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52718 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:59:35,248 - lava_callback - INFO - Received callback for node 69ab8683efa461f55e59df9c INFO: 172.21.0.1:57450 - "POST /node/69ab8683efa461f55e59df9c HTTP/1.0" 202 Accepted 2026-03-09 01:59:35,254 - lava_callback - INFO - Starting async job submit for node 69ab8683efa461f55e59df9c 2026-03-09 01:59:35,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:35,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8683efa461f55e59df9c HTTP/1.1" 200 None 2026-03-09 01:59:35,270 - lava_callback - DEBUG - Processing node 69ab8683efa461f55e59df9c: baseline-arm64-baylibre 2026-03-09 01:59:35,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8683efa461f55e59df9c 2026-03-09 01:59:35,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:35,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8683efa461f55e59df9c HTTP/1.1" 200 None 2026-03-09 01:59:35,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:35,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385776346680f7a28dbe0 HTTP/1.1" 200 None 2026-03-09 01:59:35,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:35,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8683efa461f55e59df9c HTTP/1.1" 200 None 2026-03-09 01:59:35,326 - lava_callback - ERROR - Error processing callback for node 69ab8683efa461f55e59df9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51126 - "GET / HTTP/1.0" 200 OK 2026-03-09 01:59:58,094 - lava_callback - INFO - Received callback for node 69ae27e98ef4424e487b4400 INFO: 172.21.0.1:51142 - "POST /node/69ae27e98ef4424e487b4400 HTTP/1.0" 202 Accepted 2026-03-09 01:59:58,101 - lava_callback - INFO - Starting async job submit for node 69ae27e98ef4424e487b4400 2026-03-09 01:59:58,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:58,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae27e98ef4424e487b4400 HTTP/1.1" 200 None 2026-03-09 01:59:58,151 - lava_callback - DEBUG - Processing node 69ae27e98ef4424e487b4400: baseline-arm64 2026-03-09 01:59:58,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:59:58,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:59:58,696 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae27e98ef4424e487b4400/log.txt.gz 2026-03-09 01:59:58,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:59:59,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:59:59,197 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae27e98ef4424e487b4400/lava-logs.yaml 2026-03-09 01:59:59,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 01:59:59,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 01:59:59,403 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae27e98ef4424e487b4400/lava_callback.json.gz DEBUG: Setup failed for 69ae27e98ef4424e487b4400. Transitting job node result: incomplete -> fail 2026-03-09 01:59:59,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:59,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae27e98ef4424e487b4400 HTTP/1.1" 200 None 2026-03-09 01:59:59,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:59,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 01:59:59,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 01:59:59,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae27e98ef4424e487b4400 HTTP/1.1" 200 None 2026-03-09 01:59:59,618 - lava_callback - INFO - Completed processing callback for node 69ae27e98ef4424e487b4400 INFO: 172.21.0.1:54402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34734 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:00:27,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:00:27,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:01:35,297 - lava_callback - INFO - Received callback for node 69ab86f6efa461f55e59dfc5 INFO: 172.21.0.1:52758 - "POST /node/69ab86f6efa461f55e59dfc5 HTTP/1.0" 202 Accepted 2026-03-09 02:01:35,303 - lava_callback - INFO - Starting async job submit for node 69ab86f6efa461f55e59dfc5 2026-03-09 02:01:35,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:01:35,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab86f6efa461f55e59dfc5 HTTP/1.1" 200 None 2026-03-09 02:01:35,323 - lava_callback - DEBUG - Processing node 69ab86f6efa461f55e59dfc5: baseline-arm64-baylibre 2026-03-09 02:01:35,324 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab86f6efa461f55e59dfc5 2026-03-09 02:01:35,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:01:35,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab86f6efa461f55e59dfc5 HTTP/1.1" 200 None 2026-03-09 02:01:35,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:01:35,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385776346680f7a28dbe0 HTTP/1.1" 200 None 2026-03-09 02:01:35,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:01:35,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab86f6efa461f55e59dfc5 HTTP/1.1" 200 None 2026-03-09 02:01:35,375 - lava_callback - ERROR - Error processing callback for node 69ab86f6efa461f55e59dfc5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60588 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:05:40,564 - lava_callback - INFO - Received callback for node 69ae29a78ef4424e487b44a2 INFO: 172.21.0.1:49942 - "POST /node/69ae29a78ef4424e487b44a2 HTTP/1.0" 202 Accepted 2026-03-09 02:05:40,571 - lava_callback - INFO - Starting async job submit for node 69ae29a78ef4424e487b44a2 2026-03-09 02:05:40,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:05:40,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae29a78ef4424e487b44a2 HTTP/1.1" 200 None 2026-03-09 02:05:40,681 - lava_callback - DEBUG - Processing node 69ae29a78ef4424e487b44a2: baseline-arm64 2026-03-09 02:05:41,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:05:41,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:05:41,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae29a78ef4424e487b44a2/log.txt.gz 2026-03-09 02:05:41,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:05:41,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:05:41,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae29a78ef4424e487b44a2/lava-logs.yaml 2026-03-09 02:05:41,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:05:41,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:05:41,844 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae29a78ef4424e487b44a2/lava_callback.json.gz DEBUG: Setup failed for 69ae29a78ef4424e487b44a2. Transitting job node result: incomplete -> fail 2026-03-09 02:05:41,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:05:41,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae29a78ef4424e487b44a2 HTTP/1.1" 200 None 2026-03-09 02:05:41,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:05:41,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 02:05:41,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:05:41,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae29a78ef4424e487b44a2 HTTP/1.1" 200 None 2026-03-09 02:05:42,038 - lava_callback - INFO - Completed processing callback for node 69ae29a78ef4424e487b44a2 INFO: 172.21.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45746 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:05:57,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:05:57,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54702 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 02:06:15,569 - lava_callback - INFO - Received callback for node 69ab8807efa461f55e59e031 INFO: 172.21.0.1:54712 - "POST /node/69ab8807efa461f55e59e031 HTTP/1.0" 202 Accepted 2026-03-09 02:06:15,574 - lava_callback - INFO - Starting async job submit for node 69ab8807efa461f55e59e031 2026-03-09 02:06:15,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:06:15,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8807efa461f55e59e031 HTTP/1.1" 200 None 2026-03-09 02:06:15,591 - lava_callback - DEBUG - Processing node 69ab8807efa461f55e59e031: baseline-arm64-baylibre 2026-03-09 02:06:15,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab8807efa461f55e59e031 2026-03-09 02:06:15,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:06:15,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab8807efa461f55e59e031 HTTP/1.1" 200 None 2026-03-09 02:06:15,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:06:15,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a385776346680f7a28dbe0 HTTP/1.1" 200 None 2026-03-09 02:06:15,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:06:15,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab8807efa461f55e59e031 HTTP/1.1" 200 None 2026-03-09 02:06:15,642 - lava_callback - ERROR - Error processing callback for node 69ab8807efa461f55e59e031: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39956 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:08:08,636 - lava_callback - INFO - Received callback for node 69ae17758ef4424e487b3450 INFO: 172.21.0.1:39972 - "POST /node/69ae17758ef4424e487b3450 HTTP/1.0" 202 Accepted 2026-03-09 02:08:08,642 - lava_callback - INFO - Starting async job submit for node 69ae17758ef4424e487b3450 2026-03-09 02:08:08,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:08,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17758ef4424e487b3450 HTTP/1.1" 200 None 2026-03-09 02:08:08,680 - lava_callback - DEBUG - Processing node 69ae17758ef4424e487b3450: baseline-arm-broonie 2026-03-09 02:08:08,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:09,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:09,111 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17758ef4424e487b3450/log.txt.gz 2026-03-09 02:08:09,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:09,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:09,445 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17758ef4424e487b3450/lava-logs.yaml 2026-03-09 02:08:09,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:09,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:09,579 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17758ef4424e487b3450/lava_callback.json.gz 2026-03-09 02:08:09,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:09,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17758ef4424e487b3450 HTTP/1.1" 200 None 2026-03-09 02:08:09,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:09,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 02:08:09,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:09,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17758ef4424e487b3450 HTTP/1.1" 200 None 2026-03-09 02:08:09,667 - lava_callback - INFO - Completed processing callback for node 69ae17758ef4424e487b3450 2026-03-09 02:08:14,872 - lava_callback - INFO - Received callback for node 69ae17768ef4424e487b3453 INFO: 172.21.0.1:38306 - "POST /node/69ae17768ef4424e487b3453 HTTP/1.0" 202 Accepted 2026-03-09 02:08:14,878 - lava_callback - INFO - Starting async job submit for node 69ae17768ef4424e487b3453 2026-03-09 02:08:14,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:14,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17768ef4424e487b3453 HTTP/1.1" 200 None 2026-03-09 02:08:14,938 - lava_callback - DEBUG - Processing node 69ae17768ef4424e487b3453: baseline-arm-broonie INFO: 172.21.0.1:38322 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 02:08:15,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:15,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:15,323 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17768ef4424e487b3453/log.txt.gz 2026-03-09 02:08:15,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:15,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:15,668 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17768ef4424e487b3453/lava-logs.yaml 2026-03-09 02:08:15,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:15,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:15,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae17768ef4424e487b3453/lava_callback.json.gz 2026-03-09 02:08:15,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:15,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17768ef4424e487b3453 HTTP/1.1" 200 None 2026-03-09 02:08:15,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:15,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 02:08:15,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:15,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17768ef4424e487b3453 HTTP/1.1" 200 None 2026-03-09 02:08:15,931 - lava_callback - INFO - Completed processing callback for node 69ae17768ef4424e487b3453 INFO: 172.21.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:08:27,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:27,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:08:45,501 - lava_callback - INFO - Received callback for node 69ae1ef48ef4424e487b3e72 INFO: 172.21.0.1:50286 - "POST /node/69ae1ef48ef4424e487b3e72 HTTP/1.0" 202 Accepted 2026-03-09 02:08:45,510 - lava_callback - INFO - Starting async job submit for node 69ae1ef48ef4424e487b3e72 2026-03-09 02:08:45,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:45,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef48ef4424e487b3e72 HTTP/1.1" 200 None 2026-03-09 02:08:45,571 - lava_callback - DEBUG - Processing node 69ae1ef48ef4424e487b3e72: baseline-nfs-arm64 INFO: 172.21.0.1:50294 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:08:46,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:46,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:46,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef48ef4424e487b3e72/log.txt.gz 2026-03-09 02:08:47,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:47,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:47,900 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef48ef4424e487b3e72/lava-logs.yaml 2026-03-09 02:08:47,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:08:48,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:08:48,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef48ef4424e487b3e72/lava_callback.json.gz DEBUG: Setup failed for 69ae1ef48ef4424e487b3e72. Transitting job node result: incomplete -> fail 2026-03-09 02:08:48,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:48,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef48ef4424e487b3e72 HTTP/1.1" 200 None 2026-03-09 02:08:48,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:48,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 02:08:48,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:48,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef48ef4424e487b3e72 HTTP/1.1" 200 None 2026-03-09 02:08:48,363 - lava_callback - INFO - Completed processing callback for node 69ae1ef48ef4424e487b3e72 INFO: 172.21.0.1:43996 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:08:57,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:08:57,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46438 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:11:35,131 - lava_callback - INFO - Received callback for node 69ae17a38ef4424e487b3535 INFO: 172.21.0.1:57196 - "POST /node/69ae17a38ef4424e487b3535 HTTP/1.0" 202 Accepted 2026-03-09 02:11:35,138 - lava_callback - INFO - Starting async job submit for node 69ae17a38ef4424e487b3535 2026-03-09 02:11:35,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:35,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17a38ef4424e487b3535 HTTP/1.1" 200 None 2026-03-09 02:11:35,180 - lava_callback - DEBUG - Processing node 69ae17a38ef4424e487b3535: ltp-syscalls 2026-03-09 02:11:35,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:11:36,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:11:36,101 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69ae17a38ef4424e487b3535/log.txt.gz 2026-03-09 02:11:36,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:57208 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:11:36,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:11:36,741 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69ae17a38ef4424e487b3535/lava-logs.yaml 2026-03-09 02:11:36,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:11:36,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:11:36,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69ae17a38ef4424e487b3535/lava_callback.json.gz 2026-03-09 02:11:36,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:36,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17a38ef4424e487b3535 HTTP/1.1" 200 None 2026-03-09 02:11:36,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:36,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 02:11:36,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:37,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17a38ef4424e487b3535 HTTP/1.1" 200 None 2026-03-09 02:11:37,022 - lava_callback - INFO - Completed processing callback for node 69ae17a38ef4424e487b3535 2026-03-09 02:11:40,068 - lava_callback - INFO - Received callback for node 69ae25878ef4424e487b42f1 INFO: 172.21.0.1:48862 - "POST /node/69ae25878ef4424e487b42f1 HTTP/1.0" 202 Accepted 2026-03-09 02:11:40,122 - lava_callback - INFO - Starting async job submit for node 69ae25878ef4424e487b42f1 2026-03-09 02:11:40,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:40,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25878ef4424e487b42f1 HTTP/1.1" 200 None 2026-03-09 02:11:40,190 - lava_callback - DEBUG - Processing node 69ae25878ef4424e487b42f1: baseline-arm-pengutronix 2026-03-09 02:11:40,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:11:40,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:11:40,415 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae25878ef4424e487b42f1/log.txt.gz 2026-03-09 02:11:40,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:11:40,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:11:40,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae25878ef4424e487b42f1/lava-logs.yaml 2026-03-09 02:11:40,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:11:40,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:11:40,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae25878ef4424e487b42f1/lava_callback.json.gz DEBUG: Setup failed for 69ae25878ef4424e487b42f1. Transitting job node result: incomplete -> fail 2026-03-09 02:11:40,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:40,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25878ef4424e487b42f1 HTTP/1.1" 200 None 2026-03-09 02:11:40,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:40,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 02:11:40,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:40,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae25878ef4424e487b42f1 HTTP/1.1" 200 None 2026-03-09 02:11:40,993 - lava_callback - INFO - Completed processing callback for node 69ae25878ef4424e487b42f1 INFO: 172.21.0.1:48872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60900 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:11:57,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:11:57,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42014 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 02:14:16,133 - lava_callback - INFO - Received callback for node 69ab89f5efa461f55e59e0fc INFO: 172.21.0.1:42026 - "POST /node/69ab89f5efa461f55e59e0fc HTTP/1.0" 202 Accepted 2026-03-09 02:14:16,139 - lava_callback - INFO - Starting async job submit for node 69ab89f5efa461f55e59e0fc 2026-03-09 02:14:16,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:14:16,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab89f5efa461f55e59e0fc HTTP/1.1" 200 None 2026-03-09 02:14:16,154 - lava_callback - DEBUG - Processing node 69ab89f5efa461f55e59e0fc: baseline-arm64-baylibre 2026-03-09 02:14:16,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab89f5efa461f55e59e0fc 2026-03-09 02:14:16,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:14:16,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab89f5efa461f55e59e0fc HTTP/1.1" 200 None 2026-03-09 02:14:16,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:14:16,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 02:14:16,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:14:16,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab89f5efa461f55e59e0fc HTTP/1.1" 200 None 2026-03-09 02:14:16,212 - lava_callback - ERROR - Error processing callback for node 69ab89f5efa461f55e59e0fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57144 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:16:28,060 - lava_callback - INFO - Received callback for node 69ae25528ef4424e487b42c7 INFO: 172.21.0.1:57156 - "POST /node/69ae25528ef4424e487b42c7 HTTP/1.0" 202 Accepted 2026-03-09 02:16:28,066 - lava_callback - INFO - Starting async job submit for node 69ae25528ef4424e487b42c7 2026-03-09 02:16:28,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:16:28,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25528ef4424e487b42c7 HTTP/1.1" 200 None 2026-03-09 02:16:28,125 - lava_callback - DEBUG - Processing node 69ae25528ef4424e487b42c7: baseline-arm 2026-03-09 02:16:28,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:16:28,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:16:28,807 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae25528ef4424e487b42c7/log.txt.gz 2026-03-09 02:16:29,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:16:29,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:16:29,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae25528ef4424e487b42c7/lava-logs.yaml 2026-03-09 02:16:29,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:16:29,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:16:29,574 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae25528ef4424e487b42c7/lava_callback.json.gz DEBUG: Setup failed for 69ae25528ef4424e487b42c7. Transitting job node result: incomplete -> fail 2026-03-09 02:16:29,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:16:29,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae25528ef4424e487b42c7 HTTP/1.1" 200 None 2026-03-09 02:16:29,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:16:29,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 02:16:29,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:16:29,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae25528ef4424e487b42c7 HTTP/1.1" 200 None 2026-03-09 02:16:29,827 - lava_callback - INFO - Completed processing callback for node 69ae25528ef4424e487b42c7 INFO: 172.21.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:16:57,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:16:57,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52192 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:20:30,705 - lava_callback - INFO - Received callback for node 69ae1a838ef4424e487b39a9 INFO: 172.21.0.1:60434 - "POST /node/69ae1a838ef4424e487b39a9 HTTP/1.0" 202 Accepted 2026-03-09 02:20:30,712 - lava_callback - INFO - Starting async job submit for node 69ae1a838ef4424e487b39a9 2026-03-09 02:20:30,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:20:30,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a838ef4424e487b39a9 HTTP/1.1" 200 None 2026-03-09 02:20:30,790 - lava_callback - DEBUG - Processing node 69ae1a838ef4424e487b39a9: ltp-smoke 2026-03-09 02:20:31,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:20:31,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:20:31,481 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae1a838ef4424e487b39a9/log.txt.gz 2026-03-09 02:20:31,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:20:32,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:20:32,031 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae1a838ef4424e487b39a9/lava-logs.yaml 2026-03-09 02:20:32,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:20:32,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:20:32,283 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae1a838ef4424e487b39a9/lava_callback.json.gz 2026-03-09 02:20:32,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:20:32,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a838ef4424e487b39a9 HTTP/1.1" 200 None 2026-03-09 02:20:32,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:20:32,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae116d8ef4424e487b2ea8 HTTP/1.1" 200 None 2026-03-09 02:20:32,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:20:32,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a838ef4424e487b39a9 HTTP/1.1" 200 None 2026-03-09 02:20:32,571 - lava_callback - INFO - Completed processing callback for node 69ae1a838ef4424e487b39a9 INFO: 172.21.0.1:60440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38436 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:20:57,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:20:58,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36838 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:31:20,839 - lava_callback - INFO - Received callback for node 69ae2c6c8ef4424e487b455d INFO: 172.21.0.1:55312 - "POST /node/69ae2c6c8ef4424e487b455d HTTP/1.0" 202 Accepted 2026-03-09 02:31:20,890 - lava_callback - INFO - Starting async job submit for node 69ae2c6c8ef4424e487b455d 2026-03-09 02:31:20,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:31:20,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2c6c8ef4424e487b455d HTTP/1.1" 200 None 2026-03-09 02:31:20,956 - lava_callback - DEBUG - Processing node 69ae2c6c8ef4424e487b455d: baseline-arm-pengutronix 2026-03-09 02:31:21,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:31:21,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:31:21,355 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae2c6c8ef4424e487b455d/log.txt.gz 2026-03-09 02:31:21,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:31:21,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:31:21,598 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae2c6c8ef4424e487b455d/lava-logs.yaml 2026-03-09 02:31:21,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:31:21,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:31:21,739 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69ae2c6c8ef4424e487b455d/lava_callback.json.gz DEBUG: Setup failed for 69ae2c6c8ef4424e487b455d. Transitting job node result: incomplete -> fail 2026-03-09 02:31:21,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:31:21,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2c6c8ef4424e487b455d HTTP/1.1" 200 None 2026-03-09 02:31:21,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:31:21,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 02:31:21,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:31:21,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae2c6c8ef4424e487b455d HTTP/1.1" 200 None 2026-03-09 02:31:21,904 - lava_callback - INFO - Completed processing callback for node 69ae2c6c8ef4424e487b455d INFO: 172.21.0.1:55314 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:31:28,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:31:28,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:36:58,541 - lava_callback - INFO - Received callback for node 69ae2d7e8ef4424e487b45cd INFO: 172.21.0.1:35634 - "POST /node/69ae2d7e8ef4424e487b45cd HTTP/1.0" 202 Accepted 2026-03-09 02:36:58,547 - lava_callback - INFO - Starting async job submit for node 69ae2d7e8ef4424e487b45cd 2026-03-09 02:36:58,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:36:58,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2d7e8ef4424e487b45cd HTTP/1.1" 200 None 2026-03-09 02:36:58,784 - lava_callback - DEBUG - Processing node 69ae2d7e8ef4424e487b45cd: baseline-arm 2026-03-09 02:36:59,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:36:59,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:36:59,348 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69ae2d7e8ef4424e487b45cd/log.txt.gz 2026-03-09 02:36:59,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:36:59,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:36:59,861 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69ae2d7e8ef4424e487b45cd/lava-logs.yaml 2026-03-09 02:36:59,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:37:00,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:37:00,073 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69ae2d7e8ef4424e487b45cd/lava_callback.json.gz DEBUG: Setup failed for 69ae2d7e8ef4424e487b45cd. Transitting job node result: incomplete -> fail 2026-03-09 02:37:00,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:37:00,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2d7e8ef4424e487b45cd HTTP/1.1" 200 None 2026-03-09 02:37:00,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:37:00,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-09 02:37:00,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:37:00,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae2d7e8ef4424e487b45cd HTTP/1.1" 200 None 2026-03-09 02:37:00,315 - lava_callback - INFO - Completed processing callback for node 69ae2d7e8ef4424e487b45cd INFO: 172.21.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:37:28,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:37:28,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41110 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:38:29,829 - lava_callback - INFO - Received callback for node 69ae2e728ef4424e487b4630 INFO: 172.21.0.1:57592 - "POST /node/69ae2e728ef4424e487b4630 HTTP/1.0" 202 Accepted 2026-03-09 02:38:29,836 - lava_callback - INFO - Starting async job submit for node 69ae2e728ef4424e487b4630 2026-03-09 02:38:29,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:38:29,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2e728ef4424e487b4630 HTTP/1.1" 200 None 2026-03-09 02:38:29,909 - lava_callback - DEBUG - Processing node 69ae2e728ef4424e487b4630: ltp-smoke 2026-03-09 02:38:30,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:38:30,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:38:30,569 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae2e728ef4424e487b4630/log.txt.gz 2026-03-09 02:38:30,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:38:31,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:38:31,107 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae2e728ef4424e487b4630/lava-logs.yaml 2026-03-09 02:38:31,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:38:31,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:38:31,367 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae2e728ef4424e487b4630/lava_callback.json.gz 2026-03-09 02:38:31,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:38:31,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2e728ef4424e487b4630 HTTP/1.1" 200 None 2026-03-09 02:38:31,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:38:31,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae116d8ef4424e487b2ea8 HTTP/1.1" 200 None 2026-03-09 02:38:31,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:38:31,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae2e728ef4424e487b4630 HTTP/1.1" 200 None 2026-03-09 02:38:31,712 - lava_callback - INFO - Completed processing callback for node 69ae2e728ef4424e487b4630 INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44556 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:38:58,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:38:58,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39836 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:44:37,657 - lava_callback - INFO - Received callback for node 69ab9113efa461f55e59e2ec INFO: 172.21.0.1:39838 - "POST /node/69ab9113efa461f55e59e2ec HTTP/1.0" 202 Accepted 2026-03-09 02:44:37,662 - lava_callback - INFO - Starting async job submit for node 69ab9113efa461f55e59e2ec 2026-03-09 02:44:37,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:44:37,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9113efa461f55e59e2ec HTTP/1.1" 200 None 2026-03-09 02:44:37,685 - lava_callback - DEBUG - Processing node 69ab9113efa461f55e59e2ec: baseline-arm64-baylibre 2026-03-09 02:44:37,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab9113efa461f55e59e2ec 2026-03-09 02:44:37,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:44:37,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9113efa461f55e59e2ec HTTP/1.1" 200 None 2026-03-09 02:44:37,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:44:37,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 02:44:37,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:44:37,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab9113efa461f55e59e2ec HTTP/1.1" 200 None 2026-03-09 02:44:37,737 - lava_callback - ERROR - Error processing callback for node 69ab9113efa461f55e59e2ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50316 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:48:00,854 - lava_callback - INFO - Received callback for node 69ab91daefa461f55e59e35b INFO: 172.21.0.1:47088 - "POST /node/69ab91daefa461f55e59e35b HTTP/1.0" 202 Accepted 2026-03-09 02:48:00,860 - lava_callback - INFO - Starting async job submit for node 69ab91daefa461f55e59e35b 2026-03-09 02:48:00,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:00,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab91daefa461f55e59e35b HTTP/1.1" 200 None 2026-03-09 02:48:00,876 - lava_callback - DEBUG - Processing node 69ab91daefa461f55e59e35b: baseline-arm-clabbe 2026-03-09 02:48:00,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab91daefa461f55e59e35b 2026-03-09 02:48:00,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:00,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab91daefa461f55e59e35b HTTP/1.1" 200 None 2026-03-09 02:48:00,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:00,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-09 02:48:00,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:00,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab91daefa461f55e59e35b HTTP/1.1" 200 None 2026-03-09 02:48:00,969 - lava_callback - ERROR - Error processing callback for node 69ab91daefa461f55e59e35b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:48:10,821 - lava_callback - INFO - Received callback for node 69ae1f1d8ef4424e487b3f10 INFO: 172.21.0.1:51856 - "POST /node/69ae1f1d8ef4424e487b3f10 HTTP/1.0" 202 Accepted 2026-03-09 02:48:10,827 - lava_callback - INFO - Starting async job submit for node 69ae1f1d8ef4424e487b3f10 2026-03-09 02:48:10,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:10,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f1d8ef4424e487b3f10 HTTP/1.1" 200 None 2026-03-09 02:48:10,866 - lava_callback - DEBUG - Processing node 69ae1f1d8ef4424e487b3f10: baseline-arm64-broonie 2026-03-09 02:48:11,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:48:11,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:48:11,259 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f1d8ef4424e487b3f10/log.txt.gz 2026-03-09 02:48:11,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:48:11,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:48:11,671 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f1d8ef4424e487b3f10/lava-logs.yaml 2026-03-09 02:48:11,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:48:11,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:48:11,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f1d8ef4424e487b3f10/lava_callback.json.gz 2026-03-09 02:48:11,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:11,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f1d8ef4424e487b3f10 HTTP/1.1" 200 None 2026-03-09 02:48:11,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:11,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 02:48:11,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:11,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f1d8ef4424e487b3f10 HTTP/1.1" 200 None 2026-03-09 02:48:11,917 - lava_callback - INFO - Completed processing callback for node 69ae1f1d8ef4424e487b3f10 INFO: 172.21.0.1:51866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37430 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:48:28,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:28,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:48:57,907 - lava_callback - INFO - Received callback for node 69ab9206efa461f55e59e36d INFO: 172.21.0.1:52736 - "POST /node/69ab9206efa461f55e59e36d HTTP/1.0" 202 Accepted 2026-03-09 02:48:57,912 - lava_callback - INFO - Starting async job submit for node 69ab9206efa461f55e59e36d 2026-03-09 02:48:57,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:57,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9206efa461f55e59e36d HTTP/1.1" 200 None 2026-03-09 02:48:57,928 - lava_callback - DEBUG - Processing node 69ab9206efa461f55e59e36d: baseline-arm64-baylibre 2026-03-09 02:48:57,929 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab9206efa461f55e59e36d 2026-03-09 02:48:57,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:57,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9206efa461f55e59e36d HTTP/1.1" 200 None 2026-03-09 02:48:57,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:57,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 02:48:57,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:48:58,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab9206efa461f55e59e36d HTTP/1.1" 200 None 2026-03-09 02:48:58,042 - lava_callback - ERROR - Error processing callback for node 69ab9206efa461f55e59e36d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36200 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:54:41,164 - lava_callback - INFO - Received callback for node 69ab936eefa461f55e59e3f2 INFO: 172.21.0.1:48292 - "POST /node/69ab936eefa461f55e59e3f2 HTTP/1.0" 202 Accepted 2026-03-09 02:54:41,169 - lava_callback - INFO - Starting async job submit for node 69ab936eefa461f55e59e3f2 2026-03-09 02:54:41,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:54:41,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab936eefa461f55e59e3f2 HTTP/1.1" 200 None 2026-03-09 02:54:41,185 - lava_callback - DEBUG - Processing node 69ab936eefa461f55e59e3f2: baseline-arm-clabbe 2026-03-09 02:54:41,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab936eefa461f55e59e3f2 2026-03-09 02:54:41,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:54:41,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab936eefa461f55e59e3f2 HTTP/1.1" 200 None 2026-03-09 02:54:41,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:54:41,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-09 02:54:41,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:54:41,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab936eefa461f55e59e3f2 HTTP/1.1" 200 None 2026-03-09 02:54:41,239 - lava_callback - ERROR - Error processing callback for node 69ab936eefa461f55e59e3f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51616 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:56:30,551 - lava_callback - INFO - Received callback for node 69ae32a98ef4424e487b4726 INFO: 172.21.0.1:59696 - "POST /node/69ae32a98ef4424e487b4726 HTTP/1.0" 202 Accepted 2026-03-09 02:56:30,561 - lava_callback - INFO - Starting async job submit for node 69ae32a98ef4424e487b4726 2026-03-09 02:56:30,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:56:30,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae32a98ef4424e487b4726 HTTP/1.1" 200 None 2026-03-09 02:56:30,639 - lava_callback - DEBUG - Processing node 69ae32a98ef4424e487b4726: ltp-smoke 2026-03-09 02:56:31,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:56:31,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:56:31,302 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae32a98ef4424e487b4726/log.txt.gz 2026-03-09 02:56:31,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:56:31,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:56:31,830 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae32a98ef4424e487b4726/lava-logs.yaml 2026-03-09 02:56:31,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 02:56:31,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 02:56:31,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69ae32a98ef4424e487b4726/lava_callback.json.gz 2026-03-09 02:56:32,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:56:32,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae32a98ef4424e487b4726 HTTP/1.1" 200 None 2026-03-09 02:56:32,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:56:32,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae116d8ef4424e487b2ea8 HTTP/1.1" 200 None 2026-03-09 02:56:32,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:56:32,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae32a98ef4424e487b4726 HTTP/1.1" 200 None 2026-03-09 02:56:32,289 - lava_callback - INFO - Completed processing callback for node 69ae32a98ef4424e487b4726 INFO: 172.21.0.1:59706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40408 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:56:58,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:56:58,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55570 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:57:58,414 - lava_callback - INFO - Received callback for node 69ab942cefa461f55e59e437 INFO: 172.21.0.1:55572 - "POST /node/69ab942cefa461f55e59e437 HTTP/1.0" 202 Accepted 2026-03-09 02:57:58,420 - lava_callback - INFO - Starting async job submit for node 69ab942cefa461f55e59e437 2026-03-09 02:57:58,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:57:58,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab942cefa461f55e59e437 HTTP/1.1" 200 None 2026-03-09 02:57:58,436 - lava_callback - DEBUG - Processing node 69ab942cefa461f55e59e437: baseline-arm64-baylibre 2026-03-09 02:57:58,436 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab942cefa461f55e59e437 2026-03-09 02:57:58,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:57:58,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab942cefa461f55e59e437 HTTP/1.1" 200 None 2026-03-09 02:57:58,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:57:58,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-09 02:57:58,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:57:58,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab942cefa461f55e59e437 HTTP/1.1" 200 None 2026-03-09 02:57:58,497 - lava_callback - ERROR - Error processing callback for node 69ab942cefa461f55e59e437: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET / HTTP/1.0" 200 OK 2026-03-09 02:59:01,377 - lava_callback - INFO - Received callback for node 69ab946eefa461f55e59e447 INFO: 172.21.0.1:33874 - "POST /node/69ab946eefa461f55e59e447 HTTP/1.0" 202 Accepted 2026-03-09 02:59:01,382 - lava_callback - INFO - Starting async job submit for node 69ab946eefa461f55e59e447 2026-03-09 02:59:01,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:59:01,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab946eefa461f55e59e447 HTTP/1.1" 200 None 2026-03-09 02:59:01,404 - lava_callback - DEBUG - Processing node 69ab946eefa461f55e59e447: baseline-arm-clabbe 2026-03-09 02:59:01,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab946eefa461f55e59e447 2026-03-09 02:59:01,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:59:01,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab946eefa461f55e59e447 HTTP/1.1" 200 None 2026-03-09 02:59:01,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:59:01,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-09 02:59:01,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 02:59:01,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab946eefa461f55e59e447 HTTP/1.1" 200 None 2026-03-09 02:59:01,473 - lava_callback - ERROR - Error processing callback for node 69ab946eefa461f55e59e447: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37736 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:00:01,416 - lava_callback - INFO - Received callback for node 69ab949fefa461f55e59e456 INFO: 172.21.0.1:37214 - "POST /node/69ab949fefa461f55e59e456 HTTP/1.0" 202 Accepted 2026-03-09 03:00:01,422 - lava_callback - INFO - Starting async job submit for node 69ab949fefa461f55e59e456 2026-03-09 03:00:01,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:00:01,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab949fefa461f55e59e456 HTTP/1.1" 200 None 2026-03-09 03:00:01,475 - lava_callback - DEBUG - Processing node 69ab949fefa461f55e59e456: baseline-arm64-clabbe 2026-03-09 03:00:01,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab949fefa461f55e59e456 2026-03-09 03:00:01,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:00:01,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab949fefa461f55e59e456 HTTP/1.1" 200 None 2026-03-09 03:00:01,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:00:01,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-09 03:00:01,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:00:01,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab949fefa461f55e59e456 HTTP/1.1" 200 None 2026-03-09 03:00:01,531 - lava_callback - ERROR - Error processing callback for node 69ab949fefa461f55e59e456: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:01:01,433 - lava_callback - INFO - Received callback for node 69ab94eaefa461f55e59e466 INFO: 172.21.0.1:41448 - "POST /node/69ab94eaefa461f55e59e466 HTTP/1.0" 202 Accepted 2026-03-09 03:01:01,439 - lava_callback - INFO - Starting async job submit for node 69ab94eaefa461f55e59e466 2026-03-09 03:01:01,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:01:01,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab94eaefa461f55e59e466 HTTP/1.1" 200 None 2026-03-09 03:01:01,454 - lava_callback - DEBUG - Processing node 69ab94eaefa461f55e59e466: baseline-arm64-clabbe 2026-03-09 03:01:01,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab94eaefa461f55e59e466 2026-03-09 03:01:01,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:01:01,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab94eaefa461f55e59e466 HTTP/1.1" 200 None 2026-03-09 03:01:01,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:01:01,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-09 03:01:01,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:01:01,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab94eaefa461f55e59e466 HTTP/1.1" 200 None 2026-03-09 03:01:01,507 - lava_callback - ERROR - Error processing callback for node 69ab94eaefa461f55e59e466: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46350 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:04:38,756 - lava_callback - INFO - Received callback for node 69ab95c0efa461f55e59e4bf INFO: 172.21.0.1:46352 - "POST /node/69ab95c0efa461f55e59e4bf HTTP/1.0" 202 Accepted 2026-03-09 03:04:38,762 - lava_callback - INFO - Starting async job submit for node 69ab95c0efa461f55e59e4bf 2026-03-09 03:04:38,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:04:38,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab95c0efa461f55e59e4bf HTTP/1.1" 200 None 2026-03-09 03:04:38,778 - lava_callback - DEBUG - Processing node 69ab95c0efa461f55e59e4bf: baseline-arm64-baylibre 2026-03-09 03:04:38,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab95c0efa461f55e59e4bf 2026-03-09 03:04:38,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:04:38,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab95c0efa461f55e59e4bf HTTP/1.1" 200 None 2026-03-09 03:04:38,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:04:38,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 03:04:38,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:04:38,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab95c0efa461f55e59e4bf HTTP/1.1" 200 None 2026-03-09 03:04:38,840 - lava_callback - ERROR - Error processing callback for node 69ab95c0efa461f55e59e4bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59654 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:07:18,870 - lava_callback - INFO - Received callback for node 69ab9653efa461f55e59e4f1 INFO: 172.21.0.1:59656 - "POST /node/69ab9653efa461f55e59e4f1 HTTP/1.0" 202 Accepted 2026-03-09 03:07:18,876 - lava_callback - INFO - Starting async job submit for node 69ab9653efa461f55e59e4f1 2026-03-09 03:07:18,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:07:18,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9653efa461f55e59e4f1 HTTP/1.1" 200 None 2026-03-09 03:07:18,892 - lava_callback - DEBUG - Processing node 69ab9653efa461f55e59e4f1: baseline-arm64-baylibre 2026-03-09 03:07:18,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab9653efa461f55e59e4f1 2026-03-09 03:07:18,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:07:18,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9653efa461f55e59e4f1 HTTP/1.1" 200 None 2026-03-09 03:07:18,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:07:18,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 03:07:18,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:07:18,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab9653efa461f55e59e4f1 HTTP/1.1" 200 None 2026-03-09 03:07:18,954 - lava_callback - ERROR - Error processing callback for node 69ab9653efa461f55e59e4f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42260 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 03:09:15,553 - lava_callback - INFO - Received callback for node 69ae2bba8ef4424e487b4524 INFO: 172.21.0.1:42274 - "POST /node/69ae2bba8ef4424e487b4524 HTTP/1.0" 202 Accepted 2026-03-09 03:09:15,562 - lava_callback - INFO - Starting async job submit for node 69ae2bba8ef4424e487b4524 2026-03-09 03:09:15,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:15,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2bba8ef4424e487b4524 HTTP/1.1" 200 None 2026-03-09 03:09:15,632 - lava_callback - DEBUG - Processing node 69ae2bba8ef4424e487b4524: baseline-nfs-arm64 INFO: 172.21.0.1:42286 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:09:16,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:09:16,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:09:16,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae2bba8ef4424e487b4524/log.txt.gz 2026-03-09 03:09:17,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:09:17,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:09:17,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae2bba8ef4424e487b4524/lava-logs.yaml 2026-03-09 03:09:17,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:09:18,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:09:18,044 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae2bba8ef4424e487b4524/lava_callback.json.gz DEBUG: Setup failed for 69ae2bba8ef4424e487b4524. Transitting job node result: incomplete -> fail 2026-03-09 03:09:18,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:18,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae2bba8ef4424e487b4524 HTTP/1.1" 200 None 2026-03-09 03:09:18,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:18,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 03:09:18,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:18,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae2bba8ef4424e487b4524 HTTP/1.1" 200 None 2026-03-09 03:09:18,285 - lava_callback - INFO - Completed processing callback for node 69ae2bba8ef4424e487b4524 INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:09:28,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:28,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44662 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:09:39,014 - lava_callback - INFO - Received callback for node 69ab96e6efa461f55e59e534 INFO: 172.21.0.1:44668 - "POST /node/69ab96e6efa461f55e59e534 HTTP/1.0" 202 Accepted 2026-03-09 03:09:39,020 - lava_callback - INFO - Starting async job submit for node 69ab96e6efa461f55e59e534 2026-03-09 03:09:39,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:39,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab96e6efa461f55e59e534 HTTP/1.1" 200 None 2026-03-09 03:09:39,041 - lava_callback - DEBUG - Processing node 69ab96e6efa461f55e59e534: baseline-arm64-baylibre 2026-03-09 03:09:39,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab96e6efa461f55e59e534 2026-03-09 03:09:39,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:39,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab96e6efa461f55e59e534 HTTP/1.1" 200 None 2026-03-09 03:09:39,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:39,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 03:09:39,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:09:39,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab96e6efa461f55e59e534 HTTP/1.1" 200 None 2026-03-09 03:09:39,099 - lava_callback - ERROR - Error processing callback for node 69ab96e6efa461f55e59e534: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56816 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:10:59,138 - lava_callback - INFO - Received callback for node 69ab9736efa461f55e59e55b INFO: 172.21.0.1:56822 - "POST /node/69ab9736efa461f55e59e55b HTTP/1.0" 202 Accepted 2026-03-09 03:10:59,144 - lava_callback - INFO - Starting async job submit for node 69ab9736efa461f55e59e55b 2026-03-09 03:10:59,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:10:59,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9736efa461f55e59e55b HTTP/1.1" 200 None 2026-03-09 03:10:59,160 - lava_callback - DEBUG - Processing node 69ab9736efa461f55e59e55b: baseline-arm64-baylibre 2026-03-09 03:10:59,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab9736efa461f55e59e55b 2026-03-09 03:10:59,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:10:59,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9736efa461f55e59e55b HTTP/1.1" 200 None 2026-03-09 03:10:59,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:10:59,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 03:10:59,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:10:59,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab9736efa461f55e59e55b HTTP/1.1" 200 None 2026-03-09 03:10:59,214 - lava_callback - ERROR - Error processing callback for node 69ab9736efa461f55e59e55b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:12:51,017 - lava_callback - INFO - Received callback for node 69ae1f168ef4424e487b3eea INFO: 172.21.0.1:51526 - "POST /node/69ae1f168ef4424e487b3eea HTTP/1.0" 202 Accepted 2026-03-09 03:12:51,023 - lava_callback - INFO - Starting async job submit for node 69ae1f168ef4424e487b3eea 2026-03-09 03:12:51,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:12:51,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f168ef4424e487b3eea HTTP/1.1" 200 None 2026-03-09 03:12:51,063 - lava_callback - DEBUG - Processing node 69ae1f168ef4424e487b3eea: baseline-arm64-broonie 2026-03-09 03:12:51,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:12:51,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:12:51,476 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f168ef4424e487b3eea/log.txt.gz 2026-03-09 03:12:51,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:12:51,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:12:51,848 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f168ef4424e487b3eea/lava-logs.yaml 2026-03-09 03:12:51,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:12:52,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:12:52,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f168ef4424e487b3eea/lava_callback.json.gz 2026-03-09 03:12:52,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:12:52,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f168ef4424e487b3eea HTTP/1.1" 200 None 2026-03-09 03:12:52,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:12:52,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 03:12:52,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:12:52,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f168ef4424e487b3eea HTTP/1.1" 200 None 2026-03-09 03:12:52,219 - lava_callback - INFO - Completed processing callback for node 69ae1f168ef4424e487b3eea INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:12:58,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:12:58,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49228 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:13:42,679 - lava_callback - INFO - Received callback for node 69ae1ede8ef4424e487b3e39 INFO: 172.21.0.1:51372 - "POST /node/69ae1ede8ef4424e487b3e39 HTTP/1.0" 202 Accepted 2026-03-09 03:13:42,685 - lava_callback - INFO - Starting async job submit for node 69ae1ede8ef4424e487b3e39 2026-03-09 03:13:42,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:13:42,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ede8ef4424e487b3e39 HTTP/1.1" 200 None 2026-03-09 03:13:42,717 - lava_callback - DEBUG - Processing node 69ae1ede8ef4424e487b3e39: baseline-arm64 2026-03-09 03:13:43,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:13:43,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:13:43,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ede8ef4424e487b3e39/log.txt.gz 2026-03-09 03:13:43,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:13:43,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:13:43,616 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ede8ef4424e487b3e39/lava-logs.yaml 2026-03-09 03:13:43,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:13:43,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:13:43,783 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69ae1ede8ef4424e487b3e39/lava_callback.json.gz 2026-03-09 03:13:43,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:13:43,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ede8ef4424e487b3e39 HTTP/1.1" 200 None 2026-03-09 03:13:43,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:13:43,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 03:13:43,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:13:43,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ede8ef4424e487b3e39 HTTP/1.1" 200 None 2026-03-09 03:13:43,869 - lava_callback - INFO - Completed processing callback for node 69ae1ede8ef4424e487b3e39 INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:13:58,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:13:58,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52982 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:14:19,365 - lava_callback - INFO - Received callback for node 69ab97fcefa461f55e59e598 INFO: 172.21.0.1:52998 - "POST /node/69ab97fcefa461f55e59e598 HTTP/1.0" 202 Accepted 2026-03-09 03:14:19,371 - lava_callback - INFO - Starting async job submit for node 69ab97fcefa461f55e59e598 2026-03-09 03:14:19,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:14:19,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab97fcefa461f55e59e598 HTTP/1.1" 200 None 2026-03-09 03:14:19,413 - lava_callback - DEBUG - Processing node 69ab97fcefa461f55e59e598: baseline-arm64-baylibre 2026-03-09 03:14:19,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab97fcefa461f55e59e598 2026-03-09 03:14:19,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:14:19,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab97fcefa461f55e59e598 HTTP/1.1" 200 None 2026-03-09 03:14:19,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:14:19,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-09 03:14:19,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:14:19,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab97fcefa461f55e59e598 HTTP/1.1" 200 None 2026-03-09 03:14:19,492 - lava_callback - ERROR - Error processing callback for node 69ab97fcefa461f55e59e598: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43422 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:15:39,352 - lava_callback - INFO - Received callback for node 69ab9847efa461f55e59e5be INFO: 172.21.0.1:43428 - "POST /node/69ab9847efa461f55e59e5be HTTP/1.0" 202 Accepted 2026-03-09 03:15:39,357 - lava_callback - INFO - Starting async job submit for node 69ab9847efa461f55e59e5be 2026-03-09 03:15:39,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:15:39,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9847efa461f55e59e5be HTTP/1.1" 200 None 2026-03-09 03:15:39,373 - lava_callback - DEBUG - Processing node 69ab9847efa461f55e59e5be: baseline-arm64-baylibre 2026-03-09 03:15:39,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab9847efa461f55e59e5be 2026-03-09 03:15:39,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:15:39,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab9847efa461f55e59e5be HTTP/1.1" 200 None 2026-03-09 03:15:39,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:15:39,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a5e0fe32aaccb4c112eb82 HTTP/1.1" 200 None 2026-03-09 03:15:39,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:15:39,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab9847efa461f55e59e5be HTTP/1.1" 200 None 2026-03-09 03:15:39,433 - lava_callback - ERROR - Error processing callback for node 69ab9847efa461f55e59e5be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43716 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:17:32,606 - lava_callback - INFO - Received callback for node 69ae1ef18ef4424e487b3e69 INFO: 172.21.0.1:54690 - "POST /node/69ae1ef18ef4424e487b3e69 HTTP/1.0" 202 Accepted 2026-03-09 03:17:32,612 - lava_callback - INFO - Starting async job submit for node 69ae1ef18ef4424e487b3e69 2026-03-09 03:17:32,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:17:32,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef18ef4424e487b3e69 HTTP/1.1" 200 None 2026-03-09 03:17:32,646 - lava_callback - DEBUG - Processing node 69ae1ef18ef4424e487b3e69: baseline-nfs-arm64 2026-03-09 03:17:33,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:17:33,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:17:33,246 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef18ef4424e487b3e69/log.txt.gz 2026-03-09 03:17:33,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:17:33,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:17:33,734 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef18ef4424e487b3e69/lava-logs.yaml 2026-03-09 03:17:33,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:17:33,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:17:33,912 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae1ef18ef4424e487b3e69/lava_callback.json.gz 2026-03-09 03:17:33,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:17:33,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ef18ef4424e487b3e69 HTTP/1.1" 200 None 2026-03-09 03:17:33,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:17:33,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 03:17:33,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:17:34,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ef18ef4424e487b3e69 HTTP/1.1" 200 None 2026-03-09 03:17:34,004 - lava_callback - INFO - Completed processing callback for node 69ae1ef18ef4424e487b3e69 INFO: 172.21.0.1:54700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46748 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:17:58,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:17:58,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 03:18:02,210 - lava_callback - INFO - Received callback for node 69ab98d4efa461f55e59e5ec INFO: 172.21.0.1:39160 - "POST /node/69ab98d4efa461f55e59e5ec HTTP/1.0" 202 Accepted 2026-03-09 03:18:02,216 - lava_callback - INFO - Starting async job submit for node 69ab98d4efa461f55e59e5ec 2026-03-09 03:18:02,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:18:02,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab98d4efa461f55e59e5ec HTTP/1.1" 200 None 2026-03-09 03:18:02,234 - lava_callback - DEBUG - Processing node 69ab98d4efa461f55e59e5ec: baseline-arm64-clabbe 2026-03-09 03:18:02,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab98d4efa461f55e59e5ec 2026-03-09 03:18:02,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:18:02,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab98d4efa461f55e59e5ec HTTP/1.1" 200 None 2026-03-09 03:18:02,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:18:02,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-09 03:18:02,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:18:02,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab98d4efa461f55e59e5ec HTTP/1.1" 200 None 2026-03-09 03:18:02,339 - lava_callback - ERROR - Error processing callback for node 69ab98d4efa461f55e59e5ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59292 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:19:02,238 - lava_callback - INFO - Received callback for node 69ab991fefa461f55e59e60e INFO: 172.21.0.1:57282 - "POST /node/69ab991fefa461f55e59e60e HTTP/1.0" 202 Accepted 2026-03-09 03:19:02,243 - lava_callback - INFO - Starting async job submit for node 69ab991fefa461f55e59e60e 2026-03-09 03:19:02,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:19:02,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab991fefa461f55e59e60e HTTP/1.1" 200 None 2026-03-09 03:19:02,260 - lava_callback - DEBUG - Processing node 69ab991fefa461f55e59e60e: baseline-arm64-clabbe 2026-03-09 03:19:02,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ab991fefa461f55e59e60e 2026-03-09 03:19:02,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:19:02,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab991fefa461f55e59e60e HTTP/1.1" 200 None 2026-03-09 03:19:02,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:19:02,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-09 03:19:02,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:19:02,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ab991fefa461f55e59e60e HTTP/1.1" 200 None 2026-03-09 03:19:02,316 - lava_callback - ERROR - Error processing callback for node 69ab991fefa461f55e59e60e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:29:27,766 - lava_callback - INFO - Received callback for node 69ae39e68ef4424e487b4a58 INFO: 172.21.0.1:41252 - "POST /node/69ae39e68ef4424e487b4a58 HTTP/1.0" 202 Accepted 2026-03-09 03:29:27,775 - lava_callback - INFO - Starting async job submit for node 69ae39e68ef4424e487b4a58 2026-03-09 03:29:27,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:29:27,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae39e68ef4424e487b4a58 HTTP/1.1" 200 None 2026-03-09 03:29:27,841 - lava_callback - DEBUG - Processing node 69ae39e68ef4424e487b4a58: baseline-nfs-arm64 2026-03-09 03:29:28,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:29:29,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:29:29,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae39e68ef4424e487b4a58/log.txt.gz 2026-03-09 03:29:29,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:29:30,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:29:30,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae39e68ef4424e487b4a58/lava-logs.yaml 2026-03-09 03:29:30,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:29:30,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:29:30,483 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae39e68ef4424e487b4a58/lava_callback.json.gz DEBUG: Setup failed for 69ae39e68ef4424e487b4a58. Transitting job node result: incomplete -> fail 2026-03-09 03:29:30,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:29:30,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae39e68ef4424e487b4a58 HTTP/1.1" 200 None 2026-03-09 03:29:30,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:29:30,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 03:29:30,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:29:30,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae39e68ef4424e487b4a58 HTTP/1.1" 200 None 2026-03-09 03:29:30,724 - lava_callback - INFO - Completed processing callback for node 69ae39e68ef4424e487b4a58 INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36134 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:29:58,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:29:58,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52408 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:49:36,854 - lava_callback - INFO - Received callback for node 69ae3ea28ef4424e487b4ba9 INFO: 172.21.0.1:52422 - "POST /node/69ae3ea28ef4424e487b4ba9 HTTP/1.0" 202 Accepted 2026-03-09 03:49:36,862 - lava_callback - INFO - Starting async job submit for node 69ae3ea28ef4424e487b4ba9 2026-03-09 03:49:36,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:49:36,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3ea28ef4424e487b4ba9 HTTP/1.1" 200 None 2026-03-09 03:49:36,920 - lava_callback - DEBUG - Processing node 69ae3ea28ef4424e487b4ba9: baseline-nfs-arm64 2026-03-09 03:49:38,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:49:38,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:49:38,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae3ea28ef4424e487b4ba9/log.txt.gz 2026-03-09 03:49:38,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:49:39,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:49:39,142 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae3ea28ef4424e487b4ba9/lava-logs.yaml 2026-03-09 03:49:39,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 03:49:39,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 03:49:39,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69ae3ea28ef4424e487b4ba9/lava_callback.json.gz DEBUG: Setup failed for 69ae3ea28ef4424e487b4ba9. Transitting job node result: incomplete -> fail 2026-03-09 03:49:39,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:49:39,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3ea28ef4424e487b4ba9 HTTP/1.1" 200 None 2026-03-09 03:49:39,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:49:39,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 03:49:39,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:49:39,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae3ea28ef4424e487b4ba9 HTTP/1.1" 200 None 2026-03-09 03:49:39,713 - lava_callback - INFO - Completed processing callback for node 69ae3ea28ef4424e487b4ba9 INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35550 - "GET / HTTP/1.0" 200 OK 2026-03-09 03:49:58,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 03:49:58,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK 2026-03-09 04:26:00,005 - lava_callback - INFO - Received callback for node 69ae1f1b8ef4424e487b3f0d INFO: 172.21.0.1:41512 - "POST /node/69ae1f1b8ef4424e487b3f0d HTTP/1.0" 202 Accepted 2026-03-09 04:26:00,011 - lava_callback - INFO - Starting async job submit for node 69ae1f1b8ef4424e487b3f0d 2026-03-09 04:26:00,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:26:00,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f1b8ef4424e487b3f0d HTTP/1.1" 200 None 2026-03-09 04:26:00,049 - lava_callback - DEBUG - Processing node 69ae1f1b8ef4424e487b3f0d: baseline-arm64-broonie 2026-03-09 04:26:00,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 04:26:00,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 04:26:00,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f1b8ef4424e487b3f0d/log.txt.gz 2026-03-09 04:26:00,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 04:26:00,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 04:26:00,804 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f1b8ef4424e487b3f0d/lava-logs.yaml 2026-03-09 04:26:00,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 04:26:00,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 04:26:00,967 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f1b8ef4424e487b3f0d/lava_callback.json.gz 2026-03-09 04:26:00,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:26:01,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f1b8ef4424e487b3f0d HTTP/1.1" 200 None 2026-03-09 04:26:01,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:26:01,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 04:26:01,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:26:01,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f1b8ef4424e487b3f0d HTTP/1.1" 200 None 2026-03-09 04:26:01,074 - lava_callback - INFO - Completed processing callback for node 69ae1f1b8ef4424e487b3f0d INFO: 172.21.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59448 - "GET / HTTP/1.0" 200 OK 2026-03-09 04:26:28,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:26:28,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK 2026-03-09 04:37:20,484 - lava_callback - INFO - Received callback for node 69ae1f148ef4424e487b3ee7 INFO: 172.21.0.1:45160 - "POST /node/69ae1f148ef4424e487b3ee7 HTTP/1.0" 202 Accepted 2026-03-09 04:37:20,490 - lava_callback - INFO - Starting async job submit for node 69ae1f148ef4424e487b3ee7 2026-03-09 04:37:20,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:37:20,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f148ef4424e487b3ee7 HTTP/1.1" 200 None 2026-03-09 04:37:20,527 - lava_callback - DEBUG - Processing node 69ae1f148ef4424e487b3ee7: baseline-arm64-broonie 2026-03-09 04:37:20,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 04:37:20,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 04:37:20,957 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f148ef4424e487b3ee7/log.txt.gz 2026-03-09 04:37:21,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 04:37:21,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 04:37:21,369 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f148ef4424e487b3ee7/lava-logs.yaml 2026-03-09 04:37:21,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 04:37:21,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 04:37:21,542 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f148ef4424e487b3ee7/lava_callback.json.gz 2026-03-09 04:37:21,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:37:21,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f148ef4424e487b3ee7 HTTP/1.1" 200 None 2026-03-09 04:37:21,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:37:21,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 04:37:21,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:37:21,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f148ef4424e487b3ee7 HTTP/1.1" 200 None 2026-03-09 04:37:21,639 - lava_callback - INFO - Completed processing callback for node 69ae1f148ef4424e487b3ee7 INFO: 172.21.0.1:45168 - "GET / HTTP/1.0" 200 OK 2026-03-09 04:37:28,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 04:37:28,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46236 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:00:05,350 - lava_callback - INFO - Received callback for node 69abb0cdefa461f55e59ebb2 INFO: 172.21.0.1:40358 - "POST /node/69abb0cdefa461f55e59ebb2 HTTP/1.0" 202 Accepted 2026-03-09 05:00:05,356 - lava_callback - INFO - Starting async job submit for node 69abb0cdefa461f55e59ebb2 2026-03-09 05:00:05,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:00:05,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb0cdefa461f55e59ebb2 HTTP/1.1" 200 None 2026-03-09 05:00:05,377 - lava_callback - DEBUG - Processing node 69abb0cdefa461f55e59ebb2: baseline-arm-baylibre 2026-03-09 05:00:05,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abb0cdefa461f55e59ebb2 2026-03-09 05:00:05,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:00:05,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb0cdefa461f55e59ebb2 HTTP/1.1" 200 None 2026-03-09 05:00:05,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:00:05,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-09 05:00:05,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:00:05,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abb0cdefa461f55e59ebb2 HTTP/1.1" 200 None 2026-03-09 05:00:05,430 - lava_callback - ERROR - Error processing callback for node 69abb0cdefa461f55e59ebb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48228 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:02:06,722 - lava_callback - INFO - Received callback for node 69abb13cefa461f55e59ebeb INFO: 172.21.0.1:48240 - "POST /node/69abb13cefa461f55e59ebeb HTTP/1.0" 202 Accepted 2026-03-09 05:02:06,727 - lava_callback - INFO - Starting async job submit for node 69abb13cefa461f55e59ebeb 2026-03-09 05:02:06,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:06,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb13cefa461f55e59ebeb HTTP/1.1" 200 None 2026-03-09 05:02:06,744 - lava_callback - DEBUG - Processing node 69abb13cefa461f55e59ebeb: baseline-arm-clabbe 2026-03-09 05:02:06,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abb13cefa461f55e59ebeb 2026-03-09 05:02:06,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:06,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb13cefa461f55e59ebeb HTTP/1.1" 200 None 2026-03-09 05:02:06,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:06,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-09 05:02:06,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:06,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abb13cefa461f55e59ebeb HTTP/1.1" 200 None 2026-03-09 05:02:06,800 - lava_callback - ERROR - Error processing callback for node 69abb13cefa461f55e59ebeb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51058 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:02:46,702 - lava_callback - INFO - Received callback for node 69abb160efa461f55e59ec06 INFO: 172.21.0.1:51064 - "POST /node/69abb160efa461f55e59ec06 HTTP/1.0" 202 Accepted 2026-03-09 05:02:46,707 - lava_callback - INFO - Starting async job submit for node 69abb160efa461f55e59ec06 2026-03-09 05:02:46,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:46,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb160efa461f55e59ec06 HTTP/1.1" 200 None 2026-03-09 05:02:46,724 - lava_callback - DEBUG - Processing node 69abb160efa461f55e59ec06: baseline-arm-clabbe 2026-03-09 05:02:46,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abb160efa461f55e59ec06 2026-03-09 05:02:46,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:46,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb160efa461f55e59ec06 HTTP/1.1" 200 None 2026-03-09 05:02:46,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:46,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-09 05:02:46,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:02:46,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abb160efa461f55e59ec06 HTTP/1.1" 200 None 2026-03-09 05:02:46,787 - lava_callback - ERROR - Error processing callback for node 69abb160efa461f55e59ec06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38990 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:03:06,740 - lava_callback - INFO - Received callback for node 69abb185efa461f55e59ec17 INFO: 172.21.0.1:38998 - "POST /node/69abb185efa461f55e59ec17 HTTP/1.0" 202 Accepted 2026-03-09 05:03:06,746 - lava_callback - INFO - Starting async job submit for node 69abb185efa461f55e59ec17 2026-03-09 05:03:06,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:03:06,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb185efa461f55e59ec17 HTTP/1.1" 200 None 2026-03-09 05:03:06,766 - lava_callback - DEBUG - Processing node 69abb185efa461f55e59ec17: baseline-arm-clabbe 2026-03-09 05:03:06,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abb185efa461f55e59ec17 2026-03-09 05:03:06,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:03:06,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb185efa461f55e59ec17 HTTP/1.1" 200 None 2026-03-09 05:03:06,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:03:06,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-09 05:03:06,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:03:06,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abb185efa461f55e59ec17 HTTP/1.1" 200 None 2026-03-09 05:03:06,838 - lava_callback - ERROR - Error processing callback for node 69abb185efa461f55e59ec17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40438 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:04:05,467 - lava_callback - INFO - Received callback for node 69abb1b4efa461f55e59ec32 INFO: 172.21.0.1:54848 - "POST /node/69abb1b4efa461f55e59ec32 HTTP/1.0" 202 Accepted 2026-03-09 05:04:05,473 - lava_callback - INFO - Starting async job submit for node 69abb1b4efa461f55e59ec32 2026-03-09 05:04:05,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:04:05,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb1b4efa461f55e59ec32 HTTP/1.1" 200 None 2026-03-09 05:04:05,489 - lava_callback - DEBUG - Processing node 69abb1b4efa461f55e59ec32: baseline-arm64-baylibre 2026-03-09 05:04:05,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abb1b4efa461f55e59ec32 2026-03-09 05:04:05,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:04:05,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb1b4efa461f55e59ec32 HTTP/1.1" 200 None 2026-03-09 05:04:05,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:04:05,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a85aff440c1da21c2371fe HTTP/1.1" 200 None 2026-03-09 05:04:05,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:04:05,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abb1b4efa461f55e59ec32 HTTP/1.1" 200 None 2026-03-09 05:04:05,546 - lava_callback - ERROR - Error processing callback for node 69abb1b4efa461f55e59ec32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:05:05,436 - lava_callback - INFO - Received callback for node 69abb1fcefa461f55e59ec55 INFO: 172.21.0.1:56300 - "POST /node/69abb1fcefa461f55e59ec55 HTTP/1.0" 202 Accepted 2026-03-09 05:05:05,446 - lava_callback - INFO - Starting async job submit for node 69abb1fcefa461f55e59ec55 2026-03-09 05:05:05,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:05:05,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb1fcefa461f55e59ec55 HTTP/1.1" 200 None 2026-03-09 05:05:05,482 - lava_callback - DEBUG - Processing node 69abb1fcefa461f55e59ec55: baseline-arm64-baylibre 2026-03-09 05:05:05,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abb1fcefa461f55e59ec55 2026-03-09 05:05:05,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:05:05,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abb1fcefa461f55e59ec55 HTTP/1.1" 200 None 2026-03-09 05:05:05,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:05:05,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a85aff440c1da21c2371fe HTTP/1.1" 200 None 2026-03-09 05:05:05,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:05:05,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abb1fcefa461f55e59ec55 HTTP/1.1" 200 None 2026-03-09 05:05:05,554 - lava_callback - ERROR - Error processing callback for node 69abb1fcefa461f55e59ec55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36624 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:34:36,392 - lava_callback - INFO - Received callback for node 69ae1f0f8ef4424e487b3ed4 INFO: 172.21.0.1:49230 - "POST /node/69ae1f0f8ef4424e487b3ed4 HTTP/1.0" 202 Accepted 2026-03-09 05:34:36,398 - lava_callback - INFO - Starting async job submit for node 69ae1f0f8ef4424e487b3ed4 2026-03-09 05:34:36,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:34:36,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f0f8ef4424e487b3ed4 HTTP/1.1" 200 None 2026-03-09 05:34:36,440 - lava_callback - DEBUG - Processing node 69ae1f0f8ef4424e487b3ed4: baseline-arm64-broonie INFO: 172.21.0.1:49246 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:34:36,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 05:34:36,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 05:34:36,820 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f0f8ef4424e487b3ed4/log.txt.gz 2026-03-09 05:34:36,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 05:34:37,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 05:34:37,169 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f0f8ef4424e487b3ed4/lava-logs.yaml 2026-03-09 05:34:37,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 05:34:37,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 05:34:37,342 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f0f8ef4424e487b3ed4/lava_callback.json.gz 2026-03-09 05:34:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:34:37,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f0f8ef4424e487b3ed4 HTTP/1.1" 200 None 2026-03-09 05:34:37,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:34:37,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 05:34:37,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:34:37,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f0f8ef4424e487b3ed4 HTTP/1.1" 200 None 2026-03-09 05:34:37,423 - lava_callback - INFO - Completed processing callback for node 69ae1f0f8ef4424e487b3ed4 INFO: 172.21.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35894 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:34:58,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:34:58,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:46:36,676 - lava_callback - INFO - Received callback for node 69ae1f178ef4424e487b3eed INFO: 172.21.0.1:59614 - "POST /node/69ae1f178ef4424e487b3eed HTTP/1.0" 202 Accepted 2026-03-09 05:46:36,682 - lava_callback - INFO - Starting async job submit for node 69ae1f178ef4424e487b3eed 2026-03-09 05:46:36,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:46:36,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f178ef4424e487b3eed HTTP/1.1" 200 None 2026-03-09 05:46:36,743 - lava_callback - DEBUG - Processing node 69ae1f178ef4424e487b3eed: baseline-arm64-broonie 2026-03-09 05:46:36,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 05:46:37,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 05:46:37,138 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f178ef4424e487b3eed/log.txt.gz 2026-03-09 05:46:37,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 05:46:37,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 05:46:37,509 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f178ef4424e487b3eed/lava-logs.yaml 2026-03-09 05:46:37,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 05:46:37,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 05:46:37,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f178ef4424e487b3eed/lava_callback.json.gz 2026-03-09 05:46:37,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:46:37,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f178ef4424e487b3eed HTTP/1.1" 200 None 2026-03-09 05:46:37,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:46:37,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 05:46:37,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:46:37,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f178ef4424e487b3eed HTTP/1.1" 200 None 2026-03-09 05:46:37,797 - lava_callback - INFO - Completed processing callback for node 69ae1f178ef4424e487b3eed INFO: 172.21.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41290 - "GET / HTTP/1.0" 200 OK 2026-03-09 05:46:58,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 05:46:58,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51456 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:37:58,515 - lava_callback - INFO - Received callback for node 69ae1f0d8ef4424e487b3ed1 INFO: 172.21.0.1:41712 - "POST /node/69ae1f0d8ef4424e487b3ed1 HTTP/1.0" 202 Accepted 2026-03-09 06:37:58,521 - lava_callback - INFO - Starting async job submit for node 69ae1f0d8ef4424e487b3ed1 2026-03-09 06:37:58,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:37:58,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f0d8ef4424e487b3ed1 HTTP/1.1" 200 None 2026-03-09 06:37:58,575 - lava_callback - DEBUG - Processing node 69ae1f0d8ef4424e487b3ed1: baseline-arm64-broonie 2026-03-09 06:37:58,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 06:37:58,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 06:37:58,970 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f0d8ef4424e487b3ed1/log.txt.gz 2026-03-09 06:37:59,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 06:37:59,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 06:37:59,288 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f0d8ef4424e487b3ed1/lava-logs.yaml 2026-03-09 06:37:59,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 06:37:59,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 06:37:59,441 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f0d8ef4424e487b3ed1/lava_callback.json.gz 2026-03-09 06:37:59,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:37:59,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f0d8ef4424e487b3ed1 HTTP/1.1" 200 None 2026-03-09 06:37:59,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:37:59,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 06:37:59,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:37:59,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f0d8ef4424e487b3ed1 HTTP/1.1" 200 None 2026-03-09 06:37:59,522 - lava_callback - INFO - Completed processing callback for node 69ae1f0d8ef4424e487b3ed1 INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44926 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:38:28,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:38:28,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54978 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:39:50,667 - lava_callback - INFO - Received callback for node 69abc82fefa461f55e59f0e0 INFO: 172.21.0.1:60722 - "POST /node/69abc82fefa461f55e59f0e0 HTTP/1.0" 202 Accepted 2026-03-09 06:39:50,672 - lava_callback - INFO - Starting async job submit for node 69abc82fefa461f55e59f0e0 2026-03-09 06:39:50,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:39:50,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc82fefa461f55e59f0e0 HTTP/1.1" 200 None 2026-03-09 06:39:50,694 - lava_callback - DEBUG - Processing node 69abc82fefa461f55e59f0e0: baseline-arm-baylibre 2026-03-09 06:39:50,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc82fefa461f55e59f0e0 2026-03-09 06:39:50,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:39:50,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc82fefa461f55e59f0e0 HTTP/1.1" 200 None 2026-03-09 06:39:50,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:39:50,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:39:50,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:39:50,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc82fefa461f55e59f0e0 HTTP/1.1" 200 None 2026-03-09 06:39:50,750 - lava_callback - ERROR - Error processing callback for node 69abc82fefa461f55e59f0e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:41:50,735 - lava_callback - INFO - Received callback for node 69abc8a8efa461f55e59f0fc INFO: 172.21.0.1:33752 - "POST /node/69abc8a8efa461f55e59f0fc HTTP/1.0" 202 Accepted 2026-03-09 06:41:50,740 - lava_callback - INFO - Starting async job submit for node 69abc8a8efa461f55e59f0fc 2026-03-09 06:41:50,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:41:50,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc8a8efa461f55e59f0fc HTTP/1.1" 200 None 2026-03-09 06:41:50,764 - lava_callback - DEBUG - Processing node 69abc8a8efa461f55e59f0fc: baseline-arm-baylibre 2026-03-09 06:41:50,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc8a8efa461f55e59f0fc 2026-03-09 06:41:50,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:41:50,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc8a8efa461f55e59f0fc HTTP/1.1" 200 None 2026-03-09 06:41:50,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:41:50,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:41:50,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:41:50,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc8a8efa461f55e59f0fc HTTP/1.1" 200 None 2026-03-09 06:41:50,826 - lava_callback - ERROR - Error processing callback for node 69abc8a8efa461f55e59f0fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47192 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:42:30,748 - lava_callback - INFO - Received callback for node 69abc8cdefa461f55e59f10d INFO: 172.21.0.1:58518 - "POST /node/69abc8cdefa461f55e59f10d HTTP/1.0" 202 Accepted 2026-03-09 06:42:30,754 - lava_callback - INFO - Starting async job submit for node 69abc8cdefa461f55e59f10d 2026-03-09 06:42:30,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:42:30,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc8cdefa461f55e59f10d HTTP/1.1" 200 None 2026-03-09 06:42:30,770 - lava_callback - DEBUG - Processing node 69abc8cdefa461f55e59f10d: baseline-arm-baylibre 2026-03-09 06:42:30,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc8cdefa461f55e59f10d 2026-03-09 06:42:30,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:42:30,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc8cdefa461f55e59f10d HTTP/1.1" 200 None 2026-03-09 06:42:30,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:42:30,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:42:30,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:42:30,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc8cdefa461f55e59f10d HTTP/1.1" 200 None 2026-03-09 06:42:30,827 - lava_callback - ERROR - Error processing callback for node 69abc8cdefa461f55e59f10d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49532 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:43:10,861 - lava_callback - INFO - Received callback for node 69abc8f1efa461f55e59f12d INFO: 172.21.0.1:33992 - "POST /node/69abc8f1efa461f55e59f12d HTTP/1.0" 202 Accepted 2026-03-09 06:43:10,867 - lava_callback - INFO - Starting async job submit for node 69abc8f1efa461f55e59f12d 2026-03-09 06:43:10,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc8f1efa461f55e59f12d HTTP/1.1" 200 None 2026-03-09 06:43:10,883 - lava_callback - DEBUG - Processing node 69abc8f1efa461f55e59f12d: baseline-arm-baylibre 2026-03-09 06:43:10,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc8f1efa461f55e59f12d 2026-03-09 06:43:10,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:10,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc8f1efa461f55e59f12d HTTP/1.1" 200 None 2026-03-09 06:43:10,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:10,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:43:10,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:10,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc8f1efa461f55e59f12d HTTP/1.1" 200 None 2026-03-09 06:43:10,940 - lava_callback - ERROR - Error processing callback for node 69abc8f1efa461f55e59f12d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:43:50,811 - lava_callback - INFO - Received callback for node 69abc916efa461f55e59f13b INFO: 172.21.0.1:43582 - "POST /node/69abc916efa461f55e59f13b HTTP/1.0" 202 Accepted 2026-03-09 06:43:50,817 - lava_callback - INFO - Starting async job submit for node 69abc916efa461f55e59f13b 2026-03-09 06:43:50,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:50,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc916efa461f55e59f13b HTTP/1.1" 200 None 2026-03-09 06:43:50,833 - lava_callback - DEBUG - Processing node 69abc916efa461f55e59f13b: baseline-arm-baylibre 2026-03-09 06:43:50,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc916efa461f55e59f13b 2026-03-09 06:43:50,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:50,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc916efa461f55e59f13b HTTP/1.1" 200 None 2026-03-09 06:43:50,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:50,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:43:50,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:43:50,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc916efa461f55e59f13b HTTP/1.1" 200 None 2026-03-09 06:43:50,921 - lava_callback - ERROR - Error processing callback for node 69abc916efa461f55e59f13b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57302 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:44:30,909 - lava_callback - INFO - Received callback for node 69abc93aefa461f55e59f151 INFO: 172.21.0.1:55682 - "POST /node/69abc93aefa461f55e59f151 HTTP/1.0" 202 Accepted 2026-03-09 06:44:30,915 - lava_callback - INFO - Starting async job submit for node 69abc93aefa461f55e59f151 2026-03-09 06:44:30,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:30,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc93aefa461f55e59f151 HTTP/1.1" 200 None 2026-03-09 06:44:30,934 - lava_callback - DEBUG - Processing node 69abc93aefa461f55e59f151: baseline-arm-baylibre 2026-03-09 06:44:30,934 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc93aefa461f55e59f151 2026-03-09 06:44:30,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:30,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc93aefa461f55e59f151 HTTP/1.1" 200 None 2026-03-09 06:44:30,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:30,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:44:30,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:30,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc93aefa461f55e59f151 HTTP/1.1" 200 None 2026-03-09 06:44:31,001 - lava_callback - ERROR - Error processing callback for node 69abc93aefa461f55e59f151: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34958 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:44:50,863 - lava_callback - INFO - Received callback for node 69abc95eefa461f55e59f159 INFO: 172.21.0.1:48998 - "POST /node/69abc95eefa461f55e59f159 HTTP/1.0" 202 Accepted 2026-03-09 06:44:50,868 - lava_callback - INFO - Starting async job submit for node 69abc95eefa461f55e59f159 2026-03-09 06:44:50,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:50,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc95eefa461f55e59f159 HTTP/1.1" 200 None 2026-03-09 06:44:50,884 - lava_callback - DEBUG - Processing node 69abc95eefa461f55e59f159: baseline-arm-baylibre 2026-03-09 06:44:50,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc95eefa461f55e59f159 2026-03-09 06:44:50,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:50,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc95eefa461f55e59f159 HTTP/1.1" 200 None 2026-03-09 06:44:50,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:50,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:44:50,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:44:50,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc95eefa461f55e59f159 HTTP/1.1" 200 None 2026-03-09 06:44:50,945 - lava_callback - ERROR - Error processing callback for node 69abc95eefa461f55e59f159: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48134 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:45:31,018 - lava_callback - INFO - Received callback for node 69abc985efa461f55e59f169 INFO: 172.21.0.1:50716 - "POST /node/69abc985efa461f55e59f169 HTTP/1.0" 202 Accepted 2026-03-09 06:45:31,027 - lava_callback - INFO - Starting async job submit for node 69abc985efa461f55e59f169 2026-03-09 06:45:31,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:45:31,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc985efa461f55e59f169 HTTP/1.1" 200 None 2026-03-09 06:45:31,050 - lava_callback - DEBUG - Processing node 69abc985efa461f55e59f169: baseline-arm-baylibre 2026-03-09 06:45:31,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc985efa461f55e59f169 2026-03-09 06:45:31,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:45:31,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc985efa461f55e59f169 HTTP/1.1" 200 None 2026-03-09 06:45:31,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:45:31,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:45:31,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:45:31,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc985efa461f55e59f169 HTTP/1.1" 200 None 2026-03-09 06:45:31,144 - lava_callback - ERROR - Error processing callback for node 69abc985efa461f55e59f169: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:46:10,962 - lava_callback - INFO - Received callback for node 69abc9a9efa461f55e59f178 INFO: 172.21.0.1:39744 - "POST /node/69abc9a9efa461f55e59f178 HTTP/1.0" 202 Accepted 2026-03-09 06:46:10,967 - lava_callback - INFO - Starting async job submit for node 69abc9a9efa461f55e59f178 2026-03-09 06:46:10,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:10,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc9a9efa461f55e59f178 HTTP/1.1" 200 None 2026-03-09 06:46:10,987 - lava_callback - DEBUG - Processing node 69abc9a9efa461f55e59f178: baseline-arm-baylibre 2026-03-09 06:46:10,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc9a9efa461f55e59f178 2026-03-09 06:46:10,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:11,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc9a9efa461f55e59f178 HTTP/1.1" 200 None 2026-03-09 06:46:11,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:11,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:46:11,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:11,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc9a9efa461f55e59f178 HTTP/1.1" 200 None 2026-03-09 06:46:11,045 - lava_callback - ERROR - Error processing callback for node 69abc9a9efa461f55e59f178: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54650 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:46:51,015 - lava_callback - INFO - Received callback for node 69abc9caefa461f55e59f18b INFO: 172.21.0.1:58044 - "POST /node/69abc9caefa461f55e59f18b HTTP/1.0" 202 Accepted 2026-03-09 06:46:51,020 - lava_callback - INFO - Starting async job submit for node 69abc9caefa461f55e59f18b 2026-03-09 06:46:51,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:51,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc9caefa461f55e59f18b HTTP/1.1" 200 None 2026-03-09 06:46:51,037 - lava_callback - DEBUG - Processing node 69abc9caefa461f55e59f18b: baseline-arm-baylibre 2026-03-09 06:46:51,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc9caefa461f55e59f18b 2026-03-09 06:46:51,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:51,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc9caefa461f55e59f18b HTTP/1.1" 200 None 2026-03-09 06:46:51,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:51,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:46:51,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:46:51,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc9caefa461f55e59f18b HTTP/1.1" 200 None 2026-03-09 06:46:51,089 - lava_callback - ERROR - Error processing callback for node 69abc9caefa461f55e59f18b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54932 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:47:31,055 - lava_callback - INFO - Received callback for node 69abc9eeefa461f55e59f194 INFO: 172.21.0.1:59848 - "POST /node/69abc9eeefa461f55e59f194 HTTP/1.0" 202 Accepted 2026-03-09 06:47:31,060 - lava_callback - INFO - Starting async job submit for node 69abc9eeefa461f55e59f194 2026-03-09 06:47:31,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:31,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc9eeefa461f55e59f194 HTTP/1.1" 200 None 2026-03-09 06:47:31,077 - lava_callback - DEBUG - Processing node 69abc9eeefa461f55e59f194: baseline-arm-baylibre 2026-03-09 06:47:31,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abc9eeefa461f55e59f194 2026-03-09 06:47:31,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:31,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abc9eeefa461f55e59f194 HTTP/1.1" 200 None 2026-03-09 06:47:31,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:31,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:47:31,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:31,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abc9eeefa461f55e59f194 HTTP/1.1" 200 None 2026-03-09 06:47:31,129 - lava_callback - ERROR - Error processing callback for node 69abc9eeefa461f55e59f194: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57364 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:47:51,058 - lava_callback - INFO - Received callback for node 69abca12efa461f55e59f1a1 INFO: 172.21.0.1:48382 - "POST /node/69abca12efa461f55e59f1a1 HTTP/1.0" 202 Accepted 2026-03-09 06:47:51,064 - lava_callback - INFO - Starting async job submit for node 69abca12efa461f55e59f1a1 2026-03-09 06:47:51,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:51,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abca12efa461f55e59f1a1 HTTP/1.1" 200 None 2026-03-09 06:47:51,081 - lava_callback - DEBUG - Processing node 69abca12efa461f55e59f1a1: baseline-arm-baylibre 2026-03-09 06:47:51,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abca12efa461f55e59f1a1 2026-03-09 06:47:51,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:51,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abca12efa461f55e59f1a1 HTTP/1.1" 200 None 2026-03-09 06:47:51,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:51,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-09 06:47:51,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:47:51,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abca12efa461f55e59f1a1 HTTP/1.1" 200 None 2026-03-09 06:47:51,138 - lava_callback - ERROR - Error processing callback for node 69abca12efa461f55e59f1a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57652 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:51:11,286 - lava_callback - INFO - Received callback for node 69abcacfefa461f55e59f20c INFO: 172.21.0.1:39334 - "POST /node/69abcacfefa461f55e59f20c HTTP/1.0" 202 Accepted 2026-03-09 06:51:11,296 - lava_callback - INFO - Starting async job submit for node 69abcacfefa461f55e59f20c 2026-03-09 06:51:11,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:11,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcacfefa461f55e59f20c HTTP/1.1" 200 None 2026-03-09 06:51:11,323 - lava_callback - DEBUG - Processing node 69abcacfefa461f55e59f20c: baseline-arm-baylibre 2026-03-09 06:51:11,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcacfefa461f55e59f20c 2026-03-09 06:51:11,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:11,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcacfefa461f55e59f20c HTTP/1.1" 200 None 2026-03-09 06:51:11,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:11,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:51:11,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:11,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcacfefa461f55e59f20c HTTP/1.1" 200 None 2026-03-09 06:51:11,405 - lava_callback - ERROR - Error processing callback for node 69abcacfefa461f55e59f20c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:51:51,258 - lava_callback - INFO - Received callback for node 69abcaf2efa461f55e59f220 INFO: 172.21.0.1:36288 - "POST /node/69abcaf2efa461f55e59f220 HTTP/1.0" 202 Accepted 2026-03-09 06:51:51,267 - lava_callback - INFO - Starting async job submit for node 69abcaf2efa461f55e59f220 2026-03-09 06:51:51,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:51,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcaf2efa461f55e59f220 HTTP/1.1" 200 None 2026-03-09 06:51:51,289 - lava_callback - DEBUG - Processing node 69abcaf2efa461f55e59f220: baseline-arm-baylibre 2026-03-09 06:51:51,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcaf2efa461f55e59f220 2026-03-09 06:51:51,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:51,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcaf2efa461f55e59f220 HTTP/1.1" 200 None 2026-03-09 06:51:51,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:51,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:51:51,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:51:51,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcaf2efa461f55e59f220 HTTP/1.1" 200 None 2026-03-09 06:51:51,373 - lava_callback - ERROR - Error processing callback for node 69abcaf2efa461f55e59f220: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38280 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:52:11,284 - lava_callback - INFO - Received callback for node 69abcb16efa461f55e59f230 INFO: 172.21.0.1:50376 - "POST /node/69abcb16efa461f55e59f230 HTTP/1.0" 202 Accepted 2026-03-09 06:52:11,290 - lava_callback - INFO - Starting async job submit for node 69abcb16efa461f55e59f230 2026-03-09 06:52:11,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:11,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcb16efa461f55e59f230 HTTP/1.1" 200 None 2026-03-09 06:52:11,306 - lava_callback - DEBUG - Processing node 69abcb16efa461f55e59f230: baseline-arm-baylibre 2026-03-09 06:52:11,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcb16efa461f55e59f230 2026-03-09 06:52:11,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:11,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcb16efa461f55e59f230 HTTP/1.1" 200 None 2026-03-09 06:52:11,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:11,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:52:11,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:11,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcb16efa461f55e59f230 HTTP/1.1" 200 None 2026-03-09 06:52:11,364 - lava_callback - ERROR - Error processing callback for node 69abcb16efa461f55e59f230: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35840 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:52:51,401 - lava_callback - INFO - Received callback for node 69abcb3aefa461f55e59f240 INFO: 172.21.0.1:44994 - "POST /node/69abcb3aefa461f55e59f240 HTTP/1.0" 202 Accepted 2026-03-09 06:52:51,406 - lava_callback - INFO - Starting async job submit for node 69abcb3aefa461f55e59f240 2026-03-09 06:52:51,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:51,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcb3aefa461f55e59f240 HTTP/1.1" 200 None 2026-03-09 06:52:51,422 - lava_callback - DEBUG - Processing node 69abcb3aefa461f55e59f240: baseline-arm-baylibre 2026-03-09 06:52:51,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcb3aefa461f55e59f240 2026-03-09 06:52:51,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:51,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcb3aefa461f55e59f240 HTTP/1.1" 200 None 2026-03-09 06:52:51,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:51,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:52:51,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:52:51,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcb3aefa461f55e59f240 HTTP/1.1" 200 None 2026-03-09 06:52:51,488 - lava_callback - ERROR - Error processing callback for node 69abcb3aefa461f55e59f240: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49296 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:53:31,360 - lava_callback - INFO - Received callback for node 69abcb5eefa461f55e59f254 INFO: 172.21.0.1:33744 - "POST /node/69abcb5eefa461f55e59f254 HTTP/1.0" 202 Accepted 2026-03-09 06:53:31,365 - lava_callback - INFO - Starting async job submit for node 69abcb5eefa461f55e59f254 2026-03-09 06:53:31,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:53:31,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcb5eefa461f55e59f254 HTTP/1.1" 200 None 2026-03-09 06:53:31,390 - lava_callback - DEBUG - Processing node 69abcb5eefa461f55e59f254: baseline-arm-baylibre 2026-03-09 06:53:31,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcb5eefa461f55e59f254 2026-03-09 06:53:31,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:53:31,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcb5eefa461f55e59f254 HTTP/1.1" 200 None 2026-03-09 06:53:31,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:53:31,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:53:31,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:53:31,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcb5eefa461f55e59f254 HTTP/1.1" 200 None 2026-03-09 06:53:31,452 - lava_callback - ERROR - Error processing callback for node 69abcb5eefa461f55e59f254: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34102 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:54:45,774 - lava_callback - INFO - Received callback for node 69ae1f128ef4424e487b3ee2 INFO: 172.21.0.1:35124 - "POST /node/69ae1f128ef4424e487b3ee2 HTTP/1.0" 202 Accepted 2026-03-09 06:54:45,780 - lava_callback - INFO - Starting async job submit for node 69ae1f128ef4424e487b3ee2 2026-03-09 06:54:45,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:54:45,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f128ef4424e487b3ee2 HTTP/1.1" 200 None 2026-03-09 06:54:45,821 - lava_callback - DEBUG - Processing node 69ae1f128ef4424e487b3ee2: baseline-arm64-broonie 2026-03-09 06:54:46,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 06:54:46,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 06:54:46,258 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f128ef4424e487b3ee2/log.txt.gz 2026-03-09 06:54:46,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35136 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:54:46,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 06:54:46,673 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f128ef4424e487b3ee2/lava-logs.yaml 2026-03-09 06:54:46,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 06:54:46,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 06:54:46,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69ae1f128ef4424e487b3ee2/lava_callback.json.gz 2026-03-09 06:54:46,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:54:46,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f128ef4424e487b3ee2 HTTP/1.1" 200 None 2026-03-09 06:54:46,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:54:46,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-09 06:54:46,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:54:46,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f128ef4424e487b3ee2 HTTP/1.1" 200 None 2026-03-09 06:54:46,925 - lava_callback - INFO - Completed processing callback for node 69ae1f128ef4424e487b3ee2 INFO: 172.21.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60574 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:54:58,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:54:58,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38466 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:55:11,783 - lava_callback - INFO - Received callback for node 69abcbc9efa461f55e59f28f INFO: 172.21.0.1:45728 - "POST /node/69abcbc9efa461f55e59f28f HTTP/1.0" 202 Accepted 2026-03-09 06:55:11,788 - lava_callback - INFO - Starting async job submit for node 69abcbc9efa461f55e59f28f 2026-03-09 06:55:11,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:11,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcbc9efa461f55e59f28f HTTP/1.1" 200 None 2026-03-09 06:55:11,811 - lava_callback - DEBUG - Processing node 69abcbc9efa461f55e59f28f: baseline-arm-clabbe 2026-03-09 06:55:11,812 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcbc9efa461f55e59f28f 2026-03-09 06:55:11,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:11,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcbc9efa461f55e59f28f HTTP/1.1" 200 None 2026-03-09 06:55:11,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:11,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:55:11,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:11,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcbc9efa461f55e59f28f HTTP/1.1" 200 None 2026-03-09 06:55:11,873 - lava_callback - ERROR - Error processing callback for node 69abcbc9efa461f55e59f28f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47236 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:55:51,605 - lava_callback - INFO - Received callback for node 69abcbedefa461f55e59f2a0 INFO: 172.21.0.1:59292 - "POST /node/69abcbedefa461f55e59f2a0 HTTP/1.0" 202 Accepted 2026-03-09 06:55:51,610 - lava_callback - INFO - Starting async job submit for node 69abcbedefa461f55e59f2a0 2026-03-09 06:55:51,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:51,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcbedefa461f55e59f2a0 HTTP/1.1" 200 None 2026-03-09 06:55:51,631 - lava_callback - DEBUG - Processing node 69abcbedefa461f55e59f2a0: baseline-arm-clabbe 2026-03-09 06:55:51,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcbedefa461f55e59f2a0 2026-03-09 06:55:51,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:51,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcbedefa461f55e59f2a0 HTTP/1.1" 200 None 2026-03-09 06:55:51,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:51,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:55:51,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:55:51,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcbedefa461f55e59f2a0 HTTP/1.1" 200 None 2026-03-09 06:55:51,684 - lava_callback - ERROR - Error processing callback for node 69abcbedefa461f55e59f2a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51158 - "GET / HTTP/1.0" 200 OK 2026-03-09 06:56:31,670 - lava_callback - INFO - Received callback for node 69abcc12efa461f55e59f2b3 INFO: 172.21.0.1:33542 - "POST /node/69abcc12efa461f55e59f2b3 HTTP/1.0" 202 Accepted 2026-03-09 06:56:31,675 - lava_callback - INFO - Starting async job submit for node 69abcc12efa461f55e59f2b3 2026-03-09 06:56:31,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:56:31,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcc12efa461f55e59f2b3 HTTP/1.1" 200 None 2026-03-09 06:56:31,691 - lava_callback - DEBUG - Processing node 69abcc12efa461f55e59f2b3: baseline-arm-clabbe 2026-03-09 06:56:31,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcc12efa461f55e59f2b3 2026-03-09 06:56:31,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:56:31,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcc12efa461f55e59f2b3 HTTP/1.1" 200 None 2026-03-09 06:56:31,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:56:31,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 06:56:31,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 06:56:31,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcc12efa461f55e59f2b3 HTTP/1.1" 200 None 2026-03-09 06:56:31,779 - lava_callback - ERROR - Error processing callback for node 69abcc12efa461f55e59f2b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57438 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:08:12,262 - lava_callback - INFO - Received callback for node 69abced1efa461f55e59f368 INFO: 172.21.0.1:46500 - "POST /node/69abced1efa461f55e59f368 HTTP/1.0" 202 Accepted 2026-03-09 07:08:12,268 - lava_callback - INFO - Starting async job submit for node 69abced1efa461f55e59f368 2026-03-09 07:08:12,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:08:12,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abced1efa461f55e59f368 HTTP/1.1" 200 None 2026-03-09 07:08:12,284 - lava_callback - DEBUG - Processing node 69abced1efa461f55e59f368: baseline-arm64-baylibre 2026-03-09 07:08:12,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abced1efa461f55e59f368 2026-03-09 07:08:12,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:08:12,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abced1efa461f55e59f368 HTTP/1.1" 200 None 2026-03-09 07:08:12,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:08:12,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-09 07:08:12,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:08:12,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abced1efa461f55e59f368 HTTP/1.1" 200 None 2026-03-09 07:08:12,359 - lava_callback - ERROR - Error processing callback for node 69abced1efa461f55e59f368: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58522 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:10:12,389 - lava_callback - INFO - Received callback for node 69abcf47efa461f55e59f375 INFO: 172.21.0.1:39816 - "POST /node/69abcf47efa461f55e59f375 HTTP/1.0" 202 Accepted 2026-03-09 07:10:12,395 - lava_callback - INFO - Starting async job submit for node 69abcf47efa461f55e59f375 2026-03-09 07:10:12,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:10:12,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcf47efa461f55e59f375 HTTP/1.1" 200 None 2026-03-09 07:10:12,418 - lava_callback - DEBUG - Processing node 69abcf47efa461f55e59f375: baseline-arm64-baylibre 2026-03-09 07:10:12,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcf47efa461f55e59f375 2026-03-09 07:10:12,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:10:12,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcf47efa461f55e59f375 HTTP/1.1" 200 None 2026-03-09 07:10:12,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:10:12,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-09 07:10:12,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:10:12,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcf47efa461f55e59f375 HTTP/1.1" 200 None 2026-03-09 07:10:12,493 - lava_callback - ERROR - Error processing callback for node 69abcf47efa461f55e59f375: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35022 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:11:32,270 - lava_callback - INFO - Received callback for node 69abcf99efa461f55e59f39b INFO: 172.21.0.1:50184 - "POST /node/69abcf99efa461f55e59f39b HTTP/1.0" 202 Accepted 2026-03-09 07:11:32,276 - lava_callback - INFO - Starting async job submit for node 69abcf99efa461f55e59f39b 2026-03-09 07:11:32,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:11:32,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcf99efa461f55e59f39b HTTP/1.1" 200 None 2026-03-09 07:11:32,292 - lava_callback - DEBUG - Processing node 69abcf99efa461f55e59f39b: baseline-arm64-clabbe 2026-03-09 07:11:32,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcf99efa461f55e59f39b 2026-03-09 07:11:32,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:11:32,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcf99efa461f55e59f39b HTTP/1.1" 200 None 2026-03-09 07:11:32,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:11:32,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-09 07:11:32,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:11:32,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcf99efa461f55e59f39b HTTP/1.1" 200 None 2026-03-09 07:11:32,354 - lava_callback - ERROR - Error processing callback for node 69abcf99efa461f55e59f39b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56036 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:12:32,325 - lava_callback - INFO - Received callback for node 69abcfd8efa461f55e59f3ad INFO: 172.21.0.1:43622 - "POST /node/69abcfd8efa461f55e59f3ad HTTP/1.0" 202 Accepted 2026-03-09 07:12:32,331 - lava_callback - INFO - Starting async job submit for node 69abcfd8efa461f55e59f3ad 2026-03-09 07:12:32,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:12:32,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcfd8efa461f55e59f3ad HTTP/1.1" 200 None 2026-03-09 07:12:32,348 - lava_callback - DEBUG - Processing node 69abcfd8efa461f55e59f3ad: baseline-arm-clabbe 2026-03-09 07:12:32,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcfd8efa461f55e59f3ad 2026-03-09 07:12:32,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:12:32,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcfd8efa461f55e59f3ad HTTP/1.1" 200 None 2026-03-09 07:12:32,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:12:32,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-09 07:12:32,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:12:32,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcfd8efa461f55e59f3ad HTTP/1.1" 200 None 2026-03-09 07:12:32,415 - lava_callback - ERROR - Error processing callback for node 69abcfd8efa461f55e59f3ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:13:12,317 - lava_callback - INFO - Received callback for node 69abcffdefa461f55e59f3c1 INFO: 172.21.0.1:33310 - "POST /node/69abcffdefa461f55e59f3c1 HTTP/1.0" 202 Accepted 2026-03-09 07:13:12,323 - lava_callback - INFO - Starting async job submit for node 69abcffdefa461f55e59f3c1 2026-03-09 07:13:12,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:13:12,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcffdefa461f55e59f3c1 HTTP/1.1" 200 None 2026-03-09 07:13:12,342 - lava_callback - DEBUG - Processing node 69abcffdefa461f55e59f3c1: baseline-arm-clabbe 2026-03-09 07:13:12,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abcffdefa461f55e59f3c1 2026-03-09 07:13:12,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:13:12,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abcffdefa461f55e59f3c1 HTTP/1.1" 200 None 2026-03-09 07:13:12,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:13:12,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-09 07:13:12,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:13:12,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abcffdefa461f55e59f3c1 HTTP/1.1" 200 None 2026-03-09 07:13:12,416 - lava_callback - ERROR - Error processing callback for node 69abcffdefa461f55e59f3c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39810 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:14:12,393 - lava_callback - INFO - Received callback for node 69abd030efa461f55e59f3d2 INFO: 172.21.0.1:57094 - "POST /node/69abd030efa461f55e59f3d2 HTTP/1.0" 202 Accepted 2026-03-09 07:14:12,398 - lava_callback - INFO - Starting async job submit for node 69abd030efa461f55e59f3d2 2026-03-09 07:14:12,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:14:12,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd030efa461f55e59f3d2 HTTP/1.1" 200 None 2026-03-09 07:14:12,414 - lava_callback - DEBUG - Processing node 69abd030efa461f55e59f3d2: baseline-arm64-clabbe 2026-03-09 07:14:12,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd030efa461f55e59f3d2 2026-03-09 07:14:12,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:14:12,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd030efa461f55e59f3d2 HTTP/1.1" 200 None 2026-03-09 07:14:12,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:14:12,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-09 07:14:12,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:14:12,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd030efa461f55e59f3d2 HTTP/1.1" 200 None 2026-03-09 07:14:12,466 - lava_callback - ERROR - Error processing callback for node 69abd030efa461f55e59f3d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48784 - "GET /node/698f5dda3013abef95cf9047%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49840 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:18:32,496 - lava_callback - INFO - Received callback for node 69abd13aefa461f55e59f415 INFO: 172.21.0.1:42054 - "POST /node/69abd13aefa461f55e59f415 HTTP/1.0" 202 Accepted 2026-03-09 07:18:32,502 - lava_callback - INFO - Starting async job submit for node 69abd13aefa461f55e59f415 2026-03-09 07:18:32,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:18:32,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd13aefa461f55e59f415 HTTP/1.1" 200 None 2026-03-09 07:18:32,518 - lava_callback - DEBUG - Processing node 69abd13aefa461f55e59f415: baseline-arm-clabbe 2026-03-09 07:18:32,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd13aefa461f55e59f415 2026-03-09 07:18:32,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:18:32,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd13aefa461f55e59f415 HTTP/1.1" 200 None 2026-03-09 07:18:32,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:18:32,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-09 07:18:32,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:18:32,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd13aefa461f55e59f415 HTTP/1.1" 200 None 2026-03-09 07:18:32,572 - lava_callback - ERROR - Error processing callback for node 69abd13aefa461f55e59f415: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36666 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:24:12,768 - lava_callback - INFO - Received callback for node 69abd297efa461f55e59f4af INFO: 172.21.0.1:58502 - "POST /node/69abd297efa461f55e59f4af HTTP/1.0" 202 Accepted 2026-03-09 07:24:12,774 - lava_callback - INFO - Starting async job submit for node 69abd297efa461f55e59f4af 2026-03-09 07:24:12,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:12,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd297efa461f55e59f4af HTTP/1.1" 200 None 2026-03-09 07:24:12,790 - lava_callback - DEBUG - Processing node 69abd297efa461f55e59f4af: baseline-arm-clabbe 2026-03-09 07:24:12,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd297efa461f55e59f4af 2026-03-09 07:24:12,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:12,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd297efa461f55e59f4af HTTP/1.1" 200 None 2026-03-09 07:24:12,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:12,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-09 07:24:12,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:12,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd297efa461f55e59f4af HTTP/1.1" 200 None 2026-03-09 07:24:12,854 - lava_callback - ERROR - Error processing callback for node 69abd297efa461f55e59f4af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40788 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:24:53,280 - lava_callback - INFO - Received callback for node 69abd2c2efa461f55e59f4c8 INFO: 172.21.0.1:36422 - "POST /node/69abd2c2efa461f55e59f4c8 HTTP/1.0" 202 Accepted 2026-03-09 07:24:53,286 - lava_callback - INFO - Starting async job submit for node 69abd2c2efa461f55e59f4c8 2026-03-09 07:24:53,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:53,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd2c2efa461f55e59f4c8 HTTP/1.1" 200 None 2026-03-09 07:24:53,306 - lava_callback - DEBUG - Processing node 69abd2c2efa461f55e59f4c8: baseline-arm64-baylibre 2026-03-09 07:24:53,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd2c2efa461f55e59f4c8 2026-03-09 07:24:53,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:53,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd2c2efa461f55e59f4c8 HTTP/1.1" 200 None 2026-03-09 07:24:53,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:53,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:24:53,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:24:53,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd2c2efa461f55e59f4c8 HTTP/1.1" 200 None 2026-03-09 07:24:53,363 - lava_callback - ERROR - Error processing callback for node 69abd2c2efa461f55e59f4c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44872 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:26:13,248 - lava_callback - INFO - Received callback for node 69abd30defa461f55e59f4d9 INFO: 172.21.0.1:35958 - "POST /node/69abd30defa461f55e59f4d9 HTTP/1.0" 202 Accepted 2026-03-09 07:26:13,254 - lava_callback - INFO - Starting async job submit for node 69abd30defa461f55e59f4d9 2026-03-09 07:26:13,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:26:13,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd30defa461f55e59f4d9 HTTP/1.1" 200 None 2026-03-09 07:26:13,272 - lava_callback - DEBUG - Processing node 69abd30defa461f55e59f4d9: baseline-arm64-baylibre 2026-03-09 07:26:13,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd30defa461f55e59f4d9 2026-03-09 07:26:13,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:26:13,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd30defa461f55e59f4d9 HTTP/1.1" 200 None 2026-03-09 07:26:13,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:35962 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 07:26:17,300 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69ab6e44efa461f55e59cdc8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-03-09 07:26:17,300 - urllib3.connectionpool - WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError(": Failed to resolve 'staging.kernelci.org' ([Errno -5] No address associated with hostname)")': /latest/node/69ab6e44efa461f55e59cdc8 2026-03-09 07:26:17,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): staging.kernelci.org:9000 2026-03-09 07:26:17,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:26:17,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:35968 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:26:18,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd30defa461f55e59f4d9 HTTP/1.1" 200 None 2026-03-09 07:26:18,588 - lava_callback - ERROR - Error processing callback for node 69abd30defa461f55e59f4d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:27:33,358 - lava_callback - INFO - Received callback for node 69abd358efa461f55e59f4ec INFO: 172.21.0.1:43470 - "POST /node/69abd358efa461f55e59f4ec HTTP/1.0" 202 Accepted 2026-03-09 07:27:33,364 - lava_callback - INFO - Starting async job submit for node 69abd358efa461f55e59f4ec 2026-03-09 07:27:33,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:27:33,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd358efa461f55e59f4ec HTTP/1.1" 200 None 2026-03-09 07:27:33,380 - lava_callback - DEBUG - Processing node 69abd358efa461f55e59f4ec: baseline-arm64-baylibre 2026-03-09 07:27:33,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd358efa461f55e59f4ec 2026-03-09 07:27:33,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:27:33,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd358efa461f55e59f4ec HTTP/1.1" 200 None 2026-03-09 07:27:33,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:27:33,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:27:33,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:27:33,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd358efa461f55e59f4ec HTTP/1.1" 200 None 2026-03-09 07:27:33,442 - lava_callback - ERROR - Error processing callback for node 69abd358efa461f55e59f4ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39510 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:28:53,456 - lava_callback - INFO - Received callback for node 69abd3a4efa461f55e59f4fd INFO: 172.21.0.1:57108 - "POST /node/69abd3a4efa461f55e59f4fd HTTP/1.0" 202 Accepted 2026-03-09 07:28:53,462 - lava_callback - INFO - Starting async job submit for node 69abd3a4efa461f55e59f4fd 2026-03-09 07:28:53,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:28:53,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd3a4efa461f55e59f4fd HTTP/1.1" 200 None 2026-03-09 07:28:53,478 - lava_callback - DEBUG - Processing node 69abd3a4efa461f55e59f4fd: baseline-arm64-baylibre 2026-03-09 07:28:53,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd3a4efa461f55e59f4fd 2026-03-09 07:28:53,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:28:53,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd3a4efa461f55e59f4fd HTTP/1.1" 200 None 2026-03-09 07:28:53,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:28:53,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:28:53,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:28:53,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd3a4efa461f55e59f4fd HTTP/1.1" 200 None 2026-03-09 07:28:53,559 - lava_callback - ERROR - Error processing callback for node 69abd3a4efa461f55e59f4fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54836 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:30:13,634 - lava_callback - INFO - Received callback for node 69abd3f1efa461f55e59f517 INFO: 172.21.0.1:51580 - "POST /node/69abd3f1efa461f55e59f517 HTTP/1.0" 202 Accepted 2026-03-09 07:30:13,639 - lava_callback - INFO - Starting async job submit for node 69abd3f1efa461f55e59f517 2026-03-09 07:30:13,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:30:13,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd3f1efa461f55e59f517 HTTP/1.1" 200 None 2026-03-09 07:30:13,656 - lava_callback - DEBUG - Processing node 69abd3f1efa461f55e59f517: baseline-arm64-baylibre 2026-03-09 07:30:13,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd3f1efa461f55e59f517 2026-03-09 07:30:13,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:30:13,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd3f1efa461f55e59f517 HTTP/1.1" 200 None 2026-03-09 07:30:13,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:30:13,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:30:13,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:30:13,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd3f1efa461f55e59f517 HTTP/1.1" 200 None 2026-03-09 07:30:13,745 - lava_callback - ERROR - Error processing callback for node 69abd3f1efa461f55e59f517: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40912 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:31:13,503 - lava_callback - INFO - Received callback for node 69abd43cefa461f55e59f525 INFO: 172.21.0.1:55074 - "POST /node/69abd43cefa461f55e59f525 HTTP/1.0" 202 Accepted 2026-03-09 07:31:13,508 - lava_callback - INFO - Starting async job submit for node 69abd43cefa461f55e59f525 2026-03-09 07:31:13,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:31:13,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd43cefa461f55e59f525 HTTP/1.1" 200 None 2026-03-09 07:31:13,544 - lava_callback - DEBUG - Processing node 69abd43cefa461f55e59f525: baseline-arm64-baylibre 2026-03-09 07:31:13,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd43cefa461f55e59f525 2026-03-09 07:31:13,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:31:13,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd43cefa461f55e59f525 HTTP/1.1" 200 None 2026-03-09 07:31:13,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:31:13,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:31:13,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:31:13,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd43cefa461f55e59f525 HTTP/1.1" 200 None 2026-03-09 07:31:13,600 - lava_callback - ERROR - Error processing callback for node 69abd43cefa461f55e59f525: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53792 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:32:33,147 - lava_callback - INFO - Received callback for node 69abd48defa461f55e59f53d INFO: 172.21.0.1:44130 - "POST /node/69abd48defa461f55e59f53d HTTP/1.0" 202 Accepted 2026-03-09 07:32:33,153 - lava_callback - INFO - Starting async job submit for node 69abd48defa461f55e59f53d 2026-03-09 07:32:33,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:32:33,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd48defa461f55e59f53d HTTP/1.1" 200 None 2026-03-09 07:32:33,169 - lava_callback - DEBUG - Processing node 69abd48defa461f55e59f53d: baseline-arm64-clabbe 2026-03-09 07:32:33,169 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd48defa461f55e59f53d 2026-03-09 07:32:33,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:32:33,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd48defa461f55e59f53d HTTP/1.1" 200 None 2026-03-09 07:32:33,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:32:33,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:32:33,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:32:33,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd48defa461f55e59f53d HTTP/1.1" 200 None 2026-03-09 07:32:33,227 - lava_callback - ERROR - Error processing callback for node 69abd48defa461f55e59f53d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59840 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:33:53,271 - lava_callback - INFO - Received callback for node 69abd4daefa461f55e59f54a INFO: 172.21.0.1:40198 - "POST /node/69abd4daefa461f55e59f54a HTTP/1.0" 202 Accepted 2026-03-09 07:33:53,277 - lava_callback - INFO - Starting async job submit for node 69abd4daefa461f55e59f54a 2026-03-09 07:33:53,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:33:53,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd4daefa461f55e59f54a HTTP/1.1" 200 None 2026-03-09 07:33:53,292 - lava_callback - DEBUG - Processing node 69abd4daefa461f55e59f54a: baseline-arm64-clabbe 2026-03-09 07:33:53,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd4daefa461f55e59f54a 2026-03-09 07:33:53,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:33:53,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd4daefa461f55e59f54a HTTP/1.1" 200 None 2026-03-09 07:33:53,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:33:53,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 07:33:53,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:33:53,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd4daefa461f55e59f54a HTTP/1.1" 200 None 2026-03-09 07:33:53,347 - lava_callback - ERROR - Error processing callback for node 69abd4daefa461f55e59f54a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46102 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:35:13,767 - lava_callback - INFO - Received callback for node 69abd520efa461f55e59f560 INFO: 172.21.0.1:36844 - "POST /node/69abd520efa461f55e59f560 HTTP/1.0" 202 Accepted 2026-03-09 07:35:13,772 - lava_callback - INFO - Starting async job submit for node 69abd520efa461f55e59f560 2026-03-09 07:35:13,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:35:13,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd520efa461f55e59f560 HTTP/1.1" 200 None 2026-03-09 07:35:13,789 - lava_callback - DEBUG - Processing node 69abd520efa461f55e59f560: baseline-arm64-baylibre 2026-03-09 07:35:13,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd520efa461f55e59f560 2026-03-09 07:35:13,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:35:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd520efa461f55e59f560 HTTP/1.1" 200 None 2026-03-09 07:35:13,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:35:13,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:35:13,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:35:13,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd520efa461f55e59f560 HTTP/1.1" 200 None 2026-03-09 07:35:13,852 - lava_callback - ERROR - Error processing callback for node 69abd520efa461f55e59f560: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47482 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:36:33,871 - lava_callback - INFO - Received callback for node 69abd56eefa461f55e59f572 INFO: 172.21.0.1:37874 - "POST /node/69abd56eefa461f55e59f572 HTTP/1.0" 202 Accepted 2026-03-09 07:36:33,877 - lava_callback - INFO - Starting async job submit for node 69abd56eefa461f55e59f572 2026-03-09 07:36:33,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:36:33,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd56eefa461f55e59f572 HTTP/1.1" 200 None 2026-03-09 07:36:33,893 - lava_callback - DEBUG - Processing node 69abd56eefa461f55e59f572: baseline-arm64-baylibre 2026-03-09 07:36:33,893 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd56eefa461f55e59f572 2026-03-09 07:36:33,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:36:33,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd56eefa461f55e59f572 HTTP/1.1" 200 None 2026-03-09 07:36:33,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:36:33,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:36:33,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:36:33,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd56eefa461f55e59f572 HTTP/1.1" 200 None 2026-03-09 07:36:33,949 - lava_callback - ERROR - Error processing callback for node 69abd56eefa461f55e59f572: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55308 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:37:53,894 - lava_callback - INFO - Received callback for node 69abd5c1efa461f55e59f58a INFO: 172.21.0.1:48672 - "POST /node/69abd5c1efa461f55e59f58a HTTP/1.0" 202 Accepted 2026-03-09 07:37:53,900 - lava_callback - INFO - Starting async job submit for node 69abd5c1efa461f55e59f58a 2026-03-09 07:37:53,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:37:53,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd5c1efa461f55e59f58a HTTP/1.1" 200 None 2026-03-09 07:37:53,916 - lava_callback - DEBUG - Processing node 69abd5c1efa461f55e59f58a: baseline-arm64-baylibre 2026-03-09 07:37:53,916 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd5c1efa461f55e59f58a 2026-03-09 07:37:53,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:37:53,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd5c1efa461f55e59f58a HTTP/1.1" 200 None 2026-03-09 07:37:53,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:37:53,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:37:53,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:37:53,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd5c1efa461f55e59f58a HTTP/1.1" 200 None 2026-03-09 07:37:53,973 - lava_callback - ERROR - Error processing callback for node 69abd5c1efa461f55e59f58a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43890 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:39:13,972 - lava_callback - INFO - Received callback for node 69abd60fefa461f55e59f59e INFO: 172.21.0.1:48788 - "POST /node/69abd60fefa461f55e59f59e HTTP/1.0" 202 Accepted 2026-03-09 07:39:13,977 - lava_callback - INFO - Starting async job submit for node 69abd60fefa461f55e59f59e 2026-03-09 07:39:13,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:39:13,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd60fefa461f55e59f59e HTTP/1.1" 200 None 2026-03-09 07:39:13,993 - lava_callback - DEBUG - Processing node 69abd60fefa461f55e59f59e: baseline-arm64-baylibre 2026-03-09 07:39:13,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd60fefa461f55e59f59e 2026-03-09 07:39:13,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:39:14,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd60fefa461f55e59f59e HTTP/1.1" 200 None 2026-03-09 07:39:14,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:39:14,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:39:14,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:39:14,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd60fefa461f55e59f59e HTTP/1.1" 200 None 2026-03-09 07:39:14,054 - lava_callback - ERROR - Error processing callback for node 69abd60fefa461f55e59f59e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:40:34,123 - lava_callback - INFO - Received callback for node 69abd670efa461f55e59f5b6 INFO: 172.21.0.1:56344 - "POST /node/69abd670efa461f55e59f5b6 HTTP/1.0" 202 Accepted 2026-03-09 07:40:34,129 - lava_callback - INFO - Starting async job submit for node 69abd670efa461f55e59f5b6 2026-03-09 07:40:34,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:40:34,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd670efa461f55e59f5b6 HTTP/1.1" 200 None 2026-03-09 07:40:34,151 - lava_callback - DEBUG - Processing node 69abd670efa461f55e59f5b6: baseline-arm64-baylibre 2026-03-09 07:40:34,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd670efa461f55e59f5b6 2026-03-09 07:40:34,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:40:34,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd670efa461f55e59f5b6 HTTP/1.1" 200 None 2026-03-09 07:40:34,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:40:34,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:40:34,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:40:34,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd670efa461f55e59f5b6 HTTP/1.1" 200 None 2026-03-09 07:40:34,213 - lava_callback - ERROR - Error processing callback for node 69abd670efa461f55e59f5b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59972 - "GET /node/6991f7c201ff1e8e0859d442%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54204 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:41:54,194 - lava_callback - INFO - Received callback for node 69abd6bbefa461f55e59f5cb INFO: 172.21.0.1:53434 - "POST /node/69abd6bbefa461f55e59f5cb HTTP/1.0" 202 Accepted 2026-03-09 07:41:54,199 - lava_callback - INFO - Starting async job submit for node 69abd6bbefa461f55e59f5cb 2026-03-09 07:41:54,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:41:54,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd6bbefa461f55e59f5cb HTTP/1.1" 200 None 2026-03-09 07:41:54,224 - lava_callback - DEBUG - Processing node 69abd6bbefa461f55e59f5cb: baseline-arm64-baylibre 2026-03-09 07:41:54,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd6bbefa461f55e59f5cb 2026-03-09 07:41:54,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:41:54,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd6bbefa461f55e59f5cb HTTP/1.1" 200 None 2026-03-09 07:41:54,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:41:54,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:41:54,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:41:54,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd6bbefa461f55e59f5cb HTTP/1.1" 200 None 2026-03-09 07:41:54,280 - lava_callback - ERROR - Error processing callback for node 69abd6bbefa461f55e59f5cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42538 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59330 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:50850 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:43:14,241 - lava_callback - INFO - Received callback for node 69abd70eefa461f55e59f5ec INFO: 172.21.0.1:56806 - "POST /node/69abd70eefa461f55e59f5ec HTTP/1.0" 202 Accepted 2026-03-09 07:43:14,246 - lava_callback - INFO - Starting async job submit for node 69abd70eefa461f55e59f5ec 2026-03-09 07:43:14,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:43:14,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd70eefa461f55e59f5ec HTTP/1.1" 200 None 2026-03-09 07:43:14,277 - lava_callback - DEBUG - Processing node 69abd70eefa461f55e59f5ec: baseline-arm64-baylibre 2026-03-09 07:43:14,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd70eefa461f55e59f5ec 2026-03-09 07:43:14,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:43:14,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd70eefa461f55e59f5ec HTTP/1.1" 200 None 2026-03-09 07:43:14,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:43:14,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:43:14,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:43:14,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd70eefa461f55e59f5ec HTTP/1.1" 200 None 2026-03-09 07:43:14,339 - lava_callback - ERROR - Error processing callback for node 69abd70eefa461f55e59f5ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40672 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:44:34,300 - lava_callback - INFO - Received callback for node 69abd758efa461f55e59f609 INFO: 172.21.0.1:49740 - "POST /node/69abd758efa461f55e59f609 HTTP/1.0" 202 Accepted 2026-03-09 07:44:34,306 - lava_callback - INFO - Starting async job submit for node 69abd758efa461f55e59f609 2026-03-09 07:44:34,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:44:34,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd758efa461f55e59f609 HTTP/1.1" 200 None 2026-03-09 07:44:34,323 - lava_callback - DEBUG - Processing node 69abd758efa461f55e59f609: baseline-arm64-baylibre 2026-03-09 07:44:34,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd758efa461f55e59f609 2026-03-09 07:44:34,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:44:34,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd758efa461f55e59f609 HTTP/1.1" 200 None 2026-03-09 07:44:34,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:44:34,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:44:34,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:44:34,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd758efa461f55e59f609 HTTP/1.1" 200 None 2026-03-09 07:44:34,381 - lava_callback - ERROR - Error processing callback for node 69abd758efa461f55e59f609: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34824 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:47:14,461 - lava_callback - INFO - Received callback for node 69abd7f6efa461f55e59f638 INFO: 172.21.0.1:56042 - "POST /node/69abd7f6efa461f55e59f638 HTTP/1.0" 202 Accepted 2026-03-09 07:47:14,467 - lava_callback - INFO - Starting async job submit for node 69abd7f6efa461f55e59f638 2026-03-09 07:47:14,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:47:14,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd7f6efa461f55e59f638 HTTP/1.1" 200 None 2026-03-09 07:47:14,483 - lava_callback - DEBUG - Processing node 69abd7f6efa461f55e59f638: baseline-arm64-baylibre 2026-03-09 07:47:14,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd7f6efa461f55e59f638 2026-03-09 07:47:14,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:47:14,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd7f6efa461f55e59f638 HTTP/1.1" 200 None 2026-03-09 07:47:14,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:47:14,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:47:14,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:47:14,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd7f6efa461f55e59f638 HTTP/1.1" 200 None 2026-03-09 07:47:14,546 - lava_callback - ERROR - Error processing callback for node 69abd7f6efa461f55e59f638: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:50:14,635 - lava_callback - INFO - Received callback for node 69abd8abefa461f55e59f662 INFO: 172.21.0.1:48148 - "POST /node/69abd8abefa461f55e59f662 HTTP/1.0" 202 Accepted 2026-03-09 07:50:14,641 - lava_callback - INFO - Starting async job submit for node 69abd8abefa461f55e59f662 2026-03-09 07:50:14,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:50:14,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd8abefa461f55e59f662 HTTP/1.1" 200 None 2026-03-09 07:50:14,658 - lava_callback - DEBUG - Processing node 69abd8abefa461f55e59f662: baseline-arm64-baylibre 2026-03-09 07:50:14,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd8abefa461f55e59f662 2026-03-09 07:50:14,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:50:14,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd8abefa461f55e59f662 HTTP/1.1" 200 None 2026-03-09 07:50:14,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:50:14,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:50:14,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:50:14,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd8abefa461f55e59f662 HTTP/1.1" 200 None 2026-03-09 07:50:14,718 - lava_callback - ERROR - Error processing callback for node 69abd8abefa461f55e59f662: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:51:34,653 - lava_callback - INFO - Received callback for node 69abd8f8efa461f55e59f679 INFO: 172.21.0.1:35614 - "POST /node/69abd8f8efa461f55e59f679 HTTP/1.0" 202 Accepted 2026-03-09 07:51:34,659 - lava_callback - INFO - Starting async job submit for node 69abd8f8efa461f55e59f679 2026-03-09 07:51:34,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:51:34,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd8f8efa461f55e59f679 HTTP/1.1" 200 None 2026-03-09 07:51:34,675 - lava_callback - DEBUG - Processing node 69abd8f8efa461f55e59f679: baseline-arm64-baylibre 2026-03-09 07:51:34,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd8f8efa461f55e59f679 2026-03-09 07:51:34,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:51:34,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd8f8efa461f55e59f679 HTTP/1.1" 200 None 2026-03-09 07:51:34,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:51:34,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:51:34,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:51:34,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd8f8efa461f55e59f679 HTTP/1.1" 200 None 2026-03-09 07:51:34,742 - lava_callback - ERROR - Error processing callback for node 69abd8f8efa461f55e59f679: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43230 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:54:54,872 - lava_callback - INFO - Received callback for node 69abd9cbefa461f55e59f6b7 INFO: 172.21.0.1:46208 - "POST /node/69abd9cbefa461f55e59f6b7 HTTP/1.0" 202 Accepted 2026-03-09 07:54:54,878 - lava_callback - INFO - Starting async job submit for node 69abd9cbefa461f55e59f6b7 2026-03-09 07:54:54,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:54:54,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd9cbefa461f55e59f6b7 HTTP/1.1" 200 None 2026-03-09 07:54:54,894 - lava_callback - DEBUG - Processing node 69abd9cbefa461f55e59f6b7: baseline-arm64-baylibre 2026-03-09 07:54:54,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abd9cbefa461f55e59f6b7 2026-03-09 07:54:54,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:54:54,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abd9cbefa461f55e59f6b7 HTTP/1.1" 200 None 2026-03-09 07:54:54,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:54:54,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:54:54,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:54:54,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abd9cbefa461f55e59f6b7 HTTP/1.1" 200 None 2026-03-09 07:54:54,952 - lava_callback - ERROR - Error processing callback for node 69abd9cbefa461f55e59f6b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43718 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:56:15,015 - lava_callback - INFO - Received callback for node 69abda18efa461f55e59f6ce INFO: 172.21.0.1:52874 - "POST /node/69abda18efa461f55e59f6ce HTTP/1.0" 202 Accepted 2026-03-09 07:56:15,022 - lava_callback - INFO - Starting async job submit for node 69abda18efa461f55e59f6ce 2026-03-09 07:56:15,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:56:15,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abda18efa461f55e59f6ce HTTP/1.1" 200 None 2026-03-09 07:56:15,044 - lava_callback - DEBUG - Processing node 69abda18efa461f55e59f6ce: baseline-arm64-baylibre 2026-03-09 07:56:15,044 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abda18efa461f55e59f6ce 2026-03-09 07:56:15,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:56:15,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abda18efa461f55e59f6ce HTTP/1.1" 200 None 2026-03-09 07:56:15,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:56:15,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:56:15,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:56:15,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abda18efa461f55e59f6ce HTTP/1.1" 200 None 2026-03-09 07:56:15,120 - lava_callback - ERROR - Error processing callback for node 69abda18efa461f55e59f6ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49128 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:57:35,052 - lava_callback - INFO - Received callback for node 69abda64efa461f55e59f6ea INFO: 172.21.0.1:46762 - "POST /node/69abda64efa461f55e59f6ea HTTP/1.0" 202 Accepted 2026-03-09 07:57:35,057 - lava_callback - INFO - Starting async job submit for node 69abda64efa461f55e59f6ea 2026-03-09 07:57:35,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:57:35,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abda64efa461f55e59f6ea HTTP/1.1" 200 None 2026-03-09 07:57:35,074 - lava_callback - DEBUG - Processing node 69abda64efa461f55e59f6ea: baseline-arm64-baylibre 2026-03-09 07:57:35,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abda64efa461f55e59f6ea 2026-03-09 07:57:35,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:57:35,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abda64efa461f55e59f6ea HTTP/1.1" 200 None 2026-03-09 07:57:35,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:57:35,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:57:35,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:57:35,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abda64efa461f55e59f6ea HTTP/1.1" 200 None 2026-03-09 07:57:35,133 - lava_callback - ERROR - Error processing callback for node 69abda64efa461f55e59f6ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49282 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:58:55,095 - lava_callback - INFO - Received callback for node 69abdaaeefa461f55e59f6f6 INFO: 172.21.0.1:36416 - "POST /node/69abdaaeefa461f55e59f6f6 HTTP/1.0" 202 Accepted 2026-03-09 07:58:55,101 - lava_callback - INFO - Starting async job submit for node 69abdaaeefa461f55e59f6f6 2026-03-09 07:58:55,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:58:55,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdaaeefa461f55e59f6f6 HTTP/1.1" 200 None 2026-03-09 07:58:55,118 - lava_callback - DEBUG - Processing node 69abdaaeefa461f55e59f6f6: baseline-arm64-baylibre 2026-03-09 07:58:55,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abdaaeefa461f55e59f6f6 2026-03-09 07:58:55,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:58:55,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdaaeefa461f55e59f6f6 HTTP/1.1" 200 None 2026-03-09 07:58:55,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:58:55,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:58:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:58:55,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abdaaeefa461f55e59f6f6 HTTP/1.1" 200 None 2026-03-09 07:58:55,195 - lava_callback - ERROR - Error processing callback for node 69abdaaeefa461f55e59f6f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58178 - "GET / HTTP/1.0" 200 OK 2026-03-09 07:59:55,099 - lava_callback - INFO - Received callback for node 69abdaf9efa461f55e59f719 INFO: 172.21.0.1:37318 - "POST /node/69abdaf9efa461f55e59f719 HTTP/1.0" 202 Accepted 2026-03-09 07:59:55,105 - lava_callback - INFO - Starting async job submit for node 69abdaf9efa461f55e59f719 2026-03-09 07:59:55,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:59:55,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdaf9efa461f55e59f719 HTTP/1.1" 200 None 2026-03-09 07:59:55,139 - lava_callback - DEBUG - Processing node 69abdaf9efa461f55e59f719: baseline-arm64-baylibre 2026-03-09 07:59:55,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69abdaf9efa461f55e59f719 2026-03-09 07:59:55,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:59:55,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdaf9efa461f55e59f719 HTTP/1.1" 200 None 2026-03-09 07:59:55,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:59:55,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-09 07:59:55,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 07:59:55,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69abdaf9efa461f55e59f719 HTTP/1.1" 200 None 2026-03-09 07:59:55,214 - lava_callback - ERROR - Error processing callback for node 69abdaf9efa461f55e59f719: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]